It started with an interface dream. What if a student could do math and get help with math at the same time? With Khanmigo and ChatGPT, students have no place to write and draw. I wanted tutoring to feel like sharing a whiteboard with a tutor - with students solving problems at the center.

My good friend was on sabbatical from Schoolhouse, an offshoot of Khan Academy. We were both interested in whether LLMs were capable of good tutoring, and we felt potential in interfaces beyond chatbots. So we teamed up, built an iPad app prototype, and called it frog tutor.

We first built out a “problem mode” where you upload your own problem, work through it independently, and get tips when you need them.** I showed the “problem mode” but we built out a lot more, including a “learn anything mode” and diagrams. If there’s interest, perhaps I’ll do a writeup showing the full interface as well as our approach to finding students, prompting + evals, and diagrams. Then, we got students to try it. Alongside that, we tutored students to remind ourselves what they really need.

Let’s see some examples of students using frog, with two questions in mind. Is frog doing the mental work for the student? Is frog encouraging interesting thinking?

Example 1
Arush: How should I go about solving this?

Frog: Start by recalling what “root” even means for an equation. What does it tell you about the equation when you plug in 4?

Arush: The root is the zeros, the, the x-intercepts.

Frog: Yep, exactly right.

Arush: So would I do y equals a times x minus 4 times x minus 4? But I don’t think I can do that, can I?

Frog: That actually works. What’s making you doubt it?

Arush: Okay. I just thought it has to be like 2 different numbers, and, um, I don’t know, I don’t know. I don’t think you could do x minus 4 and x minus 4. I knew you could do the same thing, but I thought it would be minus 4 plus 4 or something like that.

Frog: If you used minus 4 and plus 4, plugging in negative 4 would also give zero, so you’d have two roots, not one. Repeating the same factor is exactly how you force 4 to be the only one.

Arush: All right, thank you.

Arush starts off stuck but after a small prod, he tries it and gets it right. Frog probes Arush’s doubt, which Arush verbalizes and frog clears up. It would’ve been nice if frog drew a quadratic with one root and a quadratic with two roots and asked Arush some questions - but overall pretty decent.

Example 2
David: “I don’t have a calculator, so I don’t know.”

Frog: “No calculator needed. The absolute value of 4 is just 4, so what’s F of 4?”

David: “Is f of 4, 4?”

Frog: “Yes, exactly right.”

This is not good tutoring but unfortunately it does happen somewhat often. David feels he needs a calculator to find the absolute value of 4 - this shows he doesn’t understand the basic mechanics. A good tutor would probe and review these mechanics, along with the core concept. Instead frog gave him the tiny step he needed, and he solved the problem and moved on.

Example 3
Jessica: Okay, let’s do it. One of these coins is randomly chosen and flipped 10 times. What’s the probability that the coin lands on heads on exactly 7 of the 10 flips? Okay...

Frog: Nice setup. Now think about how to combine those two cases for part a.

Jessica: Oh. We’re gonna let X₁ be the number of heads given we chose coin one, and X₂ the number of heads given we chose coin two... Our k is going to be exactly seven. And so it’s going to be probability of X₁ equals seven times probability of one, plus probability of X₂ equals seven times probability of two. So let’s look at that.

Frog: Looks great, go for it.

Jessica would share her understanding by writing out her work and explaining her reasoning. She would then use frog to check her reasoning. Instead of feeling “maybe this setup is right but I’m fuzzy” and then “maybe this use of conditional probability is right but I’m fuzzy” and then further stacking uncertainty, she builds a solution confidently one step at a time.

Jessica has used frog for dozens of hours and >500 messages. For her, it brings out interesting and critical thinking, and it’s more engaging than a textbook. She develops her intuition and confidence. She texted us, “I’ve been using it to study my probability class! It’s been very helpful, so thank you!! I’m super impressed it can understand my handwriting, the problem context, and process my audio at the same time to give an answer that makes sense & is correct.”

Other students weren’t as comfortable writing math on an iPad, especially the younger middle schoolers. Still others didn’t want to engage with an AI tutor at all, even though they would work with me and my friend.

We eventually lost momentum, and I’m wrapping up frog in a few days. Students used it for schoolwork, then summer came and nearly all students stopped using it. The initial interface dream resonated, but a poor model tip could break the spell. When I read through dozens of student transcripts, I felt that the tips and conversational flow were just ok. We used Opus 4.8 at the time, and now Fable 5 feels like a step up in quality. I’m optimistic about AI tutors in the next year or two.