Rebuilding Allie:
Allen's AI powered
doubt solver
How I rebuilt Allen's AI doubt solver to recover from a 35% decline after ChatGPT Study Mode and Gemini reshaped what students expect from AI.
Please view in
desktop for now
Rebuilding Allie: Allen's AI-powered doubt solver
How I rebuilt Allen's AI doubt solver to recover from a 35% decline after ChatGPT Study Mode and Gemini reshaped what students expect from AI.
meet Allie
In eight weeks, Allie lost a third of its users and monthly doubts fell 35%. Nothing in the product broke. The market reset, and students left for better products. As the sole product designer, I redesigned the experience around four product initiatives.
Students who stayed also asked more often.
The Market Reset
Two launches, six weeks, a third of our users gone.
Overnight, every student had a frontier AI tutor for free, and Allie's one-shot answer (one length, one language, one method) became the weakest option they had.
The Diagnosis
Students weren't using Allie less. They were leaving.
Before designing anything, I needed to know which problem this was: were students asking fewer doubts, or leaving entirely?
Students who stayed behaved almost exactly as before.
Why They Left
A survey, interviews and two benchmarks asked why students left. A journey analysis showed where doubts were getting lost.
churned students surveyed over WhatsApp
in-depth interviews
benchmarks against the products students left for
of doubts started right after a test
Journey analysisThe survey: ranking the complaints
AI answers were correct but hard to understand
The highest-cited reason. Formatting, structure and pedagogy were the issue, not accuracy.
- 2Answers felt slow: 15–30s to first response
- 3No video solution for many questions
- 4AI answers were wrong (a smaller cohort)
- 5Teacher solutions weren't helpful
What interviews revealed
The power user
Allie ignored custom formatting instructions and missed errors.
The speed purist
15–30s latency and messy image replies. Now solves 90% of doubts on Google.
The below-average student
Lengthy, inaccurate answers and zero Hinglish support.
Every interview pointed to the same pattern:
Trust broke after ~2 bad answers
Try, get a bad answer, try once more, never return. 3 of 4 said it unprompted.
Understanding, not answers
They wanted the approach, the concept, the mistake.
Teach like their teacher
A method that differed from class felt wrong.
“Approach nahi samjha, concept nahi samjha.
Tell me what mistake I made.”
Recurring phrases across churned-student interviews
Student interview
Student interviewBenchmark: Allie fails quietly, ChatGPT fails confidently
Allie, ChatGPT and Filo on the same 26 JEE & NEET doubts.
Failed quietly: under-explained answers hid decent accuracy.
Failed confidently: beautifully formatted, factually wrong.
Consistently correct and structured.
Benchmark 2: Defining a quality standard
The same 10+ class 6 to 10 doubts on Allie, Gemini and GPAI, judged on correctness, scannability, depth and visual aids.
Journey analysis: where doubts were dying
Analysing the doubt journey showed that 70% of doubts started from an MCQ image right after a test, yet asking one meant leaving review and re-uploading the question. Interviews confirmed it: doubts weren't disappearing, they were dying on the way.
Each dot is a doubt, and the dashed ones are lost at that step. The counts are illustrative, not real data.
The Solution
Four initiatives, one per churn driver, each shippable on its own so early data could steer the harder work.
Solve doubts where they happen
Removing context re-entry unlocks lost doubts
See initiative → 02RigidityLet students choose how they're taught
Choice of format stops defection
See initiative → 03QualityImprove the structure of answers
Fixing comprehension rebuilds trust
See initiative → 04Dead endsSend fewer, better doubts to teachers
Higher-intent escalations let teachers keep pace
See initiative →Solve doubts where they happen
Embed Allie inside test review and pass the question context automatically. If the context-switch hypothesis was right, the lost doubts would come back.
Doubts are born on many screens; one floater covers them all without a separate build for each.
Next: make it draggable, or shrink on scroll, so it never hides the solution.
Impact of redesign
The doubts were always there.
The flow was losing them.
Let students choose how they're taught
The same student wants a ten-second check on Tuesday and a full walkthrough on Thursday. The power users we lost said it directly: they couldn't shape the response.
Detailed
The full solution, for students who want the complete approach.
Guided
A Socratic walkthrough that leads the student to the answer.
Short
A concise answer for quick checks and confirmation.
Pill in composerA mode pill inside the composer. Cleanest, but easy to miss.
Tabs on topMode tabs at the top. The subtext changes with the mode.
Tabs below AllieThe same tabs, placed below Allie.
Tab carouselA carousel above the composer with subtext for each mode. Fully visible.
Tabs in composerTabs inside the composer. The placeholder text changes with the mode.
Detailed already works for most students. The pill keeps “type and send” simple, and modes reappear after the first answer, when students actually decide.
Trade-off: it's the least noticeable of the five.
Impact of redesign
Improve the structure of answers
I turned the benchmarks into response guidelines, so “quality” became a checklist instead of an opinion.
36+ doubts scored across five solvers
Buried answers, no bifurcation, no visuals, method drift
Headers, pointers, bifurcation, solved like an Allen teacher
Data Science tuned model output to the guidelines
Send fewer, better doubts to teachers
Teacher answers ranked #5 among churn reasons, hurt NPS and took 24+ hours. Quality wasn't a product problem, so I didn't redesign the destination; I changed who reached it and what they expected.
Reduced prominence
“Ask teacher” moved from a quick action to a secondary pill.
Deliberate friction
A mandatory note filters low-intent escalations and gives teachers context.
Honest wait time
Expected turnaround is shown upfront, so waiting is a choice.
Impact of redesign
What still needs fixing
Latency (15–30s)
Owned by engineering and data science. Pre-caching helps, but it needs a real fix.
Video coverage
I'm designing AI-generated explainers for popular questions and validating them through usability testing.
Language selection
A key ask from research. Ready to launch, but paused on technical issues.
Guided mode for concepts
Guided only works for problem-solving questions; it loses its Socratic approach on conceptual ones.
Key Takeaways
Diagnose before you design
Splitting a 35% drop into user churn vs. usage frequency showed the real issue: students were leaving for better AI alternatives.
External benchmarks reset the bar, fast
Benchmark against evolving AI experiences, not just direct competitors.
Layered initiatives beat one big swing
Four independent initiatives let early wins validate the strategy before the harder work.
