Retention study: Diagnosing Why a Popular International Language-Learning App Was Losing India

Client: A popular international language-learning app (name withheld under NDA)

Agency: 1990 Research Labs

My role: One of two researchers on the study (my manager and me), with a research-ops associate. I built the AI voice agent behind the 50-user longitudinal study and ran the brand dipstick end-to-end; I split the field interviews with my manager, and we analyzed and synthesized as a team.

Timeline: 3 months

Quick snapshot

• Project: Diagnosing early churn in India for a language-learning app (under NDA)
• Goal: Identify week-one churn drivers and propose shippable fixes for early retention
• Team: 2 researchers + 1 research-ops associate
• Constraints: NDA; multi-city fieldwork; mixed language contexts; voice-agent calls capped at 15–20 minutes

TL;DR

The reframe: the retention gap wasn't primarily technical (devices, networks, offline). It was driven by expectation mismatch, localization, and habit mechanics - plus a speaking-practice experience learners loved but couldn't access.

Impact: That reframing unlocked four changes the client shipped in India (detailed below): onboarding difficulty placement, a re-ordered sign-in flow, a localized streak mechanic, and free access to the speaking feature.

What surprised stakeholders

• Device and network constraints were not the primary churn driver: learners had 5G/Wi-Fi; going offline was a choice, not a limitation.
• "Too easy" (boredom) surfaced as a real drop-off reason: the opposite of the "it's too hard / too technical" hypothesis.
• Speaking practice created real excitement, but the experience that delivered it was gated behind a paywall in the highest-acquisition market.

The story

India was the app's biggest market for new downloads — but most people disappeared within a week. Inside the team, the leading theory was technical: low-end devices, patchy networks, no offline mode.

We went into homes and campuses and watched onboarding happen live, then followed a separate cohort week by week. A different pattern emerged: the biggest drop-offs came from small but compounding mismatches - difficulty placement, localized habit cues, sign-in friction, and a speaking experience people wanted but couldn't easily reach. That reframing is what turned a vague "India is hard" into a short list of things the client could actually ship.

The challenge

Research questions

• What drives churn within the first 7 days?
• Where does onboarding / early learning create mismatch?
• Is drop-off driven by difficulty or boredom?
• What sustains use after habit mechanics kick in?

What we ran (methods)

Key pivots (why the plan changed):

• Deferred a planned survey once the qualitative + longitudinal signals were strong enough to act on.
• Adjusted sampling across Indore, nearby towns, and Pune (English-UI learners).
• Ran a quick "difficulty shift" test when boredom surfaced in the week-3 check-ins.

Findings

1 · Onboarding placed users too low, and they quit from boredom

The placement test read basic vocabulary as beginner-level knowledge, so learners who could read English but hesitated to speak got funnelled into lessons far below their level.

"I already told the app I know common words — so why is it showing me easy ones? Even fifth- or sixth-grade kids would do this. From the graphics, it feels like a kids' app." — Field participant, Indore

So what: under-challenged learners decided the app was a toy, not a tool, and left before ever reaching the advanced content.

What we did: we spotted the pattern in the longitudinal check-ins, then ran a week-4 difficulty shift, moving a subset up to intermediate, as a directional test. Engagement jumped, and near-churn users came back:

"Earlier it was getting boring… now whenever I get free time, I open it on my own." — Longitudinal participant who had stopped because it felt "boring"

2 · A localized habit term broke a core retention mechanic

Learners instantly recognised "streak" from Snapchat , but the Hindi label "Silsila" didn't trigger the same association.

Asked what "Silsila" meant, one learner guessed "maybe a type of goal, a target to finish in so many days." Asked about a "streak," the same learner described Snapchat streaks exactly. — Field participant

So what: the concept worked, but the label blocked it, a retention mechanic that was effectively switched off for Hindi users.

3 · The sign-in flow didn't match how Indians create accounts

Sign-in options were easy to miss, and the email-plus-password pattern collided with the phone-plus-OTP flow every major Indian app uses. Password creation became a wall at the exact moment motivation is most fragile.

"I normally put my name and 1234 - but it's asking me to create a strong password." — Field participant

So what: unnecessary friction at first signup (and again when returning users tried to log back in) turned fragile intent into drop-off.

4 · Learners came for speaking confidence - and the feature that delivered it was paywalled

People arrived wanting to speak, not tap tiles. The gap was confidence, not vocabulary.

"The sentence forms in my mind, everything forms. But as soon as I go to speak - I go blank." — Field participant

When participants tried the app's AI video-call speaking exercise during interviews, the reaction was consistently strong, they called it a "reality check." But it sat behind a paid subscription, out of reach for most learners in the highest-acquisition market.

So what: the one experience aligned to users' real goal- speaking under pressure, judgment-free, was the one they couldn't access.

What shipped

Building the voice agent

We needed weekly check-ins from 50 participants over four weeks, but having a researcher call everyone, and probe well, wasn't realistic at that scale. So I built a lightweight AI voice agent that could run short, high-signal conversations: reliable enough to run unsupervised, and structured enough to compare week-to-week.

• Why an agent: to make "what changed this week?" observable week by week, not reconstructed from memory later.
• Design constraint: keep calls to 15–20 minutes and front-load the highest-value questions.
• What I built: a branching question ladder with probing prompts, iterated through pilots until it held up with strangers.

Limitations

• External client project under NDA - I'm sharing outcomes and decision logic without proprietary metrics.
• In-person insights skew toward the early journey; the month-completer in-depth interviews (10 drawn from the longitudinal cohort) were added to cover later-stage retention.
• Fieldwork covered several contexts but not all of India - findings should be re-validated when expanding.

Next steps (if extending)

• Quantify the strongest hypotheses post-ship (survey or analytics).
• Usability-test the revised onboarding and sign-in sequence.
• Investigate "speaking confidence" as an explicit early value proposition.