The Shift from Text to Voice

Voice-first AI is not simply text-to-speech; it fundamentally invalidates the assumptions of traditional LLM applications. In text chat, turns are slow and context is stable. In voice, users are volatile—they interrupt, change subjects mid-sentence, and talk while performing other tasks. The Tolan team found that even a 500ms latency drift significantly degraded user metrics. To manage this, they optimized for "fewer bad interruptions" rather than "fewer interruptions," building turn detection that analyzes speech patterns to prevent premature cutoffs, even at the cost of 60ms of latency.

Pipeline Optimization and Model Routing

To maintain a conversational feel, the team measures every stage of the pipeline independently: end-of-utterance detection, transcription, time-to-first-token, and text-to-speech playback. They employ a tiered model strategy, routing turns based on emotional stakes rather than cost. High-stakes moments (onboarding, crisis intervention) use frontier models, while casual chatter is handled by smaller, faster, and cheaper models. This routing strategy is essential for unit economics, as frontier models can be five times more expensive than smaller alternatives, yet the team found that offloading a third of turns to smaller models had no measurable impact on user retention.

Dynamic Memory and Context Management

Traditional transcript-based memory fails in voice-first apps because it degrades over long sessions and leads to hallucinations. Instead, the team treats memory as a retrieval system. Facts, preferences, and emotional signals are embedded in a vector database with sub-50ms lookups. Every night, the system compresses these memories by merging duplicates and resolving contradictions. Furthermore, the team avoids reusing context across turns to prevent "drift." Instead, they reassemble the context window every turn using a combination of recent message summaries, persona cards, retrieved memories, and real-time app state.

AI-Assisted Engineering

Beyond the product itself, the team uses a fleet of agents to accelerate development. These agents handle implementation, code review, and bug triage. By treating the codebase as the primary documentation, they achieved a 99.9% crash-free rate and reduced runtime errors by over 50%. The team notes that managing these agents requires skills similar to human management: decomposing problems, delegating with checkpoints, and providing fast feedback. This approach allows for rapid iteration, such as developing and testing a new character persona in a single afternoon.