The Strategic Value of Specialist Designers

Nev Flynn argues that while early-stage startups require generalists, scaling a design team effectively requires hiring specialists. At ElevenLabs, bringing in experts—such as a dedicated sound designer and a WebGL professional—has allowed the team to transcend the limitations of standard AI-generated outputs. Flynn notes that while AI raises the floor for everyone, true differentiation comes from specialists who can push creative boundaries beyond what models provide out-of-the-box.

Engineering Artful Experiences: The Voice Orb System

One of the most significant projects at ElevenLabs involved visualizing voice data. Since voice is inherently non-visual, the team developed a system to generate unique "voice orbs" for 40 million voices. They achieved this by extracting voice embeddings—the data defining a voice's characteristics—and using them to drive real-time WebGL shaders.

This process involved:

  • Data Mapping: Using pitch and expressiveness as primary traits to influence visual output.
  • Layered Shaders: Applying six distinct mathematical passes (including fluid simulations, noise, and audio-reactive soundwaves) to a base texture.
  • Performance Optimization: Moving from an initial, costly implementation to a highly cached, performant system that reduced costs by a significant margin.

Maintaining Culture Through Creative Outlets

Flynn emphasizes that enterprise growth often leads to the loss of a company's early creative spark. To combat this, ElevenLabs intentionally carves out space for "artful expression" outside of their core product brand. This includes:

  • Event Identities: Creating unique, site-specific branding for summits that avoids generic "iconic" imagery in favor of tasteful, abstract graphical systems.
  • Physical Installations: Partnering with artists to create audio-reactive environments, such as the 11-monolith installation.
  • Experimental Web Projects: Allowing designers to build custom identities for hackathons or internal tools, which serves as a powerful magnet for top-tier design talent.

Designing for AI and Agents

As the company shifts toward agent-native design, Flynn highlights the challenge of designing interfaces for models that are constantly evolving. The design team's role is to invent coherent UIs for capabilities that didn't exist previously. This requires a shift in mindset: rather than just designing static screens, the team is building systems that can adapt to the unpredictable nature of AI-driven outputs.