Karpathy: Vibe Coding to Agentic Engineering Shift
Andrej Karpathy describes evolving from 'vibe coding'—where anyone can build quickly with AI—to 'agentic engineering,' a disciplined practice coordinating jagged LLMs as 'ghosts' to ship production-quality software faster than ever.
Software 3.0: Prompting LLMs as the New Programming Paradigm
Andrej Karpathy frames the current AI shift as Software 3.0, where LLMs become programmable interpreters. In Software 1.0, you write explicit rules in code. Software 2.0 involves curating datasets and architectures to train neural nets. Now, Software 3.0 treats massive LLMs—trained on internet-scale multitask data—as a universal computer. Programming boils down to crafting prompts and context windows to steer this interpreter through digital information space.
Karpathy illustrates with the OpenClaw installer: Traditional setups balloon into complex shell scripts for cross-platform compatibility. Instead, OpenClaw provides a text block you paste into an agent like Cursor or Claude. The agent intelligently adapts to your environment, debugs loops, and installs—leveraging the LLM's baked-in intelligence without spelling out every if-then. This isn't faster Software 1.0; it's a paradigm where your 'code' is a snippet of natural language, and the neural net handles the heavy lifting.
He contrasts his own MenuGen app—Vercel-hosted, OCRs menu photos, generates dish images via APIs—with a pure Software 3.0 version: Feed the photo to Gemini with a prompt like 'use Nanobanana to overlay images onto the menu.' Nanobanana directly inpaints visuals into the original pixels, rendering a visualized menu without intermediate apps, OCR, or UIs. MenuGen becomes obsolete; raw neural processing from image input to image output suffices. Karpathy stresses this unlocks non-programmable tasks before, like recompiling documents into personalized LLM knowledge bases—reframing unstructured data into wikis without traditional ETL pipelines.
"Software 3.0 now is kind of about your programming now turns to prompting and what's in the context window is your lever over the interpreter that is the LLM."
This paradigm extends beyond code to general information processing, enabling novel apps like on-the-fly UIs from raw video/audio via diffusion models.
Vibe Coding Raises the Floor, Agentic Engineering Raises the Ceiling
Karpathy coined 'vibe coding' last year for casual, intuitive building with early AI tools. By December, models like o1 and Claude hit a tipping point: Code chunks output cleanly, workflows cohere agentically, and corrections vanished. He dove into infinite side projects, feeling both exhilarated and unsettled—never more 'behind' as a programmer because AI handles execution flawlessly.
Vibe coding democratizes software: Anyone vibes out prototypes. But production demands more. Enter agentic engineering: Coordinating spiky, stochastic LLMs—'fable' ghosts summoned statistically—to preserve pre-AI quality bars without vulnerabilities. It's an engineering discipline magnifying productivity beyond 10x for top practitioners.
AI-native coders maximize tools: Custom setups in Cursor, Claude Code, or Codec; full feature utilization. Mediocre users treat them as ChatGPT adjuncts. Hiring must adapt—no LeetCode puzzles. Instead: "Give me a really big project... write a Twitter clone for agents... deploy it... then I'm going to use 10 codecs, 5.4x for X high to try to break your website and they should not be able to break it."
"Vibe coding is about raising the floor for everyone... agentic engineering is about preserving the quality bar of what existed before in professional software."
Jagged Intelligence: Verifiability Drives Peaks and Troughs
LLMs are 'jagged, statistical ghosts'—peaking in verifiable domains like code/math (RL-trained with clear rewards) but faltering elsewhere. Frontier labs prioritize economically valuable arenas, injecting data like chess positions, spiking capabilities. Out-of-distribution tasks stagnate.
Classic fails: Counting 'r's in 'strawberry' (now patched); advising to walk 50m to a car wash (Opus ignores driving context despite refactoring 100k-line codebases). Jaggedness stems from training: RLHF rewards verifiable outputs, data distributions, and lab focus. Users must probe: Are you in the 'circuits'? If not, fine-tune.
This explains why agents err on judgment calls, like MenuGen's agent mismatching Stripe/Google emails for user credits—lacking persistent IDs. Humans supply taste, aesthetics, oversight: Directing ghosts requires 'a new kind of taste and judgment.'
"State-of-the-art Opus 4.7 will simultaneously refactor a 100,000 line codebase... and yet tells me to walk to this car wash? This is insane."
Verifiability predicts acceleration: Code, math automate first. Professions assuming safety (e.g., basic reasoning) aren't. Everything's automatable with LLM judge councils, but verifiable domains scale easiest via RL/fine-tuning.
Founder Advice: Bet on Verifiable Domains and New Opportunities
Labs escape velocity in obvious verifiable spaces (code/math). Founders: Seek underserved RL environments for fine-tuning—tractable, data-rich niches labs overlook. Verifiability enables pulling capability levers without base-model dependency.
Don't speed up old paradigms; invent Software 3.0 natives. 2026 hindsight: 'A lot of this code shouldn't exist... neural net doing most of the work.' Expect neural-host computers: LLMs as primary compute, CPUs as appendages for determinism. Raw inputs (video/audio) yield ephemeral UIs via diffusion/tool-use hybrids.
"You can outsource your thinking but never your understanding."
Karpathy's Eureka Labs embodies this: Building AI for learning, agents everywhere.
Key Takeaways
- Paste agent prompts over complex scripts: OpenClaw's installer shows Software 3.0 trumps bash bloat—let LLMs adapt intelligently.
- Skip intermediate apps: Raw prompts to Gemini/Nanobanana visualize menus directly; audit your stack for neural-native rewrites.
- Probe jaggedness: Test LLMs on your domain—if verifiable (code/math-like), RL/fine-tune; else, supply human judgment.
- Hire for agentic scale: Assign massive projects (secure Twitter clones), stress-test with adversarial agents—not trivia puzzles.
- Master oversight: Agents are interns—humans own taste, specs, error-catching (e.g., email mismatches).
- Founders: Target verifiable niches for custom RL; build non-programmable before (knowledge bases, dynamic UIs).
- Reframe productivity: Vibe code prototypes, agentically engineer production—aim for >10x via coordination.
- Explore base models empirically: No manuals—map circuits via trials; chess spiked via data injection.
- Anticipate neural dominance: By 2026, LLMs host processes, tools as co-processors for weird, foreign apps.