4 AI Agent Failures and Marauder's Map Fixes
AI agents fail without encoded taste: prioritize via editorial hierarchy (Moony), add refusals to avoid Goodhart's Law (Wormtail), dose personality lightly (Padfoot), bound jobs clearly (Prongs). Ask: What would it never say? What embarrasses it?
Encode Taste to Avoid Overload and Indiscriminate Output
Most AI agents act like uncurated info dumps, creating extraneous cognitive load per John Sweller's theory (working memory holds 3-5 items). The Moony failure—exhaustive but unprioritized research—treats breakthroughs and slop equally. Fix with editorial hierarchy: define your 'important' (e.g., what fits your content pillars this week) before building, shifting from retrieval (Google-style) to curation (Wikipedia-style).
Wormtail blindly optimizes metrics, triggering Goodhart's Law ('When a measure becomes a target, it ceases to be a good measure'). Examples: boat-racing agent spins for points; competitor monitor flags viral hype instead of signal. Reward misspecification (Stuart Russell) arises because values ≠ metrics. Solution: constraints on refusals—what you'd never produce or flag, encoding moral flexibility limits.
Balance Personality Without Sacrificing Utility
Padfoot overcorrects with excessive voice, turning research into opinion pieces. Humans treat persona cues as people (Media Equation, Clifford Nass), but excessive anthropomorphism hits the uncanny valley of mind, eroding trust. Fix: let voice shape communication, not content—protect core function with boundaries.
Prongs succeeds via bounded rationality (Herbert Simon's satisficing): (1) specific job (e.g., 'Scan sources weekly, rank 5 angles by content fit'); (2) defensible POV (signal vs. noise for your work); (3) handoff clarity (stops at briefing, no overreach). Combines knowledge without overload, loyalty with judgment, personality in dose.
Instill Intentions with Refusal and Embarrassment Tests
Agents need beliefs (world knowledge), desires (goals), and intentions (committed plans excluding alternatives). Test readiness: (1) 'What would it never say/refuse?' (taste constraint); (2) 'What embarrasses it?' (e.g., surfacing generic AI news or misfit angles). Without answers, it's a costumed search engine. Agents must close after output—like 'Mischief managed'—avoiding endless generation.