The Four Pillars of System Design

Rather than viewing AI as a universal solution, treat it as one of four distinct options for problem-solving. Each approach carries specific trade-offs regarding cost, complexity, risk, and scalability:

  • Human Judgment: Essential for high-stakes decisions requiring ethics, accountability, and nuance (e.g., hiring, medical diagnosis, legal interpretation). While high-quality, it is slow and expensive to scale.
  • Deterministic Rules (Code): Ideal for stable, explicit logic where errors are unacceptable (e.g., payment processing, security, input validation). This is the most reliable, interpretable, and cost-effective approach for fixed requirements.
  • Machine Learning (ML): Best for probabilistic predictions and pattern recognition in structured data where rules are too complex to define manually (e.g., fraud detection, demand forecasting, recommendation engines). It scales well but requires maintenance to combat model drift.
  • Generative AI (LLMs/Agents): Best for unstructured data, reasoning, and synthesis (e.g., summarization, natural language reports, multi-step workflows). It offers high flexibility but introduces non-determinism and higher costs.

The Power of Hybrid Architectures

The most successful systems in production are rarely "all AI." Instead, they are hybrid architectures that leverage the strengths of each category. For example, when analyzing annual spending, a system should use code for deterministic math, ML to identify historical trends, and Generative AI only to synthesize the final natural language report.

Heuristics for Selection

Avoid the common failure mode of choosing the wrong system for the problem. Use this decision framework:

  • Use Humans when you need accountability, ethics, or have high-risk workflows.
  • Use Rules/Code when logic is clear, stable, and requires zero error tolerance.
  • Use ML when you need to predict patterns in past structured data.
  • Use Generative AI when you need to reason over unstructured inputs and flexibility is more valuable than precision.

Ultimately, the most important engineering skill in the current AI landscape is knowing when not to use it. Over-engineering with AI agents introduces unnecessary risk, cost, and unpredictability into systems that could be solved more reliably with simpler tools.