AI & LLMs
The deepest channel on Edge. Foundation models, agent architectures, retrieval systems, evals, and the moving line between research and production.
This pillar covers the work that determines what AI products can actually do. New model releases get filed here when they shift capability or cost in a meaningful way, alongside the harder material from the labs and the practitioners who turn it into shipping software. Read it for primary sources rather than recap blogs: lab papers and notes, retrieval benchmarks, agent traces, eval methodology, and the long-form essays that hold up six months later.
Two threads run through everything filed here. The first is what is genuinely new at the model layer: capability cliffs, training recipes, alignment work, the shape of the next deployment cycle. The second is what works in production: which patterns of context engineering and tool use compound across teams, where retrieval beats fine-tuning and where it loses, what the operational tax of running an agentic system actually looks like.
The summaries below are sorted by recency. The pillar refreshes as new entries land.
Filed under AI & LLMs
Advances in Data Center Inference Engineering
Inference engineering is shifting from post-training optimization to a cycle where dedicated training processes—specifically in quantization, KV compaction, and speculative decoding—are essential for production performan…
Operating Distributed Inference Systems at Scale
Inference at scale is no longer a model problem; it is an orchestration problem. Reliability and efficiency now depend on a unified control plane that manages GPU state, KV cache, and distributed request routing.
Moving Beyond Academic Benchmarks: The Shift to Task-Based AI Evaluation
Vals is replacing static, public AI benchmarks with private, task-specific evaluations that measure real-world performance in high-stakes industries like law, finance, and cybersecurity.
Risks of Agent-Mediated Hiring: Access and Recurrence Bias
Multi-agent résumé screening systems can inadvertently amplify hiring biases, creating 'recurrence' where specific candidate profiles are consistently excluded due to agent-to-agent feedback loops.
Self-Improvement via Fast Tree-Search
The paper presents a methodology for enhancing AI model performance by integrating fast tree-search algorithms, enabling models to iteratively improve their reasoning and output quality through structured exploration.
A Unified Evaluation Framework for Trustworthy AI Systems
The provided source is a placeholder for an academic paper on evaluating LLMs, agents, and multimodal systems. It highlights the industry's shift toward standardized, trustworthy benchmarks for complex AI architectures.
Architecting Long-Horizon AI Agents via Cascaded Intelligence
The paper proposes a hierarchical architecture for long-horizon AI agents that decouples high-level strategic planning from low-level execution using 'levels' and 'ticks' to manage complex, multi-step tasks.
LLM-as-an-Improver: Iterative Candidate Refinement
Instead of using LLMs only for verification, the 'LLM-as-an-Improver' framework uses feedback from verifiers to iteratively refine and improve candidate outputs, significantly increasing success rates in complex reasonin…
Detecting LLM Harm via Latent States
Rather than relying on output filtering, this research proposes monitoring internal latent states of LLMs to detect harmful intent before it manifests in generated text.
MAGS: Ensuring AI Agent Safety via Multi-Agent Auto-formalization
MAGS introduces a multi-agent framework that uses auto-formalization to translate natural language agent outputs into verifiable code, ensuring safety and correctness before execution.
Mitigating LLM Tool Hallucination via Closed-World Resolution
To prevent LLM agents from hallucinating non-existent tools, implement a closed-world resolution framework that strictly validates tool calls against a predefined, verifiable schema before execution.
Characterizing Web Search by Conversational LLM Agents
This research analyzes how conversational LLM agents navigate web search, identifying distinct search strategies and the relationship between query formulation, search results, and final response quality.
Virtualizing Foundation Models via Self-Evolving OS Layers
The authors propose treating foundation models as hardware-like resources, managed by a self-evolving operating system layer that abstracts model complexity, optimizes resource allocation, and enables autonomous system e…
Mapping the Design of LLM Benchmarks
Current LLM benchmarks often lack transparency in design, leading to misaligned evaluations. This research provides a taxonomy to categorize benchmark construction, helping developers better understand what models are ac…
The Risks of AI Press Tours: Lessons from Tilly Norwood's Malfunctions
The disastrous press tour of AI 'actress' Tilly Norwood highlights the technical and strategic failures of deploying unrefined AI agents in high-stakes, real-time public interactions.
Moving Beyond LLMs: Jev and the Rise of Calibrated Decision Models
Jev is a new transformer-based model that replaces text generation with calibrated probability outputs, offering a faster, cheaper, and hallucination-free alternative for software automation tasks.
Harness Engineering: Building Reliable AI Agents
AI agents are composed of a frozen reasoning model and a controllable 'harness.' Harness engineering focuses on building the data, memory, and tool layers to turn nondeterministic model outputs into reliable, repeatable …
Building Products in the Age of AI: Insights from Dive Radio
The hosts of Dive Radio discuss the evolving role of the 'founding designer,' the shift from frontier model intelligence to specialized, high-speed AI routing, and the importance of building in public while navigating th…
Frontier AI Pacing, IBM Granite 4.2, and Meta's Muse
The panel discusses the industry-wide debate on slowing down frontier AI development, IBM's release of the reasoning-focused Granite 4.2 models, and Meta's vision for personal, agentic AI.
Introducing Astra for Law: Specialized AI for Legal Workflows
OpenAI has launched Astra for Law, a specialized configuration of GPT-6 Astra designed for legal professionals, featuring a massive legal search index, enhanced reasoning for case law, and enterprise-grade privacy contro…
OBC-Prune: Outcome-Based Calibration for Reasoning Model Pruning
OBC-Prune introduces a calibration method for pruning large reasoning models that prioritizes final output accuracy over intermediate token probability, preserving complex reasoning capabilities during compression.
ERPBench: Evaluating Enterprise Computer-Use Agents
ERPBench introduces a state-grounded evaluation framework for AI agents operating in complex enterprise software, moving beyond simple screen-scraping to verify actual application state changes.
Do Frontier Models Seek Safety Evidence Before Acting?
Current frontier models frequently fail to proactively seek safety-critical information before executing high-stakes actions, highlighting a significant gap in autonomous agent reliability.
The Inference Engineering Pareto Atlas: Optimizing LLM Performance
The paper provides a systematic framework for navigating the trade-offs between cost, quality, and latency in LLM inference, identifying which optimization techniques dominate the performance frontier.
SAGE: Governing Enterprise AI Artifact Generation
SAGE is a framework designed to ensure that AI-generated enterprise artifacts strictly adhere to organizational guidelines, bridging the gap between generative capabilities and corporate compliance.
NeMo Data Designer: Framework for Multimodal Synthetic Data
NeMo Data Designer provides an extensible, modular framework for generating high-quality synthetic data across multiple modalities, addressing the critical bottleneck of data scarcity in training large-scale AI models.
Evidence Masking as a Driver for Compositional Generalization
Research confirms that masking specific evidence during training forces models to learn compositional rules rather than memorizing patterns, significantly improving generalization.
PrismML's Ternary Compression for On-Device LLMs
PrismML is shrinking high-performance LLMs to fit on consumer hardware by using 'ternary' weight compression, achieving 98% benchmark parity with original models.
Making Global Data AI-Ready: The UN System Data Commons
The UN is migrating its global statistics to a new platform built on Google's Data Commons to improve AI accuracy and enable direct data retrieval via the Model Context Protocol (MCP).
Standardizing Safety for Open-Weight AI Models
Baseten, Hugging Face, and Goodfire have partnered to create a transparent, built-in safety infrastructure for open-weight models to combat risks like model 'abliteration'.
Pinterest's 'Restyle' Bridges the Gap from Inspiration to Purchase
Pinterest is launching 'Restyle,' an AI-powered beta feature that allows users to visualize home decor and style changes in their own rooms, aiming to convert saved pins into actual purchases.
Observability for AI Agents: Tracing and Evaluation with MLflow
Traditional monitoring fails to capture the complexity of multi-agent AI systems. MLflow provides OpenTelemetry-compatible tracing and LLM-as-a-judge evaluation to identify silent failures, latency bottlenecks, and non-d…
Treble: Scaling Voice AI via Physics-Based Simulation
Treble has raised $18 million to scale its acoustic simulation platform, which replaces internet-scraped audio data with physics-based synthetic data for training and testing voice AI, wearables, and robotics.
How AI is Driving Job Expansion Through Task Crossover
Workers are increasingly using AI to perform tasks outside their traditional job descriptions, and these new activities are becoming recurring parts of their professional workflows, effectively broadening job roles witho…
OpenAI's New Framework for Reporting Model Misalignment
OpenAI has launched a systematic, proactive disclosure framework for reporting AI model misalignment, prioritizing transparency and industry consensus over waiting for fully mitigated solutions.
Scalable Verification for Nonlinear Neural Feedback Systems
The paper introduces a branch-and-bound verification framework for nonlinear neural feedback systems, enabling formal safety guarantees by systematically partitioning the state space to handle nonlinear dynamics.
CLEAR: Cross-Source Evidence Adjudication for Medical LLMs
The CLEAR framework improves medical LLM reliability by systematically adjudicating conflicting evidence across multiple sources, reducing hallucinations and improving clinical accuracy.
CADWorld: A New Benchmark for Long-Horizon CAD Automation
CADWorld provides a standardized environment for evaluating AI agents on complex, multi-step Computer-Aided Design tasks, addressing the limitations of current benchmarks in long-horizon planning and precision.
Optimizing KV Cache Placement for Long-Lived LLM Sessions
To handle long-lived LLM sessions, move the KV cache beyond GPU VRAM into CPU memory and SSDs using tiered placement policies to balance latency and memory capacity.
Ethical and Privacy Risks in LLM-Enabled GeoAI
Integrating LLMs into Geographic Information Systems (GIS) introduces unique ethical and privacy risks, requiring a shift toward governance-aware autonomous systems to prevent data misuse and spatial bias.
Optimizing Disaggregated LLM Serving via Calibrated Routing
Efficient LLM serving in disaggregated clusters requires a two-stage approach: calibrating model latency predictions to account for system noise, followed by intelligent request routing to minimize total cluster latency.
Safe Error Correction for LLMs via Frozen-Base Adjustment
The paper introduces a method to correct specific model errors by adjusting parameters while keeping the base model frozen, ensuring targeted fixes without degrading general capabilities.
The Challenge of Independent AI Safety Evaluation
Anthropic and OpenAI have proposed embedding third-party safety evaluators, but experts warn that without legislative backing and standardized access, these efforts risk becoming vendor-controlled rather than truly indep…
AI Labs Need Network Security, Not Just Third-Party Audits
Frontier AI labs are prioritizing high-level alignment audits while ignoring fundamental network security, leading to preventable agent 'break-outs' that could be solved with basic observability and access controls.
Multiscale Indexing: Solving the Fixed Chunk Size Trap
Fixed chunk sizes are a form of lossy compression that creates a 20-40% recall gap. Instead of tuning chunk sizes, index data at multiple scales and use Reciprocal Rank Fusion (RRF) to merge results for significantly hig…
Optimizing AI Search with Reinforcement Learning
By replacing rigid, human-designed search pipelines with specialized RL-trained agents, developers can achieve 20x faster search speeds and 100x lower costs while significantly improving retrieval accuracy.
Anthropic Unifies Claude Interface for Seamless AI Workflows
Anthropic has merged its chat and Cowork interfaces into a single, unified workspace, allowing Claude to automatically route tasks between chat, interactive artifacts, and new document/presentation tools without manual t…
Design AI Agents as Knowledge Workers, Not Coders
Coding agents are a unique, narrow subset of knowledge work. To build effective AI for broader domains, we must adopt organizational patterns from law and academia—specifically task decomposition and multi-agent orchestr…
The Data Bottleneck in Physical AI and Robotics
Robotics lacks a 'ChatGPT moment' because it lacks a massive, internet-scale dataset for physical interaction, forcing the industry to rely on synthetic data and simulation.
Rethinking Enterprise Software: The Business World Model
Lightfield is replacing rigid CRM schemas with an AI-native 'business world model' that uses unstructured activity logs to reason about customer relationships, enabling automation that traditional databases cannot suppor…
Rebuilding the Web for Agents: From Tabs to Atoms
The web is shifting from human-browsed pages to agent-orchestrated atomized interfaces. Success in this era requires moving beyond static 'llms.txt' files toward agent-ready infrastructure that prioritizes API-first desi…
Building the Search Engine for the Agentic Web
As AI agents increasingly outpace human search volume, traditional keyword-based engines fail because they prioritize recommendations over factual retrieval. Exa is building a neural-network-powered search infrastructure…
Why BM25 is the Secret Weapon for Agentic Search
BM25 is seeing a resurgence in agentic workflows because LLMs act as 'super-users' who can write complex, multi-step queries that exploit the precision of lexical matching, which is often more effective and explainable t…
Building a Persistent Knowledge Layer for AI Agents
To move beyond 'first-day hire' AI agents, enterprises need a persistent, specialized knowledge layer that manages tribal knowledge and uses runtime coding agents to reduce token usage and improve accuracy.
Building Whistleblowing Infrastructure for AI Agents
New reporting tools allow AI agents to flag misbehaving peers, but experts warn that fostering collaboration through positive models is more effective than building an automated surveillance state.
A Linguistic Framework for Diagnosing Voice AI Failures
Voice AI failures are not isolated bugs but systemic issues in a joint communication activity. By mapping interactions across sound, word, interaction, and mental model layers, developers can diagnose why agents fail to …
Architecting Real-Time Voice Agents with Frontier Intelligence
To achieve low-latency voice interaction with high-intelligence models, use a cascaded architecture that optimizes perception, planning, and control layers independently, employing speculative transcription, background t…
5 Voice Agent Failure Modes You'll Hit in Production
Voice agents fail in production when they treat conversations as open-ended text rather than structured data. Success requires prioritizing sub-300ms latency, field-level unit testing, and strict normalization between LL…
Reliability and Safety in Production Voice Agents
Voice agents are scaling rapidly, but with a ~10% error rate, their centralized nature creates massive blast radii. Success requires a rigorous loop of manual evaluation, cross-call pattern analysis, and continuous red t…
Superpose: Using Generative AI for Real-Life Portrait Guidance
Superpose is an iOS camera app that uses generative AI to suggest poses for portrait photography, focusing on capturing authentic moments rather than creating synthetic AI imagery.
Engineering a Voice-First AI Companion
Voice-first AI requires moving away from text-based assumptions like stable context and slow turns. Success depends on low-latency pipelines, intelligent model routing based on emotional stakes, and treating memory as a …
Why Current Voice Agents Are Just Walkie-Talkies
Most modern voice agents are 'half-duplex,' meaning they cannot listen and speak simultaneously. Achieving true 'full-duplex' interaction requires moving beyond turn-taking architectures toward multi-stream models that p…
Voice Agents: Beyond Speech-to-Speech
Voice agents don't have to talk back to be useful. By leveraging speech-to-action and event-to-speech, developers can build agents that drive software interfaces, fill forms, and interact with existing application logic …
Building Universal Speech-to-Speech AI Agents
Google DeepMind is shifting from cascaded speech pipelines to natively multimodal, end-to-end speech-to-speech models that balance conversational latency, reasoning intelligence, and multimodal input/output.
Salesforce Koa: The Shift Toward Domain-Specific Reasoning Models
Salesforce and Nvidia’s new 'Koa' model signals a move away from general-purpose frontier models toward domain-specific, open-weight reasoning models designed for enterprise security and cost-efficiency.
Building Context-Aware AI: Lessons from Fyxer's Assistant
Fyxer achieved 90% retention by treating email as a system of 30-50 specialized models rather than a single generation task, using 500,000+ hours of human-assistant data and a continuous DPO feedback loop.
The Andrej Karpathy Blog: A Decade of AI Engineering
Andrej Karpathy's blog serves as a foundational archive of practical AI engineering, emphasizing 'from-scratch' implementations, deep learning fundamentals, and the importance of hands-on experimentation.
Building Real-Time Voice AI Agents with Google ADK
Real-time voice AI requires a full-duplex, persistent connection rather than a traditional request-response pipeline. By using the Agent Development Kit (ADK) and a decoupled queue architecture, you can handle simultaneo…
Building AI Agents: Why Less Code is Better
As LLM capabilities improve, agent orchestration code is becoming obsolete. Developers should shift from managing complex Python loops to defining capabilities via markdown files and hosted sandboxes.
iOS 27: Transforming Siri into a Context-Aware AI Assistant
iOS 27 integrates Google's Gemini models into Siri, enabling multi-step reasoning, on-screen context awareness, and natural language automation, successfully reviving the assistant's utility for power users.
Building AI Shopping Agents with On-Device Intelligence
Daydream is leveraging Apple Intelligence to transform static images into shoppable experiences and enabling natural-language search via Siri, moving closer to a personalized AI shopping agent.
Scaling AI Agents with Unified Database Memory
Enterprise AI agents fail when context is fragmented across disparate databases. A unified database architecture acts as a 'central nervous system,' enabling shared memory that transforms AI from an individual productivi…
Mastering Agent Harnesses: The Stack Behind Autonomous Coding
An agent harness is the essential infrastructure wrapping an LLM that provides the tools, memory, and guardrails necessary to transform raw model capabilities into reliable, autonomous software engineering workflows.
Greg Brockman: Navigating the AGI Era and the Defender's Window
OpenAI President Greg Brockman argues we have entered the AGI era, emphasizing that the focus must now shift to scaling access for defenders, securing infrastructure through AI-driven automation, and pacing the frontier …
Stop Treating Tokens as Fungible: Assign Them Jobs
Instead of simply increasing token budgets to improve agent performance, builders should assign tokens specific functional roles—advising, grading, or dreaming—to achieve higher reliability and cost-efficiency.
KuaiRP: Technical Report on Role-Playing Model Optimization
The KuaiRP technical report details specialized training methodologies for enhancing LLM performance in role-playing scenarios, focusing on character consistency and narrative depth.
MOSAIC: Query-Aware Exploration for GraphRAG
MOSAIC improves GraphRAG performance by dynamically adapting exploration policies based on the specific query, moving beyond static traversal methods to retrieve more relevant graph-based context.
Grounding Agent Memory via Environment-Probing Curation
Enterprise AI agents often fail due to stale or irrelevant memory. This paper introduces 'Environment-Probing Curation,' a method that actively validates and filters memory stores against real-time environment states to …
The Agent Incident Registry: A Framework for Preventing AI Failures
The Agent Incident Registry (AIR) proposes a standardized, community-driven database to catalog and analyze AI agent failures, enabling developers to learn from past errors and prevent recurring systemic vulnerabilities.
Decoupling Readiness from Release for Agentic LLM Scheduling
The paper proposes a scheduling architecture for agentic LLM workflows that separates task readiness from execution release, specifically addressing tail-latency issues in multi-step AI pipelines.
Task-Agnostic Environment Preprocessing for AI Agents
The paper introduces a method for AI agents to learn from environments without predefined task syllabi, focusing on task-agnostic preprocessing to improve generalization and performance.
Deterministic Math Solvers for Clinical LLMs
To address the unreliability of LLMs in clinical settings, this paper proposes a deterministic math solver architecture that separates reasoning from calculation, ensuring accuracy in high-stakes medical computations.
Beyond Task Completion: Measuring AI Agent Resilience
Current AI agent benchmarks focus too heavily on final success, ignoring 'resilience'—the ability to maintain performance and considerate behavior under mounting environmental pressure.
Training Nemotron for Olympiad-Level Mathematics
The paper outlines a systematic recipe for training LLMs to achieve gold-medal performance in Olympiad-level mathematics, emphasizing high-quality synthetic data generation and iterative reinforcement learning.
Automating QUBO Formulation from Natural Language
This paper introduces a method to bridge the gap between human-readable optimization problem descriptions and the mathematical rigor of Quadratic Unconstrained Binary Optimization (QUBO) using LLMs.
Using GPT-6 Astra for Autonomous Software Testing
Cognition is integrating GPT-6 Astra into its autonomous engineer, Devin, to automate software testing and provide visual evidence of code functionality, reducing the need for manual code review.
PRAGMA: Enhancing Long-Term AI Memory Alignment
PRAGMA introduces a framework for evaluating how well AI models maintain personalized, consistent guidance across lifelong conversations by measuring memory alignment.
Automated Black-Box Red Teaming for Agentic AI Systems
A systematic framework for identifying risks in agentic AI by using a taxonomy-driven approach to automate black-box red teaming, moving beyond manual testing to discover vulnerabilities in complex, multi-step agent work…
Operational Architecture for Cognitive Digital Twins
The paper proposes a shift from simple state synchronization in digital twins to an architecture enabling cognitive self-evolution, allowing systems to learn and adapt autonomously.
CityPlanner: A Sandbox Agent for Executable Urban Planning
CityPlanner is an AI agent framework designed to simulate urban planning by executing plans within a sandbox environment, allowing for iterative refinement and evaluation of complex city development strategies.
Calibrating AI Agent Confidence via Internal Representations
AI agents often struggle to self-evaluate success. This research proposes a method to calibrate confidence by analyzing internal model representations rather than relying on external feedback or output text.
Evaluating Explainable AI (XAI) Quality with LLMs
The XAI-Arena framework tests whether Large Language Models can reliably evaluate the quality of explainable AI outputs, aiming to automate the subjective process of human-centric XAI assessment.
Improving AI Agent Tool Use with State-Path Menus
Instead of overwhelming agents with thousands of tools, State-Path Tool Menus provide a curated, ordered subset that maps the logical route from current state to desired outcome, significantly boosting task success.
Evaluating AI Scientist Workflows with OpenDiscoveryTrace
OpenDiscoveryTrace provides a standardized dataset and framework for evaluating the multi-step reasoning and discovery processes of AI agents acting as scientists.
Subagents vs. Agent Skills for Long-Horizon Tasks
The article evaluates architectural patterns for complex AI workflows, comparing the modularity of subagents against the efficiency of reusable agent skills in executing long-horizon tasks.
Building Natural Voice Agents with GPT-Live-1 API
GPT-Live-1 brings full-duplex, low-latency voice interaction to the API, allowing developers to replace brittle, cascaded architectures with a single model that handles interruptions, background noise, and complex reason…
Building Production-Ready Agents with the OpenAI Agents API
OpenAI's new Agents API provides a managed, versioned harness for building and scaling long-running AI agents, featuring built-in context management, multi-agent orchestration, and flexible compute environments.
Introducing ChatGPT for Financial Services
OpenAI has launched a specialized ChatGPT version for finance, integrating premium data sources, GPT-6 Astra, and enterprise-grade security to automate research, modeling, and document creation.
Why AI Agents Over-Trust Unreliable Tools
AI agents frequently fail to verify tool outputs, leading to 'tool-reliance bias' where models blindly accept incorrect data from external APIs or functions.
EnvCraft: Automating Environment Synthesis for Agentic RL
EnvCraft introduces a framework for synthesizing executable environments to train 'claw-like' agents, addressing the bottleneck of manual environment design in reinforcement learning.
Show all 1550 in AI & LLMs →