Firehose

This workshop details how to move beyond simple prompt engineering by building "harnesses" that treat AI as a component in a larger, deterministic system. The speaker argues that prose-based instructions are insufficient to overcome a model's tendency toward "slop," and instead demonstrates nine techniques—such as adversarial sub-agents, deterministic linters, and external script execution—to force divergence and maintain design quality. - Impeccable — The speaker's open-source design skill and framework. - Paul Bakaus — The speaker's personal site and portfolio.

This video argues that organizations should stop using token consumption as a primary metric for AI success, as it encourages either wasteful "tokenmaxxing" or counterproductive cost-cutting. It advocates for Valuemaxxing, a shift toward measuring operational outcomes like deployment speed and rework reduction.

This talk explains FlashNorm, a technique to speed up transformer inference by folding RMS norm layers into projection weights and parallelizing matrix and vector operations. The speaker details the CUDA implementation challenges, specifically how an implicit stream join caused a race condition that led to model output repetition.

This talk is a technical post-mortem on two silent, state-related bugs the speakers encountered while running Jamba on vLLM. It details how they used log-probability forensics and custom request-ID tracing to identify scheduler race conditions and integer overflows in the Mamba state cache that caused the model to output gibberish without crashing.

This is a technical pitch from FriendliAI explaining why standard inference stacks fail for agentic workflows. The speaker argues that because agents reuse large context prefixes across long-running tasks, inference providers need to shift from optimizing individual request latency to optimizing end-to-end task completion through techniques like prefix caching, hierarchical KV cache management, and cache-aware routing.

This talk outlines the operational bottlenecks of serving a "fleet" of small, task-specific models rather than a single large one. The speaker argues that traditional top-down routing and manual infrastructure tuning stall GPU utilization, and proposes an inverted, queue-based architecture to automate batching and model adaptation.

This talk serves as a post-publication update to the author's book, *Inference Engineering*, focusing on data center optimizations that have emerged since early 2026. It categorizes recent developments into three pillars: * Quantization — A breakdown of why techniques like TurboQuant, while effective for memory-constrained local setups, often underperform in data centers compared to standard weight quantization. * KV Compaction — An overview of moving from simple caching to learned bottlenecks that cross-attend query vectors against the cache to emit compact keys and values. * Speculative Decoding — A review of the shift toward training custom draft models, the introduction of diffusion-based drafting (DFlash), and the performance gains from continuously retraining speculators on live production prompts.

This is a high-level technical overview of how CoreWeave manages inference workloads by balancing serverless and dedicated capacity. The speaker explains how they optimize for KV cache locality and use heterogeneous GPU scheduling to treat inference like a game of Tetris, prioritizing cache hits to avoid redundant prefill computations.

This talk explains why common LLM benchmarking tools often produce misleading results due to Python's global interpreter lock, client-side latency inflation, and inconsistent data sampling. The speakers introduce Inference Perf, a CNCF project designed to provide high-fidelity, multi-process load generation and declarative workload definitions to ensure reproducible performance metrics.

This is a technical post-mortem from OpenAI engineers on evolving their inference load balancer from a reactive, feedback-loop-based controller to a global, policy-driven optimizer. It explains why proportional control loops caused cache-disrupting oscillations and how they moved to a decoupled control plane/data plane architecture to balance latency and capacity across global clusters.

This talk from Meta engineers argues that AI inference has outgrown simple model serving and now requires a dedicated control plane, similar to how cloud infrastructure evolved from virtual machines to Kubernetes. They detail how the tight coupling between routing, KV cache management, and GPU scheduling turns every request into a complex distributed transaction that demands holistic orchestration rather than edge-level fixes.

This workshop provides a conceptual framework for "harness engineering"—the practice of building a reliable, repeatable control layer around non-deterministic LLMs. The speaker outlines a seven-layer architecture for agent memory and tool use, emphasizing that while model reasoning is rented, the surrounding infrastructure is where developers can actually exert control.

This is a casual, live-streamed industry hangout where the hosts discuss their workflows, share personal anecdotes about balancing design work with travel, and spotlight fellow designers. It functions primarily as a community-driven showcase for portfolio work and design inspiration.

This is a roundtable discussion from the *Mixture of Experts* podcast covering three industry topics: the debate around slowing down frontier AI development, the release of IBM’s Granite 4.2 model suite, and Meta’s work on agentic AI assistants. It functions as a high-level industry news recap rather than a technical deep dive.

This video demonstrates how to use the Google Agent Development Kit (ADK 2.0) to replace complex prompt-chaining with deterministic graph-based workflows. The presenters walk through a "smart parser" demo, showing how to implement parallel processing and self-correcting loops using the provided GitHub repository.

This is a technical talk by John Ousterhout explaining why TCP and RDMA struggle with the small, latency-sensitive messages common in modern AI inference and agentic workloads. He introduces Homa, a receiver-driven transport protocol designed to prioritize short messages and reduce tail latency in data centers.

This video provides a conceptual overview of using MLflow for observability in multi-agent systems. It explains how to use traces and LLM-as-a-judge evaluation to debug agent behavior, followed by four practical tips for moving from a development notebook to a production-ready deployment.

This talk argues that fixed chunk sizes are a source of significant, hidden recall loss because the optimal window size is a property of the query, not the data. The proposed solution is to index the corpus at multiple scales simultaneously and use reciprocal rank fusion to merge the results.

Maximilian-David Rumpf (site) argues that current agentic search is inefficient because it relies on rigid, human-designed pipelines. He proposes replacing these with reinforcement learning models that learn to search iteratively, claiming this approach can be 20x faster and 100x cheaper than using general-purpose frontier models.

This is a technical post-mortem on scaling legal document search, detailing how the team moved from a sharded PostgreSQL setup that suffered from cache thrashing to a namespace-per-project architecture using turbopuffer. The speakers explain why decoupling storage from compute via object storage solved their multi-tenancy and encryption requirements for regulated enterprise clients.

This talk explains why Docusign and NVIDIA collaborated to build a specialized, 900-million parameter vision language model for extracting structured data from complex legal documents. They argue that generic models fail at preserving the semantic structure of nested tables, and demonstrate how a purpose-built "extractor" model provides higher accuracy and lower latency for high-volume enterprise workflows.

This talk argues that the AI industry has mistakenly used software engineering as the primary template for agent design, ignoring that code is uniquely structured and "grep-able" compared to the messy, context-dependent nature of general knowledge work. The speaker suggests that instead of treating agents like programmers, we should model them after the historical evolution of human knowledge organizations—using iterative loops of search, memoization, and specialized roles to bridge the "oracle gap."

This is a founder-led interview with Lightfield CEO Keith Peiris, detailing his transition from building the presentation tool Tome to developing an AI-native CRM. The conversation focuses on the technical and strategic shift toward creating a "business world model" that aggregates fragmented company data, rather than offering a product demo or a deep dive into the software's specific mechanics.

This talk explains the Model Context Protocol (MCP), a standard that allows servers to inject functional UI components directly into chat interfaces. The speaker argues that this "headless" approach will eventually replace traditional browser-based workflows, as users shift from navigating individual websites to interacting with services through a single, unified AI assistant.

This is a founder-led pitch for Exa, a search engine designed specifically for AI agents rather than human users. The talk explains the company’s shift from a consumer-facing tool to an API-first platform, focusing on their technical approach to using embeddings for high-precision retrieval that avoids the "recommendation engine" bias of traditional search.

This talk explains why the 30-year-old BM25 algorithm is seeing a resurgence in agentic workflows: because LLMs are better at writing long, specific queries than humans ever were. The speaker argues that reasoning isn't the bottleneck for agents—retrieval quality is—and suggests treating retrieved documents as a file system for the model to query rather than relying on complex embedding-based setups.

This is a high-level conceptual pitch from Edo Liberty regarding the limitations of current RAG-based enterprise agents. He argues that agents currently lack "tribal knowledge"—the dynamic, non-document-based context of how a company actually operates—and proposes a persistent, self-updating "knowledge layer" that uses a runtime coding agent to iterate on answers rather than relying on static retrieval.

This talk argues that voice AI teams should stop obsessing over model accuracy and instead optimize for "user pain" by treating system errors as having different costs. The speaker introduces a heuristic called OUCH (Outcome User Cost Heuristic) to mathematically determine when an assistant should act, decline, or confirm based on confidence thresholds.

This talk provides a linguistic framework for diagnosing why voice AI agents fail, arguing that developers should stop treating these issues as isolated bugs and start viewing them as failures in a "joint activity." The speaker, Midam Kim, maps these interactions across four levels—sounds, words, interaction, and mental models—to help engineers build more coherent, context-aware systems.

This talk breaks down the engineering stack behind EliseAI's voice agents, focusing on three specific latency-reduction techniques: speculative transcription, background tool-calling, and audio prefix caching. It’s a technical look at how to make high-latency, high-intelligence models feel instantaneous in a conversational voice loop.