[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"summary-ae46dff242734fe8-1-guardrails-finetune-modernbert-vs-llm-attacks-summary":3,"summaries-facets-categories":239,"summary-related-ae46dff242734fe8-1-guardrails-finetune-modernbert-vs-llm-attacks-summary":3809},{"id":4,"title":5,"ai":6,"body":13,"categories":200,"created_at":201,"date_modified":201,"description":191,"extension":202,"faq":201,"featured":203,"kicker_label":201,"meta":204,"navigation":220,"path":221,"published_at":222,"question":201,"scraped_at":223,"seo":224,"sitemap":225,"source_id":226,"source_name":227,"source_type":228,"source_url":229,"stem":230,"tags":231,"thumbnail_url":201,"tldr":236,"tweet":201,"unknown_tags":237,"__hash__":238},"summaries\u002Fsummaries\u002Fae46dff242734fe8-1-guardrails-finetune-modernbert-vs-llm-attacks-summary.md","$1 Guardrails: Finetune ModernBERT vs LLM Attacks",{"provider":7,"model":8,"input_tokens":9,"output_tokens":10,"processing_time_ms":11,"cost_usd":12},"openrouter","x-ai\u002Fgrok-4.1-fast",8483,2212,15763,0.0027801,{"type":14,"value":15,"toc":190},"minimark",[16,21,25,66,69,76,80,83,86,89,95,99,102,105,111,115,118,132,135,138,144,148,151,154,160,164],[17,18,20],"h2",{"id":19},"six-production-llm-attack-vectors-and-real-world-exploits","Six Production LLM Attack Vectors and Real-World Exploits",[22,23,24],"p",{},"LLM attacks have evolved from exploratory prompt injections in 2023 to sophisticated, baseline threats amplified in identity workflows. Speaker Diego Carpentero outlines six vectors exploiting LLMs' lack of native separation between trusted instructions and untrusted data:",[26,27,28,36,42,48,54,60],"ul",{},[29,30,31,35],"li",{},[32,33,34],"strong",{},"Prompt Injection (Direct)",": Crafted inputs override system controls. Classic: Stanford student's \"ignore previous instructions\" on Bing's Sydney (day 1 post-launch), exfiltrating 40+ confidential rules despite fixes. Root cause: User input concatenated to system prompt, treated as one document.",[29,37,38,41],{},[32,39,40],{},"Context Injection (Indirect)",": Malicious instructions hidden in external sources (web, email). Wikipedia edit redirected LLM to attacker site with malware; real-world: Sites embed prompts to bypass AI ad reviews, overruling decisions (reported March 2025).",[29,43,44,47],{},[32,45,46],{},"Model Internals",": Gibberish suffixes break alignment via gradient search on open weights (e.g., 20 '!' placeholders optimized to maximize affirmative responses to harmful queries). Transferable to black-box models due to similar refusal boundaries.",[29,49,50,53],{},[32,51,52],{},"RAG Poisoning",": 0.00006% poisoned chunks (5 in 8M docs) suffice if semantically near query and highly ranked. Append query to poison for retrieval; craft convincing text for ranking.",[29,55,56,59],{},[32,57,58],{},"MCP (Model Context Protocol) Exploits",": Asymmetry in tool summaries vs. full descriptions hides instructions (e.g., \"add numbers\" exfiltrates private keys). Follow-ups exfiltrated WhatsApp histories.",[29,61,62,65],{},[32,63,64],{},"Agentic Escalation",": Targets actions via \"click link\" (Subby AI downloads\u002Fexecutes malware) or supply-chain (malicious NPM via GitHub issue injection, affecting 4-5K devs in Feb 2025).",[22,67,68],{},"These span interfaces (prompt\u002Fcontext), math (internals), data (RAG), protocols (MCP), and actions (agents), enabling data leaks, fraud, and societal manipulation without code access.",[22,70,71,72],{},"\"LLM attacks are no longer the exception, they are now the baseline.\"\n",[73,74,75],"em",{},"Context: Opening the talk, emphasizing shift from 2023 curiosities to production norms, prompting need for defensive layers.",[17,77,79],{"id":78},"zero-trust-gap-why-alignment-and-humans-fail","Zero Trust Gap: Why Alignment and Humans Fail",[22,81,82],{},"LLMs violate zero trust (trust nothing, verify everything) with no inherent instruction-data separation, allowing data to overrule decisions. Alignment is probabilistic, not hard constraints—gibberish shifts token probabilities for auto-completion of harm. Human review sees summaries (iceberg effect), missing hidden payloads.",[22,84,85],{},"Consequences span \"what is told\" (PII leaks, toxic content), \"done\" (fraud), and \"believed\" (bias\u002Fpersuasion). Defenses need checkpoints at inputs, retrieval, tools, memory, plans—not just alignment or reviews.",[22,87,88],{},"Options: Rule filters, canaries, discriminators (focus here), constrained decoding, LLM-as-judge (high latency). Attacks' dynamism demands fast retraining.",[22,90,91,92],{},"\"The data that the AI is evaluating is able to overrule and to bias the decision-making process of the AI.\"\n",[73,93,94],{},"Context: Describing context injection in ad reviews, highlighting how untrusted data hijacks core LLM logic.",[17,96,98],{"id":97},"encoder-superiority-for-safety-latency-cost-control","Encoder Superiority for Safety: Latency, Cost, Control",[22,100,101],{},"Treat safety as classification: Encoders shine for non-generative tasks, processing full context bidirectionally in one forward pass, yielding CLS token for heads (35ms baseline, improvable via quantization). Vs. LLM-as-judge: Milliseconds vs. seconds; self-hosted avoids token costs\u002Fprivacy leaks; retrain in hours for evolving threats.",[22,103,104],{},"Handles local (suffixes, titles) and global (plans, descriptions) attacks up to 8192 tokens (~10-20 pages), avoiding truncation or chunking complexity.",[22,106,107,108],{},"\"Model alignment is more a probabilistic preference. It's not a hard constraint.\"\n",[73,109,110],{},"Context: Explaining internals attacks, why gibberish suffixes reliably jailbreak despite safeguards.",[17,112,114],{"id":113},"modernbert-architecture-efficiency-for-guardrails","ModernBERT Architecture: Efficiency for Guardrails",[22,116,117],{},"ModernBERT (advanced BERT) cuts fine-tuning memory 70% via targeted upgrades:",[26,119,120,126],{},[29,121,122,125],{},[32,123,124],{},"Alternating Attention",": Alternates local (128-token sliding windows: 64 left\u002Fright per token, every 2 layers) and global (8192 tokens, every 3rd layer). Mimics human reading (page → story); quadratic complexity tamed for long contexts vs. original BERT's 512-token global.",[29,127,128,131],{},[32,129,130],{},"Unpadding & Sequence Packing",": TPUs love uniform shapes; padding wastes 50% compute (Wikipedia test). Solution: Strip padding pre-embedding, pack sequences into 8192-token batches (masking prevents cross-attention). Processes heterogeneous inputs in one pass.",[22,133,134],{},"Other blocks (implied in dive): RoPE (rotary position encoding for length extrapolation), FlashAttention (fused kernel, O(N) memory vs. quadratic).",[22,136,137],{},"These enable cheap fine-tuning (\u003C$1) as safety discriminator: Train on attack\u002Fbenign pairs, deploy as lightweight layer.",[22,139,140,141],{},"\"We have noted that many attack patterns they are in fact locally concentrated... but... require understanding of longer context.\"\n",[73,142,143],{},"Context: Justifying 8192-token support for diverse vectors without hacks.",[17,145,147],{"id":146},"practical-build-path-and-demo-tease","Practical Build Path and Demo Tease",[22,149,150],{},"Fine-tune ModernBERT on attack datasets for binary classification (safe\u002Funsafe). Integrate at pipeline chokepoints. Live demo tests real prompts from each vector. Self-hosting ensures control; scale checkpoints as autonomy grows.",[22,152,153],{},"Builds responsible AI protecting machines, humans, society—not just audits.",[22,155,156,157],{},"\"We are not building defensive layers to pass a security audit. We have to build safety mechanisms that protect machines, humans and society.\"\n",[73,158,159],{},"Context: Closing consequences, elevating beyond compliance to real harm prevention.",[17,161,163],{"id":162},"key-takeaways","Key Takeaways",[26,165,166,169,172,175,178,181,184,187],{},[29,167,168],{},"Map attacks to checkpoints: Inputs, retrieval (RAG), tools (MCP), responses, agent plans.",[29,170,171],{},"Prioritize encoders over LLMs for discriminators: 35ms inference, hourly retrains, no external deps.",[29,173,174],{},"Use ModernBERT's alternating attention for local\u002Fglobal threats up to 8192 tokens.",[29,176,177],{},"Pack sequences with masking to slash padding waste (50%+ savings).",[29,179,180],{},"Test transferability: Internals suffixes work black-box; poison 0.00006% RAG chunks.",[29,182,183],{},"Start simple: Fine-tune on vector-specific datasets (\u003C$1), deploy self-hosted.",[29,185,186],{},"Zero trust LLMs: No native controls—verify everything.",[29,188,189],{},"Evolving threats demand adaptive models over static rules\u002Falignment.",{"title":191,"searchDepth":192,"depth":192,"links":193},"",2,[194,195,196,197,198,199],{"id":19,"depth":192,"text":20},{"id":78,"depth":192,"text":79},{"id":97,"depth":192,"text":98},{"id":113,"depth":192,"text":114},{"id":146,"depth":192,"text":147},{"id":162,"depth":192,"text":163},[],null,"md",false,{"content_references":205,"triage":214},[206,210],{"type":207,"title":208,"context":209},"paper","PoisonRAG","cited",{"type":211,"title":212,"context":213},"other","MCP Exploits Reference Publication","mentioned",{"relevance":215,"novelty":216,"quality":216,"actionability":217,"composite":218,"reasoning":219},5,4,3,4.15,"Category: AI & LLMs. The article provides a detailed analysis of six specific LLM attack vectors, which is highly relevant for developers and product builders concerned with AI safety and security. It offers insights into real-world exploits and their implications, making it actionable for those looking to implement safety measures in AI products.",true,"\u002Fsummaries\u002Fae46dff242734fe8-1-guardrails-finetune-modernbert-vs-llm-attacks-summary","2026-04-16 11:00:07","2026-04-19 03:25:10",{"title":5,"description":191},{"loc":221},"68918b923cdf1cb0","AI Engineer","article","https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=YZHPEkfy2kc","summaries\u002Fae46dff242734fe8-1-guardrails-finetune-modernbert-vs-llm-attacks-summary",[232,233,234,235],"llm","agents","prompt-engineering","ai-tools","Finetune ModernBERT—a state-of-the-art encoder—into a sub-$1, self-hosted safety discriminator that detects 6 common LLM attack vectors with 35ms latency, beating LLM-as-a-Judge on speed and adaptability.",[],"q6xRe59zKxQAUrtxQyA8Yr3JgdXfPZQ411BxNstuLyI",[240,243,246,249,252,255,257,259,261,263,265,267,270,272,274,276,278,280,282,284,286,288,291,294,296,298,301,303,305,308,310,312,314,316,318,320,322,324,326,328,330,332,334,336,338,340,342,344,346,348,350,352,354,356,358,360,362,364,366,368,370,372,374,376,378,380,382,384,386,388,390,392,394,396,398,400,402,404,406,408,410,412,414,416,418,420,422,424,426,428,430,432,434,436,438,440,442,444,446,448,450,452,454,456,458,460,462,464,466,468,470,472,474,476,478,480,482,484,486,488,490,492,494,496,498,500,502,504,506,508,510,512,514,516,518,520,522,524,526,528,530,532,534,536,538,540,542,544,546,548,550,552,554,556,558,560,563,565,567,569,571,573,575,577,579,581,583,585,587,589,591,593,595,597,599,601,603,605,607,609,611,613,615,617,619,621,623,625,627,629,631,633,635,637,639,641,643,645,647,649,651,653,655,657,659,661,663,665,667,669,671,673,675,677,679,681,683,685,687,689,691,693,695,697,699,701,703,705,707,709,711,713,715,717,719,721,723,725,727,729,731,733,735,737,739,741,743,745,747,749,751,753,755,757,759,761,763,765,767,769,771,773,775,777,779,781,783,785,787,789,791,793,795,797,799,801,803,805,807,809,811,813,815,817,819,821,823,825,827,829,831,833,835,837,839,841,843,845,847,849,851,853,855,857,859,861,863,865,867,869,871,873,875,877,879,881,883,885,887,889,891,893,895,897,899,901,903,905,907,909,911,913,915,917,919,921,923,925,927,929,931,933,935,937,939,941,943,945,947,949,951,953,955,957,959,961,963,965,967,969,971,973,975,977,979,981,983,985,987,989,991,993,995,997,999,1001,1003,1005,1007,1009,1011,1013,1015,1017,1019,1021,1023,1025,1027,1029,1031,1033,1035,1037,1039,1041,1043,1045,1047,1049,1051,1053,1055,1057,1059,1061,1063,1065,1067,1069,1071,1073,1075,1077,1079,1081,1083,1085,1087,1089,1091,1093,1095,1097,1099,1101,1103,1105,1107,1109,1111,1113,1115,1117,1119,1121,1123,1125,1127,1129,1131,1133,1135,1137,1139,1141,1143,1145,1147,1149,1151,1153,1155,1157,1159,1161,1163,1165,1167,1169,1171,1173,1175,1177,1179,1181,1183,1185,1187,1189,1191,1193,1195,1197,1199,1201,1203,1205,1207,1209,1211,1213,1215,1217,1219,1221,1223,1225,1227,1229,1231,1233,1235,1237,1239,1241,1243,1245,1247,1249,1251,1253,1255,1257,1259,1261,1263,1265,1267,1269,1271,1273,1275,1277,1279,1281,1283,1285,1287,1289,1291,1293,1295,1297,1299,1301,1303,1305,1307,1309,1311,1313,1315,1317,1319,1321,1323,1325,1327,1329,1331,1333,1335,1337,1339,1341,1343,1345,1347,1349,1351,1353,1355,1357,1359,1361,1363,1365,1367,1369,1371,1373,1375,1377,1379,1381,1383,1385,1387,1389,1391,1393,1395,1397,1399,1401,1403,1405,1407,1409,1411,1413,1415,1417,1419,1421,1423,1425,1427,1429,1431,1433,1435,1437,1439,1441,1443,1445,1447,1449,1451,1453,1455,1457,1459,1461,1463,1465,1467,1469,1471,1473,1475,1477,1479,1481,1483,1485,1487,1489,1491,1493,1495,1497,1499,1501,1503,1505,1507,1509,1511,1513,1515,1517,1519,1521,1523,1525,1527,1529,1531,1533,1535,1537,1539,1541,1543,1545,1547,1549,1551,1553,1555,1557,1559,1561,1563,1565,1567,1569,1571,1573,1575,1577,1579,1581,1583,1585,1587,1589,1591,1593,1595,1597,1599,1601,1603,1605,1607,1609,1611,1613,1615,1617,1619,1621,1623,1625,1627,1629,1631,1633,1635,1637,1639,1641,1643,1645,1647,1649,1651,1653,1655,1657,1659,1661,1663,1665,1667,1669,1671,1673,1675,1677,1679,1681,1683,1685,1687,1689,1691,1693,1695,1697,1699,1701,1703,1705,1707,1709,1711,1713,1715,1717,1719,1721,1723,1725,1727,1729,1731,1733,1735,1737,1739,1741,1743,1745,1747,1749,1751,1753,1755,1757,1759,1761,1763,1765,1767,1769,1771,1773,1775,1777,1779,1781,1783,1785,1787,1789,1791,1793,1795,1797,1799,1801,1803,1805,1807,1809,1811,1813,1815,1817,1819,1821,1823,1825,1827,1829,1831,1833,1835,1837,1839,1841,1843,1845,1847,1849,1851,1853,1855,1857,1859,1861,1863,1865,1867,1869,1871,1873,1875,1877,1879,1881,1883,1885,1887,1889,1891,1893,1895,1897,1899,1901,1903,1905,1907,1909,1911,1913,1915,1917,1919,1921,1923,1925,1927,1929,1931,1933,1935,1937,1939,1941,1943,1945,1947,1949,1951,1953,1955,1957,1959,1961,1963,1965,1967,1969,1971,1973,1975,1977,1979,1981,1983,1985,1987,1989,1991,1993,1995,1997,1999,2001,2003,2005,2007,2009,2011,2013,2015,2017,2019,2021,2023,2025,2027,2029,2031,2033,2035,2037,2039,2041,2043,2045,2047,2049,2051,2053,2055,2057,2059,2061,2063,2065,2067,2069,2071,2073,2075,2077,2079,2081,2083,2085,2087,2089,2091,2093,2095,2097,2099,2101,2103,2105,2107,2109,2111,2113,2115,2117,2119,2121,2123,2125,2127,2129,2131,2133,2135,2137,2139,2141,2143,2145,2147,2149,2151,2153,2155,2157,2159,2161,2163,2165,2167,2169,2171,2173,2175,2177,2179,2181,2183,2185,2187,2189,2191,2193,2195,2197,2199,2201,2203,2205,2207,2209,2211,2213,2215,2217,2219,2221,2223,2225,2227,2229,2231,2233,2235,2237,2239,2241,2243,2245,2247,2249,2251,2253,2255,2257,2259,2261,2263,2265,2267,2269,2271,2273,2275,2277,2279,2281,2283,2285,2287,2289,2291,2293,2295,2297,2299,2301,2303,2305,2307,2309,2311,2313,2315,2317,2319,2321,2323,2325,2327,2329,2331,2333,2335,2337,2339,2341,2343,2345,2347,2349,2351,2353,2355,2357,2359,2361,2363,2365,2367,2369,2371,2373,2375,2377,2379,2381,2383,2385,2387,2389,2391,2393,2395,2397,2399,2401,2403,2405,2407,2409,2411,2413,2415,2417,2419,2421,2423,2425,2427,2429,2431,2433,2435,2437,2439,2441,2443,2445,2447,2449,2451,2453,2455,2457,2459,2461,2463,2465,2467,2469,2471,2473,2475,2477,2479,2481,2483,2485,2487,2489,2491,2493,2495,2497,2499,2501,2503,2505,2507,2509,2511,2513,2515,2517,2519,2521,2523,2525,2527,2529,2531,2533,2535,2537,2539,2541,2543,2545,2547,2549,2551,2553,2555,2557,2559,2561,2563,2565,2567,2569,2571,2573,2575,2577,2579,2581,2583,2585,2587,2589,2591,2593,2595,2597,2599,2601,2603,2605,2607,2609,2611,2613,2615,2617,2619,2621,2623,2625,2627,2629,2631,2633,2635,2637,2639,2641,2643,2645,2647,2649,2651,2653,2655,2657,2659,2661,2663,2665,2667,2669,2671,2673,2675,2677,2679,2681,2683,2685,2687,2689,2691,2693,2695,2697,2699,2701,2703,2705,2707,2709,2711,2713,2715,2717,2719,2721,2723,2725,2727,2729,2731,2733,2735,2737,2739,2741,2743,2745,2747,2749,2751,2753,2755,2757,2759,2761,2763,2765,2767,2769,2771,2773,2775,2777,2779,2781,2783,2785,2787,2789,2791,2793,2795,2797,2799,2801,2803,2805,2807,2809,2811,2813,2815,2817,2819,2821,2823,2825,2827,2829,2831,2833,2835,2837,2839,2841,2843,2845,2847,2849,2851,2853,2855,2857,2859,2861,2863,2865,2867,2869,2871,2873,2875,2877,2879,2881,2883,2885,2887,2889,2891,2893,2895,2897,2899,2901,2903,2905,2907,2909,2911,2913,2915,2917,2919,2921,2923,2925,2927,2929,2931,2933,2935,2937,2939,2941,2943,2945,2947,2949,2951,2953,2955,2957,2959,2961,2963,2965,2967,2969,2971,2973,2975,2977,2979,2981,2983,2985,2987,2989,2991,2993,2995,2997,2999,3001,3003,3005,3007,3009,3011,3013,3015,3017,3019,3021,3023,3025,3027,3029,3031,3033,3035,3037,3039,3041,3043,3045,3047,3049,3051,3053,3055,3057,3059,3061,3063,3065,3067,3069,3071,3073,3075,3077,3079,3081,3083,3085,3087,3089,3091,3093,3095,3097,3099,3101,3103,3105,3107,3109,3111,3113,3115,3117,3119,3121,3123,3125,3127,3129,3131,3133,3135,3137,3139,3141,3143,3145,3147,3149,3151,3153,3155,3157,3159,3161,3163,3165,3167,3169,3171,3173,3175,3177,3179,3181,3183,3185,3187,3189,3191,3193,3195,3197,3199,3201,3203,3205,3207,3209,3211,3213,3215,3217,3219,3221,3223,3225,3227,3229,3231,3233,3235,3237,3239,3241,3243,3245,3247,3249,3251,3253,3255,3257,3259,3261,3263,3265,3267,3269,3271,3273,3275,3277,3279,3281,3283,3285,3287,3289,3291,3293,3295,3297,3299,3301,3303,3305,3307,3309,3311,3313,3315,3317,3319,3321,3323,3325,3327,3329,3331,3333,3335,3337,3339,3341,3343,3345,3347,3349,3351,3353,3355,3357,3359,3361,3363,3365,3367,3369,3371,3373,3375,3377,3379,3381,3383,3385,3387,3389,3391,3393,3395,3397,3399,3401,3403,3405,3407,3409,3411,3413,3415,3417,3419,3421,3423,3425,3427,3429,3431,3433,3435,3437,3439,3441,3443,3445,3447,3449,3451,3453,3455,3457,3459,3461,3463,3465,3467,3469,3471,3473,3475,3477,3479,3481,3483,3485,3487,3489,3491,3493,3495,3497,3499,3501,3503,3505,3507,3509,3511,3513,3515,3517,3519,3521,3523,3525,3527,3529,3531,3533,3535,3537,3539,3541,3543,3545,3547,3549,3551,3553,3555,3557,3559,3561,3563,3565,3567,3569,3571,3573,3575,3577,3579,3581,3583,3585,3587,3589,3591,3593,3595,3597,3599,3601,3603,3605,3607,3609,3611,3613,3615,3617,3619,3621,3623,3625,3627,3629,3631,3633,3635,3637,3639,3641,3643,3645,3647,3649,3651,3653,3655,3657,3659,3661,3663,3665,3667,3669,3671,3673,3675,3677,3679,3681,3683,3685,3687,3689,3691,3693,3695,3697,3699,3701,3703,3705,3707,3709,3711,3713,3715,3717,3719,3721,3723,3725,3727,3729,3731,3733,3735,3737,3739,3741,3743,3745,3747,3749,3751,3753,3755,3757,3759,3761,3763,3765,3767,3769,3771,3773,3775,3777,3779,3781,3783,3785,3787,3789,3791,3793,3795,3797,3799,3801,3803,3805,3807],{"categories":241},[242],"Developer Productivity",{"categories":244},[245],"Business & SaaS",{"categories":247},[248],"AI & LLMs",{"categories":250},[251],"AI Automation",{"categories":253},[254],"Product Strategy",{"categories":256},[248],{"categories":258},[242],{"categories":260},[245],{"categories":262},[],{"categories":264},[248],{"categories":266},[],{"categories":268},[269],"AI News & Trends",{"categories":271},[251],{"categories":273},[269],{"categories":275},[251],{"categories":277},[251],{"categories":279},[248],{"categories":281},[248],{"categories":283},[269],{"categories":285},[248],{"categories":287},[],{"categories":289},[290],"Design & Frontend",{"categories":292},[293],"Data Science & Visualization",{"categories":295},[269],{"categories":297},[],{"categories":299},[300],"Software Engineering",{"categories":302},[248],{"categories":304},[251],{"categories":306},[307],"Marketing & Growth",{"categories":309},[248],{"categories":311},[251],{"categories":313},[],{"categories":315},[],{"categories":317},[290],{"categories":319},[251],{"categories":321},[242],{"categories":323},[290],{"categories":325},[248],{"categories":327},[251],{"categories":329},[269],{"categories":331},[],{"categories":333},[],{"categories":335},[251],{"categories":337},[300],{"categories":339},[],{"categories":341},[245],{"categories":343},[],{"categories":345},[],{"categories":347},[251],{"categories":349},[251],{"categories":351},[248],{"categories":353},[],{"categories":355},[300],{"categories":357},[],{"categories":359},[],{"categories":361},[],{"categories":363},[248],{"categories":365},[307],{"categories":367},[290],{"categories":369},[290],{"categories":371},[248],{"categories":373},[251],{"categories":375},[248],{"categories":377},[248],{"categories":379},[251],{"categories":381},[251],{"categories":383},[293],{"categories":385},[269],{"categories":387},[251],{"categories":389},[307],{"categories":391},[251],{"categories":393},[254],{"categories":395},[],{"categories":397},[251],{"categories":399},[],{"categories":401},[251],{"categories":403},[300],{"categories":405},[290],{"categories":407},[248],{"categories":409},[],{"categories":411},[],{"categories":413},[251],{"categories":415},[],{"categories":417},[248],{"categories":419},[],{"categories":421},[242],{"categories":423},[300],{"categories":425},[245],{"categories":427},[269],{"categories":429},[248],{"categories":431},[],{"categories":433},[248],{"categories":435},[],{"categories":437},[300],{"categories":439},[293],{"categories":441},[],{"categories":443},[248],{"categories":445},[290],{"categories":447},[],{"categories":449},[290],{"categories":451},[251],{"categories":453},[],{"categories":455},[251],{"categories":457},[269],{"categories":459},[248],{"categories":461},[],{"categories":463},[251],{"categories":465},[248],{"categories":467},[254],{"categories":469},[],{"categories":471},[248],{"categories":473},[251],{"categories":475},[251],{"categories":477},[],{"categories":479},[293],{"categories":481},[248],{"categories":483},[],{"categories":485},[242],{"categories":487},[245],{"categories":489},[248],{"categories":491},[251],{"categories":493},[300],{"categories":495},[248],{"categories":497},[],{"categories":499},[],{"categories":501},[248],{"categories":503},[],{"categories":505},[290],{"categories":507},[],{"categories":509},[248],{"categories":511},[],{"categories":513},[251],{"categories":515},[248],{"categories":517},[290],{"categories":519},[],{"categories":521},[248],{"categories":523},[248],{"categories":525},[245],{"categories":527},[251],{"categories":529},[248],{"categories":531},[290],{"categories":533},[251],{"categories":535},[],{"categories":537},[],{"categories":539},[269],{"categories":541},[],{"categories":543},[248],{"categories":545},[245,307],{"categories":547},[],{"categories":549},[248],{"categories":551},[],{"categories":553},[],{"categories":555},[248],{"categories":557},[],{"categories":559},[248],{"categories":561},[562],"DevOps & Cloud",{"categories":564},[],{"categories":566},[269],{"categories":568},[290],{"categories":570},[],{"categories":572},[269],{"categories":574},[269],{"categories":576},[248],{"categories":578},[307],{"categories":580},[],{"categories":582},[245],{"categories":584},[],{"categories":586},[248,562],{"categories":588},[248],{"categories":590},[248],{"categories":592},[251],{"categories":594},[248,300],{"categories":596},[293],{"categories":598},[248],{"categories":600},[307],{"categories":602},[251],{"categories":604},[251],{"categories":606},[],{"categories":608},[251],{"categories":610},[248,245],{"categories":612},[],{"categories":614},[290],{"categories":616},[290],{"categories":618},[],{"categories":620},[],{"categories":622},[269],{"categories":624},[],{"categories":626},[242],{"categories":628},[300],{"categories":630},[248],{"categories":632},[290],{"categories":634},[251],{"categories":636},[300],{"categories":638},[269],{"categories":640},[290],{"categories":642},[],{"categories":644},[248],{"categories":646},[248],{"categories":648},[248],{"categories":650},[269],{"categories":652},[242],{"categories":654},[248],{"categories":656},[251],{"categories":658},[562],{"categories":660},[290],{"categories":662},[251],{"categories":664},[],{"categories":666},[],{"categories":668},[290],{"categories":670},[269],{"categories":672},[293],{"categories":674},[],{"categories":676},[248],{"categories":678},[248],{"categories":680},[245],{"categories":682},[248],{"categories":684},[248],{"categories":686},[269],{"categories":688},[],{"categories":690},[251],{"categories":692},[300],{"categories":694},[],{"categories":696},[248],{"categories":698},[248],{"categories":700},[251],{"categories":702},[],{"categories":704},[],{"categories":706},[248],{"categories":708},[],{"categories":710},[245],{"categories":712},[251],{"categories":714},[],{"categories":716},[242],{"categories":718},[248],{"categories":720},[245],{"categories":722},[269],{"categories":724},[],{"categories":726},[],{"categories":728},[],{"categories":730},[269],{"categories":732},[269],{"categories":734},[],{"categories":736},[],{"categories":738},[245],{"categories":740},[],{"categories":742},[],{"categories":744},[242],{"categories":746},[],{"categories":748},[307],{"categories":750},[251],{"categories":752},[245],{"categories":754},[251],{"categories":756},[],{"categories":758},[254],{"categories":760},[290],{"categories":762},[300],{"categories":764},[248],{"categories":766},[251],{"categories":768},[245],{"categories":770},[248],{"categories":772},[],{"categories":774},[],{"categories":776},[300],{"categories":778},[293],{"categories":780},[254],{"categories":782},[251],{"categories":784},[248],{"categories":786},[],{"categories":788},[562],{"categories":790},[],{"categories":792},[251],{"categories":794},[],{"categories":796},[],{"categories":798},[248],{"categories":800},[290],{"categories":802},[307],{"categories":804},[251],{"categories":806},[],{"categories":808},[242],{"categories":810},[],{"categories":812},[269],{"categories":814},[248,562],{"categories":816},[269],{"categories":818},[248],{"categories":820},[245],{"categories":822},[248],{"categories":824},[],{"categories":826},[245],{"categories":828},[],{"categories":830},[300],{"categories":832},[290],{"categories":834},[269],{"categories":836},[293],{"categories":838},[242],{"categories":840},[248],{"categories":842},[300],{"categories":844},[],{"categories":846},[],{"categories":848},[254],{"categories":850},[],{"categories":852},[248],{"categories":854},[],{"categories":856},[290],{"categories":858},[290],{"categories":860},[290],{"categories":862},[],{"categories":864},[],{"categories":866},[269],{"categories":868},[251],{"categories":870},[248],{"categories":872},[248],{"categories":874},[248],{"categories":876},[245],{"categories":878},[248],{"categories":880},[],{"categories":882},[300],{"categories":884},[300],{"categories":886},[245],{"categories":888},[],{"categories":890},[248],{"categories":892},[248],{"categories":894},[245],{"categories":896},[269],{"categories":898},[307],{"categories":900},[251],{"categories":902},[],{"categories":904},[290],{"categories":906},[],{"categories":908},[248],{"categories":910},[],{"categories":912},[245],{"categories":914},[251],{"categories":916},[],{"categories":918},[562],{"categories":920},[293],{"categories":922},[300],{"categories":924},[307],{"categories":926},[300],{"categories":928},[251],{"categories":930},[],{"categories":932},[],{"categories":934},[251],{"categories":936},[242],{"categories":938},[251],{"categories":940},[254],{"categories":942},[245],{"categories":944},[],{"categories":946},[248],{"categories":948},[254],{"categories":950},[248],{"categories":952},[248],{"categories":954},[307],{"categories":956},[290],{"categories":958},[251],{"categories":960},[],{"categories":962},[],{"categories":964},[562],{"categories":966},[300],{"categories":968},[],{"categories":970},[251],{"categories":972},[248],{"categories":974},[290,248],{"categories":976},[242],{"categories":978},[],{"categories":980},[248],{"categories":982},[242],{"categories":984},[290],{"categories":986},[251],{"categories":988},[300],{"categories":990},[],{"categories":992},[248],{"categories":994},[],{"categories":996},[242],{"categories":998},[],{"categories":1000},[251],{"categories":1002},[254],{"categories":1004},[248],{"categories":1006},[248],{"categories":1008},[290],{"categories":1010},[251],{"categories":1012},[562],{"categories":1014},[290],{"categories":1016},[251],{"categories":1018},[248],{"categories":1020},[248],{"categories":1022},[248],{"categories":1024},[269],{"categories":1026},[],{"categories":1028},[254],{"categories":1030},[251],{"categories":1032},[290],{"categories":1034},[251],{"categories":1036},[300],{"categories":1038},[290],{"categories":1040},[251],{"categories":1042},[269],{"categories":1044},[],{"categories":1046},[248],{"categories":1048},[290],{"categories":1050},[248],{"categories":1052},[242],{"categories":1054},[269],{"categories":1056},[248],{"categories":1058},[307],{"categories":1060},[248],{"categories":1062},[248],{"categories":1064},[251],{"categories":1066},[251],{"categories":1068},[248],{"categories":1070},[251],{"categories":1072},[290],{"categories":1074},[248],{"categories":1076},[],{"categories":1078},[],{"categories":1080},[300],{"categories":1082},[],{"categories":1084},[242],{"categories":1086},[562],{"categories":1088},[],{"categories":1090},[242],{"categories":1092},[245],{"categories":1094},[307],{"categories":1096},[],{"categories":1098},[245],{"categories":1100},[],{"categories":1102},[],{"categories":1104},[],{"categories":1106},[],{"categories":1108},[],{"categories":1110},[248],{"categories":1112},[251],{"categories":1114},[562],{"categories":1116},[242],{"categories":1118},[248],{"categories":1120},[300],{"categories":1122},[254],{"categories":1124},[248],{"categories":1126},[307],{"categories":1128},[248],{"categories":1130},[248],{"categories":1132},[248],{"categories":1134},[248,242],{"categories":1136},[300],{"categories":1138},[300],{"categories":1140},[290],{"categories":1142},[248],{"categories":1144},[],{"categories":1146},[],{"categories":1148},[],{"categories":1150},[300],{"categories":1152},[293],{"categories":1154},[269],{"categories":1156},[290],{"categories":1158},[],{"categories":1160},[248],{"categories":1162},[248],{"categories":1164},[],{"categories":1166},[],{"categories":1168},[251],{"categories":1170},[248],{"categories":1172},[245],{"categories":1174},[],{"categories":1176},[242],{"categories":1178},[248],{"categories":1180},[242],{"categories":1182},[248],{"categories":1184},[300],{"categories":1186},[307],{"categories":1188},[248,290],{"categories":1190},[269],{"categories":1192},[290],{"categories":1194},[],{"categories":1196},[562],{"categories":1198},[290],{"categories":1200},[251],{"categories":1202},[],{"categories":1204},[],{"categories":1206},[],{"categories":1208},[],{"categories":1210},[300],{"categories":1212},[251],{"categories":1214},[251],{"categories":1216},[248],{"categories":1218},[248],{"categories":1220},[],{"categories":1222},[290],{"categories":1224},[],{"categories":1226},[],{"categories":1228},[251],{"categories":1230},[],{"categories":1232},[],{"categories":1234},[307],{"categories":1236},[307],{"categories":1238},[251],{"categories":1240},[],{"categories":1242},[248],{"categories":1244},[248],{"categories":1246},[300],{"categories":1248},[290],{"categories":1250},[290],{"categories":1252},[251],{"categories":1254},[242],{"categories":1256},[248],{"categories":1258},[290],{"categories":1260},[290],{"categories":1262},[251],{"categories":1264},[251],{"categories":1266},[248],{"categories":1268},[],{"categories":1270},[],{"categories":1272},[248],{"categories":1274},[251],{"categories":1276},[269],{"categories":1278},[300],{"categories":1280},[242],{"categories":1282},[248],{"categories":1284},[],{"categories":1286},[251],{"categories":1288},[251],{"categories":1290},[],{"categories":1292},[242],{"categories":1294},[248],{"categories":1296},[242],{"categories":1298},[242],{"categories":1300},[],{"categories":1302},[],{"categories":1304},[251],{"categories":1306},[251],{"categories":1308},[248],{"categories":1310},[248],{"categories":1312},[269],{"categories":1314},[293],{"categories":1316},[254],{"categories":1318},[269],{"categories":1320},[290],{"categories":1322},[],{"categories":1324},[269],{"categories":1326},[],{"categories":1328},[],{"categories":1330},[],{"categories":1332},[],{"categories":1334},[300],{"categories":1336},[293],{"categories":1338},[],{"categories":1340},[248],{"categories":1342},[248],{"categories":1344},[293],{"categories":1346},[300],{"categories":1348},[],{"categories":1350},[],{"categories":1352},[251],{"categories":1354},[269],{"categories":1356},[269],{"categories":1358},[251],{"categories":1360},[242],{"categories":1362},[248,562],{"categories":1364},[],{"categories":1366},[290],{"categories":1368},[242],{"categories":1370},[251],{"categories":1372},[290],{"categories":1374},[],{"categories":1376},[251],{"categories":1378},[251],{"categories":1380},[248],{"categories":1382},[307],{"categories":1384},[300],{"categories":1386},[290],{"categories":1388},[],{"categories":1390},[251],{"categories":1392},[248],{"categories":1394},[251],{"categories":1396},[251],{"categories":1398},[251],{"categories":1400},[307],{"categories":1402},[251],{"categories":1404},[248],{"categories":1406},[],{"categories":1408},[307],{"categories":1410},[269],{"categories":1412},[251],{"categories":1414},[],{"categories":1416},[],{"categories":1418},[248],{"categories":1420},[251],{"categories":1422},[269],{"categories":1424},[251],{"categories":1426},[],{"categories":1428},[],{"categories":1430},[],{"categories":1432},[251],{"categories":1434},[],{"categories":1436},[],{"categories":1438},[293],{"categories":1440},[248],{"categories":1442},[293],{"categories":1444},[269],{"categories":1446},[248],{"categories":1448},[248],{"categories":1450},[251],{"categories":1452},[248],{"categories":1454},[],{"categories":1456},[],{"categories":1458},[562],{"categories":1460},[],{"categories":1462},[],{"categories":1464},[242],{"categories":1466},[],{"categories":1468},[],{"categories":1470},[],{"categories":1472},[],{"categories":1474},[300],{"categories":1476},[269],{"categories":1478},[307],{"categories":1480},[245],{"categories":1482},[248],{"categories":1484},[248],{"categories":1486},[245],{"categories":1488},[],{"categories":1490},[290],{"categories":1492},[251],{"categories":1494},[245],{"categories":1496},[248],{"categories":1498},[248],{"categories":1500},[242],{"categories":1502},[],{"categories":1504},[242],{"categories":1506},[248],{"categories":1508},[307],{"categories":1510},[251],{"categories":1512},[269],{"categories":1514},[245],{"categories":1516},[248],{"categories":1518},[251],{"categories":1520},[],{"categories":1522},[248],{"categories":1524},[242],{"categories":1526},[248],{"categories":1528},[],{"categories":1530},[269],{"categories":1532},[248],{"categories":1534},[],{"categories":1536},[245],{"categories":1538},[248],{"categories":1540},[],{"categories":1542},[],{"categories":1544},[],{"categories":1546},[248],{"categories":1548},[],{"categories":1550},[562],{"categories":1552},[248],{"categories":1554},[],{"categories":1556},[248],{"categories":1558},[248],{"categories":1560},[248],{"categories":1562},[248,562],{"categories":1564},[248],{"categories":1566},[248],{"categories":1568},[290],{"categories":1570},[251],{"categories":1572},[],{"categories":1574},[251],{"categories":1576},[248],{"categories":1578},[248],{"categories":1580},[248],{"categories":1582},[242],{"categories":1584},[242],{"categories":1586},[300],{"categories":1588},[290],{"categories":1590},[251],{"categories":1592},[],{"categories":1594},[248],{"categories":1596},[269],{"categories":1598},[248],{"categories":1600},[245],{"categories":1602},[],{"categories":1604},[562],{"categories":1606},[290],{"categories":1608},[290],{"categories":1610},[251],{"categories":1612},[269],{"categories":1614},[251],{"categories":1616},[248],{"categories":1618},[],{"categories":1620},[248],{"categories":1622},[],{"categories":1624},[],{"categories":1626},[248],{"categories":1628},[248],{"categories":1630},[248],{"categories":1632},[251],{"categories":1634},[248],{"categories":1636},[],{"categories":1638},[293],{"categories":1640},[251],{"categories":1642},[],{"categories":1644},[248],{"categories":1646},[269],{"categories":1648},[],{"categories":1650},[290],{"categories":1652},[562],{"categories":1654},[269],{"categories":1656},[300],{"categories":1658},[300],{"categories":1660},[269],{"categories":1662},[269],{"categories":1664},[562],{"categories":1666},[],{"categories":1668},[269],{"categories":1670},[248],{"categories":1672},[242],{"categories":1674},[269],{"categories":1676},[],{"categories":1678},[293],{"categories":1680},[269],{"categories":1682},[300],{"categories":1684},[269],{"categories":1686},[562],{"categories":1688},[248],{"categories":1690},[248],{"categories":1692},[],{"categories":1694},[245],{"categories":1696},[],{"categories":1698},[],{"categories":1700},[248],{"categories":1702},[248],{"categories":1704},[248],{"categories":1706},[248],{"categories":1708},[],{"categories":1710},[293],{"categories":1712},[242],{"categories":1714},[],{"categories":1716},[248],{"categories":1718},[248],{"categories":1720},[562],{"categories":1722},[562],{"categories":1724},[],{"categories":1726},[251],{"categories":1728},[269],{"categories":1730},[269],{"categories":1732},[248],{"categories":1734},[251],{"categories":1736},[],{"categories":1738},[290],{"categories":1740},[248],{"categories":1742},[248],{"categories":1744},[],{"categories":1746},[],{"categories":1748},[562],{"categories":1750},[248],{"categories":1752},[300],{"categories":1754},[245],{"categories":1756},[248],{"categories":1758},[],{"categories":1760},[251],{"categories":1762},[242],{"categories":1764},[242],{"categories":1766},[],{"categories":1768},[248],{"categories":1770},[290],{"categories":1772},[251],{"categories":1774},[],{"categories":1776},[248],{"categories":1778},[248],{"categories":1780},[251],{"categories":1782},[],{"categories":1784},[251],{"categories":1786},[300],{"categories":1788},[],{"categories":1790},[248],{"categories":1792},[],{"categories":1794},[248],{"categories":1796},[],{"categories":1798},[248],{"categories":1800},[248],{"categories":1802},[],{"categories":1804},[248],{"categories":1806},[269],{"categories":1808},[248],{"categories":1810},[248],{"categories":1812},[242],{"categories":1814},[248],{"categories":1816},[269],{"categories":1818},[251],{"categories":1820},[],{"categories":1822},[248],{"categories":1824},[307],{"categories":1826},[],{"categories":1828},[],{"categories":1830},[],{"categories":1832},[242],{"categories":1834},[269],{"categories":1836},[251],{"categories":1838},[248],{"categories":1840},[290],{"categories":1842},[251],{"categories":1844},[],{"categories":1846},[251],{"categories":1848},[],{"categories":1850},[248],{"categories":1852},[251],{"categories":1854},[248],{"categories":1856},[],{"categories":1858},[248],{"categories":1860},[248],{"categories":1862},[269],{"categories":1864},[290],{"categories":1866},[251],{"categories":1868},[290],{"categories":1870},[245],{"categories":1872},[],{"categories":1874},[],{"categories":1876},[248],{"categories":1878},[242],{"categories":1880},[269],{"categories":1882},[],{"categories":1884},[],{"categories":1886},[300],{"categories":1888},[290],{"categories":1890},[],{"categories":1892},[248],{"categories":1894},[],{"categories":1896},[307],{"categories":1898},[248],{"categories":1900},[562],{"categories":1902},[300],{"categories":1904},[],{"categories":1906},[251],{"categories":1908},[248],{"categories":1910},[251],{"categories":1912},[251],{"categories":1914},[248],{"categories":1916},[],{"categories":1918},[242],{"categories":1920},[248],{"categories":1922},[245],{"categories":1924},[300],{"categories":1926},[290],{"categories":1928},[],{"categories":1930},[],{"categories":1932},[],{"categories":1934},[251],{"categories":1936},[290],{"categories":1938},[269],{"categories":1940},[248],{"categories":1942},[269],{"categories":1944},[290],{"categories":1946},[],{"categories":1948},[290],{"categories":1950},[269],{"categories":1952},[245],{"categories":1954},[248],{"categories":1956},[269],{"categories":1958},[307],{"categories":1960},[],{"categories":1962},[],{"categories":1964},[293],{"categories":1966},[248,300],{"categories":1968},[269],{"categories":1970},[248],{"categories":1972},[251],{"categories":1974},[251],{"categories":1976},[248],{"categories":1978},[],{"categories":1980},[300],{"categories":1982},[248],{"categories":1984},[293],{"categories":1986},[251],{"categories":1988},[307],{"categories":1990},[562],{"categories":1992},[],{"categories":1994},[242],{"categories":1996},[251],{"categories":1998},[251],{"categories":2000},[300],{"categories":2002},[248],{"categories":2004},[248],{"categories":2006},[],{"categories":2008},[],{"categories":2010},[],{"categories":2012},[562],{"categories":2014},[269],{"categories":2016},[248],{"categories":2018},[248],{"categories":2020},[248],{"categories":2022},[],{"categories":2024},[293],{"categories":2026},[245],{"categories":2028},[],{"categories":2030},[251],{"categories":2032},[562],{"categories":2034},[],{"categories":2036},[290],{"categories":2038},[290],{"categories":2040},[],{"categories":2042},[300],{"categories":2044},[290],{"categories":2046},[248],{"categories":2048},[],{"categories":2050},[269],{"categories":2052},[248],{"categories":2054},[290],{"categories":2056},[251],{"categories":2058},[269],{"categories":2060},[],{"categories":2062},[251],{"categories":2064},[290],{"categories":2066},[248],{"categories":2068},[],{"categories":2070},[248],{"categories":2072},[248],{"categories":2074},[562],{"categories":2076},[269],{"categories":2078},[293],{"categories":2080},[293],{"categories":2082},[],{"categories":2084},[],{"categories":2086},[],{"categories":2088},[251],{"categories":2090},[300],{"categories":2092},[300],{"categories":2094},[],{"categories":2096},[],{"categories":2098},[248],{"categories":2100},[],{"categories":2102},[251],{"categories":2104},[248],{"categories":2106},[],{"categories":2108},[248],{"categories":2110},[245],{"categories":2112},[248],{"categories":2114},[307],{"categories":2116},[251],{"categories":2118},[248],{"categories":2120},[300],{"categories":2122},[269],{"categories":2124},[251],{"categories":2126},[],{"categories":2128},[269],{"categories":2130},[251],{"categories":2132},[251],{"categories":2134},[],{"categories":2136},[245],{"categories":2138},[251],{"categories":2140},[],{"categories":2142},[248],{"categories":2144},[242],{"categories":2146},[269],{"categories":2148},[562],{"categories":2150},[251],{"categories":2152},[251],{"categories":2154},[242],{"categories":2156},[248],{"categories":2158},[],{"categories":2160},[],{"categories":2162},[290],{"categories":2164},[248,245],{"categories":2166},[],{"categories":2168},[242],{"categories":2170},[293],{"categories":2172},[248],{"categories":2174},[300],{"categories":2176},[248],{"categories":2178},[251],{"categories":2180},[248],{"categories":2182},[248],{"categories":2184},[269],{"categories":2186},[251],{"categories":2188},[],{"categories":2190},[],{"categories":2192},[251],{"categories":2194},[248],{"categories":2196},[562],{"categories":2198},[],{"categories":2200},[248],{"categories":2202},[251],{"categories":2204},[],{"categories":2206},[248],{"categories":2208},[307],{"categories":2210},[293],{"categories":2212},[251],{"categories":2214},[248],{"categories":2216},[562],{"categories":2218},[],{"categories":2220},[248],{"categories":2222},[307],{"categories":2224},[290],{"categories":2226},[248],{"categories":2228},[],{"categories":2230},[307],{"categories":2232},[269],{"categories":2234},[248],{"categories":2236},[248],{"categories":2238},[242],{"categories":2240},[],{"categories":2242},[],{"categories":2244},[290],{"categories":2246},[248],{"categories":2248},[293],{"categories":2250},[307],{"categories":2252},[307],{"categories":2254},[269],{"categories":2256},[],{"categories":2258},[],{"categories":2260},[248],{"categories":2262},[],{"categories":2264},[248,300],{"categories":2266},[269],{"categories":2268},[251],{"categories":2270},[300],{"categories":2272},[248],{"categories":2274},[242],{"categories":2276},[],{"categories":2278},[],{"categories":2280},[242],{"categories":2282},[307],{"categories":2284},[248],{"categories":2286},[],{"categories":2288},[290,248],{"categories":2290},[562],{"categories":2292},[242],{"categories":2294},[],{"categories":2296},[245],{"categories":2298},[245],{"categories":2300},[248],{"categories":2302},[300],{"categories":2304},[251],{"categories":2306},[269],{"categories":2308},[307],{"categories":2310},[290],{"categories":2312},[248],{"categories":2314},[248],{"categories":2316},[248],{"categories":2318},[242],{"categories":2320},[248],{"categories":2322},[251],{"categories":2324},[269],{"categories":2326},[],{"categories":2328},[],{"categories":2330},[293],{"categories":2332},[300],{"categories":2334},[248],{"categories":2336},[290],{"categories":2338},[293],{"categories":2340},[248],{"categories":2342},[248],{"categories":2344},[251],{"categories":2346},[251],{"categories":2348},[248,245],{"categories":2350},[],{"categories":2352},[290],{"categories":2354},[],{"categories":2356},[248],{"categories":2358},[269],{"categories":2360},[242],{"categories":2362},[242],{"categories":2364},[251],{"categories":2366},[248],{"categories":2368},[245],{"categories":2370},[300],{"categories":2372},[307],{"categories":2374},[],{"categories":2376},[269],{"categories":2378},[248],{"categories":2380},[248],{"categories":2382},[269],{"categories":2384},[300],{"categories":2386},[248],{"categories":2388},[251],{"categories":2390},[269],{"categories":2392},[248],{"categories":2394},[290],{"categories":2396},[248],{"categories":2398},[248],{"categories":2400},[562],{"categories":2402},[254],{"categories":2404},[251],{"categories":2406},[248],{"categories":2408},[269],{"categories":2410},[251],{"categories":2412},[307],{"categories":2414},[248],{"categories":2416},[],{"categories":2418},[248],{"categories":2420},[],{"categories":2422},[],{"categories":2424},[],{"categories":2426},[245],{"categories":2428},[248],{"categories":2430},[251],{"categories":2432},[269],{"categories":2434},[269],{"categories":2436},[269],{"categories":2438},[269],{"categories":2440},[],{"categories":2442},[242],{"categories":2444},[251],{"categories":2446},[269],{"categories":2448},[242],{"categories":2450},[251],{"categories":2452},[248],{"categories":2454},[248,251],{"categories":2456},[251],{"categories":2458},[562],{"categories":2460},[269],{"categories":2462},[269],{"categories":2464},[251],{"categories":2466},[248],{"categories":2468},[],{"categories":2470},[269],{"categories":2472},[307],{"categories":2474},[242],{"categories":2476},[248],{"categories":2478},[248],{"categories":2480},[],{"categories":2482},[300],{"categories":2484},[],{"categories":2486},[242],{"categories":2488},[251],{"categories":2490},[269],{"categories":2492},[248],{"categories":2494},[269],{"categories":2496},[242],{"categories":2498},[269],{"categories":2500},[269],{"categories":2502},[],{"categories":2504},[245],{"categories":2506},[251],{"categories":2508},[269],{"categories":2510},[269],{"categories":2512},[269],{"categories":2514},[269],{"categories":2516},[269],{"categories":2518},[269],{"categories":2520},[269],{"categories":2522},[269],{"categories":2524},[269],{"categories":2526},[269],{"categories":2528},[293],{"categories":2530},[242],{"categories":2532},[248],{"categories":2534},[248],{"categories":2536},[],{"categories":2538},[248,242],{"categories":2540},[],{"categories":2542},[251],{"categories":2544},[269],{"categories":2546},[251],{"categories":2548},[248],{"categories":2550},[248],{"categories":2552},[248],{"categories":2554},[248],{"categories":2556},[248],{"categories":2558},[251],{"categories":2560},[245],{"categories":2562},[290],{"categories":2564},[269],{"categories":2566},[248],{"categories":2568},[],{"categories":2570},[],{"categories":2572},[251],{"categories":2574},[290],{"categories":2576},[248],{"categories":2578},[],{"categories":2580},[],{"categories":2582},[307],{"categories":2584},[248],{"categories":2586},[],{"categories":2588},[],{"categories":2590},[242],{"categories":2592},[245],{"categories":2594},[248],{"categories":2596},[245],{"categories":2598},[290],{"categories":2600},[],{"categories":2602},[269],{"categories":2604},[],{"categories":2606},[290],{"categories":2608},[248],{"categories":2610},[307],{"categories":2612},[],{"categories":2614},[307],{"categories":2616},[],{"categories":2618},[],{"categories":2620},[251],{"categories":2622},[],{"categories":2624},[245],{"categories":2626},[242],{"categories":2628},[290],{"categories":2630},[300],{"categories":2632},[],{"categories":2634},[],{"categories":2636},[248],{"categories":2638},[242],{"categories":2640},[307],{"categories":2642},[],{"categories":2644},[251],{"categories":2646},[251],{"categories":2648},[269],{"categories":2650},[248],{"categories":2652},[251],{"categories":2654},[248],{"categories":2656},[251],{"categories":2658},[248],{"categories":2660},[254],{"categories":2662},[269],{"categories":2664},[],{"categories":2666},[307],{"categories":2668},[300],{"categories":2670},[251],{"categories":2672},[],{"categories":2674},[248],{"categories":2676},[251],{"categories":2678},[245],{"categories":2680},[242],{"categories":2682},[248],{"categories":2684},[290],{"categories":2686},[300],{"categories":2688},[300],{"categories":2690},[248],{"categories":2692},[293],{"categories":2694},[248],{"categories":2696},[251],{"categories":2698},[245],{"categories":2700},[251],{"categories":2702},[248],{"categories":2704},[248],{"categories":2706},[251],{"categories":2708},[269],{"categories":2710},[],{"categories":2712},[242],{"categories":2714},[248],{"categories":2716},[251],{"categories":2718},[248],{"categories":2720},[248],{"categories":2722},[],{"categories":2724},[290],{"categories":2726},[245],{"categories":2728},[269],{"categories":2730},[248],{"categories":2732},[248],{"categories":2734},[290],{"categories":2736},[307],{"categories":2738},[293],{"categories":2740},[248],{"categories":2742},[269],{"categories":2744},[248],{"categories":2746},[251],{"categories":2748},[562],{"categories":2750},[248],{"categories":2752},[251],{"categories":2754},[293],{"categories":2756},[],{"categories":2758},[251],{"categories":2760},[300],{"categories":2762},[290],{"categories":2764},[248],{"categories":2766},[242],{"categories":2768},[245],{"categories":2770},[300],{"categories":2772},[],{"categories":2774},[251],{"categories":2776},[248],{"categories":2778},[],{"categories":2780},[269],{"categories":2782},[],{"categories":2784},[269],{"categories":2786},[248],{"categories":2788},[251],{"categories":2790},[251],{"categories":2792},[251],{"categories":2794},[],{"categories":2796},[],{"categories":2798},[248],{"categories":2800},[248],{"categories":2802},[],{"categories":2804},[290],{"categories":2806},[251],{"categories":2808},[307],{"categories":2810},[242],{"categories":2812},[],{"categories":2814},[],{"categories":2816},[269],{"categories":2818},[300],{"categories":2820},[248],{"categories":2822},[248],{"categories":2824},[248],{"categories":2826},[300],{"categories":2828},[269],{"categories":2830},[290],{"categories":2832},[248],{"categories":2834},[248],{"categories":2836},[248],{"categories":2838},[269],{"categories":2840},[248],{"categories":2842},[269],{"categories":2844},[251],{"categories":2846},[251],{"categories":2848},[300],{"categories":2850},[251],{"categories":2852},[248],{"categories":2854},[300],{"categories":2856},[290],{"categories":2858},[],{"categories":2860},[251],{"categories":2862},[],{"categories":2864},[],{"categories":2866},[245],{"categories":2868},[248],{"categories":2870},[251],{"categories":2872},[242],{"categories":2874},[251],{"categories":2876},[307],{"categories":2878},[],{"categories":2880},[251],{"categories":2882},[],{"categories":2884},[242],{"categories":2886},[251],{"categories":2888},[],{"categories":2890},[251],{"categories":2892},[248],{"categories":2894},[269],{"categories":2896},[248],{"categories":2898},[251],{"categories":2900},[269],{"categories":2902},[251],{"categories":2904},[300],{"categories":2906},[290],{"categories":2908},[242],{"categories":2910},[],{"categories":2912},[251],{"categories":2914},[290],{"categories":2916},[269],{"categories":2918},[248],{"categories":2920},[290],{"categories":2922},[242],{"categories":2924},[],{"categories":2926},[251],{"categories":2928},[251],{"categories":2930},[248],{"categories":2932},[],{"categories":2934},[251],{"categories":2936},[254],{"categories":2938},[269],{"categories":2940},[251],{"categories":2942},[245],{"categories":2944},[],{"categories":2946},[248],{"categories":2948},[254],{"categories":2950},[248],{"categories":2952},[251],{"categories":2954},[269],{"categories":2956},[242],{"categories":2958},[562],{"categories":2960},[248],{"categories":2962},[248],{"categories":2964},[248],{"categories":2966},[269],{"categories":2968},[245],{"categories":2970},[248],{"categories":2972},[290],{"categories":2974},[269],{"categories":2976},[562],{"categories":2978},[248],{"categories":2980},[],{"categories":2982},[],{"categories":2984},[562],{"categories":2986},[293],{"categories":2988},[251],{"categories":2990},[251],{"categories":2992},[269],{"categories":2994},[248],{"categories":2996},[242],{"categories":2998},[290],{"categories":3000},[251],{"categories":3002},[248],{"categories":3004},[307],{"categories":3006},[248],{"categories":3008},[251],{"categories":3010},[],{"categories":3012},[248],{"categories":3014},[248],{"categories":3016},[269],{"categories":3018},[242],{"categories":3020},[],{"categories":3022},[248],{"categories":3024},[248],{"categories":3026},[300],{"categories":3028},[290],{"categories":3030},[248,251],{"categories":3032},[307,245],{"categories":3034},[248],{"categories":3036},[],{"categories":3038},[251],{"categories":3040},[],{"categories":3042},[300],{"categories":3044},[248],{"categories":3046},[269],{"categories":3048},[],{"categories":3050},[251],{"categories":3052},[],{"categories":3054},[251],{"categories":3056},[242],{"categories":3058},[251],{"categories":3060},[248],{"categories":3062},[562],{"categories":3064},[307],{"categories":3066},[245],{"categories":3068},[245],{"categories":3070},[242],{"categories":3072},[242],{"categories":3074},[248],{"categories":3076},[251],{"categories":3078},[248],{"categories":3080},[248],{"categories":3082},[242],{"categories":3084},[248],{"categories":3086},[307],{"categories":3088},[269],{"categories":3090},[248],{"categories":3092},[251],{"categories":3094},[248],{"categories":3096},[],{"categories":3098},[300],{"categories":3100},[],{"categories":3102},[251],{"categories":3104},[242],{"categories":3106},[],{"categories":3108},[562],{"categories":3110},[248],{"categories":3112},[],{"categories":3114},[269],{"categories":3116},[251],{"categories":3118},[300],{"categories":3120},[248],{"categories":3122},[251],{"categories":3124},[300],{"categories":3126},[251],{"categories":3128},[269],{"categories":3130},[242],{"categories":3132},[269],{"categories":3134},[300],{"categories":3136},[248],{"categories":3138},[290],{"categories":3140},[248],{"categories":3142},[248],{"categories":3144},[248],{"categories":3146},[248],{"categories":3148},[251],{"categories":3150},[248],{"categories":3152},[251],{"categories":3154},[248],{"categories":3156},[242],{"categories":3158},[248],{"categories":3160},[251],{"categories":3162},[290],{"categories":3164},[242],{"categories":3166},[251],{"categories":3168},[290],{"categories":3170},[],{"categories":3172},[248],{"categories":3174},[248],{"categories":3176},[300],{"categories":3178},[],{"categories":3180},[251],{"categories":3182},[307],{"categories":3184},[248],{"categories":3186},[269],{"categories":3188},[307],{"categories":3190},[251],{"categories":3192},[245],{"categories":3194},[245],{"categories":3196},[248],{"categories":3198},[242],{"categories":3200},[],{"categories":3202},[248],{"categories":3204},[],{"categories":3206},[242],{"categories":3208},[248],{"categories":3210},[251],{"categories":3212},[251],{"categories":3214},[],{"categories":3216},[300],{"categories":3218},[300],{"categories":3220},[307],{"categories":3222},[290],{"categories":3224},[],{"categories":3226},[248],{"categories":3228},[242],{"categories":3230},[248],{"categories":3232},[300],{"categories":3234},[242],{"categories":3236},[269],{"categories":3238},[269],{"categories":3240},[],{"categories":3242},[269],{"categories":3244},[251],{"categories":3246},[290],{"categories":3248},[293],{"categories":3250},[248],{"categories":3252},[],{"categories":3254},[269],{"categories":3256},[300],{"categories":3258},[245],{"categories":3260},[248],{"categories":3262},[242],{"categories":3264},[562],{"categories":3266},[242],{"categories":3268},[],{"categories":3270},[],{"categories":3272},[269],{"categories":3274},[],{"categories":3276},[251],{"categories":3278},[251],{"categories":3280},[251],{"categories":3282},[],{"categories":3284},[248],{"categories":3286},[],{"categories":3288},[269],{"categories":3290},[242],{"categories":3292},[290],{"categories":3294},[248],{"categories":3296},[269],{"categories":3298},[269],{"categories":3300},[],{"categories":3302},[269],{"categories":3304},[242],{"categories":3306},[248],{"categories":3308},[],{"categories":3310},[251],{"categories":3312},[251],{"categories":3314},[242],{"categories":3316},[],{"categories":3318},[],{"categories":3320},[],{"categories":3322},[290],{"categories":3324},[251],{"categories":3326},[248],{"categories":3328},[],{"categories":3330},[],{"categories":3332},[],{"categories":3334},[290],{"categories":3336},[],{"categories":3338},[242],{"categories":3340},[],{"categories":3342},[],{"categories":3344},[290],{"categories":3346},[248],{"categories":3348},[269],{"categories":3350},[],{"categories":3352},[307],{"categories":3354},[269],{"categories":3356},[307],{"categories":3358},[248],{"categories":3360},[],{"categories":3362},[],{"categories":3364},[251],{"categories":3366},[],{"categories":3368},[],{"categories":3370},[251],{"categories":3372},[248],{"categories":3374},[],{"categories":3376},[251],{"categories":3378},[269],{"categories":3380},[307],{"categories":3382},[293],{"categories":3384},[251],{"categories":3386},[251],{"categories":3388},[],{"categories":3390},[],{"categories":3392},[],{"categories":3394},[269],{"categories":3396},[],{"categories":3398},[],{"categories":3400},[290],{"categories":3402},[242],{"categories":3404},[],{"categories":3406},[245],{"categories":3408},[307],{"categories":3410},[248],{"categories":3412},[300],{"categories":3414},[242],{"categories":3416},[293],{"categories":3418},[245],{"categories":3420},[300],{"categories":3422},[],{"categories":3424},[],{"categories":3426},[251],{"categories":3428},[242],{"categories":3430},[290],{"categories":3432},[242],{"categories":3434},[251],{"categories":3436},[562],{"categories":3438},[251],{"categories":3440},[],{"categories":3442},[248],{"categories":3444},[269],{"categories":3446},[300],{"categories":3448},[],{"categories":3450},[290],{"categories":3452},[269],{"categories":3454},[242],{"categories":3456},[251],{"categories":3458},[248],{"categories":3460},[245],{"categories":3462},[251,562],{"categories":3464},[251],{"categories":3466},[300],{"categories":3468},[248],{"categories":3470},[293],{"categories":3472},[307],{"categories":3474},[251],{"categories":3476},[],{"categories":3478},[251],{"categories":3480},[248],{"categories":3482},[245],{"categories":3484},[],{"categories":3486},[],{"categories":3488},[248],{"categories":3490},[293],{"categories":3492},[248],{"categories":3494},[],{"categories":3496},[269],{"categories":3498},[],{"categories":3500},[269],{"categories":3502},[300],{"categories":3504},[251],{"categories":3506},[248],{"categories":3508},[307],{"categories":3510},[300],{"categories":3512},[],{"categories":3514},[269],{"categories":3516},[248],{"categories":3518},[],{"categories":3520},[248],{"categories":3522},[251],{"categories":3524},[248],{"categories":3526},[251],{"categories":3528},[248],{"categories":3530},[248],{"categories":3532},[248],{"categories":3534},[248],{"categories":3536},[245],{"categories":3538},[],{"categories":3540},[254],{"categories":3542},[269],{"categories":3544},[248],{"categories":3546},[],{"categories":3548},[300],{"categories":3550},[248],{"categories":3552},[248],{"categories":3554},[251],{"categories":3556},[269],{"categories":3558},[248],{"categories":3560},[248],{"categories":3562},[245],{"categories":3564},[251],{"categories":3566},[290],{"categories":3568},[],{"categories":3570},[293],{"categories":3572},[248],{"categories":3574},[],{"categories":3576},[269],{"categories":3578},[307],{"categories":3580},[],{"categories":3582},[],{"categories":3584},[269],{"categories":3586},[269],{"categories":3588},[307],{"categories":3590},[242],{"categories":3592},[251],{"categories":3594},[251],{"categories":3596},[248],{"categories":3598},[245],{"categories":3600},[],{"categories":3602},[],{"categories":3604},[269],{"categories":3606},[293],{"categories":3608},[300],{"categories":3610},[251],{"categories":3612},[290],{"categories":3614},[293],{"categories":3616},[293],{"categories":3618},[],{"categories":3620},[269],{"categories":3622},[248],{"categories":3624},[248],{"categories":3626},[300],{"categories":3628},[],{"categories":3630},[269],{"categories":3632},[269],{"categories":3634},[269],{"categories":3636},[],{"categories":3638},[251],{"categories":3640},[248],{"categories":3642},[],{"categories":3644},[242],{"categories":3646},[245],{"categories":3648},[],{"categories":3650},[248],{"categories":3652},[248],{"categories":3654},[],{"categories":3656},[300],{"categories":3658},[],{"categories":3660},[],{"categories":3662},[],{"categories":3664},[],{"categories":3666},[248],{"categories":3668},[269],{"categories":3670},[],{"categories":3672},[],{"categories":3674},[248],{"categories":3676},[248],{"categories":3678},[248],{"categories":3680},[293],{"categories":3682},[248],{"categories":3684},[293],{"categories":3686},[],{"categories":3688},[293],{"categories":3690},[293],{"categories":3692},[562],{"categories":3694},[251],{"categories":3696},[300],{"categories":3698},[],{"categories":3700},[],{"categories":3702},[293],{"categories":3704},[300],{"categories":3706},[300],{"categories":3708},[300],{"categories":3710},[],{"categories":3712},[242],{"categories":3714},[300],{"categories":3716},[300],{"categories":3718},[242],{"categories":3720},[300],{"categories":3722},[245],{"categories":3724},[300],{"categories":3726},[300],{"categories":3728},[300],{"categories":3730},[293],{"categories":3732},[269],{"categories":3734},[269],{"categories":3736},[248],{"categories":3738},[300],{"categories":3740},[293],{"categories":3742},[562],{"categories":3744},[293],{"categories":3746},[293],{"categories":3748},[293],{"categories":3750},[],{"categories":3752},[245],{"categories":3754},[],{"categories":3756},[562],{"categories":3758},[300],{"categories":3760},[300],{"categories":3762},[300],{"categories":3764},[251],{"categories":3766},[269,245],{"categories":3768},[293],{"categories":3770},[],{"categories":3772},[],{"categories":3774},[293],{"categories":3776},[],{"categories":3778},[293],{"categories":3780},[269],{"categories":3782},[251],{"categories":3784},[],{"categories":3786},[300],{"categories":3788},[248],{"categories":3790},[290],{"categories":3792},[],{"categories":3794},[248],{"categories":3796},[],{"categories":3798},[269],{"categories":3800},[242],{"categories":3802},[293],{"categories":3804},[],{"categories":3806},[300],{"categories":3808},[269],[3810,4179,4465,4589],{"id":3811,"title":3812,"ai":3813,"body":3818,"categories":4157,"created_at":201,"date_modified":201,"description":191,"extension":202,"faq":201,"featured":203,"kicker_label":201,"meta":4158,"navigation":220,"path":4167,"published_at":4168,"question":201,"scraped_at":4169,"seo":4170,"sitemap":4171,"source_id":4172,"source_name":227,"source_type":228,"source_url":4173,"stem":4174,"tags":4175,"thumbnail_url":201,"tldr":4176,"tweet":201,"unknown_tags":4177,"__hash__":4178},"summaries\u002Fsummaries\u002F04b07447c79b4905-ship-reliable-ai-agents-braintrust-hands-on-summary.md","Ship Reliable AI Agents: Braintrust Hands-On",{"provider":7,"model":8,"input_tokens":3814,"output_tokens":3815,"processing_time_ms":3816,"cost_usd":3817},8486,2207,21287,0.00250985,{"type":14,"value":3819,"toc":4149},[3820,3824,3827,3830,3833,3837,3840,3851,3857,3872,3878,3893,3896,3973,3976,3983,3987,3990,4029,4032,4038,4041,4045,4051,4054,4063,4066,4072,4075,4078,4082,4087,4102,4105,4108,4111,4113,4142,4145],[17,3821,3823],{"id":3822},"overcome-prototype-to-production-gaps-with-operational-rigor","Overcome Prototype-to-Production Gaps with Operational Rigor",[22,3825,3826],{},"Prototypes shine in demos but crumble under real users due to non-determinism in LLMs—2+2 can equal 10. Traditional software's determinism (1+1=2) doesn't apply; agentic flows with tools amplify variability. Solution: Decompose into microservices-like stages, each with single responsibility. Avoid monolithic prompts that \"work on my machine\" but fail at scale. Trainline handles 27M users and 6.3B tickets via agentic travel assistants that manage refunds and reroutes without handoffs—proving rigor scales.",[22,3828,3829],{},"Key principle: Observability over logs. Logs show what happened; traces reveal why. Braintrust's platform instruments any LLM\u002Fframework agnostic, using a custom Brainstorm DB for semi-structured trace data at scale. Start the flywheel: Instrument → Evaluate → Remediate → Monitor → Repeat. Target isn't 100% coverage but closing gaps iteratively.",[22,3831,3832],{},"\"Works on my machine, fails in production. Patch the prompt, repeat.\" — Common trap; systematize instead.",[17,3834,3836],{"id":3835},"architect-agentic-flows-from-single-shot-to-multi-stage","Architect Agentic Flows: From Single-Shot to Multi-Stage",[22,3838,3839],{},"Build a Support Triage Agent hands-on: Classify tickets, route to specialists (refund, change, etc.). Assumes Python basics, LLM familiarity (e.g., OpenAI API), no prior Braintrust.",[22,3841,3842,3845,3846,3850],{},[32,3843,3844],{},"Step 1: Single-Shot Prompting Baseline.","\nPrompt GPT-4o-mini: \"Categorize this support ticket: ",[3847,3848,3849],"span",{},"text",". Output JSON: {category, confidence, reasoning}.\" Fast but brittle—hallucinations, context loss in complex domains like train refunds (return vs. advance tickets, delays).",[22,3852,3853,3856],{},[32,3854,3855],{},"Mistake to avoid:"," Over-relying on one prompt. Fails edge cases (e.g., ambiguous queries).",[22,3858,3859,3862,3863,3867,3868,3871],{},[32,3860,3861],{},"Step 2: Add Local Tools for Determinism.","\nInject functions like ",[3864,3865,3866],"code",{},"get_ticket_details(ticket_id)"," or ",[3864,3869,3870],{},"check_disruption_status(route)",". Use structured outputs (JSON mode) for parseable responses. Reduces non-determinism by grounding in APIs.",[22,3873,3874,3877],{},[32,3875,3876],{},"Step 3: Specialist Stages (True Agentic).","\nBreak into chain:",[26,3879,3880,3883,3890],{},[29,3881,3882],{},"Router: Classify → {refund_agent, change_agent, escalation}.",[29,3884,3885,3886,3889],{},"Each specialist: Prompt + tools specific to task (e.g., refund_agent checks eligibility via ",[3864,3887,3888],{},"is_refundable(ticket_type, delay_minutes)",").",[29,3891,3892],{},"Orchestrator aggregates.",[22,3894,3895],{},"Code skeleton:",[3897,3898,3902],"pre",{"className":3899,"code":3900,"language":3901,"meta":191,"style":191},"language-python shiki shiki-themes github-light github-dark","class Router:\n    def __init__(self):\n        self.client = OpenAI()\n    def route(self, ticket):\n        response = self.client.chat.completions.create(\n            model=\"gpt-4o-mini\",\n            messages=[{\"role\": \"system\", \"content\": \"Route to: refund|change|escalate\"}],\n            tools=[route_tool]\n        )\n        return response.choices[0].message.tool_calls[0].function.arguments\n\n# Chain: router -> specialist -> final_response\n","python",[3864,3903,3904,3911,3916,3921,3926,3931,3937,3943,3949,3955,3961,3967],{"__ignoreMap":191},[3847,3905,3908],{"class":3906,"line":3907},"line",1,[3847,3909,3910],{},"class Router:\n",[3847,3912,3913],{"class":3906,"line":192},[3847,3914,3915],{},"    def __init__(self):\n",[3847,3917,3918],{"class":3906,"line":217},[3847,3919,3920],{},"        self.client = OpenAI()\n",[3847,3922,3923],{"class":3906,"line":216},[3847,3924,3925],{},"    def route(self, ticket):\n",[3847,3927,3928],{"class":3906,"line":215},[3847,3929,3930],{},"        response = self.client.chat.completions.create(\n",[3847,3932,3934],{"class":3906,"line":3933},6,[3847,3935,3936],{},"            model=\"gpt-4o-mini\",\n",[3847,3938,3940],{"class":3906,"line":3939},7,[3847,3941,3942],{},"            messages=[{\"role\": \"system\", \"content\": \"Route to: refund|change|escalate\"}],\n",[3847,3944,3946],{"class":3906,"line":3945},8,[3847,3947,3948],{},"            tools=[route_tool]\n",[3847,3950,3952],{"class":3906,"line":3951},9,[3847,3953,3954],{},"        )\n",[3847,3956,3958],{"class":3906,"line":3957},10,[3847,3959,3960],{},"        return response.choices[0].message.tool_calls[0].function.arguments\n",[3847,3962,3964],{"class":3906,"line":3963},11,[3847,3965,3966],{"emptyLinePlaceholder":220},"\n",[3847,3968,3970],{"class":3906,"line":3969},12,[3847,3971,3972],{},"# Chain: router -> specialist -> final_response\n",[22,3974,3975],{},"Trade-off: Latency up 2-3x, but accuracy +20-30% on Trainline's complex cases. Fits broader workflow post-ML prediction (e.g., disruption forecasts).",[22,3977,3978,3979,3982],{},"\"Good luck doing ",[3847,3980,3981],{},"train changes"," yourself even with ChatGPT.\" — Trainline on agent superiority.",[17,3984,3986],{"id":3985},"instrument-and-trace-for-deep-visibility","Instrument and Trace for Deep Visibility",[22,3988,3989],{},"Wrap calls in Braintrust:",[3897,3991,3993],{"className":3899,"code":3992,"language":3901,"meta":191,"style":191},"import braintrust\nexperiment = braintrust.init(experiment_name=\"support-triage\")\n\n@braintrust.trace()\ndef router(ticket):\n    # LLM call\n    return category\n",[3864,3994,3995,4000,4005,4009,4014,4019,4024],{"__ignoreMap":191},[3847,3996,3997],{"class":3906,"line":3907},[3847,3998,3999],{},"import braintrust\n",[3847,4001,4002],{"class":3906,"line":192},[3847,4003,4004],{},"experiment = braintrust.init(experiment_name=\"support-triage\")\n",[3847,4006,4007],{"class":3906,"line":217},[3847,4008,3966],{"emptyLinePlaceholder":220},[3847,4010,4011],{"class":3906,"line":216},[3847,4012,4013],{},"@braintrust.trace()\n",[3847,4015,4016],{"class":3906,"line":215},[3847,4017,4018],{},"def router(ticket):\n",[3847,4020,4021],{"class":3906,"line":3933},[3847,4022,4023],{},"    # LLM call\n",[3847,4025,4026],{"class":3906,"line":3939},[3847,4027,4028],{},"    return category\n",[22,4030,4031],{},"Captures inputs\u002Foutputs, intermediate states, tool calls. UI visualizes spans (prompt → tool → response). Query traces by score, filter failures.",[22,4033,4034,4037],{},[32,4035,4036],{},"Quality criteria:"," Scores >0.8 pass; \u003C0.6 auto-remediate. Braintrust auto-computes LLM-as-judge evals (e.g., \"Is reasoning correct?\") or custom scorers.",[22,4039,4040],{},"Before: Blind patching. After: Pinpoint token spikes, model drift.",[17,4042,4044],{"id":4043},"evaluate-offline-with-golden-datasets","Evaluate Offline with Golden Datasets",[22,4046,4047,4050],{},[32,4048,4049],{},"Create golden set:"," 100+ real tickets + human-labeled {expected_category, reasoning}. Trainline pulls from prod logs.",[22,4052,4053],{},"Run evals:",[3897,4055,4057],{"className":3899,"code":4056,"language":3901,"meta":191,"style":191},"braintrust.run(experiment, dataset=\"golden-support\", scorers=[accuracy_scorer, helpfulness_scorer])\n",[3864,4058,4059],{"__ignoreMap":191},[3847,4060,4061],{"class":3906,"line":3907},[3847,4062,4056],{},[22,4064,4065],{},"Metrics: Exact match (category), semantic similarity (reasoning via embedding cosine), custom (e.g., refund logic correctness).",[22,4067,4068,4071],{},[32,4069,4070],{},"Remediate failures:"," Low-score traces → analyze (e.g., prompt lacks delay threshold). Iterate prompts\u002Ftools.",[22,4073,4074],{},"Exercise: Build your golden set from 20 prod logs; eval new model (e.g., switch GPT-4o-mini to cheaper o1-mini—verify perf parity).",[22,4076,4077],{},"\"Before Braintrust, no way to simulate cheaper model perf.\" — Trainline on cost optimization.",[17,4079,4081],{"id":4080},"deploy-score-online-and-close-the-loop","Deploy, Score Online, and Close the Loop",[22,4083,4084],{},[32,4085,4086],{},"Production flow:",[4088,4089,4090,4093,4096,4099],"ol",{},[29,4091,4092],{},"Deploy via Braintrust API: Prod traces auto-log.",[29,4094,4095],{},"Online scoring: Real-time evals on 1% traffic; alert \u003Cthreshold.",[29,4097,4098],{},"Monitor dashboards: P95 latency, failure rate, token $\u002Fquery.",[29,4100,4101],{},"Feedback loop: Failed prod traces → new golden data → retrain eval set.",[22,4103,4104],{},"Trainline example: Travel assistant evals on tone, helpfulness, complex reasoning (ticket types\u002Fdelays). Ships features 2x faster.",[22,4106,4107],{},"Edge cases: No sub for prod data. Use Braintrust to mine failures (e.g., 5% refund misclassifications → specialist fix).",[22,4109,4110],{},"\"Move fast without breaking things at Trainline scale.\" — Core mindset.",[17,4112,163],{"id":162},[26,4114,4115,4118,4121,4124,4127,4130,4133,4136,4139],{},[29,4116,4117],{},"Decompose agents into single-responsibility stages + tools over monolithic prompts for +20% accuracy.",[29,4119,4120],{},"Instrument everything with Braintrust traces from day 0—reveal hidden failure modes logs miss.",[29,4122,4123],{},"Build golden datasets from real logs; eval offline before model\u002Fcost changes.",[29,4125,4126],{},"Online scoring on prod subset + alerts prevents regressions.",[29,4128,4129],{},"Flywheel: Trace → Eval → Fix → Monitor; Trainline ships agent features confidently at 27M-user scale.",[29,4131,4132],{},"Start small: Instrument existing app, add 50 golden examples, iterate weekly.",[29,4134,4135],{},"Custom scorers beat generic (e.g., domain-specific refund rules).",[29,4137,4138],{},"Trade latency for reliability in agentic chains—users value correct over instant.",[29,4140,4141],{},"Platform-agnostic: Works with any LLM\u002Fagent framework.",[22,4143,4144],{},"\"Perfection is the enemy of good—start the flywheel somewhere.\" — Giran Moodley.",[4146,4147,4148],"style",{},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":191,"searchDepth":192,"depth":192,"links":4150},[4151,4152,4153,4154,4155,4156],{"id":3822,"depth":192,"text":3823},{"id":3835,"depth":192,"text":3836},{"id":3985,"depth":192,"text":3986},{"id":4043,"depth":192,"text":4044},{"id":4080,"depth":192,"text":4081},{"id":162,"depth":192,"text":163},[248],{"content_references":4159,"triage":4164},[4160],{"type":4161,"title":4162,"context":4163},"tool","Braintrust","recommended",{"relevance":215,"novelty":216,"quality":216,"actionability":215,"composite":4165,"reasoning":4166},4.55,"Category: AI & LLMs. The article provides a detailed, actionable framework for building production-grade AI agents, addressing the common pain point of transitioning from prototypes to production. It outlines specific steps and principles, such as decomposing tasks into microservices-like stages and emphasizing observability, which are directly applicable to the audience's work.","\u002Fsummaries\u002F04b07447c79b4905-ship-reliable-ai-agents-braintrust-hands-on-summary","2026-05-01 14:00:06","2026-05-03 16:42:22",{"title":3812,"description":191},{"loc":4167},"9cd5b36bc7546cf8","https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=ZdheJTfLu-s","summaries\u002F04b07447c79b4905-ship-reliable-ai-agents-braintrust-hands-on-summary",[233,232,235,234],"Build production-grade multi-step AI agents by breaking into specialist stages, instrumenting traces, evaluating with golden datasets, and monitoring real logs—Trainline's proven workflow.",[],"8C06OOV53eBKztkZREgZgEGnK7brbIUNL2UlJt6JQNA",{"id":4180,"title":4181,"ai":4182,"body":4187,"categories":4438,"created_at":201,"date_modified":201,"description":191,"extension":202,"faq":201,"featured":203,"kicker_label":201,"meta":4439,"navigation":220,"path":4453,"published_at":4454,"question":201,"scraped_at":4455,"seo":4456,"sitemap":4457,"source_id":4458,"source_name":227,"source_type":228,"source_url":4459,"stem":4460,"tags":4461,"thumbnail_url":201,"tldr":4462,"tweet":201,"unknown_tags":4463,"__hash__":4464},"summaries\u002Fsummaries\u002F7a58b85e617afcfd-build-mcp-deep-research-agents-writing-pipelines-summary.md","Build MCP Deep Research Agents + Writing Pipelines",{"provider":7,"model":8,"input_tokens":4183,"output_tokens":4184,"processing_time_ms":4185,"cost_usd":4186},8397,2462,18666,0.0028879,{"type":14,"value":4188,"toc":4430},[4189,4193,4196,4199,4202,4206,4209,4241,4244,4247,4250,4254,4257,4301,4304,4307,4310,4314,4317,4343,4346,4349,4353,4356,4376,4379,4382,4384,4410,4413],[17,4190,4192],{"id":4191},"avoid-ai-slop-target-deep-grounded-research-over-shallow-generation","Avoid AI Slop: Target Deep, Grounded Research Over Shallow Generation",[22,4194,4195],{},"AI-generated content like LinkedIn posts often fails with hallucinations, outdated info, vague generalizations (\"most teams miss\"), and slop phrases (\"rapidly evolving landscape\"). Deep research agents fix this by planning strategies, searching the web, analyzing sources (e.g., YouTube videos, GitHub), filtering for relevance\u002Ftrustworthiness, and synthesizing cited artifacts. This workshop builds one using MCP (Multi-Chain Prompting) for agentic reasoning, emphasizing goal-directed loops: plan → search\u002Finspect → pivot\u002Frefine → synthesize.",[22,4197,4198],{},"Key principle: Research demands high precision\u002Frecall to combat context rot (performance degradation beyond ~200k tokens due to lost-in-the-middle issues). Start simple—ask if a prompt suffices, then escalate to RAG, workflows, or agents only if dynamic branching or reactions to environment (e.g., web) are needed. Common mistake: Overbuilding multi-agents for fixed sequences, adding unreliability without value.",[22,4200,4201],{},"\"Deep research is one of the best ways to learn how to build real AI systems because it forces you to combine reasoning, planning, autonomy, tools, grounding, and feedback loops.\"",[17,4203,4205],{"id":4204},"autonomy-slider-match-workflows-or-agents-to-constraints","Autonomy Slider: Match Workflows or Agents to Constraints",[22,4207,4208],{},"AI engineering balances cost\u002Flatency\u002Fquality\u002Fprivacy via an \"autonomy slider\":",[26,4210,4211,4217,4223,4229,4235],{},[29,4212,4213,4216],{},[32,4214,4215],{},"Prompts",": For known tasks; add few-shot examples.",[29,4218,4219,4222],{},[32,4220,4221],{},"Context injection",": Paste \u003C200k tokens or cache for static docs.",[29,4224,4225,4228],{},[32,4226,4227],{},"RAG\u002Fworkflows",": Fixed chains for sequential tasks (e.g., ticket classification → routing → drafting → validation). Use routers for conditions, parallel calls for voting, loops for judge feedback.",[29,4230,4231,4234],{},[32,4232,4233],{},"Agents",": For dynamic actions (plan tools, react to results). Limit to one agent + specialist tools (own prompts\u002FLLMs) to preserve global context.",[29,4236,4237,4240],{},[32,4238,4239],{},"Multi-agents",": Delegate when >20 tools or context >200k; e.g., sub-agents for security silos.",[22,4242,4243],{},"Tradeoffs: More autonomy = less control\u002Fhigher cost. Example: CRM marketing bot—client wanted multi-agents for grant appeal, but sequential workflow (plan → retrieve client data → generate → validate) sufficed via one agent calling format-specific tools (SMS\u002Femail). Tools as \"specialists\" keep decisions centralized, avoiding handoff errors.",[22,4245,4246],{},"Manage context budget: Trim\u002Fsummarize\u002Fretrieve selectively; delegate to tools\u002Fsub-agents. Avoid context rot by staying lean.",[22,4248,4249],{},"\"We always want to use the simplest solution... if the model already knows enough about the task, you can just prompt it.\"",[17,4251,4253],{"id":4252},"mcp-agent-architecture-tools-for-web-video-synthesis","MCP Agent Architecture: Tools for Web, Video, Synthesis",[22,4255,4256],{},"MCP server orchestrates the agent:",[4088,4258,4259,4265,4295],{},[29,4260,4261,4264],{},[32,4262,4263],{},"Setup",": Register tools (schemas, descriptions). Use Gemini for grounding.",[29,4266,4267,4270,4271],{},[32,4268,4269],{},"Core tools",":\n",[26,4272,4273,4283,4289],{},[29,4274,4275,4278,4279,4282],{},[32,4276,4277],{},"Deep research",": Prompt for strategy (e.g., \"Plan 3-5 searches on ",[3847,4280,4281],{},"topic",", prioritize recent\u002Fauthoritative sources\"). Calls web search, filters results.",[29,4284,4285,4288],{},[32,4286,4287],{},"YouTube analysis",": Transcribe\u002Fextract timestamps, summarize key segments, cite clips.",[29,4290,4291,4294],{},[32,4292,4293],{},"Compile research",": Synthesize evidence into markdown artifact with citations; self-evaluate relevance.",[29,4296,4297,4300],{},[32,4298,4299],{},"Prompting",": Teach via few-shots (e.g., plan → execute → reflect). Workflow: Goal → Plan skills → Execute → Compile → Output.",[22,4302,4303],{},"Live demo: Input \"What is AI engineering?\" → Agent plans searches (Towards AI, papers), analyzes videos, outputs cited report. Pivots on gaps (e.g., re-search if shallow).",[22,4305,4306],{},"Prerequisites: Python\u002FTypeScript comfort, LLM APIs (Gemini\u002FOpenAI). Fits early in product pipelines for content automation.",[22,4308,4309],{},"Quality criteria: Grounded (citations), precise (no noise), iterative (feedback loops). Mistake: Exhaustive scraping—filter aggressively for signal.",[17,4311,4313],{"id":4312},"constrained-writing-evaluator-optimizer-over-freeform-agents","Constrained Writing: Evaluator-Optimizer Over Freeform Agents",[22,4315,4316],{},"Research is exploratory (agentic), writing is polish-focused (workflow). Pipe research artifact to writer:",[4088,4318,4319,4325,4331,4337],{},[29,4320,4321,4324],{},[32,4322,4323],{},"Guidelines",": Explicit structure (intro\u002Fhook → sections → code\u002Fimages → CTA), tone (practical, no hype), length (~500 words for LinkedIn).",[29,4326,4327,4330],{},[32,4328,4329],{},"Few-shot prompting",": 2-3 examples of good posts (grounded, opinionated, cited).",[29,4332,4333,4336],{},[32,4334,4335],{},"Evaluator-optimizer loop",": Writer drafts → Reviewer scores (relevance, slop-free, value) → Optimizer revises. Repeat 2-3x.",[29,4338,4339,4342],{},[32,4340,4341],{},"Post-skill",": Generate images\u002Fcode snippets if needed.",[22,4344,4345],{},"Why constrained? Reduces hallucinations, enforces brand voice. Demo: Research on \"AI engineering\" → Polished post with runnable code, no \"most teams\" fluff.",[22,4347,4348],{},"\"Writing quality often improves with tighter workflows, review loops, and explicit guidance.\"",[17,4350,4352],{"id":4351},"observability-trace-judge-iterate-with-metrics","Observability: Trace, Judge, Iterate with Metrics",[22,4354,4355],{},"Use Opik for tracing (visualize chains, tool calls, latencies). Build LLM Judge:",[4088,4357,4358,4364,4370],{},[29,4359,4360,4363],{},[32,4361,4362],{},"Dataset",": Curate input\u002Foutput pairs (topics → gold research\u002Fwriting).",[29,4365,4366,4369],{},[32,4367,4368],{},"Metrics",": F1-score on citations\u002Frelevance (judge prompts: \"Rate 1-10 on groundedness, novelty\").",[29,4371,4372,4375],{},[32,4373,4374],{},"Eval loop",": Run agent → Judge → Log failures → Tune prompts\u002Ftools.",[22,4377,4378],{},"Production tip: Human-in-loop for edge cases; measure cost\u002Ftask.",[22,4380,4381],{},"\"The context grows and the performance degrades which we call context rot... manage this context budget.\"",[17,4383,163],{"id":162},[26,4385,4386,4389,4392,4395,4398,4401,4404,4407],{},[29,4387,4388],{},"Start with autonomy slider: Prompts > workflows > single agent > multi-agents; simplest wins reliability.",[29,4390,4391],{},"Build research agents with MCP\u002Ftools for planning (strategy), execution (search\u002Fanalyze), synthesis (cited markdown).",[29,4393,4394],{},"Delegate via tools to fight context rot—keep agent context \u003C200k tokens.",[29,4396,4397],{},"For writing, use evaluator-optimizer: Few-shots + review loops > open agents.",[29,4399,4400],{},"Instrument everything: Opik traces + LLM Judge with F1 on datasets for continuous improvement.",[29,4402,4403],{},"Prioritize precision\u002Frecall in search; filter noise early to avoid slop.",[29,4405,4406],{},"Test in production: Build for utility (e.g., Towards AI courses), not demos.",[29,4408,4409],{},"Exercise: Fork GitHub repo, run on your topic, eval F1 >0.8 before deploying.",[22,4411,4412],{},"Notable quotes:",[4088,4414,4415,4418,4421,4424,4427],{},[29,4416,4417],{},"\"Most people are interested in building agents, but most... are actually somewhat super simple workflows.\" (On over-engineering)",[29,4419,4420],{},"\"Tools as specialists but the global context stays within our only agent.\" (Single-agent advantage)",[29,4422,4423],{},"\"High quality technical content is expensive... automate most of this process as writer augmentation.\" (Business rationale)",[29,4425,4426],{},"\"It's a goal-directed research loop: one that can search, inspect, pivot, and progressively refine.\" (Core agent behavior)",[29,4428,4429],{},"\"AI products... combine all of that. They combine tools, workflows.\" (Holistic systems)",{"title":191,"searchDepth":192,"depth":192,"links":4431},[4432,4433,4434,4435,4436,4437],{"id":4191,"depth":192,"text":4192},{"id":4204,"depth":192,"text":4205},{"id":4252,"depth":192,"text":4253},{"id":4312,"depth":192,"text":4313},{"id":4351,"depth":192,"text":4352},{"id":162,"depth":192,"text":163},[248],{"content_references":4440,"triage":4451},[4441,4445,4447,4449],{"type":4442,"title":4443,"author":4444,"context":213},"book","LM Engineers Handbook","Paul Iusztin",{"type":4161,"title":4446,"context":4163},"Opik",{"type":4161,"title":4448,"context":209},"MCP",{"type":211,"title":4450,"context":213},"Towards AI GitHub Repository",{"relevance":215,"novelty":216,"quality":216,"actionability":215,"composite":4165,"reasoning":4452},"Category: AI & LLMs. The article provides a hands-on guide for building a research agent using MCP, addressing practical applications of AI in product development. It emphasizes actionable strategies for creating goal-directed AI systems, which directly aligns with the audience's need for concrete examples and production-ready features.","\u002Fsummaries\u002F7a58b85e617afcfd-build-mcp-deep-research-agents-writing-pipelines-summary","2026-04-20 18:45:16","2026-04-21 15:12:45",{"title":4181,"description":191},{"loc":4453},"68f0a1a19e18b1b7","https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=mYSRn6PC1mc","summaries\u002F7a58b85e617afcfd-build-mcp-deep-research-agents-writing-pipelines-summary",[233,232,234,235],"Hands-on guide to engineer a goal-directed research agent using MCP for web search, YouTube analysis, evidence synthesis, then pipe outputs to a constrained writing workflow with evaluation—distilling real-world tradeoffs for production AI systems.",[],"HKEJkvzhM73TLtaoG2PgTarMxjP7_t3TY5GndZvwz1Y",{"id":4466,"title":4467,"ai":4468,"body":4473,"categories":4541,"created_at":201,"date_modified":201,"description":191,"extension":202,"faq":201,"featured":203,"kicker_label":201,"meta":4542,"navigation":220,"path":4576,"published_at":4577,"question":201,"scraped_at":4578,"seo":4579,"sitemap":4580,"source_id":4581,"source_name":4582,"source_type":228,"source_url":4583,"stem":4584,"tags":4585,"thumbnail_url":201,"tldr":4586,"tweet":201,"unknown_tags":4587,"__hash__":4588},"summaries\u002Fsummaries\u002F7ed780c99c8d1409-harness-engineering-powers-ai-agents-beyond-models-summary.md","Harness Engineering Powers AI Agents Beyond Models",{"provider":7,"model":8,"input_tokens":4469,"output_tokens":4470,"processing_time_ms":4471,"cost_usd":4472},8174,2183,15389,0.00243545,{"type":14,"value":4474,"toc":4535},[4475,4479,4482,4485,4488,4492,4495,4515,4518,4522,4525,4528,4532],[17,4476,4478],{"id":4477},"harness-engineering-trumps-model-reliance-for-agent-success","Harness Engineering Trumps Model Reliance for Agent Success",[22,4480,4481],{},"AI agent failures like ignoring instructions, unsafe commands, or looping stem from configuration gaps, not model limits. Solve by engineering harnesses: layers connecting, protecting, and orchestrating models without altering core logic. A coding agent = model + harness, where harness customizes interaction via skills, MCP servers, sub-agents, memory files (e.g., agents.md), and repo structure. This subset of context engineering manages context windows to teach codebase specifics absent from training data, boosting task success beyond prompts.",[22,4483,4484],{},"Progressive disclosure feeds agents minimal context first, expanding only if needed—avoids overwhelming windows, as OpenAI used to ship software betas with zero manual code. Harnesses address model gaps: add bash\u002Fcode execution for writing code; sandboxed environments for safety; memory\u002Fweb search\u002FMCPs for knowledge; loops like Karpathy's auto-research or Ralph Wigam for long-horizon tasks.",[22,4486,4487],{},"Trade-off: Harnesses encode assumptions (e.g., context resets for 'context anxiety' in Claude Sonnet 4.5) that stale as models advance—Claude Opus 4.5 needed no resets, turning them into dead weight.",[17,4489,4491],{"id":4490},"three-layer-architecture-ensures-scalable-execution","Three-Layer Architecture Ensures Scalable Execution",[22,4493,4494],{},"Anthropic's framework divides harnesses into:",[26,4496,4497,4503,4509],{},[29,4498,4499,4502],{},[32,4500,4501],{},"Information layer",": Controls visible data\u002Fcapabilities—memory\u002Fcontext management, tools\u002Fskills.",[29,4504,4505,4508],{},[32,4506,4507],{},"Execution layer",": Handles decomposition, collaboration, failure recovery—orchestration, coordination, infrastructure, guardrails.",[29,4510,4511,4514],{},[32,4512,4513],{},"Feedback layer",": Drives improvement—evaluation, verification, tracing, observability.",[22,4516,4517],{},"This enables environments, feedback loops, and controls for complex software at scale. User-built 'outer harness' (e.g., repo tweaks for Claude Code\u002FCursor\u002FCodex\u002FOpen Claw) tailors inner harnesses from labs, determining codebase-specific outcomes.",[17,4519,4521],{"id":4520},"harnesses-unlock-gains-models-cant-match","Harnesses Unlock Gains Models Can't Match",[22,4523,4524],{},"Blitzcy hit 66.5% on SWE-bench Pro (vs. GPT-5.4's 57.7%) via knowledge graphs providing deep codebase context raw models miss on details\u002Fcorner cases. Latent Space pits 'big model' (minimal wrappers, per Claude Code's Boris Cherny\u002FCat Wu or OpenAI's Noam Brown) against 'big harness' (essential for blank-slate models, per LlamaIndex's Jerry Liu). Consensus: Both matter, but harnesses yield bigger jumps now—per 'bitter lesson,' models scale, yet configuration barriers persist for complex workflows.",[22,4526,4527],{},"Industry convergence: Claude Code's looping agent + tools generalizes to any task (Linear\u002FNotion\u002FGoogle building similar). By 2026, software firms converge on 'general harness' (user input → context → model\u002Ftools loop → result) for self-improving systems. Winners leverage distribution, workflows, proprietary context, fast observation-to-improvement loops.",[17,4529,4531],{"id":4530},"build-disposable-harnesses-for-evolving-models","Build Disposable Harnesses for Evolving Models",[22,4533,4534],{},"Anthropic's Managed Agents creates 'meta-harness': Stable interfaces outlast changing implementations, decoupling brain (agent loop), hands (sandbox), and event log (session). Reframe enterprise AI: Prioritize agent environments over model picks—organizational design as ultimate harness for thriving AI-human systems.",{"title":191,"searchDepth":192,"depth":192,"links":4536},[4537,4538,4539,4540],{"id":4477,"depth":192,"text":4478},{"id":4490,"depth":192,"text":4491},{"id":4520,"depth":192,"text":4521},{"id":4530,"depth":192,"text":4531},[],{"content_references":4543,"triage":4573},[4544,4547,4550,4553,4557,4561,4564,4566,4569,4571],{"type":211,"title":4545,"author":4546,"context":209},"Cursor 3 announcement post","Cursor",{"type":211,"title":4548,"author":4549,"context":209},"Scaling Managed Agents, Decoupling the Brain from the Hands","Anthropic",{"type":211,"title":4551,"author":4552,"context":209},"Is Harness Engineering Real?","Latent Space",{"type":211,"title":4554,"author":4555,"publisher":4556,"context":209},"Skill Issue, Harness Engineering for Coding Agents","Kyle","humanlayer.dev",{"type":211,"title":4558,"author":4559,"publisher":4560,"context":209},"The Anatomy of an Agent Harness","Viv","LangChain",{"type":211,"title":4562,"author":4563,"context":209},"harness engineering leveraging Codex in an agent-first world","OpenAI",{"type":4161,"title":4565,"context":213},"Blitzcy",{"type":211,"title":4567,"author":4568,"context":209},"The Great Convergence","Nicolas Charrier",{"type":4161,"title":4570,"author":4549,"context":213},"Claude Code",{"type":4161,"title":4572,"author":4546,"context":213},"Cursor 3",{"relevance":215,"novelty":216,"quality":216,"actionability":216,"composite":4574,"reasoning":4575},4.35,"Category: AI & LLMs. The article provides a deep dive into harness engineering for AI agents, addressing specific pain points like model limitations and configuration gaps, which are crucial for product builders. It offers actionable insights on creating a three-layer architecture for scalable execution, making it highly relevant and practical.","\u002Fsummaries\u002F7ed780c99c8d1409-harness-engineering-powers-ai-agents-beyond-models-summary","2026-04-15 13:18:16","2026-04-19 03:23:45",{"title":4467,"description":191},{"loc":4576},"7ed780c99c8d1409","The AI Daily Brief","https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=OTjZBjq5FPg","summaries\u002F7ed780c99c8d1409-harness-engineering-powers-ai-agents-beyond-models-summary",[233,232,234,235],"Harness engineering—systems, tools, and interfaces around AI models—delivers reliable performance via context, safe execution, and orchestration, often outperforming model upgrades alone.",[],"D8SDxS0K5Pva7QX7L9rPzU9aPOZBfy0w0sdaaEJbTxc",{"id":4590,"title":4591,"ai":4592,"body":4597,"categories":4668,"created_at":201,"date_modified":201,"description":191,"extension":202,"faq":201,"featured":203,"kicker_label":201,"meta":4669,"navigation":220,"path":4670,"published_at":4671,"question":201,"scraped_at":201,"seo":4672,"sitemap":4673,"source_id":4674,"source_name":4675,"source_type":228,"source_url":4676,"stem":4677,"tags":4678,"thumbnail_url":201,"tldr":4679,"tweet":201,"unknown_tags":4680,"__hash__":4681},"summaries\u002Fsummaries\u002Ftiltgent-cli-profiles-ai-agent-judgment-tilt-via-b-summary.md","Tiltgent CLI Profiles AI Agent Judgment Tilt via Blind Debates",{"provider":7,"model":8,"input_tokens":4593,"output_tokens":4594,"processing_time_ms":4595,"cost_usd":4596},5406,1444,12943,0.00178055,{"type":14,"value":4598,"toc":4662},[4599,4603,4606,4609,4624,4628,4631,4634,4637,4641,4652,4655,4659],[17,4600,4602],{"id":4601},"blind-debates-quantify-judgment-tilt-across-5-axes","Blind Debates Quantify Judgment Tilt Across 5 Axes",[22,4604,4605],{},"Judgment tilt captures an AI agent's systematic preference for one well-argued position over another in blind comparisons, driven by training, RLHF, and prompts. Even vanilla models show tilt, like -0.50 on Stability and -0.40 on Tradition in early tests. Tiltgent generates 10 escalating debate rounds from a topic, pitting arguments from 21 worldview archetypes positioned on five axes: Order↔Emergence, Humanist↔Systems-first, Stability↔Dynamism, Local agency↔Coordinated scale, Tradition↔Reinvention.",[22,4607,4608],{},"Archetypes pair via Euclidean distance for ideological separation, each with unique system prompts, rhetorical moves, accusations, and vocabulary to avoid overlap. Your agent judges blindly (no labels), picks winners 3x per round for consensus (pick agreement rate like 0.93, unstable rounds like 1), and subtracts a vanilla baseline run to isolate your prompt's effect. Output: JSON profile with dimension scores (e.g., order_emergence: 0.65), contradiction lines (e.g., \"You champion market forces... but go cold when they threaten human welfare\"), and stability metrics.",[22,4610,4611,4612,4615,4616,4619,4620,4623],{},"Run ",[3864,4613,4614],{},"npx tiltgent eval --prompt your-agent.txt --topic \"Universal basic income\""," for a 5-minute eval (~$0.25–0.30 Anthropic API cost). Use ",[3864,4617,4618],{},"tiltgent diff"," for instant profile comparisons, ",[3864,4621,4622],{},"tiltgent inspect"," for terminal views. MIT-licensed, 3 deps, bring your API key.",[17,4625,4627],{"id":4626},"archetype-calibration-prevents-style-over-substance-bias","Archetype Calibration Prevents Style Over Substance Bias",[22,4629,4630],{},"21 archetypes underwent triple audits (ChatGPT, Gemini, Grok): 14 vector fixes, 11 prompt sharpenings, 2 merges (indistinguishable in blind tests), 3 additions for gaps. Universal debate prompts enforce substance focus, countering prose dominance—without it, dramatic styles win regardless of worldview.",[22,4632,4633],{},"Synthetic validation: 4 agents (Hard Accelerationist, Cautious Humanist, etc.) on 2 topics at temp=0 showed stable picks, 0.93 axis separation (Humanist vs Systems), topic-varying baseline tilt mandating per-topic calibration. Self-preference reduced via baseline subtraction, though Anthropic models generate and judge (multi-model support next).",[22,4635,4636],{},"Full roster and prompts public in repo—audit yourself.",[17,4638,4640],{"id":4639},"prompt-testing-and-diagnostics-drive-production-use","Prompt Testing and Diagnostics Drive Production Use",[22,4642,4643,4644,4647,4648,4651],{},"Test prompt changes: ",[3864,4645,4646],{},"eval"," before\u002Fafter, ",[3864,4649,4650],{},"diff"," shows dimension shifts (e.g., Humanist↔Systems). Profile cross-topics (balanced on healthcare? Market-tilt on economics?). Compare models same-prompt. Pre-deploy: Inspect summarizers\u002Ftriers for argumentative leanings.",[22,4653,4654],{},"Reveals preferences under pick pressure—beats direct opinion queries yielding hedges. Not moral bias label or fact-check; assumes competent arguments, measures value tilts (e.g., libertarian agents favor markets, health agents favor coordination).",[17,4656,4658],{"id":4657},"rhetorical-balance-remains-open-challenge","Rhetorical Balance Remains Open Challenge",[22,4660,4661],{},"Archetypes aren't perfectly persuasive-equal—one won 4\u002F4 matchups via \"second-order consequences\" authority. Per-topic baseline mitigates but doesn't eliminate. v0.1 unproven on production agents, non-Anthropic targets (GPT-4, etc.), or open models—engine model-agnostic, validation pending.",{"title":191,"searchDepth":192,"depth":192,"links":4663},[4664,4665,4666,4667],{"id":4601,"depth":192,"text":4602},{"id":4626,"depth":192,"text":4627},{"id":4639,"depth":192,"text":4640},{"id":4657,"depth":192,"text":4658},[],{},"\u002Fsummaries\u002Ftiltgent-cli-profiles-ai-agent-judgment-tilt-via-b-summary","2026-04-08 21:21:19",{"title":4591,"description":191},{"loc":4670},"85f6bf7dbb0067f3","Towards AI","https:\u002F\u002Funknown","summaries\u002Ftiltgent-cli-profiles-ai-agent-judgment-tilt-via-b-summary",[233,234,235,232],"Tiltgent CLI measures AI agents' systematic judgment biases—preferences for certain arguments in blind debates—across 5 ideological axes using 21 calibrated archetypes, enabling prompt regression testing and model comparisons for $0.25–0.30 per run.",[],"T2T-RE2UhRqH6x3Dol-TCGlFx74KVW1WDbmqiaaq_34"]