[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"summary-1ac17c99e1a87b1f-scaling-transformer-training-to-5-million-tokens-summary":3,"summaries-facets-categories":147,"summary-related-1ac17c99e1a87b1f-scaling-transformer-training-to-5-million-tokens-summary":4622},{"id":4,"title":5,"ai":6,"body":13,"categories":106,"created_at":108,"date_modified":108,"description":99,"extension":109,"faq":108,"featured":110,"kicker_label":108,"meta":111,"navigation":126,"path":127,"published_at":128,"question":108,"scraped_at":129,"seo":130,"sitemap":131,"source_id":132,"source_name":133,"source_type":134,"source_url":135,"stem":136,"tags":137,"thumbnail_url":142,"tldr":143,"tweet":144,"unknown_tags":145,"__hash__":146},"summaries\u002Fsummaries\u002F1ac17c99e1a87b1f-scaling-transformer-training-to-5-million-tokens-summary.md","Scaling Transformer Training to 5 Million Tokens",{"provider":7,"model":8,"input_tokens":9,"output_tokens":10,"processing_time_ms":11,"cost_usd":12},"openrouter","google\u002Fgemini-3.1-flash-lite",6017,673,3862,0.00251375,{"type":14,"value":15,"toc":98},"minimark",[16,21,25,29,32,67,71,74,78],[17,18,20],"h2",{"id":19},"the-memory-bottleneck-in-long-context-training","The Memory Bottleneck in Long-Context Training",[22,23,24],"p",{},"Training standard transformer models with massive context windows (e.g., 3M+ tokens) faces two primary constraints: quadratic computational complexity and linear memory growth. Even on high-end hardware like an 8xH100 node, standard implementations fail because the model parameters and attention activations quickly exceed available GPU memory.",[17,26,28],{"id":27},"the-stack-of-optimization-techniques","The Stack of Optimization Techniques",[22,30,31],{},"To reach a 3-million token context, a layered approach is required to manage memory usage:",[33,34,35,43,49,55,61],"ul",{},[36,37,38,42],"li",{},[39,40,41],"strong",{},"Fully Sharded Data Parallelism (FSDP):"," Distributes model parameters across all available GPUs to prevent memory exhaustion from the model weights alone.",[36,44,45,48],{},[39,46,47],{},"DeepSpeed Ulysses:"," A context parallelism technique that distributes attention heads across GPUs. Instead of every GPU computing the full sequence, each GPU handles specific heads, reducing activation memory by approximately 8x.",[36,50,51,54],{},[39,52,53],{},"Activation Checkpointing:"," Recomputes activations during the backward pass rather than storing them, providing another 8x reduction in memory usage.",[36,56,57,60],{},[39,58,59],{},"CPU Offloading:"," Moves transformer block inputs to CPU memory when not actively needed for backpropagation, prefetching them just-in-time to minimize performance impact.",[36,62,63,66],{},[39,64,65],{},"Chunked Sequence Training:"," Tiles element-wise operations (like loss calculations and MLPs) across the sequence length to avoid allocating massive buffers that scale linearly with the token count.",[17,68,70],{"id":69},"untied-ulysses-pushing-to-5-million-tokens","Untied Ulysses: Pushing to 5 Million Tokens",[22,72,73],{},"To surpass the 3-million token limit, the team developed \"Untied Ulysses.\" This technique refines context parallelism by further chunking attention heads. Instead of allocating a single large buffer per head group, the system iterates through smaller chunks of heads, reusing the same memory buffers across iterations. This significantly lowers activation memory requirements with negligible impact on throughput.",[17,75,77],{"id":76},"practical-implementation-advice","Practical Implementation Advice",[33,79,80,86,92],{},[36,81,82,85],{},[39,83,84],{},"Profiling is critical:"," Use tools like the PyTorch Profiler to identify exactly where memory is being consumed, as bottlenecks often appear in unexpected places.",[36,87,88,91],{},[39,89,90],{},"Trade-offs:"," There is a direct relationship between chunk size and throughput. Larger chunks increase memory utilization but can improve overall training speed.",[36,93,94,97],{},[39,95,96],{},"Reinvesting Memory:"," By stacking these optimizations, you can free up memory that can be reinvested into other training stages or used to push context lengths even further.",{"title":99,"searchDepth":100,"depth":100,"links":101},"",2,[102,103,104,105],{"id":19,"depth":100,"text":20},{"id":27,"depth":100,"text":28},{"id":69,"depth":100,"text":70},{"id":76,"depth":100,"text":77},[107],"AI & LLMs",null,"md",false,{"content_references":112,"triage":121},[113,118],{"type":114,"title":115,"author":116,"context":117},"tool","DeepSpeed Ulysses","Microsoft","cited",{"type":114,"title":119,"context":120},"PyTorch Profiler","recommended",{"relevance":122,"novelty":123,"quality":123,"actionability":123,"composite":124,"reasoning":125},5,4,4.35,"Category: AI & LLMs. The article provides in-depth techniques for optimizing transformer training, directly addressing the audience's need for practical applications in AI model development. It discusses specific methods like Fully Sharded Data Parallelism and Untied Ulysses, which are actionable for developers looking to implement long-context training.",true,"\u002Fsummaries\u002F1ac17c99e1a87b1f-scaling-transformer-training-to-5-million-tokens-summary","2026-06-08 17:00:21","2026-06-09 12:56:17",{"title":5,"description":99},{"loc":127},"1ac17c99e1a87b1f","AI Engineer","video","https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=TUnPNY4E2fw","summaries\u002F1ac17c99e1a87b1f-scaling-transformer-training-to-5-million-tokens-summary",[138,139,140,141],"llm","machine-learning","python","ai-tools","https:\u002F\u002Fi.ytimg.com\u002Fvi\u002FTUnPNY4E2fw\u002Fhqdefault.jpg","To train models with multi-million token contexts, you must stack memory-optimization techniques—including context parallelism, activation checkpointing, and a novel method called 'Untied Ulysses'—to bypass GPU memory bottlenecks.","This is a technical breakdown of the memory-optimization stack required to train models on extremely long context windows (up to 5 million tokens). The speaker details how to combine standard techniques like [DeepSpeed Ulysses](https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002FDeepSpeed) and activation checkpointing with their own \"Untied Ulysses\" method to reduce memory overhead by chunking and reusing attention buffers.",[],"vdZh7ZzpnTzdTG3G79QASc2ywoHe_prr-WYngZH8xkI",[148,151,154,156,159,162,164,166,168,170,172,174,177,179,181,183,185,187,189,191,193,195,197,199,201,203,206,209,211,213,215,217,220,222,224,226,229,231,233,235,237,239,241,243,245,247,249,251,254,256,258,260,262,264,266,268,270,272,274,276,278,280,282,284,286,288,290,292,294,296,298,300,302,304,306,308,310,312,314,316,318,320,322,324,326,328,330,332,334,336,338,340,342,344,346,348,350,352,354,356,358,360,362,364,366,368,370,372,374,376,378,380,382,384,386,388,390,392,394,396,398,400,402,404,406,408,410,412,414,416,418,420,422,424,426,428,430,432,434,436,438,440,442,444,446,448,450,452,454,456,458,460,462,464,466,468,470,472,474,476,478,480,482,484,486,488,490,492,494,496,498,500,502,504,506,508,510,512,514,516,518,520,522,524,526,528,530,532,534,536,538,540,542,544,546,548,550,552,554,556,558,560,562,564,566,568,570,572,574,576,578,580,582,584,586,588,590,592,594,596,598,600,602,604,606,608,610,612,614,616,618,620,622,624,626,628,630,632,634,636,638,640,642,644,646,648,650,652,654,656,658,660,662,664,666,668,670,672,674,676,678,680,682,684,686,688,690,692,694,696,698,700,702,704,706,708,710,712,714,716,718,720,722,724,726,728,730,732,734,736,738,740,742,744,746,748,750,752,754,756,758,760,762,764,766,768,770,772,774,776,778,780,782,784,786,788,790,792,794,796,798,800,802,804,806,808,810,812,814,816,818,820,822,824,826,828,830,832,834,836,838,840,842,844,846,848,850,852,854,856,858,860,862,864,866,868,870,872,874,876,878,880,882,884,886,888,890,892,894,896,898,900,902,904,906,908,910,912,914,916,918,920,922,924,926,928,930,932,934,936,938,940,942,944,946,948,950,952,954,956,958,960,962,964,966,968,970,972,974,976,978,980,982,984,986,988,990,992,994,996,998,1000,1002,1004,1006,1008,1010,1012,1014,1016,1018,1020,1022,1024,1026,1028,1030,1032,1034,1036,1038,1040,1042,1044,1046,1048,1050,1052,1054,1056,1058,1060,1062,1064,1066,1068,1070,1072,1074,1076,1078,1080,1082,1084,1086,1088,1090,1092,1094,1096,1098,1100,1102,1104,1106,1108,1110,1112,1114,1116,1118,1120,1122,1124,1126,1128,1130,1132,1134,1136,1138,1140,1142,1144,1146,1148,1150,1152,1154,1156,1158,1160,1162,1164,1166,1168,1170,1172,1174,1176,1178,1180,1182,1184,1186,1188,1190,1192,1194,1196,1198,1200,1202,1204,1206,1208,1210,1212,1214,1216,1218,1220,1222,1224,1226,1228,1230,1232,1234,1236,1238,1240,1242,1244,1246,1248,1250,1252,1254,1256,1258,1260,1262,1264,1266,1268,1270,1272,1274,1276,1278,1280,1282,1284,1286,1288,1290,1292,1294,1296,1298,1300,1302,1304,1306,1308,1310,1312,1314,1316,1318,1320,1322,1324,1326,1328,1330,1332,1334,1336,1338,1340,1342,1344,1346,1348,1350,1352,1354,1356,1358,1360,1362,1364,1366,1368,1370,1372,1374,1376,1378,1380,1382,1384,1386,1388,1390,1392,1394,1396,1398,1400,1402,1404,1406,1408,1410,1412,1414,1416,1418,1420,1422,1424,1426,1428,1430,1432,1434,1436,1438,1440,1442,1444,1446,1448,1450,1452,1454,1456,1458,1460,1462,1464,1466,1468,1470,1472,1474,1476,1478,1480,1482,1484,1486,1488,1490,1492,1494,1496,1498,1500,1502,1504,1506,1508,1510,1512,1514,1516,1518,1520,1522,1524,1526,1528,1530,1532,1534,1536,1538,1540,1542,1544,1546,1548,1550,1552,1554,1556,1558,1560,1562,1564,1566,1568,1570,1572,1574,1576,1578,1580,1582,1584,1586,1588,1590,1592,1594,1596,1598,1600,1602,1604,1606,1608,1610,1612,1614,1616,1618,1620,1622,1624,1626,1628,1630,1632,1634,1636,1638,1640,1642,1644,1646,1648,1650,1652,1654,1656,1658,1660,1662,1664,1666,1668,1670,1672,1674,1676,1678,1680,1682,1684,1686,1688,1690,1692,1694,1696,1698,1700,1702,1704,1706,1708,1710,1712,1714,1716,1718,1720,1722,1724,1726,1728,1730,1732,1734,1736,1738,1740,1742,1744,1746,1748,1750,1752,1754,1756,1758,1760,1762,1764,1766,1768,1770,1772,1774,1776,1778,1780,1782,1784,1786,1788,1790,1792,1794,1796,1798,1800,1802,1804,1806,1808,1810,1812,1814,1816,1818,1820,1822,1824,1826,1828,1830,1832,1834,1836,1838,1840,1842,1844,1846,1848,1850,1852,1854,1856,1858,1860,1862,1864,1866,1868,1870,1872,1874,1876,1878,1880,1882,1884,1886,1888,1890,1892,1894,1896,1898,1900,1902,1904,1906,1908,1910,1912,1914,1916,1918,1920,1922,1924,1926,1928,1930,1932,1934,1936,1938,1940,1942,1944,1946,1948,1950,1952,1954,1956,1958,1960,1962,1964,1966,1968,1970,1972,1974,1976,1978,1980,1982,1984,1986,1988,1990,1992,1994,1996,1998,2000,2002,2004,2006,2008,2010,2012,2014,2016,2018,2020,2022,2024,2026,2028,2030,2032,2034,2036,2038,2040,2042,2044,2046,2048,2050,2052,2054,2056,2058,2060,2062,2064,2066,2068,2070,2072,2074,2076,2078,2080,2082,2084,2086,2088,2090,2092,2094,2096,2098,2100,2102,2104,2106,2108,2110,2112,2114,2116,2118,2120,2122,2124,2126,2128,2130,2132,2134,2136,2138,2140,2142,2144,2146,2148,2150,2152,2154,2156,2158,2160,2162,2164,2166,2168,2170,2172,2174,2176,2178,2180,2182,2184,2186,2188,2190,2192,2194,2196,2198,2200,2202,2204,2206,2208,2210,2212,2214,2216,2218,2220,2222,2224,2226,2228,2230,2232,2234,2236,2238,2240,2242,2244,2246,2248,2250,2252,2254,2256,2258,2260,2262,2264,2266,2268,2270,2272,2274,2276,2278,2280,2282,2284,2286,2288,2290,2292,2294,2296,2298,2300,2302,2304,2306,2308,2310,2312,2314,2316,2318,2320,2322,2324,2326,2328,2330,2332,2334,2336,2338,2340,2342,2344,2346,2348,2350,2352,2354,2356,2358,2360,2362,2364,2366,2368,2370,2372,2374,2376,2378,2380,2382,2384,2386,2388,2390,2392,2394,2396,2398,2400,2402,2404,2406,2408,2410,2412,2414,2416,2418,2420,2422,2424,2426,2428,2430,2432,2434,2436,2438,2440,2442,2444,2446,2448,2450,2452,2454,2456,2458,2460,2462,2464,2466,2468,2470,2472,2474,2476,2478,2480,2482,2484,2486,2488,2490,2492,2494,2496,2498,2500,2502,2504,2506,2508,2510,2512,2514,2516,2518,2520,2522,2524,2526,2528,2530,2532,2534,2536,2538,2540,2542,2544,2546,2548,2550,2552,2554,2556,2558,2560,2562,2564,2566,2568,2570,2572,2574,2576,2578,2580,2582,2584,2586,2588,2590,2592,2594,2596,2598,2600,2602,2604,2606,2608,2610,2612,2614,2616,2618,2620,2622,2624,2626,2628,2630,2632,2634,2636,2638,2640,2642,2644,2646,2648,2650,2652,2654,2656,2658,2660,2662,2664,2666,2668,2670,2672,2674,2676,2678,2680,2682,2684,2686,2688,2690,2692,2694,2696,2698,2700,2702,2704,2706,2708,2710,2712,2714,2716,2718,2720,2722,2724,2726,2728,2730,2732,2734,2736,2738,2740,2742,2744,2746,2748,2750,2752,2754,2756,2758,2760,2762,2764,2766,2768,2770,2772,2774,2776,2778,2780,2782,2784,2786,2788,2790,2792,2794,2796,2798,2800,2802,2804,2806,2808,2810,2812,2814,2816,2818,2820,2822,2824,2826,2828,2830,2832,2834,2836,2838,2840,2842,2844,2846,2848,2850,2852,2854,2856,2858,2860,2862,2864,2866,2868,2870,2872,2874,2876,2878,2880,2882,2884,2886,2888,2890,2892,2894,2896,2898,2900,2902,2904,2906,2908,2910,2912,2914,2916,2918,2920,2922,2924,2926,2928,2930,2932,2934,2936,2938,2940,2942,2944,2946,2948,2950,2952,2954,2956,2958,2960,2962,2964,2966,2968,2970,2972,2974,2976,2978,2980,2982,2984,2986,2988,2990,2992,2994,2996,2998,3000,3002,3004,3006,3008,3010,3012,3014,3016,3018,3020,3022,3024,3026,3028,3030,3032,3034,3036,3038,3040,3042,3044,3046,3048,3050,3052,3054,3056,3058,3060,3062,3064,3066,3068,3070,3072,3074,3076,3078,3080,3082,3084,3086,3088,3090,3092,3094,3096,3098,3100,3102,3104,3106,3108,3110,3112,3114,3116,3118,3120,3122,3124,3126,3128,3130,3132,3134,3136,3138,3140,3142,3144,3146,3148,3150,3152,3154,3156,3158,3160,3162,3164,3166,3168,3170,3172,3174,3176,3178,3180,3182,3184,3186,3188,3190,3192,3194,3196,3198,3200,3202,3204,3206,3208,3210,3212,3214,3216,3218,3220,3222,3224,3226,3228,3230,3232,3234,3236,3238,3240,3242,3244,3246,3248,3250,3252,3254,3256,3258,3260,3262,3264,3266,3268,3270,3272,3274,3276,3278,3280,3282,3284,3286,3288,3290,3292,3294,3296,3298,3300,3302,3304,3306,3308,3310,3312,3314,3316,3318,3320,3322,3324,3326,3328,3330,3332,3334,3336,3338,3340,3342,3344,3346,3348,3350,3352,3354,3356,3358,3360,3362,3364,3366,3368,3370,3372,3374,3376,3378,3380,3382,3384,3386,3388,3390,3392,3394,3396,3398,3400,3402,3404,3406,3408,3410,3412,3414,3416,3418,3420,3422,3424,3426,3428,3430,3432,3434,3436,3438,3440,3442,3444,3446,3448,3450,3452,3454,3456,3458,3460,3462,3464,3466,3468,3470,3472,3474,3476,3478,3480,3482,3484,3486,3488,3490,3492,3494,3496,3498,3500,3502,3504,3506,3508,3510,3512,3514,3516,3518,3520,3522,3524,3526,3528,3530,3532,3534,3536,3538,3540,3542,3544,3546,3548,3550,3552,3554,3556,3558,3560,3562,3564,3566,3568,3570,3572,3574,3576,3578,3580,3582,3584,3586,3588,3590,3592,3594,3596,3598,3600,3602,3604,3606,3608,3610,3612,3614,3616,3618,3620,3622,3624,3626,3628,3630,3632,3634,3636,3638,3640,3642,3644,3646,3648,3650,3652,3654,3656,3658,3660,3662,3664,3666,3668,3670,3672,3674,3676,3678,3680,3682,3684,3686,3688,3690,3692,3694,3696,3698,3700,3702,3704,3706,3708,3710,3712,3714,3716,3718,3720,3722,3724,3726,3728,3730,3732,3734,3736,3738,3740,3742,3744,3746,3748,3750,3752,3754,3756,3758,3760,3762,3764,3766,3768,3770,3772,3774,3776,3778,3780,3782,3784,3786,3788,3790,3792,3794,3796,3798,3800,3802,3804,3806,3808,3810,3812,3814,3816,3818,3820,3822,3824,3826,3828,3830,3832,3834,3836,3838,3840,3842,3844,3846,3848,3850,3852,3854,3856,3858,3860,3862,3864,3866,3868,3870,3872,3874,3876,3878,3880,3882,3884,3886,3888,3890,3892,3894,3896,3898,3900,3902,3904,3906,3908,3910,3912,3914,3916,3918,3920,3922,3924,3926,3928,3930,3932,3934,3936,3938,3940,3942,3944,3946,3948,3950,3952,3954,3956,3958,3960,3962,3964,3966,3968,3970,3972,3974,3976,3978,3980,3982,3984,3986,3988,3990,3992,3994,3996,3998,4000,4002,4004,4006,4008,4010,4012,4014,4016,4018,4020,4022,4024,4026,4028,4030,4032,4034,4036,4038,4040,4042,4044,4046,4048,4050,4052,4054,4056,4058,4060,4062,4064,4066,4068,4070,4072,4074,4076,4078,4080,4082,4084,4086,4088,4090,4092,4094,4096,4098,4100,4102,4104,4106,4108,4110,4112,4114,4116,4118,4120,4122,4124,4126,4128,4130,4132,4134,4136,4138,4140,4142,4144,4146,4148,4150,4152,4154,4156,4158,4160,4162,4164,4166,4168,4170,4172,4174,4176,4178,4180,4182,4184,4186,4188,4190,4192,4194,4196,4198,4200,4202,4204,4206,4208,4210,4212,4214,4216,4218,4220,4222,4224,4226,4228,4230,4232,4234,4236,4238,4240,4242,4244,4246,4248,4250,4252,4254,4256,4258,4260,4262,4264,4266,4268,4270,4272,4274,4276,4278,4280,4282,4284,4286,4288,4290,4292,4294,4296,4298,4300,4302,4304,4306,4308,4310,4312,4314,4316,4318,4320,4322,4324,4326,4328,4330,4332,4334,4336,4338,4340,4342,4344,4346,4348,4350,4352,4354,4356,4358,4360,4362,4364,4366,4368,4370,4372,4374,4376,4378,4380,4382,4384,4386,4388,4390,4392,4394,4396,4398,4400,4402,4404,4406,4408,4410,4412,4414,4416,4418,4420,4422,4424,4426,4428,4430,4432,4434,4436,4438,4440,4442,4444,4446,4448,4450,4452,4454,4456,4458,4460,4462,4464,4466,4468,4470,4472,4474,4476,4478,4480,4482,4484,4486,4488,4490,4492,4494,4496,4498,4500,4502,4504,4506,4508,4510,4512,4514,4516,4518,4520,4522,4524,4526,4528,4530,4532,4534,4536,4538,4540,4542,4544,4546,4548,4550,4552,4554,4556,4558,4560,4562,4564,4566,4568,4570,4572,4574,4576,4578,4580,4582,4584,4586,4588,4590,4592,4594,4596,4598,4600,4602,4604,4606,4608,4610,4612,4614,4616,4618,4620],{"categories":149},[150],"Developer Productivity",{"categories":152},[153],"Business & SaaS",{"categories":155},[107],{"categories":157},[158],"AI Automation",{"categories":160},[161],"Product Strategy",{"categories":163},[107],{"categories":165},[150],{"categories":167},[153],{"categories":169},[],{"categories":171},[107],{"categories":173},[],{"categories":175},[176],"AI News & Trends",{"categories":178},[158],{"categories":180},[158],{"categories":182},[176],{"categories":184},[158],{"categories":186},[158],{"categories":188},[158],{"categories":190},[107],{"categories":192},[107],{"categories":194},[107],{"categories":196},[176],{"categories":198},[107],{"categories":200},[107],{"categories":202},[],{"categories":204},[205],"Design & Frontend",{"categories":207},[208],"Data Science & Visualization",{"categories":210},[176],{"categories":212},[107],{"categories":214},[],{"categories":216},[107],{"categories":218},[219],"Software Engineering",{"categories":221},[107],{"categories":223},[158],{"categories":225},[107],{"categories":227},[228],"Marketing & Growth",{"categories":230},[205],{"categories":232},[107],{"categories":234},[158],{"categories":236},[],{"categories":238},[],{"categories":240},[205],{"categories":242},[158],{"categories":244},[150],{"categories":246},[219],{"categories":248},[205],{"categories":250},[107],{"categories":252},[253],"DevOps & Cloud",{"categories":255},[158],{"categories":257},[176],{"categories":259},[107],{"categories":261},[],{"categories":263},[],{"categories":265},[158],{"categories":267},[219],{"categories":269},[],{"categories":271},[153],{"categories":273},[],{"categories":275},[],{"categories":277},[107],{"categories":279},[158],{"categories":281},[107],{"categories":283},[107],{"categories":285},[158],{"categories":287},[107],{"categories":289},[107],{"categories":291},[107],{"categories":293},[],{"categories":295},[219],{"categories":297},[],{"categories":299},[],{"categories":301},[219],{"categories":303},[],{"categories":305},[219],{"categories":307},[107],{"categories":309},[107],{"categories":311},[228],{"categories":313},[205],{"categories":315},[205],{"categories":317},[107],{"categories":319},[219],{"categories":321},[158],{"categories":323},[219],{"categories":325},[107],{"categories":327},[107],{"categories":329},[158],{"categories":331},[158],{"categories":333},[208],{"categories":335},[176],{"categories":337},[158],{"categories":339},[158],{"categories":341},[228],{"categories":343},[158],{"categories":345},[161],{"categories":347},[219],{"categories":349},[],{"categories":351},[158],{"categories":353},[],{"categories":355},[158],{"categories":357},[219],{"categories":359},[253],{"categories":361},[205],{"categories":363},[107],{"categories":365},[],{"categories":367},[219],{"categories":369},[107],{"categories":371},[],{"categories":373},[158],{"categories":375},[],{"categories":377},[107],{"categories":379},[],{"categories":381},[150],{"categories":383},[219],{"categories":385},[153],{"categories":387},[107],{"categories":389},[107],{"categories":391},[176],{"categories":393},[107],{"categories":395},[],{"categories":397},[107],{"categories":399},[],{"categories":401},[219],{"categories":403},[208],{"categories":405},[],{"categories":407},[107],{"categories":409},[205],{"categories":411},[],{"categories":413},[205],{"categories":415},[158],{"categories":417},[],{"categories":419},[107],{"categories":421},[107],{"categories":423},[158],{"categories":425},[176],{"categories":427},[153],{"categories":429},[107],{"categories":431},[],{"categories":433},[219],{"categories":435},[158],{"categories":437},[107],{"categories":439},[161],{"categories":441},[],{"categories":443},[107],{"categories":445},[161],{"categories":447},[158],{"categories":449},[107],{"categories":451},[158],{"categories":453},[],{"categories":455},[208],{"categories":457},[107],{"categories":459},[],{"categories":461},[150],{"categories":463},[107],{"categories":465},[153],{"categories":467},[107],{"categories":469},[158],{"categories":471},[107],{"categories":473},[219],{"categories":475},[107],{"categories":477},[],{"categories":479},[],{"categories":481},[107],{"categories":483},[107],{"categories":485},[],{"categories":487},[205],{"categories":489},[],{"categories":491},[107],{"categories":493},[],{"categories":495},[158],{"categories":497},[107],{"categories":499},[205],{"categories":501},[],{"categories":503},[107],{"categories":505},[107],{"categories":507},[153],{"categories":509},[158],{"categories":511},[107],{"categories":513},[107],{"categories":515},[205],{"categories":517},[158],{"categories":519},[],{"categories":521},[158],{"categories":523},[],{"categories":525},[176],{"categories":527},[],{"categories":529},[107],{"categories":531},[153,228],{"categories":533},[],{"categories":535},[107],{"categories":537},[158],{"categories":539},[],{"categories":541},[],{"categories":543},[205],{"categories":545},[107],{"categories":547},[],{"categories":549},[107],{"categories":551},[253],{"categories":553},[],{"categories":555},[176],{"categories":557},[205],{"categories":559},[],{"categories":561},[176],{"categories":563},[107],{"categories":565},[158],{"categories":567},[176],{"categories":569},[107],{"categories":571},[228],{"categories":573},[],{"categories":575},[153],{"categories":577},[219],{"categories":579},[107],{"categories":581},[158],{"categories":583},[],{"categories":585},[107,253],{"categories":587},[107],{"categories":589},[107],{"categories":591},[107],{"categories":593},[158],{"categories":595},[107,219],{"categories":597},[208],{"categories":599},[107],{"categories":601},[219],{"categories":603},[228],{"categories":605},[158],{"categories":607},[107],{"categories":609},[158],{"categories":611},[],{"categories":613},[158],{"categories":615},[107],{"categories":617},[107,153],{"categories":619},[153],{"categories":621},[],{"categories":623},[205],{"categories":625},[205],{"categories":627},[],{"categories":629},[],{"categories":631},[176],{"categories":633},[],{"categories":635},[150],{"categories":637},[107],{"categories":639},[219],{"categories":641},[107],{"categories":643},[205],{"categories":645},[158],{"categories":647},[219],{"categories":649},[176],{"categories":651},[205],{"categories":653},[],{"categories":655},[107],{"categories":657},[107],{"categories":659},[107],{"categories":661},[107],{"categories":663},[107],{"categories":665},[107],{"categories":667},[176],{"categories":669},[150],{"categories":671},[107],{"categories":673},[158],{"categories":675},[253],{"categories":677},[205],{"categories":679},[107],{"categories":681},[158],{"categories":683},[],{"categories":685},[],{"categories":687},[205],{"categories":689},[176],{"categories":691},[208],{"categories":693},[],{"categories":695},[107],{"categories":697},[107],{"categories":699},[153],{"categories":701},[107],{"categories":703},[107],{"categories":705},[107],{"categories":707},[176],{"categories":709},[205],{"categories":711},[],{"categories":713},[158],{"categories":715},[219],{"categories":717},[],{"categories":719},[107],{"categories":721},[107],{"categories":723},[158],{"categories":725},[219],{"categories":727},[107],{"categories":729},[],{"categories":731},[],{"categories":733},[107],{"categories":735},[],{"categories":737},[161],{"categories":739},[153],{"categories":741},[158],{"categories":743},[158],{"categories":745},[],{"categories":747},[150],{"categories":749},[107],{"categories":751},[153],{"categories":753},[176],{"categories":755},[150],{"categories":757},[],{"categories":759},[107],{"categories":761},[],{"categories":763},[],{"categories":765},[176],{"categories":767},[176],{"categories":769},[],{"categories":771},[205],{"categories":773},[219],{"categories":775},[],{"categories":777},[153],{"categories":779},[],{"categories":781},[],{"categories":783},[150],{"categories":785},[],{"categories":787},[228],{"categories":789},[158],{"categories":791},[153],{"categories":793},[158],{"categories":795},[219],{"categories":797},[],{"categories":799},[161],{"categories":801},[205],{"categories":803},[219],{"categories":805},[107],{"categories":807},[158],{"categories":809},[153],{"categories":811},[107],{"categories":813},[],{"categories":815},[],{"categories":817},[219],{"categories":819},[208],{"categories":821},[161],{"categories":823},[158],{"categories":825},[107],{"categories":827},[],{"categories":829},[253],{"categories":831},[],{"categories":833},[158],{"categories":835},[],{"categories":837},[150],{"categories":839},[],{"categories":841},[107],{"categories":843},[107],{"categories":845},[205],{"categories":847},[228],{"categories":849},[219],{"categories":851},[158],{"categories":853},[],{"categories":855},[219],{"categories":857},[150],{"categories":859},[],{"categories":861},[176],{"categories":863},[107,253],{"categories":865},[107],{"categories":867},[176],{"categories":869},[107],{"categories":871},[107],{"categories":873},[153],{"categories":875},[107],{"categories":877},[],{"categories":879},[107],{"categories":881},[153],{"categories":883},[107],{"categories":885},[],{"categories":887},[158],{"categories":889},[219],{"categories":891},[205],{"categories":893},[176],{"categories":895},[208],{"categories":897},[107],{"categories":899},[150],{"categories":901},[107],{"categories":903},[158],{"categories":905},[219],{"categories":907},[],{"categories":909},[],{"categories":911},[158],{"categories":913},[161],{"categories":915},[],{"categories":917},[107],{"categories":919},[],{"categories":921},[205],{"categories":923},[158],{"categories":925},[219],{"categories":927},[205],{"categories":929},[107],{"categories":931},[205],{"categories":933},[],{"categories":935},[],{"categories":937},[176],{"categories":939},[158],{"categories":941},[158],{"categories":943},[107],{"categories":945},[107],{"categories":947},[107],{"categories":949},[153],{"categories":951},[107],{"categories":953},[],{"categories":955},[219],{"categories":957},[219],{"categories":959},[153],{"categories":961},[],{"categories":963},[107],{"categories":965},[107],{"categories":967},[158],{"categories":969},[150],{"categories":971},[153],{"categories":973},[176],{"categories":975},[158],{"categories":977},[228],{"categories":979},[107],{"categories":981},[158],{"categories":983},[],{"categories":985},[205],{"categories":987},[],{"categories":989},[107],{"categories":991},[107],{"categories":993},[],{"categories":995},[153],{"categories":997},[158],{"categories":999},[],{"categories":1001},[107],{"categories":1003},[253],{"categories":1005},[208],{"categories":1007},[219],{"categories":1009},[228],{"categories":1011},[107],{"categories":1013},[205],{"categories":1015},[107],{"categories":1017},[219],{"categories":1019},[158],{"categories":1021},[],{"categories":1023},[],{"categories":1025},[158],{"categories":1027},[150],{"categories":1029},[158],{"categories":1031},[161],{"categories":1033},[153],{"categories":1035},[],{"categories":1037},[107],{"categories":1039},[161],{"categories":1041},[107],{"categories":1043},[107],{"categories":1045},[107],{"categories":1047},[107],{"categories":1049},[228],{"categories":1051},[107],{"categories":1053},[107],{"categories":1055},[107],{"categories":1057},[205],{"categories":1059},[158],{"categories":1061},[],{"categories":1063},[],{"categories":1065},[253],{"categories":1067},[219],{"categories":1069},[],{"categories":1071},[158],{"categories":1073},[107],{"categories":1075},[205,107],{"categories":1077},[150],{"categories":1079},[],{"categories":1081},[107],{"categories":1083},[150],{"categories":1085},[205],{"categories":1087},[158],{"categories":1089},[219],{"categories":1091},[],{"categories":1093},[107],{"categories":1095},[],{"categories":1097},[],{"categories":1099},[107],{"categories":1101},[150],{"categories":1103},[107],{"categories":1105},[],{"categories":1107},[158],{"categories":1109},[161],{"categories":1111},[107],{"categories":1113},[107],{"categories":1115},[107],{"categories":1117},[205],{"categories":1119},[158],{"categories":1121},[253],{"categories":1123},[205],{"categories":1125},[153],{"categories":1127},[158],{"categories":1129},[107],{"categories":1131},[107],{"categories":1133},[107],{"categories":1135},[158],{"categories":1137},[219],{"categories":1139},[107],{"categories":1141},[161],{"categories":1143},[],{"categories":1145},[176],{"categories":1147},[],{"categories":1149},[161],{"categories":1151},[158],{"categories":1153},[205],{"categories":1155},[107],{"categories":1157},[107],{"categories":1159},[158],{"categories":1161},[219],{"categories":1163},[205],{"categories":1165},[158],{"categories":1167},[176],{"categories":1169},[],{"categories":1171},[107],{"categories":1173},[],{"categories":1175},[107],{"categories":1177},[205],{"categories":1179},[107],{"categories":1181},[150],{"categories":1183},[176],{"categories":1185},[107],{"categories":1187},[228],{"categories":1189},[107],{"categories":1191},[107],{"categories":1193},[158],{"categories":1195},[158],{"categories":1197},[107],{"categories":1199},[158],{"categories":1201},[158],{"categories":1203},[107],{"categories":1205},[107],{"categories":1207},[158],{"categories":1209},[205],{"categories":1211},[107],{"categories":1213},[107],{"categories":1215},[],{"categories":1217},[],{"categories":1219},[219],{"categories":1221},[],{"categories":1223},[150],{"categories":1225},[253],{"categories":1227},[107],{"categories":1229},[],{"categories":1231},[150],{"categories":1233},[153],{"categories":1235},[107],{"categories":1237},[228],{"categories":1239},[],{"categories":1241},[153],{"categories":1243},[],{"categories":1245},[107],{"categories":1247},[219],{"categories":1249},[],{"categories":1251},[],{"categories":1253},[],{"categories":1255},[],{"categories":1257},[107],{"categories":1259},[158],{"categories":1261},[253],{"categories":1263},[150],{"categories":1265},[219],{"categories":1267},[107],{"categories":1269},[107],{"categories":1271},[219],{"categories":1273},[161],{"categories":1275},[107],{"categories":1277},[228],{"categories":1279},[153],{"categories":1281},[107],{"categories":1283},[107],{"categories":1285},[107],{"categories":1287},[107,150],{"categories":1289},[219],{"categories":1291},[219],{"categories":1293},[205],{"categories":1295},[158],{"categories":1297},[107],{"categories":1299},[107],{"categories":1301},[],{"categories":1303},[],{"categories":1305},[107],{"categories":1307},[],{"categories":1309},[219],{"categories":1311},[208],{"categories":1313},[176],{"categories":1315},[205],{"categories":1317},[107],{"categories":1319},[219],{"categories":1321},[],{"categories":1323},[107],{"categories":1325},[107],{"categories":1327},[],{"categories":1329},[158],{"categories":1331},[107],{"categories":1333},[107],{"categories":1335},[],{"categories":1337},[158],{"categories":1339},[107],{"categories":1341},[153],{"categories":1343},[],{"categories":1345},[150],{"categories":1347},[107],{"categories":1349},[150],{"categories":1351},[107],{"categories":1353},[219],{"categories":1355},[228],{"categories":1357},[158],{"categories":1359},[158],{"categories":1361},[107,205],{"categories":1363},[176],{"categories":1365},[107],{"categories":1367},[205],{"categories":1369},[],{"categories":1371},[219],{"categories":1373},[253],{"categories":1375},[205],{"categories":1377},[219],{"categories":1379},[107],{"categories":1381},[107],{"categories":1383},[158],{"categories":1385},[],{"categories":1387},[],{"categories":1389},[],{"categories":1391},[],{"categories":1393},[219],{"categories":1395},[158],{"categories":1397},[158],{"categories":1399},[253],{"categories":1401},[107],{"categories":1403},[107],{"categories":1405},[158],{"categories":1407},[107],{"categories":1409},[107],{"categories":1411},[],{"categories":1413},[205],{"categories":1415},[219],{"categories":1417},[],{"categories":1419},[],{"categories":1421},[158],{"categories":1423},[],{"categories":1425},[],{"categories":1427},[228],{"categories":1429},[228],{"categories":1431},[158],{"categories":1433},[219],{"categories":1435},[],{"categories":1437},[107],{"categories":1439},[107],{"categories":1441},[219],{"categories":1443},[205],{"categories":1445},[205],{"categories":1447},[158],{"categories":1449},[150],{"categories":1451},[107],{"categories":1453},[107],{"categories":1455},[205],{"categories":1457},[205],{"categories":1459},[158],{"categories":1461},[158],{"categories":1463},[107],{"categories":1465},[],{"categories":1467},[107],{"categories":1469},[],{"categories":1471},[107],{"categories":1473},[158],{"categories":1475},[176],{"categories":1477},[219],{"categories":1479},[107],{"categories":1481},[150],{"categories":1483},[107],{"categories":1485},[],{"categories":1487},[158],{"categories":1489},[158],{"categories":1491},[],{"categories":1493},[107],{"categories":1495},[150],{"categories":1497},[107],{"categories":1499},[150],{"categories":1501},[150],{"categories":1503},[],{"categories":1505},[219],{"categories":1507},[],{"categories":1509},[158],{"categories":1511},[176],{"categories":1513},[107],{"categories":1515},[158],{"categories":1517},[107],{"categories":1519},[158],{"categories":1521},[107],{"categories":1523},[176],{"categories":1525},[208],{"categories":1527},[107],{"categories":1529},[161],{"categories":1531},[176],{"categories":1533},[205],{"categories":1535},[],{"categories":1537},[],{"categories":1539},[176],{"categories":1541},[],{"categories":1543},[],{"categories":1545},[],{"categories":1547},[],{"categories":1549},[219],{"categories":1551},[219],{"categories":1553},[208],{"categories":1555},[],{"categories":1557},[107],{"categories":1559},[107],{"categories":1561},[208],{"categories":1563},[219],{"categories":1565},[],{"categories":1567},[],{"categories":1569},[158],{"categories":1571},[158],{"categories":1573},[219],{"categories":1575},[158],{"categories":1577},[176],{"categories":1579},[176],{"categories":1581},[158],{"categories":1583},[158],{"categories":1585},[150],{"categories":1587},[107,253],{"categories":1589},[],{"categories":1591},[205],{"categories":1593},[219],{"categories":1595},[150],{"categories":1597},[158],{"categories":1599},[205],{"categories":1601},[],{"categories":1603},[158],{"categories":1605},[158],{"categories":1607},[107],{"categories":1609},[228],{"categories":1611},[219],{"categories":1613},[205],{"categories":1615},[107],{"categories":1617},[],{"categories":1619},[158],{"categories":1621},[205],{"categories":1623},[107],{"categories":1625},[158],{"categories":1627},[158],{"categories":1629},[158],{"categories":1631},[228],{"categories":1633},[208],{"categories":1635},[107],{"categories":1637},[158],{"categories":1639},[107],{"categories":1641},[],{"categories":1643},[228],{"categories":1645},[176],{"categories":1647},[219],{"categories":1649},[107],{"categories":1651},[158],{"categories":1653},[],{"categories":1655},[],{"categories":1657},[107],{"categories":1659},[158],{"categories":1661},[107],{"categories":1663},[176],{"categories":1665},[107],{"categories":1667},[158],{"categories":1669},[158],{"categories":1671},[],{"categories":1673},[107],{"categories":1675},[],{"categories":1677},[],{"categories":1679},[107],{"categories":1681},[158],{"categories":1683},[],{"categories":1685},[],{"categories":1687},[208],{"categories":1689},[107],{"categories":1691},[208],{"categories":1693},[176],{"categories":1695},[107],{"categories":1697},[107],{"categories":1699},[158],{"categories":1701},[107],{"categories":1703},[158],{"categories":1705},[],{"categories":1707},[],{"categories":1709},[253],{"categories":1711},[107],{"categories":1713},[],{"categories":1715},[],{"categories":1717},[150],{"categories":1719},[],{"categories":1721},[],{"categories":1723},[107],{"categories":1725},[],{"categories":1727},[],{"categories":1729},[219],{"categories":1731},[176],{"categories":1733},[228],{"categories":1735},[153],{"categories":1737},[107],{"categories":1739},[107],{"categories":1741},[153],{"categories":1743},[],{"categories":1745},[205],{"categories":1747},[158],{"categories":1749},[153],{"categories":1751},[107],{"categories":1753},[107],{"categories":1755},[150],{"categories":1757},[107],{"categories":1759},[],{"categories":1761},[150],{"categories":1763},[107],{"categories":1765},[228],{"categories":1767},[158],{"categories":1769},[176],{"categories":1771},[107],{"categories":1773},[153],{"categories":1775},[107],{"categories":1777},[107],{"categories":1779},[158],{"categories":1781},[],{"categories":1783},[107],{"categories":1785},[150],{"categories":1787},[107],{"categories":1789},[107],{"categories":1791},[],{"categories":1793},[176],{"categories":1795},[107],{"categories":1797},[107],{"categories":1799},[],{"categories":1801},[153],{"categories":1803},[153],{"categories":1805},[161],{"categories":1807},[107],{"categories":1809},[107],{"categories":1811},[],{"categories":1813},[219],{"categories":1815},[],{"categories":1817},[],{"categories":1819},[107],{"categories":1821},[176],{"categories":1823},[],{"categories":1825},[253],{"categories":1827},[107],{"categories":1829},[107],{"categories":1831},[],{"categories":1833},[107],{"categories":1835},[219],{"categories":1837},[107],{"categories":1839},[107],{"categories":1841},[107,253],{"categories":1843},[107],{"categories":1845},[107],{"categories":1847},[205],{"categories":1849},[158],{"categories":1851},[],{"categories":1853},[158],{"categories":1855},[158],{"categories":1857},[107],{"categories":1859},[107],{"categories":1861},[107],{"categories":1863},[107],{"categories":1865},[150],{"categories":1867},[208],{"categories":1869},[150],{"categories":1871},[219],{"categories":1873},[205],{"categories":1875},[158],{"categories":1877},[],{"categories":1879},[107],{"categories":1881},[176],{"categories":1883},[107],{"categories":1885},[158],{"categories":1887},[107],{"categories":1889},[107],{"categories":1891},[153],{"categories":1893},[],{"categories":1895},[253],{"categories":1897},[205],{"categories":1899},[205],{"categories":1901},[219],{"categories":1903},[158],{"categories":1905},[107],{"categories":1907},[153],{"categories":1909},[176],{"categories":1911},[205],{"categories":1913},[158],{"categories":1915},[107],{"categories":1917},[],{"categories":1919},[107],{"categories":1921},[107],{"categories":1923},[],{"categories":1925},[],{"categories":1927},[107],{"categories":1929},[107],{"categories":1931},[107],{"categories":1933},[107],{"categories":1935},[158],{"categories":1937},[107],{"categories":1939},[107],{"categories":1941},[],{"categories":1943},[208],{"categories":1945},[107],{"categories":1947},[158],{"categories":1949},[],{"categories":1951},[],{"categories":1953},[107],{"categories":1955},[107],{"categories":1957},[107],{"categories":1959},[176],{"categories":1961},[],{"categories":1963},[205],{"categories":1965},[107],{"categories":1967},[253],{"categories":1969},[176],{"categories":1971},[219],{"categories":1973},[219],{"categories":1975},[176],{"categories":1977},[176],{"categories":1979},[253],{"categories":1981},[],{"categories":1983},[176],{"categories":1985},[107],{"categories":1987},[150],{"categories":1989},[107],{"categories":1991},[176],{"categories":1993},[],{"categories":1995},[107],{"categories":1997},[219],{"categories":1999},[208],{"categories":2001},[107],{"categories":2003},[176],{"categories":2005},[219],{"categories":2007},[158],{"categories":2009},[176],{"categories":2011},[253],{"categories":2013},[158],{"categories":2015},[107],{"categories":2017},[107],{"categories":2019},[107],{"categories":2021},[],{"categories":2023},[153],{"categories":2025},[],{"categories":2027},[],{"categories":2029},[107],{"categories":2031},[107],{"categories":2033},[107],{"categories":2035},[107],{"categories":2037},[],{"categories":2039},[208],{"categories":2041},[150],{"categories":2043},[158],{"categories":2045},[205],{"categories":2047},[],{"categories":2049},[107],{"categories":2051},[219],{"categories":2053},[107],{"categories":2055},[253],{"categories":2057},[253],{"categories":2059},[],{"categories":2061},[158],{"categories":2063},[176],{"categories":2065},[176],{"categories":2067},[107],{"categories":2069},[158],{"categories":2071},[],{"categories":2073},[205],{"categories":2075},[107],{"categories":2077},[107],{"categories":2079},[],{"categories":2081},[107],{"categories":2083},[],{"categories":2085},[107],{"categories":2087},[219],{"categories":2089},[253],{"categories":2091},[107],{"categories":2093},[219],{"categories":2095},[153],{"categories":2097},[107],{"categories":2099},[],{"categories":2101},[158],{"categories":2103},[150],{"categories":2105},[150],{"categories":2107},[],{"categories":2109},[107],{"categories":2111},[107],{"categories":2113},[219],{"categories":2115},[205],{"categories":2117},[107],{"categories":2119},[158],{"categories":2121},[],{"categories":2123},[107],{"categories":2125},[107],{"categories":2127},[158],{"categories":2129},[],{"categories":2131},[158],{"categories":2133},[219],{"categories":2135},[],{"categories":2137},[107],{"categories":2139},[158],{"categories":2141},[153],{"categories":2143},[],{"categories":2145},[107],{"categories":2147},[],{"categories":2149},[107],{"categories":2151},[107],{"categories":2153},[],{"categories":2155},[107],{"categories":2157},[107],{"categories":2159},[176],{"categories":2161},[107],{"categories":2163},[107],{"categories":2165},[150],{"categories":2167},[107],{"categories":2169},[208],{"categories":2171},[176],{"categories":2173},[158],{"categories":2175},[],{"categories":2177},[107],{"categories":2179},[205],{"categories":2181},[228],{"categories":2183},[107],{"categories":2185},[158],{"categories":2187},[],{"categories":2189},[],{"categories":2191},[],{"categories":2193},[150],{"categories":2195},[176],{"categories":2197},[158],{"categories":2199},[107],{"categories":2201},[107],{"categories":2203},[205],{"categories":2205},[158],{"categories":2207},[],{"categories":2209},[158],{"categories":2211},[158],{"categories":2213},[],{"categories":2215},[107],{"categories":2217},[158],{"categories":2219},[107],{"categories":2221},[],{"categories":2223},[107],{"categories":2225},[107],{"categories":2227},[176],{"categories":2229},[205],{"categories":2231},[158],{"categories":2233},[205],{"categories":2235},[153],{"categories":2237},[],{"categories":2239},[],{"categories":2241},[107],{"categories":2243},[150],{"categories":2245},[176],{"categories":2247},[],{"categories":2249},[205],{"categories":2251},[],{"categories":2253},[219],{"categories":2255},[219],{"categories":2257},[205],{"categories":2259},[219],{"categories":2261},[],{"categories":2263},[107],{"categories":2265},[107],{"categories":2267},[],{"categories":2269},[228],{"categories":2271},[107],{"categories":2273},[253],{"categories":2275},[219],{"categories":2277},[],{"categories":2279},[158],{"categories":2281},[107],{"categories":2283},[150],{"categories":2285},[158],{"categories":2287},[158],{"categories":2289},[107],{"categories":2291},[107],{"categories":2293},[],{"categories":2295},[150],{"categories":2297},[107],{"categories":2299},[153],{"categories":2301},[219],{"categories":2303},[205],{"categories":2305},[],{"categories":2307},[],{"categories":2309},[],{"categories":2311},[158],{"categories":2313},[219],{"categories":2315},[205],{"categories":2317},[176],{"categories":2319},[107],{"categories":2321},[176],{"categories":2323},[158],{"categories":2325},[205],{"categories":2327},[107],{"categories":2329},[],{"categories":2331},[107],{"categories":2333},[205],{"categories":2335},[176],{"categories":2337},[153],{"categories":2339},[219],{"categories":2341},[107],{"categories":2343},[176],{"categories":2345},[228],{"categories":2347},[],{"categories":2349},[],{"categories":2351},[208],{"categories":2353},[107,219],{"categories":2355},[176],{"categories":2357},[107],{"categories":2359},[107],{"categories":2361},[158],{"categories":2363},[107],{"categories":2365},[158],{"categories":2367},[107],{"categories":2369},[107],{"categories":2371},[],{"categories":2373},[219],{"categories":2375},[107],{"categories":2377},[208],{"categories":2379},[158],{"categories":2381},[228],{"categories":2383},[253],{"categories":2385},[],{"categories":2387},[150],{"categories":2389},[158],{"categories":2391},[158],{"categories":2393},[161],{"categories":2395},[219],{"categories":2397},[107],{"categories":2399},[107],{"categories":2401},[],{"categories":2403},[],{"categories":2405},[],{"categories":2407},[253],{"categories":2409},[107],{"categories":2411},[176],{"categories":2413},[107],{"categories":2415},[107],{"categories":2417},[107],{"categories":2419},[],{"categories":2421},[208],{"categories":2423},[153],{"categories":2425},[158],{"categories":2427},[],{"categories":2429},[107],{"categories":2431},[158],{"categories":2433},[107],{"categories":2435},[253],{"categories":2437},[],{"categories":2439},[205],{"categories":2441},[205],{"categories":2443},[],{"categories":2445},[219],{"categories":2447},[107],{"categories":2449},[205],{"categories":2451},[107],{"categories":2453},[153],{"categories":2455},[],{"categories":2457},[176],{"categories":2459},[107],{"categories":2461},[107],{"categories":2463},[205],{"categories":2465},[158],{"categories":2467},[176],{"categories":2469},[],{"categories":2471},[158],{"categories":2473},[158],{"categories":2475},[205],{"categories":2477},[107],{"categories":2479},[],{"categories":2481},[107],{"categories":2483},[107],{"categories":2485},[253],{"categories":2487},[176],{"categories":2489},[208],{"categories":2491},[208],{"categories":2493},[],{"categories":2495},[],{"categories":2497},[],{"categories":2499},[158],{"categories":2501},[158],{"categories":2503},[219],{"categories":2505},[219],{"categories":2507},[107],{"categories":2509},[107],{"categories":2511},[],{"categories":2513},[],{"categories":2515},[107],{"categories":2517},[],{"categories":2519},[107],{"categories":2521},[158],{"categories":2523},[107],{"categories":2525},[],{"categories":2527},[161],{"categories":2529},[107],{"categories":2531},[205],{"categories":2533},[107],{"categories":2535},[153],{"categories":2537},[107],{"categories":2539},[228],{"categories":2541},[158],{"categories":2543},[107],{"categories":2545},[107],{"categories":2547},[107],{"categories":2549},[219],{"categories":2551},[],{"categories":2553},[176],{"categories":2555},[158],{"categories":2557},[],{"categories":2559},[176],{"categories":2561},[158],{"categories":2563},[158],{"categories":2565},[107],{"categories":2567},[158],{"categories":2569},[],{"categories":2571},[153],{"categories":2573},[158],{"categories":2575},[],{"categories":2577},[219],{"categories":2579},[107],{"categories":2581},[150],{"categories":2583},[176],{"categories":2585},[253],{"categories":2587},[158],{"categories":2589},[107],{"categories":2591},[158],{"categories":2593},[150],{"categories":2595},[],{"categories":2597},[107],{"categories":2599},[],{"categories":2601},[],{"categories":2603},[205],{"categories":2605},[107,153],{"categories":2607},[158],{"categories":2609},[107],{"categories":2611},[],{"categories":2613},[150],{"categories":2615},[208],{"categories":2617},[107],{"categories":2619},[219],{"categories":2621},[107],{"categories":2623},[158],{"categories":2625},[107],{"categories":2627},[107],{"categories":2629},[107],{"categories":2631},[176],{"categories":2633},[158],{"categories":2635},[107],{"categories":2637},[],{"categories":2639},[],{"categories":2641},[158],{"categories":2643},[107],{"categories":2645},[253],{"categories":2647},[],{"categories":2649},[107],{"categories":2651},[158],{"categories":2653},[158],{"categories":2655},[],{"categories":2657},[158],{"categories":2659},[107],{"categories":2661},[228],{"categories":2663},[208],{"categories":2665},[158],{"categories":2667},[107],{"categories":2669},[253],{"categories":2671},[],{"categories":2673},[107],{"categories":2675},[228],{"categories":2677},[205],{"categories":2679},[107],{"categories":2681},[107],{"categories":2683},[],{"categories":2685},[228],{"categories":2687},[176],{"categories":2689},[107],{"categories":2691},[107],{"categories":2693},[150],{"categories":2695},[107],{"categories":2697},[],{"categories":2699},[],{"categories":2701},[205],{"categories":2703},[107],{"categories":2705},[208],{"categories":2707},[228],{"categories":2709},[158],{"categories":2711},[228],{"categories":2713},[176],{"categories":2715},[],{"categories":2717},[107],{"categories":2719},[],{"categories":2721},[107],{"categories":2723},[158],{"categories":2725},[107],{"categories":2727},[107],{"categories":2729},[],{"categories":2731},[107,219],{"categories":2733},[176],{"categories":2735},[158],{"categories":2737},[219],{"categories":2739},[107],{"categories":2741},[150],{"categories":2743},[],{"categories":2745},[],{"categories":2747},[158],{"categories":2749},[219],{"categories":2751},[150],{"categories":2753},[219],{"categories":2755},[219],{"categories":2757},[107],{"categories":2759},[228],{"categories":2761},[107],{"categories":2763},[219],{"categories":2765},[],{"categories":2767},[205,107],{"categories":2769},[253],{"categories":2771},[150],{"categories":2773},[],{"categories":2775},[107],{"categories":2777},[153],{"categories":2779},[153],{"categories":2781},[107],{"categories":2783},[107],{"categories":2785},[107],{"categories":2787},[219],{"categories":2789},[158],{"categories":2791},[176],{"categories":2793},[228],{"categories":2795},[205],{"categories":2797},[107],{"categories":2799},[107],{"categories":2801},[107],{"categories":2803},[107],{"categories":2805},[150],{"categories":2807},[107],{"categories":2809},[158],{"categories":2811},[158],{"categories":2813},[176],{"categories":2815},[219],{"categories":2817},[],{"categories":2819},[],{"categories":2821},[208],{"categories":2823},[219],{"categories":2825},[107],{"categories":2827},[205],{"categories":2829},[107],{"categories":2831},[208],{"categories":2833},[107],{"categories":2835},[107],{"categories":2837},[107],{"categories":2839},[158],{"categories":2841},[158],{"categories":2843},[107,153],{"categories":2845},[],{"categories":2847},[205],{"categories":2849},[],{"categories":2851},[107],{"categories":2853},[176],{"categories":2855},[150],{"categories":2857},[150],{"categories":2859},[158],{"categories":2861},[158],{"categories":2863},[158],{"categories":2865},[107],{"categories":2867},[107],{"categories":2869},[153],{"categories":2871},[219],{"categories":2873},[228],{"categories":2875},[107],{"categories":2877},[],{"categories":2879},[176],{"categories":2881},[107],{"categories":2883},[107],{"categories":2885},[107],{"categories":2887},[107],{"categories":2889},[107],{"categories":2891},[219],{"categories":2893},[176],{"categories":2895},[219],{"categories":2897},[219],{"categories":2899},[107],{"categories":2901},[107],{"categories":2903},[107],{"categories":2905},[158],{"categories":2907},[176],{"categories":2909},[158],{"categories":2911},[107],{"categories":2913},[205],{"categories":2915},[107],{"categories":2917},[107],{"categories":2919},[253],{"categories":2921},[107],{"categories":2923},[161],{"categories":2925},[158],{"categories":2927},[107],{"categories":2929},[176],{"categories":2931},[158],{"categories":2933},[228],{"categories":2935},[107],{"categories":2937},[153],{"categories":2939},[107],{"categories":2941},[],{"categories":2943},[107],{"categories":2945},[107],{"categories":2947},[],{"categories":2949},[],{"categories":2951},[],{"categories":2953},[153],{"categories":2955},[107],{"categories":2957},[158],{"categories":2959},[176],{"categories":2961},[176],{"categories":2963},[176],{"categories":2965},[176],{"categories":2967},[],{"categories":2969},[150],{"categories":2971},[158],{"categories":2973},[176],{"categories":2975},[107],{"categories":2977},[150],{"categories":2979},[158],{"categories":2981},[107],{"categories":2983},[107,158],{"categories":2985},[158],{"categories":2987},[253],{"categories":2989},[176],{"categories":2991},[158],{"categories":2993},[176],{"categories":2995},[158],{"categories":2997},[107],{"categories":2999},[],{"categories":3001},[176],{"categories":3003},[228],{"categories":3005},[150],{"categories":3007},[107],{"categories":3009},[107],{"categories":3011},[],{"categories":3013},[219],{"categories":3015},[],{"categories":3017},[150],{"categories":3019},[158],{"categories":3021},[176],{"categories":3023},[107],{"categories":3025},[176],{"categories":3027},[150],{"categories":3029},[176],{"categories":3031},[176],{"categories":3033},[],{"categories":3035},[153],{"categories":3037},[158],{"categories":3039},[176],{"categories":3041},[176],{"categories":3043},[176],{"categories":3045},[176],{"categories":3047},[176],{"categories":3049},[176],{"categories":3051},[176],{"categories":3053},[176],{"categories":3055},[176],{"categories":3057},[176],{"categories":3059},[208],{"categories":3061},[150],{"categories":3063},[107],{"categories":3065},[107],{"categories":3067},[158],{"categories":3069},[158],{"categories":3071},[],{"categories":3073},[107,150],{"categories":3075},[],{"categories":3077},[158],{"categories":3079},[176],{"categories":3081},[158],{"categories":3083},[107],{"categories":3085},[107],{"categories":3087},[107],{"categories":3089},[107],{"categories":3091},[107],{"categories":3093},[158],{"categories":3095},[153],{"categories":3097},[158],{"categories":3099},[],{"categories":3101},[205],{"categories":3103},[176],{"categories":3105},[107],{"categories":3107},[],{"categories":3109},[],{"categories":3111},[158],{"categories":3113},[205],{"categories":3115},[107],{"categories":3117},[],{"categories":3119},[107],{"categories":3121},[],{"categories":3123},[228],{"categories":3125},[107],{"categories":3127},[],{"categories":3129},[],{"categories":3131},[176],{"categories":3133},[150],{"categories":3135},[107],{"categories":3137},[153],{"categories":3139},[107],{"categories":3141},[107],{"categories":3143},[153],{"categories":3145},[205],{"categories":3147},[],{"categories":3149},[107],{"categories":3151},[176],{"categories":3153},[],{"categories":3155},[205],{"categories":3157},[107],{"categories":3159},[228],{"categories":3161},[107],{"categories":3163},[253],{"categories":3165},[],{"categories":3167},[228],{"categories":3169},[],{"categories":3171},[107],{"categories":3173},[],{"categories":3175},[158],{"categories":3177},[219],{"categories":3179},[],{"categories":3181},[153],{"categories":3183},[150],{"categories":3185},[158],{"categories":3187},[205],{"categories":3189},[219],{"categories":3191},[],{"categories":3193},[],{"categories":3195},[107],{"categories":3197},[150],{"categories":3199},[107],{"categories":3201},[228],{"categories":3203},[],{"categories":3205},[158],{"categories":3207},[158],{"categories":3209},[176],{"categories":3211},[219],{"categories":3213},[107],{"categories":3215},[158],{"categories":3217},[107],{"categories":3219},[158],{"categories":3221},[107],{"categories":3223},[161],{"categories":3225},[228],{"categories":3227},[176],{"categories":3229},[],{"categories":3231},[228],{"categories":3233},[],{"categories":3235},[219],{"categories":3237},[158],{"categories":3239},[],{"categories":3241},[107],{"categories":3243},[107],{"categories":3245},[158],{"categories":3247},[153],{"categories":3249},[150],{"categories":3251},[107],{"categories":3253},[205],{"categories":3255},[219],{"categories":3257},[219],{"categories":3259},[107],{"categories":3261},[208],{"categories":3263},[158],{"categories":3265},[107],{"categories":3267},[158],{"categories":3269},[153],{"categories":3271},[205],{"categories":3273},[219],{"categories":3275},[158],{"categories":3277},[107],{"categories":3279},[107],{"categories":3281},[158],{"categories":3283},[107],{"categories":3285},[176],{"categories":3287},[],{"categories":3289},[150],{"categories":3291},[107],{"categories":3293},[107],{"categories":3295},[107],{"categories":3297},[158],{"categories":3299},[107],{"categories":3301},[107],{"categories":3303},[],{"categories":3305},[107],{"categories":3307},[205],{"categories":3309},[153],{"categories":3311},[176],{"categories":3313},[158],{"categories":3315},[107],{"categories":3317},[107],{"categories":3319},[205],{"categories":3321},[158],{"categories":3323},[107],{"categories":3325},[228],{"categories":3327},[208],{"categories":3329},[107],{"categories":3331},[176],{"categories":3333},[107],{"categories":3335},[158],{"categories":3337},[253],{"categories":3339},[107],{"categories":3341},[158],{"categories":3343},[208],{"categories":3345},[],{"categories":3347},[158],{"categories":3349},[219],{"categories":3351},[205],{"categories":3353},[107],{"categories":3355},[150],{"categories":3357},[219],{"categories":3359},[153],{"categories":3361},[219],{"categories":3363},[107],{"categories":3365},[],{"categories":3367},[158],{"categories":3369},[158],{"categories":3371},[107],{"categories":3373},[208],{"categories":3375},[],{"categories":3377},[176],{"categories":3379},[],{"categories":3381},[176],{"categories":3383},[107],{"categories":3385},[107],{"categories":3387},[158],{"categories":3389},[158],{"categories":3391},[158],{"categories":3393},[],{"categories":3395},[176],{"categories":3397},[],{"categories":3399},[107],{"categories":3401},[107],{"categories":3403},[],{"categories":3405},[205],{"categories":3407},[219],{"categories":3409},[158],{"categories":3411},[107],{"categories":3413},[228],{"categories":3415},[107],{"categories":3417},[107],{"categories":3419},[150],{"categories":3421},[],{"categories":3423},[107],{"categories":3425},[],{"categories":3427},[150],{"categories":3429},[176],{"categories":3431},[219],{"categories":3433},[107],{"categories":3435},[107],{"categories":3437},[107],{"categories":3439},[219],{"categories":3441},[176],{"categories":3443},[205],{"categories":3445},[107],{"categories":3447},[107],{"categories":3449},[107],{"categories":3451},[176],{"categories":3453},[205],{"categories":3455},[107],{"categories":3457},[176],{"categories":3459},[205],{"categories":3461},[176],{"categories":3463},[158],{"categories":3465},[158],{"categories":3467},[219],{"categories":3469},[176],{"categories":3471},[158],{"categories":3473},[158],{"categories":3475},[107],{"categories":3477},[219],{"categories":3479},[205],{"categories":3481},[107],{"categories":3483},[],{"categories":3485},[158],{"categories":3487},[],{"categories":3489},[],{"categories":3491},[],{"categories":3493},[153],{"categories":3495},[158],{"categories":3497},[107],{"categories":3499},[158],{"categories":3501},[150],{"categories":3503},[158],{"categories":3505},[228],{"categories":3507},[158],{"categories":3509},[],{"categories":3511},[158],{"categories":3513},[],{"categories":3515},[150],{"categories":3517},[158],{"categories":3519},[],{"categories":3521},[158],{"categories":3523},[107],{"categories":3525},[107],{"categories":3527},[176],{"categories":3529},[107],{"categories":3531},[158],{"categories":3533},[107],{"categories":3535},[107],{"categories":3537},[176],{"categories":3539},[158],{"categories":3541},[219],{"categories":3543},[205],{"categories":3545},[150],{"categories":3547},[],{"categories":3549},[158],{"categories":3551},[205],{"categories":3553},[253],{"categories":3555},[176],{"categories":3557},[107],{"categories":3559},[205],{"categories":3561},[107],{"categories":3563},[150],{"categories":3565},[],{"categories":3567},[158],{"categories":3569},[107],{"categories":3571},[107],{"categories":3573},[158],{"categories":3575},[107],{"categories":3577},[205],{"categories":3579},[],{"categories":3581},[158],{"categories":3583},[161],{"categories":3585},[176],{"categories":3587},[158],{"categories":3589},[153],{"categories":3591},[],{"categories":3593},[107],{"categories":3595},[161],{"categories":3597},[107],{"categories":3599},[158],{"categories":3601},[176],{"categories":3603},[150],{"categories":3605},[253],{"categories":3607},[107],{"categories":3609},[107],{"categories":3611},[107],{"categories":3613},[176],{"categories":3615},[153],{"categories":3617},[107],{"categories":3619},[205],{"categories":3621},[176],{"categories":3623},[253],{"categories":3625},[107],{"categories":3627},[],{"categories":3629},[],{"categories":3631},[107],{"categories":3633},[253],{"categories":3635},[208],{"categories":3637},[158],{"categories":3639},[158],{"categories":3641},[176],{"categories":3643},[107],{"categories":3645},[150],{"categories":3647},[107],{"categories":3649},[205],{"categories":3651},[158],{"categories":3653},[158],{"categories":3655},[107],{"categories":3657},[228],{"categories":3659},[107],{"categories":3661},[158],{"categories":3663},[],{"categories":3665},[107],{"categories":3667},[107],{"categories":3669},[107],{"categories":3671},[176],{"categories":3673},[150],{"categories":3675},[],{"categories":3677},[107],{"categories":3679},[107],{"categories":3681},[219],{"categories":3683},[205],{"categories":3685},[107,158],{"categories":3687},[228,153],{"categories":3689},[107],{"categories":3691},[107],{"categories":3693},[],{"categories":3695},[158],{"categories":3697},[],{"categories":3699},[219],{"categories":3701},[107],{"categories":3703},[],{"categories":3705},[107],{"categories":3707},[176],{"categories":3709},[],{"categories":3711},[158],{"categories":3713},[107],{"categories":3715},[],{"categories":3717},[205],{"categories":3719},[107],{"categories":3721},[158],{"categories":3723},[107],{"categories":3725},[150],{"categories":3727},[158],{"categories":3729},[107],{"categories":3731},[],{"categories":3733},[253],{"categories":3735},[228],{"categories":3737},[153],{"categories":3739},[153],{"categories":3741},[107],{"categories":3743},[150],{"categories":3745},[150],{"categories":3747},[107],{"categories":3749},[158],{"categories":3751},[107],{"categories":3753},[107],{"categories":3755},[219],{"categories":3757},[150],{"categories":3759},[107],{"categories":3761},[228],{"categories":3763},[176],{"categories":3765},[107],{"categories":3767},[107],{"categories":3769},[158],{"categories":3771},[107],{"categories":3773},[],{"categories":3775},[219],{"categories":3777},[],{"categories":3779},[219],{"categories":3781},[158],{"categories":3783},[150],{"categories":3785},[],{"categories":3787},[253],{"categories":3789},[107],{"categories":3791},[],{"categories":3793},[176],{"categories":3795},[158],{"categories":3797},[219],{"categories":3799},[107],{"categories":3801},[158],{"categories":3803},[219],{"categories":3805},[158],{"categories":3807},[176],{"categories":3809},[150],{"categories":3811},[176],{"categories":3813},[219],{"categories":3815},[107],{"categories":3817},[205],{"categories":3819},[107],{"categories":3821},[107],{"categories":3823},[107],{"categories":3825},[107],{"categories":3827},[107],{"categories":3829},[158],{"categories":3831},[107],{"categories":3833},[158],{"categories":3835},[107],{"categories":3837},[150],{"categories":3839},[107],{"categories":3841},[158],{"categories":3843},[205],{"categories":3845},[158],{"categories":3847},[150],{"categories":3849},[158],{"categories":3851},[205],{"categories":3853},[],{"categories":3855},[107],{"categories":3857},[208],{"categories":3859},[107],{"categories":3861},[107],{"categories":3863},[219],{"categories":3865},[],{"categories":3867},[158],{"categories":3869},[228],{"categories":3871},[107],{"categories":3873},[176],{"categories":3875},[228],{"categories":3877},[158],{"categories":3879},[153],{"categories":3881},[153],{"categories":3883},[107],{"categories":3885},[107],{"categories":3887},[150],{"categories":3889},[],{"categories":3891},[158],{"categories":3893},[107],{"categories":3895},[219],{"categories":3897},[],{"categories":3899},[150],{"categories":3901},[107],{"categories":3903},[158],{"categories":3905},[158],{"categories":3907},[],{"categories":3909},[219],{"categories":3911},[219],{"categories":3913},[228],{"categories":3915},[205],{"categories":3917},[],{"categories":3919},[107],{"categories":3921},[158],{"categories":3923},[150],{"categories":3925},[107],{"categories":3927},[219],{"categories":3929},[150],{"categories":3931},[176],{"categories":3933},[176],{"categories":3935},[],{"categories":3937},[176],{"categories":3939},[158],{"categories":3941},[205],{"categories":3943},[208],{"categories":3945},[107],{"categories":3947},[],{"categories":3949},[176],{"categories":3951},[219],{"categories":3953},[107],{"categories":3955},[153],{"categories":3957},[107],{"categories":3959},[150],{"categories":3961},[253],{"categories":3963},[150],{"categories":3965},[],{"categories":3967},[],{"categories":3969},[158],{"categories":3971},[176],{"categories":3973},[],{"categories":3975},[158],{"categories":3977},[158],{"categories":3979},[158],{"categories":3981},[],{"categories":3983},[107],{"categories":3985},[],{"categories":3987},[176],{"categories":3989},[150],{"categories":3991},[205],{"categories":3993},[107],{"categories":3995},[176],{"categories":3997},[107],{"categories":3999},[176],{"categories":4001},[],{"categories":4003},[176],{"categories":4005},[150],{"categories":4007},[158],{"categories":4009},[107],{"categories":4011},[],{"categories":4013},[219],{"categories":4015},[158],{"categories":4017},[161],{"categories":4019},[158],{"categories":4021},[150],{"categories":4023},[],{"categories":4025},[],{"categories":4027},[],{"categories":4029},[205],{"categories":4031},[158],{"categories":4033},[107],{"categories":4035},[107],{"categories":4037},[],{"categories":4039},[],{"categories":4041},[],{"categories":4043},[205],{"categories":4045},[],{"categories":4047},[158],{"categories":4049},[107],{"categories":4051},[150],{"categories":4053},[],{"categories":4055},[],{"categories":4057},[205],{"categories":4059},[107],{"categories":4061},[176],{"categories":4063},[],{"categories":4065},[228],{"categories":4067},[176],{"categories":4069},[228],{"categories":4071},[208],{"categories":4073},[107],{"categories":4075},[107],{"categories":4077},[],{"categories":4079},[],{"categories":4081},[158],{"categories":4083},[],{"categories":4085},[107],{"categories":4087},[],{"categories":4089},[158],{"categories":4091},[107],{"categories":4093},[],{"categories":4095},[158],{"categories":4097},[107],{"categories":4099},[176],{"categories":4101},[107],{"categories":4103},[228],{"categories":4105},[107],{"categories":4107},[107],{"categories":4109},[208],{"categories":4111},[158],{"categories":4113},[158],{"categories":4115},[],{"categories":4117},[],{"categories":4119},[107],{"categories":4121},[],{"categories":4123},[176],{"categories":4125},[],{"categories":4127},[],{"categories":4129},[205],{"categories":4131},[150],{"categories":4133},[],{"categories":4135},[153],{"categories":4137},[228],{"categories":4139},[107],{"categories":4141},[219],{"categories":4143},[150],{"categories":4145},[208],{"categories":4147},[153],{"categories":4149},[219],{"categories":4151},[219],{"categories":4153},[],{"categories":4155},[107],{"categories":4157},[],{"categories":4159},[158],{"categories":4161},[150],{"categories":4163},[205],{"categories":4165},[150],{"categories":4167},[158],{"categories":4169},[253],{"categories":4171},[107],{"categories":4173},[107],{"categories":4175},[150],{"categories":4177},[158],{"categories":4179},[],{"categories":4181},[107],{"categories":4183},[219],{"categories":4185},[176],{"categories":4187},[219],{"categories":4189},[107],{"categories":4191},[],{"categories":4193},[205],{"categories":4195},[176],{"categories":4197},[150],{"categories":4199},[107],{"categories":4201},[158],{"categories":4203},[107],{"categories":4205},[153],{"categories":4207},[158],{"categories":4209},[158,253],{"categories":4211},[158],{"categories":4213},[219],{"categories":4215},[107],{"categories":4217},[107],{"categories":4219},[208],{"categories":4221},[158],{"categories":4223},[228],{"categories":4225},[158],{"categories":4227},[],{"categories":4229},[158],{"categories":4231},[107],{"categories":4233},[153],{"categories":4235},[],{"categories":4237},[],{"categories":4239},[107],{"categories":4241},[208],{"categories":4243},[228],{"categories":4245},[107],{"categories":4247},[158],{"categories":4249},[],{"categories":4251},[176],{"categories":4253},[],{"categories":4255},[176],{"categories":4257},[219],{"categories":4259},[150],{"categories":4261},[219],{"categories":4263},[107],{"categories":4265},[158],{"categories":4267},[107],{"categories":4269},[107],{"categories":4271},[228],{"categories":4273},[219],{"categories":4275},[],{"categories":4277},[176],{"categories":4279},[107],{"categories":4281},[],{"categories":4283},[107],{"categories":4285},[107],{"categories":4287},[107],{"categories":4289},[158],{"categories":4291},[107],{"categories":4293},[161],{"categories":4295},[158],{"categories":4297},[107],{"categories":4299},[107],{"categories":4301},[107],{"categories":4303},[107],{"categories":4305},[153],{"categories":4307},[],{"categories":4309},[161],{"categories":4311},[176],{"categories":4313},[158],{"categories":4315},[107],{"categories":4317},[219],{"categories":4319},[],{"categories":4321},[219],{"categories":4323},[219],{"categories":4325},[219],{"categories":4327},[107],{"categories":4329},[107],{"categories":4331},[107],{"categories":4333},[158],{"categories":4335},[176],{"categories":4337},[107],{"categories":4339},[107],{"categories":4341},[107],{"categories":4343},[153],{"categories":4345},[107],{"categories":4347},[158],{"categories":4349},[205],{"categories":4351},[],{"categories":4353},[208],{"categories":4355},[158],{"categories":4357},[107],{"categories":4359},[],{"categories":4361},[107],{"categories":4363},[107],{"categories":4365},[176],{"categories":4367},[107],{"categories":4369},[158],{"categories":4371},[228],{"categories":4373},[],{"categories":4375},[],{"categories":4377},[176],{"categories":4379},[176],{"categories":4381},[107],{"categories":4383},[228],{"categories":4385},[107],{"categories":4387},[150],{"categories":4389},[158],{"categories":4391},[107],{"categories":4393},[158],{"categories":4395},[158],{"categories":4397},[107],{"categories":4399},[153],{"categories":4401},[],{"categories":4403},[208],{"categories":4405},[],{"categories":4407},[176],{"categories":4409},[107],{"categories":4411},[208],{"categories":4413},[107],{"categories":4415},[219],{"categories":4417},[219],{"categories":4419},[219],{"categories":4421},[158],{"categories":4423},[158],{"categories":4425},[205],{"categories":4427},[208],{"categories":4429},[208],{"categories":4431},[],{"categories":4433},[176],{"categories":4435},[107],{"categories":4437},[107],{"categories":4439},[219],{"categories":4441},[],{"categories":4443},[176],{"categories":4445},[176],{"categories":4447},[176],{"categories":4449},[],{"categories":4451},[158],{"categories":4453},[107],{"categories":4455},[],{"categories":4457},[150],{"categories":4459},[153],{"categories":4461},[],{"categories":4463},[107],{"categories":4465},[107],{"categories":4467},[],{"categories":4469},[219],{"categories":4471},[],{"categories":4473},[],{"categories":4475},[],{"categories":4477},[],{"categories":4479},[107],{"categories":4481},[176],{"categories":4483},[],{"categories":4485},[],{"categories":4487},[107],{"categories":4489},[107],{"categories":4491},[107],{"categories":4493},[208],{"categories":4495},[107],{"categories":4497},[208],{"categories":4499},[],{"categories":4501},[208],{"categories":4503},[208],{"categories":4505},[253],{"categories":4507},[158],{"categories":4509},[219],{"categories":4511},[],{"categories":4513},[],{"categories":4515},[208],{"categories":4517},[219],{"categories":4519},[219],{"categories":4521},[219],{"categories":4523},[],{"categories":4525},[150],{"categories":4527},[219],{"categories":4529},[219],{"categories":4531},[150],{"categories":4533},[219],{"categories":4535},[153],{"categories":4537},[219],{"categories":4539},[219],{"categories":4541},[219],{"categories":4543},[208],{"categories":4545},[176],{"categories":4547},[176],{"categories":4549},[107],{"categories":4551},[219],{"categories":4553},[208],{"categories":4555},[253],{"categories":4557},[208],{"categories":4559},[208],{"categories":4561},[208],{"categories":4563},[],{"categories":4565},[153],{"categories":4567},[],{"categories":4569},[253],{"categories":4571},[219],{"categories":4573},[219],{"categories":4575},[219],{"categories":4577},[158],{"categories":4579},[176,153],{"categories":4581},[208],{"categories":4583},[],{"categories":4585},[],{"categories":4587},[208],{"categories":4589},[],{"categories":4591},[208],{"categories":4593},[176],{"categories":4595},[158],{"categories":4597},[],{"categories":4599},[219],{"categories":4601},[107],{"categories":4603},[205],{"categories":4605},[],{"categories":4607},[107],{"categories":4609},[],{"categories":4611},[176],{"categories":4613},[150],{"categories":4615},[208],{"categories":4617},[],{"categories":4619},[219],{"categories":4621},[176],[4623,4727,4873,5006],{"id":4624,"title":4625,"ai":4626,"body":4631,"categories":4699,"created_at":108,"date_modified":108,"description":99,"extension":109,"faq":108,"featured":110,"kicker_label":108,"meta":4700,"navigation":126,"path":4713,"published_at":4714,"question":108,"scraped_at":4715,"seo":4716,"sitemap":4717,"source_id":4718,"source_name":4719,"source_type":4720,"source_url":4721,"stem":4722,"tags":4723,"thumbnail_url":108,"tldr":4724,"tweet":108,"unknown_tags":4725,"__hash__":4726},"summaries\u002Fsummaries\u002Fbeb19294561867ca-benchmarking-llm-compression-fp8-gptq-and-smoothqu-summary.md","Benchmarking LLM Compression: FP8, GPTQ, and SmoothQuant",{"provider":7,"model":8,"input_tokens":4627,"output_tokens":4628,"processing_time_ms":4629,"cost_usd":4630},10756,674,2766,0.0037,{"type":14,"value":4632,"toc":4695},[4633,4637,4645,4665,4669,4672,4692],[17,4634,4636],{"id":4635},"quantization-strategies-for-llm-efficiency","Quantization Strategies for LLM Efficiency",[22,4638,4639,4640,4644],{},"Post-training quantization (PTQ) is essential for deploying LLMs in resource-constrained environments. This tutorial demonstrates three distinct approaches using the ",[4641,4642,4643],"code",{},"llmcompressor"," library to reduce model footprint and improve inference speed while maintaining output quality:",[33,4646,4647,4653,4659],{},[36,4648,4649,4652],{},[39,4650,4651],{},"FP8 Dynamic Quantization:"," A data-free approach that compresses linear layers into 8-bit precision while keeping the language modeling head in higher precision. It is the fastest to implement and provides a baseline for efficiency gains.",[36,4654,4655,4658],{},[39,4656,4657],{},"GPTQ W4A16:"," A more aggressive compression method that reduces weights to 4-bit while maintaining 16-bit activation precision. This requires a calibration dataset (in this case, 256 samples from UltraChat) to minimize reconstruction error, resulting in significantly smaller model sizes.",[36,4660,4661,4664],{},[39,4662,4663],{},"SmoothQuant + GPTQ W8A8:"," An advanced pipeline that addresses activation outliers using SmoothQuant (smoothing strength 0.8) before applying 8-bit quantization. This combination balances accuracy recovery with the performance benefits of 8-bit operations.",[17,4666,4668],{"id":4667},"benchmarking-and-deployment-workflow","Benchmarking and Deployment Workflow",[22,4670,4671],{},"To evaluate these methods, the implementation establishes a standardized benchmarking suite that measures:",[33,4673,4674,4680,4686],{},[36,4675,4676,4679],{},[39,4677,4678],{},"Disk Size:"," Total storage footprint in GB.",[36,4681,4682,4685],{},[39,4683,4684],{},"Perplexity (PPL):"," Evaluated on the WikiText-2 dataset to ensure compression hasn't degraded model reasoning.",[36,4687,4688,4691],{},[39,4689,4690],{},"Generation Latency & Throughput:"," Measured in seconds and tokens per second (tok\u002Fs) using a consistent prompt.",[22,4693,4694],{},"The workflow emphasizes a \"save-and-test\" cycle, where each compressed model is saved as a reusable artifact. By comparing the FP16 baseline against these quantized variants, developers can make informed trade-offs between model size and inference performance, creating a repeatable pipeline for production-ready model deployment.",{"title":99,"searchDepth":100,"depth":100,"links":4696},[4697,4698],{"id":4635,"depth":100,"text":4636},{"id":4667,"depth":100,"text":4668},[107],{"content_references":4701,"triage":4710},[4702,4704,4708],{"type":114,"title":4643,"url":4703,"context":120},"https:\u002F\u002Fgithub.com\u002Fvllm-project\u002Fllm-compressor",{"type":4705,"title":4706,"url":4707,"context":117},"dataset","UltraChat 200k","https:\u002F\u002Fhuggingface.co\u002Fdatasets\u002FHuggingFaceH4\u002Fultrachat_200k",{"type":4705,"title":4709,"context":117},"WikiText-2",{"relevance":122,"novelty":123,"quality":123,"actionability":122,"composite":4711,"reasoning":4712},4.55,"Category: AI & LLMs. The article provides a detailed practical guide on compressing LLMs, addressing a core topic of AI engineering with actionable insights on quantization strategies. It includes specific methods and a benchmarking workflow that developers can implement directly in their projects.","\u002Fsummaries\u002Fbeb19294561867ca-benchmarking-llm-compression-fp8-gptq-and-smoothqu-summary","2026-05-17 18:19:09","2026-05-17 18:48:19",{"title":4625,"description":99},{"loc":4713},"beb19294561867ca","MarkTechPost","article","https:\u002F\u002Fwww.marktechpost.com\u002F2026\u002F05\u002F17\u002Fa-coding-implementation-to-compress-and-benchmark-instruction-tuned-llms-with-fp8-gptq-and-smoothquant-quantization-using-llmcompressor\u002F","summaries\u002Fbeb19294561867ca-benchmarking-llm-compression-fp8-gptq-and-smoothqu-summary",[138,141,140,139],"A practical guide to compressing instruction-tuned LLMs using llmcompressor, comparing FP8 dynamic, GPTQ W4A16, and SmoothQuant W8A8 quantization strategies across size, latency, and perplexity.",[],"30L_xT8fTg-Uhmof_yAXXdldKTf6tCZLD87bHm4DkII",{"id":4728,"title":4729,"ai":4730,"body":4736,"categories":4833,"created_at":108,"date_modified":108,"description":99,"extension":109,"faq":108,"featured":110,"kicker_label":108,"meta":4834,"navigation":126,"path":4861,"published_at":108,"question":108,"scraped_at":4862,"seo":4863,"sitemap":4864,"source_id":4865,"source_name":4866,"source_type":4720,"source_url":4867,"stem":4868,"tags":4869,"thumbnail_url":108,"tldr":4870,"tweet":108,"unknown_tags":4871,"__hash__":4872},"summaries\u002Fsummaries\u002Fbb2ba5cfd07cd36e-flashattention-2-4x-faster-exact-attention-on-gpus-summary.md","FlashAttention: 2-4x Faster Exact Attention on GPUs",{"provider":7,"model":4731,"input_tokens":4732,"output_tokens":4733,"processing_time_ms":4734,"cost_usd":4735},"x-ai\u002Fgrok-4.1-fast",9962,2114,53702,0.0025421,{"type":14,"value":4737,"toc":4827},[4738,4742,4745,4748,4752,4763,4774,4778,4797,4816,4820],[17,4739,4741],{"id":4740},"io-aware-kernel-design-cuts-memory-and-boosts-speed","IO-Aware Kernel Design Cuts Memory and Boosts Speed",[22,4743,4744],{},"FlashAttention computes exact attention without storing the full N^2 attention matrix or gradients, using GPU tiling to maximize SRAM usage and minimize HBM reads\u002Fwrites. This yields 2-4x end-to-end speedups in transformer training on A100 GPUs (e.g., 2.4x for GPT-2 style models) and 3-5x memory savings, enabling longer sequences like 64k tokens on single A100 vs. 16k baseline. Backward pass fuses dP computation with dV, avoiding extra softmax. FlashAttention-2 improves parallelism with better work partitioning (50-73% TFLOPS utilization on A100), supports bf16 on Ampere+, head dims to 256, causal masks aligned to bottom-right for decoder use, and sliding window attention (window_size=(left,right)).",[22,4746,4747],{},"Trade-offs: Requires Ampere+ GPUs (A100\u002FRTX30\u002F40\u002FH100); head dim >192 backward needed A100\u002FH100 originally but now works on consumer GPUs without dropout since v2.5.5. Deterministic backward option trades minor speed\u002Fmemory for reproducibility.",[17,4749,4751],{"id":4750},"installation-matches-hardware-for-peak-performance","Installation Matches Hardware for Peak Performance",[22,4753,4754,4755,4758,4759,4762],{},"Install via ",[4641,4756,4757],{},"pip install flash-attn --no-build-isolation"," (3-5 min compile with ninja on 64-core, CUDA 12+). Needs PyTorch 2.2+, packaging\u002Fpsutil\u002Fninja. Limit jobs with ",[4641,4760,4761],{},"MAX_JOBS=4"," on low-RAM machines. ROCm 6.0+ supports MI200+\u002FRDNA3\u002F4 GPUs via composable_kernel (default, fp16\u002Fbf16 fwd\u002Fbwd) or Triton backend (fp16\u002Fbf16\u002Ffp32, causal\u002FMQA\u002FGQA\u002Fpaged\u002FFP8). Use Nvidia\u002FROCm PyTorch containers for deps.",[22,4764,4765,4766,4769,4770,4773],{},"Beta FlashAttention-3 (H100\u002FH800, CUDA 12.3+, FP16\u002FBF16 fwd\u002Fbwd, FP8 fwd) via separate install; FlashAttention-4 (CuTeDSL, H100\u002FB200, ",[4641,4767,4768],{},"pip install flash-attn-4[cu13]",") for Hopper\u002FBlackwell. Huggingface kernels offer drop-in via ",[4641,4771,4772],{},"get_kernel('kernels-community\u002Fflash-attn2')",".",[17,4775,4777],{"id":4776},"usage-replaces-standard-attention-with-kv-cache-support","Usage Replaces Standard Attention with KV Cache Support",[22,4779,4780,4781,4784,4785,4788,4789,4792,4793,4796],{},"Core: ",[4641,4782,4783],{},"out = flash_attn_func(q, k, v, softmax_scale=1\u002Fmath.sqrt(d), causal=True, dropout_p=0.0)"," or ",[4641,4786,4787],{},"flash_attn_qkvpacked_func(qkv)"," for packed inputs (faster bwd). Supports MQA\u002FGQA (nheads_Q % nheads_KV == 0), ALiBi (",[4641,4790,4791],{},"alibi_slopes","), softcapping (Gemma\u002FGrok), paged KV cache (",[4641,4794,4795],{},"block_table","), variable seq lens.",[22,4798,4799,4800,4803,4804,4807,4808,4811,4812,4815],{},"Inference: ",[4641,4801,4802],{},"flash_attn_with_kvcache(q, k_cache, v_cache, k=new_k, v=new_v, rotary_cos\u002Fsin, cache_seqlens)"," updates cache inplace, applies RoPE, causal\u002Flocal masks. Example causal mask for seqlen_q=2, seqlen_k=5: attends to last 2+3 positions bottom-right aligned. Integrate in MHA via ",[4641,4805,4806],{},"flash_attn\u002Fmodules\u002Fmha.py",". Set ",[4641,4809,4810],{},"dropout_p=0.0"," eval; ",[4641,4813,4814],{},"deterministic=True"," bwd for reproducibility.",[17,4817,4819],{"id":4818},"evolutions-unlock-new-workloads","Evolutions Unlock New Workloads",[22,4821,4822,4823,4826],{},"v2.0: 2x faster rewrite, ",[4641,4824,4825],{},"flash_attn_varlen_*"," for ragged batches. v2.1+: Causal realignment, inference opts (split KV load for seqlen_q=1). v2.3+: Sliding window (Mistral 7B). v2.4+: ALiBi, deterministic bwd. v2.5+: PagedAttention. v2.6+: Softcap. v2.7+: torch.compile compat. Widely adopted (usage.md lists integrations).",{"title":99,"searchDepth":100,"depth":100,"links":4828},[4829,4830,4831,4832],{"id":4740,"depth":100,"text":4741},{"id":4750,"depth":100,"text":4751},{"id":4776,"depth":100,"text":4777},{"id":4818,"depth":100,"text":4819},[107],{"content_references":4835,"triage":4859},[4836,4841,4845,4848,4851,4856],{"type":4837,"title":4838,"author":4839,"url":4840,"context":117},"paper","FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness","Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, Christopher Ré","https:\u002F\u002Farxiv.org\u002Fabs\u002F2205.14135",{"type":4837,"title":4842,"author":4843,"url":4844,"context":117},"FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning","Tri Dao","https:\u002F\u002Ftridao.me\u002Fpublications\u002Fflash2\u002Fflash2.pdf",{"type":4837,"title":4846,"author":4843,"url":4847,"context":117},"FlashAttention-3","https:\u002F\u002Ftridao.me\u002Fpublications\u002Fflash3\u002Fflash3.pdf",{"type":4837,"title":4849,"url":4850,"context":117},"PagedAttention","https:\u002F\u002Farxiv.org\u002Fabs\u002F2309.06180",{"type":4852,"title":4853,"url":4854,"context":4855},"other","IEEE Spectrum article on MLPerf 2.0","https:\u002F\u002Fspectrum.ieee.org\u002Fmlperf-rankings-2022","mentioned",{"type":114,"title":4857,"url":4858,"context":120},"huggingface\u002Fkernels","https:\u002F\u002Fgithub.com\u002Fhuggingface\u002Fkernels",{"relevance":122,"novelty":123,"quality":123,"actionability":123,"composite":124,"reasoning":4860},"Category: AI & LLMs. The article provides a detailed explanation of how to implement FlashAttention to improve transformer training efficiency, addressing a specific pain point for AI developers looking to optimize performance. It includes practical installation instructions and usage examples, making it actionable for the target audience.","\u002Fsummaries\u002Fbb2ba5cfd07cd36e-flashattention-2-4x-faster-exact-attention-on-gpus-summary","2026-04-16 03:01:06",{"title":4729,"description":99},{"loc":4861},"bb2ba5cfd07cd36e","__oneoff__","https:\u002F\u002Fgithub.com\u002FDao-AILab\u002Fflash-attention","summaries\u002Fbb2ba5cfd07cd36e-flashattention-2-4x-faster-exact-attention-on-gpus-summary",[138,139,140,141],"Replace PyTorch's scaled_dot_product_attention with FlashAttention kernels to cut transformer training memory by 3x+ and speed up by 2-4x via IO-aware tiling that fuses softmax and skips materializing N^2 attention matrix.",[],"p9Q0kYcZPBLc6PM17T6f5gjDVL-qvZ13UNiKPL_ijhY",{"id":4874,"title":4875,"ai":4876,"body":4881,"categories":4976,"created_at":108,"date_modified":108,"description":99,"extension":109,"faq":108,"featured":110,"kicker_label":108,"meta":4977,"navigation":126,"path":4993,"published_at":4994,"question":108,"scraped_at":4995,"seo":4996,"sitemap":4997,"source_id":4998,"source_name":4999,"source_type":4720,"source_url":5000,"stem":5001,"tags":5002,"thumbnail_url":108,"tldr":5003,"tweet":108,"unknown_tags":5004,"__hash__":5005},"summaries\u002Fsummaries\u002F3082c3466d222001-ground-gemini-3-in-pdb-geometry-for-hallucination-summary.md","Ground Gemini 3 in PDB Geometry for Hallucination-Free Proteomics",{"provider":7,"model":4731,"input_tokens":4877,"output_tokens":4878,"processing_time_ms":4879,"cost_usd":4880},6594,2415,25922,0.00201945,{"type":14,"value":4882,"toc":4971},[4883,4887,4890,4957,4961,4964,4968],[17,4884,4886],{"id":4885},"build-deterministic-protein-analysis-pipeline","Build Deterministic Protein Analysis Pipeline",[22,4888,4889],{},"Parse PDB files like 6M0J (SARS-CoV-2 Spike RBD bound to human ACE2) with Biopython's Bio.PDB to extract Cα backbone coordinates, reducing noise from side chains. Differentiate chains visually: Chain A (ACE2 receptor) in red, Chain E (viral Spike RBD) in blue. Use Plotly's go.Scatter3d to create connected 3D traces of the backbone, exporting as PNG for multimodal input. Configure Gemini 3 Pro API with types.ThinkingConfig(thinking_level='HIGH') and tools like run_simulation for agentic execution. Prompt combines image and text to analyze 'Red vs. Blue' spatial conflict as a molecular gateway, translating coordinates into pathogenic risk and therapeutic targets. This grounds AI in physical geometry, bypassing probabilistic text patterns.",[4891,4892,4893,4909],"table",{},[4894,4895,4896],"thead",{},[4897,4898,4899,4903,4906],"tr",{},[4900,4901,4902],"th",{},"Component",[4900,4904,4905],{},"Responsibility",[4900,4907,4908],{},"Stack",[4910,4911,4912,4924,4935,4946],"tbody",{},[4897,4913,4914,4918,4921],{},[4915,4916,4917],"td",{},"PDB Loader",[4915,4919,4920],{},"Retrieves ground truth data",[4915,4922,4923],{},"Biopython",[4897,4925,4926,4929,4932],{},[4915,4927,4928],{},"Geometric Engine",[4915,4930,4931],{},"Maps to 3D colored chains",[4915,4933,4934],{},"Plotly",[4897,4936,4937,4940,4943],{},[4915,4938,4939],{},"Multimodal Processor",[4915,4941,4942],{},"Interprets conflict",[4915,4944,4945],{},"Gemini 3 Pro (High Thinking)",[4897,4947,4948,4951,4954],{},[4915,4949,4950],{},"Agentic Controller",[4915,4952,4953],{},"Calls simulations",[4915,4955,4956],{},"Gemini SDK",[17,4958,4960],{"id":4959},"extract-actionable-insights-from-binding-interfaces","Extract Actionable Insights from Binding Interfaces",[22,4962,4963],{},"Gemini identifies the red-blue merge as the high-affinity contact zone enabling viral membrane fusion, the key target for neutralizing antibodies and vaccines. It frames ACE2 as cellular 'gateway' and Spike RBD as 'key', emphasizing physical obstruction for immunity. For drug discovery, it highlights PPIs' flat surfaces as traditionally undruggable but spots subtle energetic hotspots via coordinate precision. This accelerates in silico design of small-molecule inhibitors that wedge into the interface, cutting wet-lab costs and carbon footprint before trials. Aligns 6M0J as training data for AlphaFold 3, enabling AI to predict 'druggable pockets' invisible in static models.",[17,4965,4967],{"id":4966},"enforce-geometric-governance-to-kill-hallucinations","Enforce Geometric Governance to Kill Hallucinations",[22,4969,4970],{},"Anchor multimodal LLMs in PDB coordinates for verifiable reasoning: AI measures Cα distances, not linguistic probabilities, creating auditable 'ground truth' trails. Visual Plotly renders allow human experts to verify contact zones. H2E framework demands this accountability, evolving agents from observers to executors via tools. Scales to Sovereign AI with local A100\u002FL4 GPUs and vLLM quantization for data privacy and low-latency in aerospace (e.g., Orion ECLSS) or proteomics. Shifts from black-box hallucinations to physics-based certainty, blueprint for safety-critical domains like molecular diagnostics.",{"title":99,"searchDepth":100,"depth":100,"links":4972},[4973,4974,4975],{"id":4885,"depth":100,"text":4886},{"id":4959,"depth":100,"text":4960},{"id":4966,"depth":100,"text":4967},[],{"content_references":4978,"triage":4991},[4979,4982,4984,4987,4989],{"type":4852,"title":4980,"url":4981,"context":117},"ALPHAFOLD3_GEMINI3.ipynb","https:\u002F\u002Fgithub.com\u002Ffrank-morales2020\u002FMLxDL\u002Fblob\u002Fmain\u002FALPHAFOLD3_GEMINI3.ipynb",{"type":4705,"title":4983,"context":117},"6M0J PDB structure",{"type":4852,"title":4985,"url":4986,"context":117},"The Wall Before the Word: H2E Geometric Governance and the Future of AI Government","https:\u002F\u002Fmedium.com\u002Fai-simplified-in-plain-english\u002Fthe-wall-before-the-word-h2e-geometric-governance-and-the-future-of-ai-government-89ff82c7598a",{"type":114,"title":4988,"context":4855},"AlphaFold 3",{"type":114,"title":4990,"context":117},"Gemini 3 Pro",{"relevance":122,"novelty":123,"quality":123,"actionability":123,"composite":124,"reasoning":4992},"Category: AI & LLMs. The article provides a detailed approach to building a deterministic protein analysis pipeline using AI tools, which directly addresses the audience's need for practical applications in AI-powered product development. It includes specific tools like Biopython and Plotly, and actionable insights for drug discovery, making it highly relevant and actionable.","\u002Fsummaries\u002F3082c3466d222001-ground-gemini-3-in-pdb-geometry-for-hallucination-summary","2026-04-19 20:16:41","2026-04-21 15:26:18",{"title":4875,"description":99},{"loc":4993},"3082c3466d222001","AI Simplified in Plain English","https:\u002F\u002Fmedium.com\u002Fai-simplified-in-plain-english\u002Fthe-convergence-of-geometric-governance-and-multimodal-ai-in-safety-critical-proteomics-with-fa8c6ba20303?source=rss----f37ab7d4e76b---4","summaries\u002F3082c3466d222001-ground-gemini-3-in-pdb-geometry-for-hallucination-summary",[138,141,139,140],"Use Biopython and Plotly to feed 3D protein structures (Red ACE2 vs. Blue Spike RBD in 6M0J PDB) into Gemini 3 Pro's high-thinking mode, enabling deterministic analysis of binding interfaces for drug discovery and safety-critical diagnostics.",[],"o-uBIXwAi0RF4UM-9FRaIfNxARtfo32iiM51omEjd0w",{"id":5007,"title":5008,"ai":5009,"body":5014,"categories":5048,"created_at":108,"date_modified":108,"description":99,"extension":109,"faq":108,"featured":110,"kicker_label":108,"meta":5049,"navigation":126,"path":5058,"published_at":108,"question":108,"scraped_at":5059,"seo":5060,"sitemap":5061,"source_id":5062,"source_name":4866,"source_type":4720,"source_url":5063,"stem":5064,"tags":5065,"thumbnail_url":108,"tldr":5066,"tweet":108,"unknown_tags":5067,"__hash__":5068},"summaries\u002Fsummaries\u002F9c41ec860da9ed62-turboquant-doubles-llm-context-via-3b-2b-kv-quanti-summary.md","TurboQuant Doubles LLM Context via 3b\u002F2b KV Quantization",{"provider":7,"model":4731,"input_tokens":5010,"output_tokens":5011,"processing_time_ms":5012,"cost_usd":5013},6519,1934,14125,0.00224815,{"type":14,"value":5015,"toc":5043},[5016,5020,5023,5026,5030,5033,5036,5040],[17,5017,5019],{"id":5018},"kv-cache-compression-delivers-massive-vram-savings","KV Cache Compression Delivers Massive VRAM Savings",[22,5021,5022],{},"TurboQuant quantizes KV cache entries to 3-bit keys and 2-bit values using Lloyd-Max codebooks optimized for Beta-distributed attention vectors, random orthogonal rotations, and QJL projections for unbiased inner product estimation. On RTX 5090 with Qwen3.5-27B-AWQ (4-bit weights, 16\u002F64 full-attention layers), it frees 30GB KV cache across 4 GPUs at 30k context, doubling max token capacity from 457k to 914k tokens while boosting prefill throughput 5.7% (1,804 to 1,907 tok\u002Fs) and decode 3.1% (1.264 to 1.303 tok\u002Fs), reducing peak activations 7% (644MB to 599MB).",[22,5024,5025],{},"On 8x RTX 3090 with Qwen3.5-35B-A3B MoE (205 experts pruned, TP=8, 10\u002F40 full-attention layers), it saves 30.9% KV cache per GPU (e.g., 755MB to 522MB at 131k context, 234MB freed), extending baseline 1.41M total tokens to 2.04M (1.45x) or supporting 3 extra 131k requests. Baseline decode holds at 98-133 tok\u002Fs up to 131k context; TQ maintains quality without throughput regression. Freed VRAM per GPU scales linearly: 17MB at 8k, 59MB at 32k, 179MB at 100k, 234MB at 131k contexts.",[17,5027,5029],{"id":5028},"quality-preserved-with-theoretical-guarantees","Quality Preserved with Theoretical Guarantees",[22,5031,5032],{},"Cosine similarity stays near-lossless for 3\u002F4-bit keys (1.000) but drops to 0.940 for 2-bit values (dominant bottleneck; 4-bit values hit 0.997). Combined 3b\u002F2b yields 0.940 sim. Needle-in-haystack passes single needle across 512-131k, 5\u002F5 multi-needle at max context, 3\u002F3 multi-fact coherence, golden ratio completion (perplexity 1.05-1.35), and math reasoning. Recall@8=0.55 (3-bit, N=4096, exceeds paper's 0.40 threshold); Spearman rank rho >0.85 (N=2048). Paper theorems validated: MSE bounds hold for unit-norm vectors, 1\u002F4^b distortion scaling (2b=0.70x bound, 3b=0.82x, 4b=0.97x), \u003C0.1% bias, 4.41x compression at head_dim=256.",[22,5034,5035],{},"Adversarial audit confirms 2x context on dense models and ~4.6-5x compression (misleading paper claim ignores Pi\u002FS matrices\u002Fring buffer), but notes low recall@1=38%, hybrid decode dequantizes to float32 (storage win, no compute save), and needle tests are easy (query≠key copies). GPU util near 100% idle-free at scale, power 130-142W.",[17,5037,5039],{"id":5038},"triton-kernels-and-vllm-integration-for-production","Triton Kernels and vLLM Integration for Production",[22,5041,5042],{},"Custom Triton kernels fuse decode attention; vLLM adapter monkey-patches KV hooks for quantization, flat compressed store, and hybrid decode. Architecture modular: codebook.py (Beta quantizers), rotation.py (projections), quantizer.py (TurboQuantMSE\u002FProd algos), kv_cache.py (bit-packing), score.py (compressed scoring). Supports dense\u002FMoE, compresses only full-attention layers. All 35+ tests pass (7 core quantizer, 19 modular, 9 theorem validations). Install via pip from setup.py; benchmark with benchmark.py\u002Fproof.py. Tested on RTX 3090\u002F5090, vLLM 0.18.0, AMD EPYC.",{"title":99,"searchDepth":100,"depth":100,"links":5044},[5045,5046,5047],{"id":5018,"depth":100,"text":5019},{"id":5028,"depth":100,"text":5029},{"id":5038,"depth":100,"text":5039},[],{"content_references":5050,"triage":5054},[5051],{"type":4837,"title":5052,"url":5053,"context":117},"TurboQuant KV cache compression","https:\u002F\u002Farxiv.org\u002Fabs\u002F2504.19874",{"relevance":123,"novelty":5055,"quality":123,"actionability":100,"composite":5056,"reasoning":5057},3,3.4,"Category: AI & LLMs. The article discusses a specific technique for optimizing KV cache in LLMs, which addresses a pain point for developers looking to improve AI model performance. However, while it presents some new insights, the practical application details are limited, making it less actionable for immediate implementation.","\u002Fsummaries\u002F9c41ec860da9ed62-turboquant-doubles-llm-context-via-3b-2b-kv-quanti-summary","2026-04-16 03:08:31",{"title":5008,"description":99},{"loc":5058},"9c41ec860da9ed62","https:\u002F\u002Fgithub.com\u002F0xSero\u002Fturboquant.git","summaries\u002F9c41ec860da9ed62-turboquant-doubles-llm-context-via-3b-2b-kv-quanti-summary",[138,140,141,139],"Compresses KV cache to 3-bit keys\u002F2-bit values with Triton kernels and vLLM integration, freeing 30GB VRAM on RTX 5090 (2x max tokens) and 233MB\u002FGPU on 8x3090 (1.45x context, 30.9% savings), passing needle tests and paper theorems.",[],"qWkisdtNrpadeQ_iDyPoaXmRtQWfTyD6ufLeRrLlGWA"]