[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"summary-5d6a69b9b1714e2b-snowflake-native-fraud-ml-pipeline-train-to-monito-summary":3,"summaries-facets-categories":83,"summary-related-5d6a69b9b1714e2b-snowflake-native-fraud-ml-pipeline-train-to-monito-summary":3652},{"id":4,"title":5,"ai":6,"body":13,"categories":52,"created_at":54,"date_modified":54,"description":47,"extension":55,"faq":54,"featured":56,"kicker_label":54,"meta":57,"navigation":64,"path":65,"published_at":66,"question":54,"scraped_at":67,"seo":68,"sitemap":69,"source_id":70,"source_name":71,"source_type":72,"source_url":73,"stem":74,"tags":75,"thumbnail_url":54,"tldr":80,"tweet":54,"unknown_tags":81,"__hash__":82},"summaries\u002Fsummaries\u002F5d6a69b9b1714e2b-snowflake-native-fraud-ml-pipeline-train-to-monito-summary.md","Snowflake-Native Fraud ML Pipeline: Train to Monitor",{"provider":7,"model":8,"input_tokens":9,"output_tokens":10,"processing_time_ms":11,"cost_usd":12},"openrouter","x-ai\u002Fgrok-4.1-fast",9925,1740,12771,0.00283235,{"type":14,"value":15,"toc":46},"minimark",[16,21,30,33,36,40,43],[17,18,20],"h2",{"id":19},"overcome-data-gravity-and-class-imbalance-in-fraud-detection","Overcome Data Gravity and Class Imbalance in Fraud Detection",[22,23,24,25,29],"p",{},"Keep all ML stages—EDA, training, inference, monitoring—inside Snowflake to eliminate data movement risks like security gaps and lineage breaks. Start with SQL summaries on 100k transaction rows showing 0.5-2% fraud rate, then visualize patterns: fraud peaks 00:00-05:00 (high-risk hour flag), channel\u002Fmerchant risks, and correlations (e.g., VELOCITY_SCORE, low DEVICE_TRUST_SCORE strongest). Engineer five key features: AMOUNT_TO_AVG_RATIO for deviation detection, IS_HIGH_RISK_HOUR binary, RISK_COMPOSITE (0.3",[26,27,28],"em",{},"VELOCITY_SCORE + 0.3","(1-DEVICE_TRUST_SCORE) + 0.2*(FAILED_TRANSACTIONS_LAST_24H\u002F10) + 0.2*(DISTINCT_COUNTRIES_7D\u002F5)) as prior signal, LOG_AMOUNT for skew, CREDIT_SCORE_BIN (0-500=0, 500-650=1, etc.). One-hot encode categoricals (CHANNEL, MERCHANT_CATEGORY, etc.), yielding 39 features after stratified 80\u002F20 split (80000 train w\u002F2797 fraud, 20000 test w\u002F699 fraud).",[22,31,32],{},"Train XGBoost with imbalance fix: scale_pos_weight = legit\u002Ffraud ratio (27.60), params like n_estimators=500, max_depth=6, learning_rate=0.05, eval_metric='aucpr' (prioritizes precision-recall over ROC-AUC for rare events), early_stopping_rounds=50. Use Snowflake ExperimentTracking to log params\u002Fmetrics automatically. Result: best_iteration=7, ROC-AUC=0.7275, Average Precision=0.4907 (discriminates better on imbalance), default F1=0.5096. Optimize threshold by sweeping 0.1-0.9: 0.58 maximizes F1=0.5874 (Fraud precision=0.90, recall=0.43), balancing false positives (customer friction) vs. negatives (financial loss).",[22,34,35],{},"Top importances: RISK_COMPOSITE, VELOCITY_SCORE, DEVICE_TRUST_SCORE confirm engineered signals boost trees.",[17,37,39],{"id":38},"productionize-models-with-registry-inference-and-observability","Productionize Models with Registry, Inference, and Observability",[22,41,42],{},"Register via Snowflake Registry: log_model with metrics, sample_input for schema inference, task=TABULAR_BINARY_CLASSIFICATION. Gets versioned artifact (FRAUD_DETECTION_XGBOOST V1) with audit trail, no external stores. For batch inference on new 1000 txns, reapply exact feature pipeline + column alignment (pad missing dummies to 39 cols). Call registered model.run(predict_proba), apply threshold, save predictions (FRAUD_PROBABILITY, FRAUD_PREDICTION) + metadata to governed table ML.PRODUCTION.FRAUD_PREDICTIONS. Flags 25.7% as fraud; top risks show ATM\u002Fonline\u002Fphone patterns.",[22,44,45],{},"Enable observability: create ModelMonitor on scored table for daily drift checks (numeric\u002Fcategorical distributions) and score distribution shifts. Alerts on evolving fraud tactics without separate dashboards—model degrades silently otherwise. Entire pipeline runs in Snowflake Notebooks: Snowpark for compute, no creds\u002Fcontext switches. Trade-off: warehouse costs scale with data size, but unified governance outweighs external stack fragility.",{"title":47,"searchDepth":48,"depth":48,"links":49},"",2,[50,51],{"id":19,"depth":48,"text":20},{"id":38,"depth":48,"text":39},[53],"Data Science & Visualization",null,"md",false,{"content_references":58,"triage":59},[],{"relevance":60,"novelty":61,"quality":61,"actionability":60,"composite":62,"reasoning":63},5,4,4.55,"Category: AI Automation. The article provides a detailed, actionable guide on building a fraud detection pipeline using Snowflake, addressing specific pain points like data gravity and class imbalance. It includes concrete steps for model training and monitoring, making it highly relevant for product builders looking to implement AI solutions.",true,"\u002Fsummaries\u002F5d6a69b9b1714e2b-snowflake-native-fraud-ml-pipeline-train-to-monito-summary","2026-04-13 05:55:09","2026-04-13 17:53:11",{"title":5,"description":47},{"loc":65},"5d6a69b9b1714e2b","Towards AI","article","https:\u002F\u002Fpub.towardsai.net\u002Fbuilding-a-production-grade-fraud-detection-pipeline-inside-snowflake-end-to-end-684b94b6983c?source=rss----98111c9905da---4","summaries\u002F5d6a69b9b1714e2b-snowflake-native-fraud-ml-pipeline-train-to-monito-summary",[76,77,78,79],"machine-learning","data-science","automation","devops-cloud","Build end-to-end fraud detection with XGBoost in Snowflake ML—data loading to drift monitoring—avoiding data gravity, handling 0.5-2% imbalance via scale_pos_weight=27.6, achieving ROC-AUC=0.7275 and optimal F1=0.5874 at threshold=0.58.",[79],"EQ9hWWp8cu56Q5QIls3ttrO2IYvpneLhHS_Tn4t6kkY",[84,87,90,93,96,99,101,103,105,107,109,111,114,116,118,120,122,124,126,128,130,132,135,137,139,141,144,146,148,151,153,155,157,159,161,163,165,167,169,171,173,175,177,179,181,183,185,187,189,191,193,195,197,199,201,203,205,207,209,211,213,215,217,219,221,223,225,227,229,231,233,235,237,239,241,243,245,247,249,251,253,255,257,259,261,263,265,267,269,271,273,275,277,279,281,283,285,287,289,291,293,295,297,299,301,303,305,307,309,311,313,315,317,319,321,323,325,327,329,331,333,335,337,339,341,343,345,347,349,351,353,355,357,359,361,363,365,367,369,371,373,375,377,379,381,383,385,387,389,391,393,395,397,399,401,403,406,408,410,412,414,416,418,420,422,424,426,428,430,432,434,436,438,440,442,444,446,448,450,452,454,456,458,460,462,464,466,468,470,472,474,476,478,480,482,484,486,488,490,492,494,496,498,500,502,504,506,508,510,512,514,516,518,520,522,524,526,528,530,532,534,536,538,540,542,544,546,548,550,552,554,556,558,560,562,564,566,568,570,572,574,576,578,580,582,584,586,588,590,592,594,596,598,600,602,604,606,608,610,612,614,616,618,620,622,624,626,628,630,632,634,636,638,640,642,644,646,648,650,652,654,656,658,660,662,664,666,668,670,672,674,676,678,680,682,684,686,688,690,692,694,696,698,700,702,704,706,708,710,712,714,716,718,720,722,724,726,728,730,732,734,736,738,740,742,744,746,748,750,752,754,756,758,760,762,764,766,768,770,772,774,776,778,780,782,784,786,788,790,792,794,796,798,800,802,804,806,808,810,812,814,816,818,820,822,824,826,828,830,832,834,836,838,840,842,844,846,848,850,852,854,856,858,860,862,864,866,868,870,872,874,876,878,880,882,884,886,888,890,892,894,896,898,900,902,904,906,908,910,912,914,916,918,920,922,924,926,928,930,932,934,936,938,940,942,944,946,948,950,952,954,956,958,960,962,964,966,968,970,972,974,976,978,980,982,984,986,988,990,992,994,996,998,1000,1002,1004,1006,1008,1010,1012,1014,1016,1018,1020,1022,1024,1026,1028,1030,1032,1034,1036,1038,1040,1042,1044,1046,1048,1050,1052,1054,1056,1058,1060,1062,1064,1066,1068,1070,1072,1074,1076,1078,1080,1082,1084,1086,1088,1090,1092,1094,1096,1098,1100,1102,1104,1106,1108,1110,1112,1114,1116,1118,1120,1122,1124,1126,1128,1130,1132,1134,1136,1138,1140,1142,1144,1146,1148,1150,1152,1154,1156,1158,1160,1162,1164,1166,1168,1170,1172,1174,1176,1178,1180,1182,1184,1186,1188,1190,1192,1194,1196,1198,1200,1202,1204,1206,1208,1210,1212,1214,1216,1218,1220,1222,1224,1226,1228,1230,1232,1234,1236,1238,1240,1242,1244,1246,1248,1250,1252,1254,1256,1258,1260,1262,1264,1266,1268,1270,1272,1274,1276,1278,1280,1282,1284,1286,1288,1290,1292,1294,1296,1298,1300,1302,1304,1306,1308,1310,1312,1314,1316,1318,1320,1322,1324,1326,1328,1330,1332,1334,1336,1338,1340,1342,1344,1346,1348,1350,1352,1354,1356,1358,1360,1362,1364,1366,1368,1370,1372,1374,1376,1378,1380,1382,1384,1386,1388,1390,1392,1394,1396,1398,1400,1402,1404,1406,1408,1410,1412,1414,1416,1418,1420,1422,1424,1426,1428,1430,1432,1434,1436,1438,1440,1442,1444,1446,1448,1450,1452,1454,1456,1458,1460,1462,1464,1466,1468,1470,1472,1474,1476,1478,1480,1482,1484,1486,1488,1490,1492,1494,1496,1498,1500,1502,1504,1506,1508,1510,1512,1514,1516,1518,1520,1522,1524,1526,1528,1530,1532,1534,1536,1538,1540,1542,1544,1546,1548,1550,1552,1554,1556,1558,1560,1562,1564,1566,1568,1570,1572,1574,1576,1578,1580,1582,1584,1586,1588,1590,1592,1594,1596,1598,1600,1602,1604,1606,1608,1610,1612,1614,1616,1618,1620,1622,1624,1626,1628,1630,1632,1634,1636,1638,1640,1642,1644,1646,1648,1650,1652,1654,1656,1658,1660,1662,1664,1666,1668,1670,1672,1674,1676,1678,1680,1682,1684,1686,1688,1690,1692,1694,1696,1698,1700,1702,1704,1706,1708,1710,1712,1714,1716,1718,1720,1722,1724,1726,1728,1730,1732,1734,1736,1738,1740,1742,1744,1746,1748,1750,1752,1754,1756,1758,1760,1762,1764,1766,1768,1770,1772,1774,1776,1778,1780,1782,1784,1786,1788,1790,1792,1794,1796,1798,1800,1802,1804,1806,1808,1810,1812,1814,1816,1818,1820,1822,1824,1826,1828,1830,1832,1834,1836,1838,1840,1842,1844,1846,1848,1850,1852,1854,1856,1858,1860,1862,1864,1866,1868,1870,1872,1874,1876,1878,1880,1882,1884,1886,1888,1890,1892,1894,1896,1898,1900,1902,1904,1906,1908,1910,1912,1914,1916,1918,1920,1922,1924,1926,1928,1930,1932,1934,1936,1938,1940,1942,1944,1946,1948,1950,1952,1954,1956,1958,1960,1962,1964,1966,1968,1970,1972,1974,1976,1978,1980,1982,1984,1986,1988,1990,1992,1994,1996,1998,2000,2002,2004,2006,2008,2010,2012,2014,2016,2018,2020,2022,2024,2026,2028,2030,2032,2034,2036,2038,2040,2042,2044,2046,2048,2050,2052,2054,2056,2058,2060,2062,2064,2066,2068,2070,2072,2074,2076,2078,2080,2082,2084,2086,2088,2090,2092,2094,2096,2098,2100,2102,2104,2106,2108,2110,2112,2114,2116,2118,2120,2122,2124,2126,2128,2130,2132,2134,2136,2138,2140,2142,2144,2146,2148,2150,2152,2154,2156,2158,2160,2162,2164,2166,2168,2170,2172,2174,2176,2178,2180,2182,2184,2186,2188,2190,2192,2194,2196,2198,2200,2202,2204,2206,2208,2210,2212,2214,2216,2218,2220,2222,2224,2226,2228,2230,2232,2234,2236,2238,2240,2242,2244,2246,2248,2250,2252,2254,2256,2258,2260,2262,2264,2266,2268,2270,2272,2274,2276,2278,2280,2282,2284,2286,2288,2290,2292,2294,2296,2298,2300,2302,2304,2306,2308,2310,2312,2314,2316,2318,2320,2322,2324,2326,2328,2330,2332,2334,2336,2338,2340,2342,2344,2346,2348,2350,2352,2354,2356,2358,2360,2362,2364,2366,2368,2370,2372,2374,2376,2378,2380,2382,2384,2386,2388,2390,2392,2394,2396,2398,2400,2402,2404,2406,2408,2410,2412,2414,2416,2418,2420,2422,2424,2426,2428,2430,2432,2434,2436,2438,2440,2442,2444,2446,2448,2450,2452,2454,2456,2458,2460,2462,2464,2466,2468,2470,2472,2474,2476,2478,2480,2482,2484,2486,2488,2490,2492,2494,2496,2498,2500,2502,2504,2506,2508,2510,2512,2514,2516,2518,2520,2522,2524,2526,2528,2530,2532,2534,2536,2538,2540,2542,2544,2546,2548,2550,2552,2554,2556,2558,2560,2562,2564,2566,2568,2570,2572,2574,2576,2578,2580,2582,2584,2586,2588,2590,2592,2594,2596,2598,2600,2602,2604,2606,2608,2610,2612,2614,2616,2618,2620,2622,2624,2626,2628,2630,2632,2634,2636,2638,2640,2642,2644,2646,2648,2650,2652,2654,2656,2658,2660,2662,2664,2666,2668,2670,2672,2674,2676,2678,2680,2682,2684,2686,2688,2690,2692,2694,2696,2698,2700,2702,2704,2706,2708,2710,2712,2714,2716,2718,2720,2722,2724,2726,2728,2730,2732,2734,2736,2738,2740,2742,2744,2746,2748,2750,2752,2754,2756,2758,2760,2762,2764,2766,2768,2770,2772,2774,2776,2778,2780,2782,2784,2786,2788,2790,2792,2794,2796,2798,2800,2802,2804,2806,2808,2810,2812,2814,2816,2818,2820,2822,2824,2826,2828,2830,2832,2834,2836,2838,2840,2842,2844,2846,2848,2850,2852,2854,2856,2858,2860,2862,2864,2866,2868,2870,2872,2874,2876,2878,2880,2882,2884,2886,2888,2890,2892,2894,2896,2898,2900,2902,2904,2906,2908,2910,2912,2914,2916,2918,2920,2922,2924,2926,2928,2930,2932,2934,2936,2938,2940,2942,2944,2946,2948,2950,2952,2954,2956,2958,2960,2962,2964,2966,2968,2970,2972,2974,2976,2978,2980,2982,2984,2986,2988,2990,2992,2994,2996,2998,3000,3002,3004,3006,3008,3010,3012,3014,3016,3018,3020,3022,3024,3026,3028,3030,3032,3034,3036,3038,3040,3042,3044,3046,3048,3050,3052,3054,3056,3058,3060,3062,3064,3066,3068,3070,3072,3074,3076,3078,3080,3082,3084,3086,3088,3090,3092,3094,3096,3098,3100,3102,3104,3106,3108,3110,3112,3114,3116,3118,3120,3122,3124,3126,3128,3130,3132,3134,3136,3138,3140,3142,3144,3146,3148,3150,3152,3154,3156,3158,3160,3162,3164,3166,3168,3170,3172,3174,3176,3178,3180,3182,3184,3186,3188,3190,3192,3194,3196,3198,3200,3202,3204,3206,3208,3210,3212,3214,3216,3218,3220,3222,3224,3226,3228,3230,3232,3234,3236,3238,3240,3242,3244,3246,3248,3250,3252,3254,3256,3258,3260,3262,3264,3266,3268,3270,3272,3274,3276,3278,3280,3282,3284,3286,3288,3290,3292,3294,3296,3298,3300,3302,3304,3306,3308,3310,3312,3314,3316,3318,3320,3322,3324,3326,3328,3330,3332,3334,3336,3338,3340,3342,3344,3346,3348,3350,3352,3354,3356,3358,3360,3362,3364,3366,3368,3370,3372,3374,3376,3378,3380,3382,3384,3386,3388,3390,3392,3394,3396,3398,3400,3402,3404,3406,3408,3410,3412,3414,3416,3418,3420,3422,3424,3426,3428,3430,3432,3434,3436,3438,3440,3442,3444,3446,3448,3450,3452,3454,3456,3458,3460,3462,3464,3466,3468,3470,3472,3474,3476,3478,3480,3482,3484,3486,3488,3490,3492,3494,3496,3498,3500,3502,3504,3506,3508,3510,3512,3514,3516,3518,3520,3522,3524,3526,3528,3530,3532,3534,3536,3538,3540,3542,3544,3546,3548,3550,3552,3554,3556,3558,3560,3562,3564,3566,3568,3570,3572,3574,3576,3578,3580,3582,3584,3586,3588,3590,3592,3594,3596,3598,3600,3602,3604,3606,3608,3610,3612,3614,3616,3618,3620,3622,3624,3626,3628,3630,3632,3634,3636,3638,3640,3642,3644,3646,3648,3650],{"categories":85},[86],"Developer Productivity",{"categories":88},[89],"Business & SaaS",{"categories":91},[92],"AI & LLMs",{"categories":94},[95],"AI Automation",{"categories":97},[98],"Product Strategy",{"categories":100},[92],{"categories":102},[86],{"categories":104},[89],{"categories":106},[],{"categories":108},[92],{"categories":110},[],{"categories":112},[113],"AI News & Trends",{"categories":115},[95],{"categories":117},[113],{"categories":119},[95],{"categories":121},[95],{"categories":123},[92],{"categories":125},[92],{"categories":127},[113],{"categories":129},[92],{"categories":131},[],{"categories":133},[134],"Design & Frontend",{"categories":136},[53],{"categories":138},[113],{"categories":140},[],{"categories":142},[143],"Software Engineering",{"categories":145},[92],{"categories":147},[95],{"categories":149},[150],"Marketing & Growth",{"categories":152},[92],{"categories":154},[95],{"categories":156},[],{"categories":158},[],{"categories":160},[134],{"categories":162},[95],{"categories":164},[86],{"categories":166},[134],{"categories":168},[92],{"categories":170},[95],{"categories":172},[113],{"categories":174},[],{"categories":176},[],{"categories":178},[95],{"categories":180},[143],{"categories":182},[],{"categories":184},[89],{"categories":186},[],{"categories":188},[],{"categories":190},[95],{"categories":192},[95],{"categories":194},[92],{"categories":196},[],{"categories":198},[143],{"categories":200},[],{"categories":202},[],{"categories":204},[],{"categories":206},[92],{"categories":208},[150],{"categories":210},[134],{"categories":212},[134],{"categories":214},[92],{"categories":216},[95],{"categories":218},[92],{"categories":220},[92],{"categories":222},[95],{"categories":224},[95],{"categories":226},[53],{"categories":228},[113],{"categories":230},[95],{"categories":232},[150],{"categories":234},[95],{"categories":236},[98],{"categories":238},[],{"categories":240},[95],{"categories":242},[],{"categories":244},[95],{"categories":246},[143],{"categories":248},[134],{"categories":250},[92],{"categories":252},[],{"categories":254},[],{"categories":256},[95],{"categories":258},[],{"categories":260},[92],{"categories":262},[],{"categories":264},[86],{"categories":266},[143],{"categories":268},[89],{"categories":270},[113],{"categories":272},[92],{"categories":274},[],{"categories":276},[92],{"categories":278},[],{"categories":280},[143],{"categories":282},[53],{"categories":284},[],{"categories":286},[92],{"categories":288},[134],{"categories":290},[],{"categories":292},[134],{"categories":294},[95],{"categories":296},[],{"categories":298},[95],{"categories":300},[113],{"categories":302},[92],{"categories":304},[],{"categories":306},[95],{"categories":308},[92],{"categories":310},[98],{"categories":312},[],{"categories":314},[92],{"categories":316},[95],{"categories":318},[95],{"categories":320},[],{"categories":322},[53],{"categories":324},[92],{"categories":326},[],{"categories":328},[86],{"categories":330},[89],{"categories":332},[92],{"categories":334},[95],{"categories":336},[143],{"categories":338},[92],{"categories":340},[],{"categories":342},[],{"categories":344},[92],{"categories":346},[],{"categories":348},[134],{"categories":350},[],{"categories":352},[92],{"categories":354},[],{"categories":356},[95],{"categories":358},[92],{"categories":360},[134],{"categories":362},[],{"categories":364},[92],{"categories":366},[92],{"categories":368},[89],{"categories":370},[95],{"categories":372},[92],{"categories":374},[134],{"categories":376},[95],{"categories":378},[],{"categories":380},[],{"categories":382},[113],{"categories":384},[],{"categories":386},[92],{"categories":388},[89,150],{"categories":390},[],{"categories":392},[92],{"categories":394},[],{"categories":396},[],{"categories":398},[92],{"categories":400},[],{"categories":402},[92],{"categories":404},[405],"DevOps & Cloud",{"categories":407},[],{"categories":409},[113],{"categories":411},[134],{"categories":413},[],{"categories":415},[113],{"categories":417},[113],{"categories":419},[92],{"categories":421},[150],{"categories":423},[],{"categories":425},[89],{"categories":427},[],{"categories":429},[92,405],{"categories":431},[92],{"categories":433},[92],{"categories":435},[95],{"categories":437},[92,143],{"categories":439},[53],{"categories":441},[92],{"categories":443},[150],{"categories":445},[95],{"categories":447},[95],{"categories":449},[],{"categories":451},[95],{"categories":453},[92,89],{"categories":455},[],{"categories":457},[134],{"categories":459},[134],{"categories":461},[],{"categories":463},[],{"categories":465},[113],{"categories":467},[],{"categories":469},[86],{"categories":471},[143],{"categories":473},[92],{"categories":475},[134],{"categories":477},[95],{"categories":479},[143],{"categories":481},[113],{"categories":483},[134],{"categories":485},[],{"categories":487},[92],{"categories":489},[92],{"categories":491},[92],{"categories":493},[113],{"categories":495},[86],{"categories":497},[92],{"categories":499},[95],{"categories":501},[405],{"categories":503},[134],{"categories":505},[95],{"categories":507},[],{"categories":509},[],{"categories":511},[134],{"categories":513},[113],{"categories":515},[53],{"categories":517},[],{"categories":519},[92],{"categories":521},[92],{"categories":523},[89],{"categories":525},[92],{"categories":527},[92],{"categories":529},[113],{"categories":531},[],{"categories":533},[95],{"categories":535},[143],{"categories":537},[],{"categories":539},[92],{"categories":541},[92],{"categories":543},[95],{"categories":545},[],{"categories":547},[],{"categories":549},[92],{"categories":551},[],{"categories":553},[89],{"categories":555},[95],{"categories":557},[],{"categories":559},[86],{"categories":561},[92],{"categories":563},[89],{"categories":565},[113],{"categories":567},[],{"categories":569},[],{"categories":571},[],{"categories":573},[113],{"categories":575},[113],{"categories":577},[],{"categories":579},[],{"categories":581},[89],{"categories":583},[],{"categories":585},[],{"categories":587},[86],{"categories":589},[],{"categories":591},[150],{"categories":593},[95],{"categories":595},[89],{"categories":597},[95],{"categories":599},[],{"categories":601},[98],{"categories":603},[134],{"categories":605},[143],{"categories":607},[92],{"categories":609},[95],{"categories":611},[89],{"categories":613},[92],{"categories":615},[],{"categories":617},[],{"categories":619},[143],{"categories":621},[53],{"categories":623},[98],{"categories":625},[95],{"categories":627},[92],{"categories":629},[],{"categories":631},[405],{"categories":633},[],{"categories":635},[95],{"categories":637},[],{"categories":639},[],{"categories":641},[92],{"categories":643},[134],{"categories":645},[150],{"categories":647},[95],{"categories":649},[],{"categories":651},[86],{"categories":653},[],{"categories":655},[113],{"categories":657},[92,405],{"categories":659},[113],{"categories":661},[92],{"categories":663},[89],{"categories":665},[92],{"categories":667},[],{"categories":669},[89],{"categories":671},[],{"categories":673},[143],{"categories":675},[134],{"categories":677},[113],{"categories":679},[53],{"categories":681},[86],{"categories":683},[92],{"categories":685},[143],{"categories":687},[],{"categories":689},[],{"categories":691},[98],{"categories":693},[],{"categories":695},[92],{"categories":697},[],{"categories":699},[134],{"categories":701},[134],{"categories":703},[134],{"categories":705},[],{"categories":707},[],{"categories":709},[113],{"categories":711},[95],{"categories":713},[92],{"categories":715},[92],{"categories":717},[92],{"categories":719},[89],{"categories":721},[92],{"categories":723},[],{"categories":725},[143],{"categories":727},[143],{"categories":729},[89],{"categories":731},[],{"categories":733},[92],{"categories":735},[92],{"categories":737},[89],{"categories":739},[113],{"categories":741},[150],{"categories":743},[95],{"categories":745},[],{"categories":747},[134],{"categories":749},[],{"categories":751},[92],{"categories":753},[],{"categories":755},[89],{"categories":757},[95],{"categories":759},[],{"categories":761},[405],{"categories":763},[53],{"categories":765},[143],{"categories":767},[150],{"categories":769},[143],{"categories":771},[95],{"categories":773},[],{"categories":775},[],{"categories":777},[95],{"categories":779},[86],{"categories":781},[95],{"categories":783},[98],{"categories":785},[89],{"categories":787},[],{"categories":789},[92],{"categories":791},[98],{"categories":793},[92],{"categories":795},[92],{"categories":797},[150],{"categories":799},[134],{"categories":801},[95],{"categories":803},[],{"categories":805},[],{"categories":807},[405],{"categories":809},[143],{"categories":811},[],{"categories":813},[95],{"categories":815},[92],{"categories":817},[134,92],{"categories":819},[86],{"categories":821},[],{"categories":823},[92],{"categories":825},[86],{"categories":827},[134],{"categories":829},[95],{"categories":831},[143],{"categories":833},[],{"categories":835},[92],{"categories":837},[],{"categories":839},[86],{"categories":841},[],{"categories":843},[95],{"categories":845},[98],{"categories":847},[92],{"categories":849},[92],{"categories":851},[134],{"categories":853},[95],{"categories":855},[405],{"categories":857},[134],{"categories":859},[95],{"categories":861},[92],{"categories":863},[92],{"categories":865},[92],{"categories":867},[113],{"categories":869},[],{"categories":871},[98],{"categories":873},[95],{"categories":875},[134],{"categories":877},[95],{"categories":879},[143],{"categories":881},[134],{"categories":883},[95],{"categories":885},[113],{"categories":887},[],{"categories":889},[92],{"categories":891},[134],{"categories":893},[92],{"categories":895},[86],{"categories":897},[113],{"categories":899},[92],{"categories":901},[150],{"categories":903},[92],{"categories":905},[92],{"categories":907},[95],{"categories":909},[95],{"categories":911},[92],{"categories":913},[95],{"categories":915},[134],{"categories":917},[92],{"categories":919},[],{"categories":921},[],{"categories":923},[143],{"categories":925},[],{"categories":927},[86],{"categories":929},[405],{"categories":931},[],{"categories":933},[86],{"categories":935},[89],{"categories":937},[150],{"categories":939},[],{"categories":941},[89],{"categories":943},[],{"categories":945},[],{"categories":947},[],{"categories":949},[],{"categories":951},[],{"categories":953},[92],{"categories":955},[95],{"categories":957},[405],{"categories":959},[86],{"categories":961},[92],{"categories":963},[143],{"categories":965},[98],{"categories":967},[92],{"categories":969},[150],{"categories":971},[92],{"categories":973},[92],{"categories":975},[92],{"categories":977},[92,86],{"categories":979},[143],{"categories":981},[143],{"categories":983},[134],{"categories":985},[92],{"categories":987},[],{"categories":989},[],{"categories":991},[],{"categories":993},[143],{"categories":995},[53],{"categories":997},[113],{"categories":999},[134],{"categories":1001},[],{"categories":1003},[92],{"categories":1005},[92],{"categories":1007},[],{"categories":1009},[],{"categories":1011},[95],{"categories":1013},[92],{"categories":1015},[89],{"categories":1017},[],{"categories":1019},[86],{"categories":1021},[92],{"categories":1023},[86],{"categories":1025},[92],{"categories":1027},[143],{"categories":1029},[150],{"categories":1031},[92,134],{"categories":1033},[113],{"categories":1035},[134],{"categories":1037},[],{"categories":1039},[405],{"categories":1041},[134],{"categories":1043},[95],{"categories":1045},[],{"categories":1047},[],{"categories":1049},[],{"categories":1051},[],{"categories":1053},[143],{"categories":1055},[95],{"categories":1057},[95],{"categories":1059},[92],{"categories":1061},[92],{"categories":1063},[],{"categories":1065},[134],{"categories":1067},[],{"categories":1069},[],{"categories":1071},[95],{"categories":1073},[],{"categories":1075},[],{"categories":1077},[150],{"categories":1079},[150],{"categories":1081},[95],{"categories":1083},[],{"categories":1085},[92],{"categories":1087},[92],{"categories":1089},[143],{"categories":1091},[134],{"categories":1093},[134],{"categories":1095},[95],{"categories":1097},[86],{"categories":1099},[92],{"categories":1101},[134],{"categories":1103},[134],{"categories":1105},[95],{"categories":1107},[95],{"categories":1109},[92],{"categories":1111},[],{"categories":1113},[],{"categories":1115},[92],{"categories":1117},[95],{"categories":1119},[113],{"categories":1121},[143],{"categories":1123},[86],{"categories":1125},[92],{"categories":1127},[],{"categories":1129},[95],{"categories":1131},[95],{"categories":1133},[],{"categories":1135},[86],{"categories":1137},[92],{"categories":1139},[86],{"categories":1141},[86],{"categories":1143},[],{"categories":1145},[],{"categories":1147},[95],{"categories":1149},[95],{"categories":1151},[92],{"categories":1153},[92],{"categories":1155},[113],{"categories":1157},[53],{"categories":1159},[98],{"categories":1161},[113],{"categories":1163},[134],{"categories":1165},[],{"categories":1167},[113],{"categories":1169},[],{"categories":1171},[],{"categories":1173},[],{"categories":1175},[],{"categories":1177},[143],{"categories":1179},[53],{"categories":1181},[],{"categories":1183},[92],{"categories":1185},[92],{"categories":1187},[53],{"categories":1189},[143],{"categories":1191},[],{"categories":1193},[],{"categories":1195},[95],{"categories":1197},[113],{"categories":1199},[113],{"categories":1201},[95],{"categories":1203},[86],{"categories":1205},[92,405],{"categories":1207},[],{"categories":1209},[134],{"categories":1211},[86],{"categories":1213},[95],{"categories":1215},[134],{"categories":1217},[],{"categories":1219},[95],{"categories":1221},[95],{"categories":1223},[92],{"categories":1225},[150],{"categories":1227},[143],{"categories":1229},[134],{"categories":1231},[],{"categories":1233},[95],{"categories":1235},[92],{"categories":1237},[95],{"categories":1239},[95],{"categories":1241},[95],{"categories":1243},[150],{"categories":1245},[95],{"categories":1247},[92],{"categories":1249},[],{"categories":1251},[150],{"categories":1253},[113],{"categories":1255},[95],{"categories":1257},[],{"categories":1259},[],{"categories":1261},[92],{"categories":1263},[95],{"categories":1265},[113],{"categories":1267},[95],{"categories":1269},[],{"categories":1271},[],{"categories":1273},[],{"categories":1275},[95],{"categories":1277},[],{"categories":1279},[],{"categories":1281},[53],{"categories":1283},[92],{"categories":1285},[53],{"categories":1287},[113],{"categories":1289},[92],{"categories":1291},[92],{"categories":1293},[95],{"categories":1295},[92],{"categories":1297},[],{"categories":1299},[],{"categories":1301},[405],{"categories":1303},[],{"categories":1305},[],{"categories":1307},[86],{"categories":1309},[],{"categories":1311},[],{"categories":1313},[],{"categories":1315},[],{"categories":1317},[143],{"categories":1319},[113],{"categories":1321},[150],{"categories":1323},[89],{"categories":1325},[92],{"categories":1327},[92],{"categories":1329},[89],{"categories":1331},[],{"categories":1333},[134],{"categories":1335},[95],{"categories":1337},[89],{"categories":1339},[92],{"categories":1341},[92],{"categories":1343},[86],{"categories":1345},[],{"categories":1347},[86],{"categories":1349},[92],{"categories":1351},[150],{"categories":1353},[95],{"categories":1355},[113],{"categories":1357},[89],{"categories":1359},[92],{"categories":1361},[95],{"categories":1363},[],{"categories":1365},[92],{"categories":1367},[86],{"categories":1369},[92],{"categories":1371},[],{"categories":1373},[113],{"categories":1375},[92],{"categories":1377},[],{"categories":1379},[89],{"categories":1381},[92],{"categories":1383},[],{"categories":1385},[],{"categories":1387},[],{"categories":1389},[92],{"categories":1391},[],{"categories":1393},[405],{"categories":1395},[92],{"categories":1397},[],{"categories":1399},[92],{"categories":1401},[92],{"categories":1403},[92],{"categories":1405},[92,405],{"categories":1407},[92],{"categories":1409},[92],{"categories":1411},[134],{"categories":1413},[95],{"categories":1415},[],{"categories":1417},[95],{"categories":1419},[92],{"categories":1421},[92],{"categories":1423},[92],{"categories":1425},[86],{"categories":1427},[86],{"categories":1429},[143],{"categories":1431},[134],{"categories":1433},[95],{"categories":1435},[],{"categories":1437},[92],{"categories":1439},[113],{"categories":1441},[92],{"categories":1443},[89],{"categories":1445},[],{"categories":1447},[405],{"categories":1449},[134],{"categories":1451},[134],{"categories":1453},[95],{"categories":1455},[113],{"categories":1457},[95],{"categories":1459},[92],{"categories":1461},[],{"categories":1463},[92],{"categories":1465},[],{"categories":1467},[],{"categories":1469},[92],{"categories":1471},[92],{"categories":1473},[92],{"categories":1475},[95],{"categories":1477},[92],{"categories":1479},[],{"categories":1481},[53],{"categories":1483},[95],{"categories":1485},[],{"categories":1487},[92],{"categories":1489},[113],{"categories":1491},[],{"categories":1493},[134],{"categories":1495},[405],{"categories":1497},[113],{"categories":1499},[143],{"categories":1501},[143],{"categories":1503},[113],{"categories":1505},[113],{"categories":1507},[405],{"categories":1509},[],{"categories":1511},[113],{"categories":1513},[92],{"categories":1515},[86],{"categories":1517},[113],{"categories":1519},[],{"categories":1521},[53],{"categories":1523},[113],{"categories":1525},[143],{"categories":1527},[113],{"categories":1529},[405],{"categories":1531},[92],{"categories":1533},[92],{"categories":1535},[],{"categories":1537},[89],{"categories":1539},[],{"categories":1541},[],{"categories":1543},[92],{"categories":1545},[92],{"categories":1547},[92],{"categories":1549},[92],{"categories":1551},[],{"categories":1553},[53],{"categories":1555},[86],{"categories":1557},[],{"categories":1559},[92],{"categories":1561},[92],{"categories":1563},[405],{"categories":1565},[405],{"categories":1567},[],{"categories":1569},[95],{"categories":1571},[113],{"categories":1573},[113],{"categories":1575},[92],{"categories":1577},[95],{"categories":1579},[],{"categories":1581},[134],{"categories":1583},[92],{"categories":1585},[92],{"categories":1587},[],{"categories":1589},[],{"categories":1591},[405],{"categories":1593},[92],{"categories":1595},[143],{"categories":1597},[89],{"categories":1599},[92],{"categories":1601},[],{"categories":1603},[95],{"categories":1605},[86],{"categories":1607},[86],{"categories":1609},[],{"categories":1611},[92],{"categories":1613},[134],{"categories":1615},[95],{"categories":1617},[],{"categories":1619},[92],{"categories":1621},[92],{"categories":1623},[95],{"categories":1625},[],{"categories":1627},[95],{"categories":1629},[143],{"categories":1631},[],{"categories":1633},[92],{"categories":1635},[],{"categories":1637},[92],{"categories":1639},[],{"categories":1641},[92],{"categories":1643},[92],{"categories":1645},[],{"categories":1647},[92],{"categories":1649},[113],{"categories":1651},[92],{"categories":1653},[92],{"categories":1655},[86],{"categories":1657},[92],{"categories":1659},[113],{"categories":1661},[95],{"categories":1663},[],{"categories":1665},[92],{"categories":1667},[150],{"categories":1669},[],{"categories":1671},[],{"categories":1673},[],{"categories":1675},[86],{"categories":1677},[113],{"categories":1679},[95],{"categories":1681},[92],{"categories":1683},[134],{"categories":1685},[95],{"categories":1687},[],{"categories":1689},[95],{"categories":1691},[],{"categories":1693},[92],{"categories":1695},[95],{"categories":1697},[92],{"categories":1699},[],{"categories":1701},[92],{"categories":1703},[92],{"categories":1705},[113],{"categories":1707},[134],{"categories":1709},[95],{"categories":1711},[134],{"categories":1713},[89],{"categories":1715},[],{"categories":1717},[],{"categories":1719},[92],{"categories":1721},[86],{"categories":1723},[113],{"categories":1725},[],{"categories":1727},[],{"categories":1729},[143],{"categories":1731},[134],{"categories":1733},[],{"categories":1735},[92],{"categories":1737},[],{"categories":1739},[150],{"categories":1741},[92],{"categories":1743},[405],{"categories":1745},[143],{"categories":1747},[],{"categories":1749},[95],{"categories":1751},[92],{"categories":1753},[95],{"categories":1755},[95],{"categories":1757},[92],{"categories":1759},[],{"categories":1761},[86],{"categories":1763},[92],{"categories":1765},[89],{"categories":1767},[143],{"categories":1769},[134],{"categories":1771},[],{"categories":1773},[],{"categories":1775},[],{"categories":1777},[95],{"categories":1779},[134],{"categories":1781},[113],{"categories":1783},[92],{"categories":1785},[113],{"categories":1787},[134],{"categories":1789},[],{"categories":1791},[134],{"categories":1793},[113],{"categories":1795},[89],{"categories":1797},[92],{"categories":1799},[113],{"categories":1801},[150],{"categories":1803},[],{"categories":1805},[],{"categories":1807},[53],{"categories":1809},[92,143],{"categories":1811},[113],{"categories":1813},[92],{"categories":1815},[95],{"categories":1817},[95],{"categories":1819},[92],{"categories":1821},[],{"categories":1823},[143],{"categories":1825},[92],{"categories":1827},[53],{"categories":1829},[95],{"categories":1831},[150],{"categories":1833},[405],{"categories":1835},[],{"categories":1837},[86],{"categories":1839},[95],{"categories":1841},[95],{"categories":1843},[143],{"categories":1845},[92],{"categories":1847},[92],{"categories":1849},[],{"categories":1851},[],{"categories":1853},[],{"categories":1855},[405],{"categories":1857},[113],{"categories":1859},[92],{"categories":1861},[92],{"categories":1863},[92],{"categories":1865},[],{"categories":1867},[53],{"categories":1869},[89],{"categories":1871},[],{"categories":1873},[95],{"categories":1875},[405],{"categories":1877},[],{"categories":1879},[134],{"categories":1881},[134],{"categories":1883},[],{"categories":1885},[143],{"categories":1887},[134],{"categories":1889},[92],{"categories":1891},[],{"categories":1893},[113],{"categories":1895},[92],{"categories":1897},[134],{"categories":1899},[95],{"categories":1901},[113],{"categories":1903},[],{"categories":1905},[95],{"categories":1907},[134],{"categories":1909},[92],{"categories":1911},[],{"categories":1913},[92],{"categories":1915},[92],{"categories":1917},[405],{"categories":1919},[113],{"categories":1921},[53],{"categories":1923},[53],{"categories":1925},[],{"categories":1927},[],{"categories":1929},[],{"categories":1931},[95],{"categories":1933},[143],{"categories":1935},[143],{"categories":1937},[],{"categories":1939},[],{"categories":1941},[92],{"categories":1943},[],{"categories":1945},[95],{"categories":1947},[92],{"categories":1949},[],{"categories":1951},[92],{"categories":1953},[89],{"categories":1955},[92],{"categories":1957},[150],{"categories":1959},[95],{"categories":1961},[92],{"categories":1963},[143],{"categories":1965},[113],{"categories":1967},[95],{"categories":1969},[],{"categories":1971},[113],{"categories":1973},[95],{"categories":1975},[95],{"categories":1977},[],{"categories":1979},[89],{"categories":1981},[95],{"categories":1983},[],{"categories":1985},[92],{"categories":1987},[86],{"categories":1989},[113],{"categories":1991},[405],{"categories":1993},[95],{"categories":1995},[95],{"categories":1997},[86],{"categories":1999},[92],{"categories":2001},[],{"categories":2003},[],{"categories":2005},[134],{"categories":2007},[92,89],{"categories":2009},[],{"categories":2011},[86],{"categories":2013},[53],{"categories":2015},[92],{"categories":2017},[143],{"categories":2019},[92],{"categories":2021},[95],{"categories":2023},[92],{"categories":2025},[92],{"categories":2027},[113],{"categories":2029},[95],{"categories":2031},[],{"categories":2033},[],{"categories":2035},[95],{"categories":2037},[92],{"categories":2039},[405],{"categories":2041},[],{"categories":2043},[92],{"categories":2045},[95],{"categories":2047},[],{"categories":2049},[92],{"categories":2051},[150],{"categories":2053},[53],{"categories":2055},[95],{"categories":2057},[92],{"categories":2059},[405],{"categories":2061},[],{"categories":2063},[92],{"categories":2065},[150],{"categories":2067},[134],{"categories":2069},[92],{"categories":2071},[],{"categories":2073},[150],{"categories":2075},[113],{"categories":2077},[92],{"categories":2079},[92],{"categories":2081},[86],{"categories":2083},[],{"categories":2085},[],{"categories":2087},[134],{"categories":2089},[92],{"categories":2091},[53],{"categories":2093},[150],{"categories":2095},[150],{"categories":2097},[113],{"categories":2099},[],{"categories":2101},[],{"categories":2103},[92],{"categories":2105},[],{"categories":2107},[92,143],{"categories":2109},[113],{"categories":2111},[95],{"categories":2113},[143],{"categories":2115},[92],{"categories":2117},[86],{"categories":2119},[],{"categories":2121},[],{"categories":2123},[86],{"categories":2125},[150],{"categories":2127},[92],{"categories":2129},[],{"categories":2131},[134,92],{"categories":2133},[405],{"categories":2135},[86],{"categories":2137},[],{"categories":2139},[89],{"categories":2141},[89],{"categories":2143},[92],{"categories":2145},[143],{"categories":2147},[95],{"categories":2149},[113],{"categories":2151},[150],{"categories":2153},[134],{"categories":2155},[92],{"categories":2157},[92],{"categories":2159},[92],{"categories":2161},[86],{"categories":2163},[92],{"categories":2165},[95],{"categories":2167},[113],{"categories":2169},[],{"categories":2171},[],{"categories":2173},[53],{"categories":2175},[143],{"categories":2177},[92],{"categories":2179},[134],{"categories":2181},[53],{"categories":2183},[92],{"categories":2185},[92],{"categories":2187},[95],{"categories":2189},[95],{"categories":2191},[92,89],{"categories":2193},[],{"categories":2195},[134],{"categories":2197},[],{"categories":2199},[92],{"categories":2201},[113],{"categories":2203},[86],{"categories":2205},[86],{"categories":2207},[95],{"categories":2209},[92],{"categories":2211},[89],{"categories":2213},[143],{"categories":2215},[150],{"categories":2217},[],{"categories":2219},[113],{"categories":2221},[92],{"categories":2223},[92],{"categories":2225},[113],{"categories":2227},[143],{"categories":2229},[92],{"categories":2231},[95],{"categories":2233},[113],{"categories":2235},[92],{"categories":2237},[134],{"categories":2239},[92],{"categories":2241},[92],{"categories":2243},[405],{"categories":2245},[98],{"categories":2247},[95],{"categories":2249},[92],{"categories":2251},[113],{"categories":2253},[95],{"categories":2255},[150],{"categories":2257},[92],{"categories":2259},[],{"categories":2261},[92],{"categories":2263},[],{"categories":2265},[],{"categories":2267},[],{"categories":2269},[89],{"categories":2271},[92],{"categories":2273},[95],{"categories":2275},[113],{"categories":2277},[113],{"categories":2279},[113],{"categories":2281},[113],{"categories":2283},[],{"categories":2285},[86],{"categories":2287},[95],{"categories":2289},[113],{"categories":2291},[86],{"categories":2293},[95],{"categories":2295},[92],{"categories":2297},[92,95],{"categories":2299},[95],{"categories":2301},[405],{"categories":2303},[113],{"categories":2305},[113],{"categories":2307},[95],{"categories":2309},[92],{"categories":2311},[],{"categories":2313},[113],{"categories":2315},[150],{"categories":2317},[86],{"categories":2319},[92],{"categories":2321},[92],{"categories":2323},[],{"categories":2325},[143],{"categories":2327},[],{"categories":2329},[86],{"categories":2331},[95],{"categories":2333},[113],{"categories":2335},[92],{"categories":2337},[113],{"categories":2339},[86],{"categories":2341},[113],{"categories":2343},[113],{"categories":2345},[],{"categories":2347},[89],{"categories":2349},[95],{"categories":2351},[113],{"categories":2353},[113],{"categories":2355},[113],{"categories":2357},[113],{"categories":2359},[113],{"categories":2361},[113],{"categories":2363},[113],{"categories":2365},[113],{"categories":2367},[113],{"categories":2369},[113],{"categories":2371},[53],{"categories":2373},[86],{"categories":2375},[92],{"categories":2377},[92],{"categories":2379},[],{"categories":2381},[92,86],{"categories":2383},[],{"categories":2385},[95],{"categories":2387},[113],{"categories":2389},[95],{"categories":2391},[92],{"categories":2393},[92],{"categories":2395},[92],{"categories":2397},[92],{"categories":2399},[92],{"categories":2401},[95],{"categories":2403},[89],{"categories":2405},[134],{"categories":2407},[113],{"categories":2409},[92],{"categories":2411},[],{"categories":2413},[],{"categories":2415},[95],{"categories":2417},[134],{"categories":2419},[92],{"categories":2421},[],{"categories":2423},[],{"categories":2425},[150],{"categories":2427},[92],{"categories":2429},[],{"categories":2431},[],{"categories":2433},[86],{"categories":2435},[89],{"categories":2437},[92],{"categories":2439},[89],{"categories":2441},[134],{"categories":2443},[],{"categories":2445},[113],{"categories":2447},[],{"categories":2449},[134],{"categories":2451},[92],{"categories":2453},[150],{"categories":2455},[],{"categories":2457},[150],{"categories":2459},[],{"categories":2461},[],{"categories":2463},[95],{"categories":2465},[],{"categories":2467},[89],{"categories":2469},[86],{"categories":2471},[134],{"categories":2473},[143],{"categories":2475},[],{"categories":2477},[],{"categories":2479},[92],{"categories":2481},[86],{"categories":2483},[150],{"categories":2485},[],{"categories":2487},[95],{"categories":2489},[95],{"categories":2491},[113],{"categories":2493},[92],{"categories":2495},[95],{"categories":2497},[92],{"categories":2499},[95],{"categories":2501},[92],{"categories":2503},[98],{"categories":2505},[113],{"categories":2507},[],{"categories":2509},[150],{"categories":2511},[143],{"categories":2513},[95],{"categories":2515},[],{"categories":2517},[92],{"categories":2519},[95],{"categories":2521},[89],{"categories":2523},[86],{"categories":2525},[92],{"categories":2527},[134],{"categories":2529},[143],{"categories":2531},[143],{"categories":2533},[92],{"categories":2535},[53],{"categories":2537},[92],{"categories":2539},[95],{"categories":2541},[89],{"categories":2543},[95],{"categories":2545},[92],{"categories":2547},[92],{"categories":2549},[95],{"categories":2551},[113],{"categories":2553},[],{"categories":2555},[86],{"categories":2557},[92],{"categories":2559},[95],{"categories":2561},[92],{"categories":2563},[92],{"categories":2565},[],{"categories":2567},[134],{"categories":2569},[89],{"categories":2571},[113],{"categories":2573},[92],{"categories":2575},[92],{"categories":2577},[134],{"categories":2579},[150],{"categories":2581},[53],{"categories":2583},[92],{"categories":2585},[113],{"categories":2587},[92],{"categories":2589},[95],{"categories":2591},[405],{"categories":2593},[92],{"categories":2595},[95],{"categories":2597},[53],{"categories":2599},[],{"categories":2601},[95],{"categories":2603},[143],{"categories":2605},[134],{"categories":2607},[92],{"categories":2609},[86],{"categories":2611},[89],{"categories":2613},[143],{"categories":2615},[],{"categories":2617},[95],{"categories":2619},[92],{"categories":2621},[],{"categories":2623},[113],{"categories":2625},[],{"categories":2627},[113],{"categories":2629},[92],{"categories":2631},[95],{"categories":2633},[95],{"categories":2635},[95],{"categories":2637},[],{"categories":2639},[],{"categories":2641},[92],{"categories":2643},[92],{"categories":2645},[],{"categories":2647},[134],{"categories":2649},[95],{"categories":2651},[150],{"categories":2653},[86],{"categories":2655},[],{"categories":2657},[],{"categories":2659},[113],{"categories":2661},[143],{"categories":2663},[92],{"categories":2665},[92],{"categories":2667},[92],{"categories":2669},[143],{"categories":2671},[113],{"categories":2673},[134],{"categories":2675},[92],{"categories":2677},[92],{"categories":2679},[92],{"categories":2681},[113],{"categories":2683},[92],{"categories":2685},[113],{"categories":2687},[95],{"categories":2689},[95],{"categories":2691},[143],{"categories":2693},[95],{"categories":2695},[92],{"categories":2697},[143],{"categories":2699},[134],{"categories":2701},[],{"categories":2703},[95],{"categories":2705},[],{"categories":2707},[],{"categories":2709},[89],{"categories":2711},[92],{"categories":2713},[95],{"categories":2715},[86],{"categories":2717},[95],{"categories":2719},[150],{"categories":2721},[],{"categories":2723},[95],{"categories":2725},[],{"categories":2727},[86],{"categories":2729},[95],{"categories":2731},[],{"categories":2733},[95],{"categories":2735},[92],{"categories":2737},[113],{"categories":2739},[92],{"categories":2741},[95],{"categories":2743},[113],{"categories":2745},[95],{"categories":2747},[143],{"categories":2749},[134],{"categories":2751},[86],{"categories":2753},[],{"categories":2755},[95],{"categories":2757},[134],{"categories":2759},[113],{"categories":2761},[92],{"categories":2763},[134],{"categories":2765},[86],{"categories":2767},[],{"categories":2769},[95],{"categories":2771},[95],{"categories":2773},[92],{"categories":2775},[],{"categories":2777},[95],{"categories":2779},[98],{"categories":2781},[113],{"categories":2783},[95],{"categories":2785},[89],{"categories":2787},[],{"categories":2789},[92],{"categories":2791},[98],{"categories":2793},[92],{"categories":2795},[95],{"categories":2797},[113],{"categories":2799},[86],{"categories":2801},[405],{"categories":2803},[92],{"categories":2805},[92],{"categories":2807},[92],{"categories":2809},[113],{"categories":2811},[89],{"categories":2813},[92],{"categories":2815},[134],{"categories":2817},[113],{"categories":2819},[405],{"categories":2821},[92],{"categories":2823},[],{"categories":2825},[],{"categories":2827},[405],{"categories":2829},[53],{"categories":2831},[95],{"categories":2833},[95],{"categories":2835},[113],{"categories":2837},[92],{"categories":2839},[86],{"categories":2841},[134],{"categories":2843},[95],{"categories":2845},[92],{"categories":2847},[150],{"categories":2849},[92],{"categories":2851},[95],{"categories":2853},[],{"categories":2855},[92],{"categories":2857},[92],{"categories":2859},[113],{"categories":2861},[86],{"categories":2863},[],{"categories":2865},[92],{"categories":2867},[92],{"categories":2869},[143],{"categories":2871},[134],{"categories":2873},[92,95],{"categories":2875},[150,89],{"categories":2877},[92],{"categories":2879},[],{"categories":2881},[95],{"categories":2883},[],{"categories":2885},[143],{"categories":2887},[92],{"categories":2889},[113],{"categories":2891},[],{"categories":2893},[95],{"categories":2895},[],{"categories":2897},[95],{"categories":2899},[86],{"categories":2901},[95],{"categories":2903},[92],{"categories":2905},[405],{"categories":2907},[150],{"categories":2909},[89],{"categories":2911},[89],{"categories":2913},[86],{"categories":2915},[86],{"categories":2917},[92],{"categories":2919},[95],{"categories":2921},[92],{"categories":2923},[92],{"categories":2925},[86],{"categories":2927},[92],{"categories":2929},[150],{"categories":2931},[113],{"categories":2933},[92],{"categories":2935},[95],{"categories":2937},[92],{"categories":2939},[],{"categories":2941},[143],{"categories":2943},[],{"categories":2945},[95],{"categories":2947},[86],{"categories":2949},[],{"categories":2951},[405],{"categories":2953},[92],{"categories":2955},[],{"categories":2957},[113],{"categories":2959},[95],{"categories":2961},[143],{"categories":2963},[92],{"categories":2965},[95],{"categories":2967},[143],{"categories":2969},[95],{"categories":2971},[113],{"categories":2973},[86],{"categories":2975},[113],{"categories":2977},[143],{"categories":2979},[92],{"categories":2981},[134],{"categories":2983},[92],{"categories":2985},[92],{"categories":2987},[92],{"categories":2989},[92],{"categories":2991},[95],{"categories":2993},[92],{"categories":2995},[95],{"categories":2997},[92],{"categories":2999},[86],{"categories":3001},[92],{"categories":3003},[95],{"categories":3005},[134],{"categories":3007},[86],{"categories":3009},[95],{"categories":3011},[134],{"categories":3013},[],{"categories":3015},[92],{"categories":3017},[92],{"categories":3019},[143],{"categories":3021},[],{"categories":3023},[95],{"categories":3025},[150],{"categories":3027},[92],{"categories":3029},[113],{"categories":3031},[150],{"categories":3033},[95],{"categories":3035},[89],{"categories":3037},[89],{"categories":3039},[92],{"categories":3041},[86],{"categories":3043},[],{"categories":3045},[92],{"categories":3047},[],{"categories":3049},[86],{"categories":3051},[92],{"categories":3053},[95],{"categories":3055},[95],{"categories":3057},[],{"categories":3059},[143],{"categories":3061},[143],{"categories":3063},[150],{"categories":3065},[134],{"categories":3067},[],{"categories":3069},[92],{"categories":3071},[86],{"categories":3073},[92],{"categories":3075},[143],{"categories":3077},[86],{"categories":3079},[113],{"categories":3081},[113],{"categories":3083},[],{"categories":3085},[113],{"categories":3087},[95],{"categories":3089},[134],{"categories":3091},[53],{"categories":3093},[92],{"categories":3095},[],{"categories":3097},[113],{"categories":3099},[143],{"categories":3101},[89],{"categories":3103},[92],{"categories":3105},[86],{"categories":3107},[405],{"categories":3109},[86],{"categories":3111},[],{"categories":3113},[],{"categories":3115},[113],{"categories":3117},[],{"categories":3119},[95],{"categories":3121},[95],{"categories":3123},[95],{"categories":3125},[],{"categories":3127},[92],{"categories":3129},[],{"categories":3131},[113],{"categories":3133},[86],{"categories":3135},[134],{"categories":3137},[92],{"categories":3139},[113],{"categories":3141},[113],{"categories":3143},[],{"categories":3145},[113],{"categories":3147},[86],{"categories":3149},[92],{"categories":3151},[],{"categories":3153},[95],{"categories":3155},[95],{"categories":3157},[86],{"categories":3159},[],{"categories":3161},[],{"categories":3163},[],{"categories":3165},[134],{"categories":3167},[95],{"categories":3169},[92],{"categories":3171},[],{"categories":3173},[],{"categories":3175},[],{"categories":3177},[134],{"categories":3179},[],{"categories":3181},[86],{"categories":3183},[],{"categories":3185},[],{"categories":3187},[134],{"categories":3189},[92],{"categories":3191},[113],{"categories":3193},[],{"categories":3195},[150],{"categories":3197},[113],{"categories":3199},[150],{"categories":3201},[92],{"categories":3203},[],{"categories":3205},[],{"categories":3207},[95],{"categories":3209},[],{"categories":3211},[],{"categories":3213},[95],{"categories":3215},[92],{"categories":3217},[],{"categories":3219},[95],{"categories":3221},[113],{"categories":3223},[150],{"categories":3225},[53],{"categories":3227},[95],{"categories":3229},[95],{"categories":3231},[],{"categories":3233},[],{"categories":3235},[],{"categories":3237},[113],{"categories":3239},[],{"categories":3241},[],{"categories":3243},[134],{"categories":3245},[86],{"categories":3247},[],{"categories":3249},[89],{"categories":3251},[150],{"categories":3253},[92],{"categories":3255},[143],{"categories":3257},[86],{"categories":3259},[53],{"categories":3261},[89],{"categories":3263},[143],{"categories":3265},[],{"categories":3267},[],{"categories":3269},[95],{"categories":3271},[86],{"categories":3273},[134],{"categories":3275},[86],{"categories":3277},[95],{"categories":3279},[405],{"categories":3281},[95],{"categories":3283},[],{"categories":3285},[92],{"categories":3287},[113],{"categories":3289},[143],{"categories":3291},[],{"categories":3293},[134],{"categories":3295},[113],{"categories":3297},[86],{"categories":3299},[95],{"categories":3301},[92],{"categories":3303},[89],{"categories":3305},[95,405],{"categories":3307},[95],{"categories":3309},[143],{"categories":3311},[92],{"categories":3313},[53],{"categories":3315},[150],{"categories":3317},[95],{"categories":3319},[],{"categories":3321},[95],{"categories":3323},[92],{"categories":3325},[89],{"categories":3327},[],{"categories":3329},[],{"categories":3331},[92],{"categories":3333},[53],{"categories":3335},[92],{"categories":3337},[],{"categories":3339},[113],{"categories":3341},[],{"categories":3343},[113],{"categories":3345},[143],{"categories":3347},[95],{"categories":3349},[92],{"categories":3351},[150],{"categories":3353},[143],{"categories":3355},[],{"categories":3357},[113],{"categories":3359},[92],{"categories":3361},[],{"categories":3363},[92],{"categories":3365},[95],{"categories":3367},[92],{"categories":3369},[95],{"categories":3371},[92],{"categories":3373},[92],{"categories":3375},[92],{"categories":3377},[92],{"categories":3379},[89],{"categories":3381},[],{"categories":3383},[98],{"categories":3385},[113],{"categories":3387},[92],{"categories":3389},[],{"categories":3391},[143],{"categories":3393},[92],{"categories":3395},[92],{"categories":3397},[95],{"categories":3399},[113],{"categories":3401},[92],{"categories":3403},[92],{"categories":3405},[89],{"categories":3407},[95],{"categories":3409},[134],{"categories":3411},[],{"categories":3413},[53],{"categories":3415},[92],{"categories":3417},[],{"categories":3419},[113],{"categories":3421},[150],{"categories":3423},[],{"categories":3425},[],{"categories":3427},[113],{"categories":3429},[113],{"categories":3431},[150],{"categories":3433},[86],{"categories":3435},[95],{"categories":3437},[95],{"categories":3439},[92],{"categories":3441},[89],{"categories":3443},[],{"categories":3445},[],{"categories":3447},[113],{"categories":3449},[53],{"categories":3451},[143],{"categories":3453},[95],{"categories":3455},[134],{"categories":3457},[53],{"categories":3459},[53],{"categories":3461},[],{"categories":3463},[113],{"categories":3465},[92],{"categories":3467},[92],{"categories":3469},[143],{"categories":3471},[],{"categories":3473},[113],{"categories":3475},[113],{"categories":3477},[113],{"categories":3479},[],{"categories":3481},[95],{"categories":3483},[92],{"categories":3485},[],{"categories":3487},[86],{"categories":3489},[89],{"categories":3491},[],{"categories":3493},[92],{"categories":3495},[92],{"categories":3497},[],{"categories":3499},[143],{"categories":3501},[],{"categories":3503},[],{"categories":3505},[],{"categories":3507},[],{"categories":3509},[92],{"categories":3511},[113],{"categories":3513},[],{"categories":3515},[],{"categories":3517},[92],{"categories":3519},[92],{"categories":3521},[92],{"categories":3523},[53],{"categories":3525},[92],{"categories":3527},[53],{"categories":3529},[],{"categories":3531},[53],{"categories":3533},[53],{"categories":3535},[405],{"categories":3537},[95],{"categories":3539},[143],{"categories":3541},[],{"categories":3543},[],{"categories":3545},[53],{"categories":3547},[143],{"categories":3549},[143],{"categories":3551},[143],{"categories":3553},[],{"categories":3555},[86],{"categories":3557},[143],{"categories":3559},[143],{"categories":3561},[86],{"categories":3563},[143],{"categories":3565},[89],{"categories":3567},[143],{"categories":3569},[143],{"categories":3571},[143],{"categories":3573},[53],{"categories":3575},[113],{"categories":3577},[113],{"categories":3579},[92],{"categories":3581},[143],{"categories":3583},[53],{"categories":3585},[405],{"categories":3587},[53],{"categories":3589},[53],{"categories":3591},[53],{"categories":3593},[],{"categories":3595},[89],{"categories":3597},[],{"categories":3599},[405],{"categories":3601},[143],{"categories":3603},[143],{"categories":3605},[143],{"categories":3607},[95],{"categories":3609},[113,89],{"categories":3611},[53],{"categories":3613},[],{"categories":3615},[],{"categories":3617},[53],{"categories":3619},[],{"categories":3621},[53],{"categories":3623},[113],{"categories":3625},[95],{"categories":3627},[],{"categories":3629},[143],{"categories":3631},[92],{"categories":3633},[134],{"categories":3635},[],{"categories":3637},[92],{"categories":3639},[],{"categories":3641},[113],{"categories":3643},[86],{"categories":3645},[53],{"categories":3647},[],{"categories":3649},[143],{"categories":3651},[113],[3653,3875,3963,4019],{"id":3654,"title":3655,"ai":3656,"body":3661,"categories":3861,"created_at":54,"date_modified":54,"description":47,"extension":55,"faq":54,"featured":56,"kicker_label":54,"meta":3862,"navigation":64,"path":3863,"published_at":3864,"question":54,"scraped_at":54,"seo":3865,"sitemap":3866,"source_id":3867,"source_name":3868,"source_type":72,"source_url":3869,"stem":3870,"tags":3871,"thumbnail_url":54,"tldr":3872,"tweet":54,"unknown_tags":3873,"__hash__":3874},"summaries\u002Fsummaries\u002Ffixing-ml-pipelines-for-databricks-constraints-summary.md","Fixing ML Pipelines for Databricks Constraints",{"provider":7,"model":8,"input_tokens":3657,"output_tokens":3658,"processing_time_ms":3659,"cost_usd":3660},4526,1389,13512,0.0015772,{"type":14,"value":3662,"toc":3855},[3663,3667,3675,3678,3715,3718,3722,3729,3754,3757,3761,3772,3813,3816,3841,3844,3848,3851],[17,3664,3666],{"id":3665},"adapt-storage-to-unity-catalog-for-governed-workflows","Adapt Storage to Unity Catalog for Governed Workflows",[22,3668,3669,3670,3674],{},"Databricks free environments disable public DBFS root, blocking traditional Delta table paths. Shift all data, checkpoints, and artifacts to Unity Catalog Volumes at ",[3671,3672,3673],"code",{},"\u002FVolumes\u002Fworkspace\u002Fecom\u002Fecom_data\u002F",". This mirrors production shifts from open file systems to governed platforms, ensuring compliance without rework.",[22,3676,3677],{},"For MLflow model logging, specify a volume-based temp dir to avoid governance errors:",[3679,3680,3684],"pre",{"className":3681,"code":3682,"language":3683,"meta":47,"style":47},"language-python shiki shiki-themes github-light github-dark","mlflow.spark.log_model(\n    spark_model=model,\n    artifact_path=\"purchase_prediction_model\",\n    dfs_tmpdir=\"\u002FVolumes\u002Fworkspace\u002Fecom\u002Fecom_data\u002Fmlflow_tmp\"\n)\n","python",[3671,3685,3686,3694,3699,3705,3710],{"__ignoreMap":47},[3687,3688,3691],"span",{"class":3689,"line":3690},"line",1,[3687,3692,3693],{},"mlflow.spark.log_model(\n",[3687,3695,3696],{"class":3689,"line":48},[3687,3697,3698],{},"    spark_model=model,\n",[3687,3700,3702],{"class":3689,"line":3701},3,[3687,3703,3704],{},"    artifact_path=\"purchase_prediction_model\",\n",[3687,3706,3707],{"class":3689,"line":61},[3687,3708,3709],{},"    dfs_tmpdir=\"\u002FVolumes\u002Fworkspace\u002Fecom\u002Fecom_data\u002Fmlflow_tmp\"\n",[3687,3711,3712],{"class":3689,"line":60},[3687,3713,3714],{},")\n",[22,3716,3717],{},"Model artifacts must align with platform storage policies, preventing deployment failures in restricted setups.",[17,3719,3721],{"id":3720},"switch-to-micro-batch-streaming-for-reliability","Switch to Micro-Batch Streaming for Reliability",[22,3723,3724,3725,3728],{},"Serverless clusters reject continuous triggers in structured streaming. Use ",[3671,3726,3727],{},"availableNow=True"," for micro-batch processing instead:",[3679,3730,3732],{"className":3681,"code":3731,"language":3683,"meta":47,"style":47},"query = stream_df.writeStream \\\n    .format(\"delta\") \\\n    .trigger(availableNow=True) \\\n    .start(\"\u002FVolumes\u002Fworkspace\u002Fecom\u002Fecom_data\u002Fstream_output\")\n",[3671,3733,3734,3739,3744,3749],{"__ignoreMap":47},[3687,3735,3736],{"class":3689,"line":3690},[3687,3737,3738],{},"query = stream_df.writeStream \\\n",[3687,3740,3741],{"class":3689,"line":48},[3687,3742,3743],{},"    .format(\"delta\") \\\n",[3687,3745,3746],{"class":3689,"line":3701},[3687,3747,3748],{},"    .trigger(availableNow=True) \\\n",[3687,3750,3751],{"class":3689,"line":61},[3687,3752,3753],{},"    .start(\"\u002FVolumes\u002Fworkspace\u002Fecom\u002Fecom_data\u002Fstream_output\")\n",[22,3755,3756],{},"This delivers production stability and cost control, as many orgs prefer micro-batches over true continuous streams to avoid instability on e-commerce event pipelines.",[17,3758,3760],{"id":3759},"handle-spark-ml-quirks-and-scale-with-subsets","Handle Spark ML Quirks and Scale with Subsets",[22,3762,3763,3764,3767,3768,3771],{},"Spark ML stores prediction probabilities as VectorUDT, not arrays, causing ",[3671,3765,3766],{},"INVALID_EXTRACT_BASE_FIELD_TYPE"," errors. Convert with ",[3671,3769,3770],{},"vector_to_array",":",[3679,3773,3775],{"className":3681,"code":3774,"language":3683,"meta":47,"style":47},"from pyspark.ml.functions import vector_to_array\n\npredictions_final = predictions.select(\n    \"user_id\",\n    vector_to_array(\"probability\")[1].alias(\"purchase_probability\"),\n    \"prediction\"\n)\n",[3671,3776,3777,3782,3787,3792,3797,3802,3808],{"__ignoreMap":47},[3687,3778,3779],{"class":3689,"line":3690},[3687,3780,3781],{},"from pyspark.ml.functions import vector_to_array\n",[3687,3783,3784],{"class":3689,"line":48},[3687,3785,3786],{"emptyLinePlaceholder":64},"\n",[3687,3788,3789],{"class":3689,"line":3701},[3687,3790,3791],{},"predictions_final = predictions.select(\n",[3687,3793,3794],{"class":3689,"line":61},[3687,3795,3796],{},"    \"user_id\",\n",[3687,3798,3799],{"class":3689,"line":60},[3687,3800,3801],{},"    vector_to_array(\"probability\")[1].alias(\"purchase_probability\"),\n",[3687,3803,3805],{"class":3689,"line":3804},6,[3687,3806,3807],{},"    \"prediction\"\n",[3687,3809,3811],{"class":3689,"line":3810},7,[3687,3812,3714],{},[22,3814,3815],{},"For recommendation models, massive user\u002Fproduct IDs trigger model size overflow. Train on top users only:",[3679,3817,3819],{"className":3681,"code":3818,"language":3683,"meta":47,"style":47},"top_users = interaction_df.groupBy(\"user_id\") \\\n    .count() \\\n    .orderBy(\"count\", ascending=False) \\\n    .limit(50000)\n",[3671,3820,3821,3826,3831,3836],{"__ignoreMap":47},[3687,3822,3823],{"class":3689,"line":3690},[3687,3824,3825],{},"top_users = interaction_df.groupBy(\"user_id\") \\\n",[3687,3827,3828],{"class":3689,"line":48},[3687,3829,3830],{},"    .count() \\\n",[3687,3832,3833],{"class":3689,"line":3701},[3687,3834,3835],{},"    .orderBy(\"count\", ascending=False) \\\n",[3687,3837,3838],{"class":3689,"line":61},[3687,3839,3840],{},"    .limit(50000)\n",[22,3842,3843],{},"This respects memory limits, turning prototypes into scalable systems without full-dataset forcing.",[17,3845,3847],{"id":3846},"production-truth-constraints-drive-engineering","Production Truth: Constraints Drive Engineering",[22,3849,3850],{},"End-to-end pipelines—from raw e-commerce ingestion, feature engineering, training, MLflow tracking, to inference—evolve through constraint-handling, not textbook ideals. Storage policies, compute limits, framework quirks, and scaling pushback separate prototypes from reliable workflows. Focus on platform adaptations yields complete, governed systems that run in real infrastructure.",[3852,3853,3854],"style",{},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":47,"searchDepth":48,"depth":48,"links":3856},[3857,3858,3859,3860],{"id":3665,"depth":48,"text":3666},{"id":3720,"depth":48,"text":3721},{"id":3759,"depth":48,"text":3760},{"id":3846,"depth":48,"text":3847},[53],{},"\u002Fsummaries\u002Ffixing-ml-pipelines-for-databricks-constraints-summary","2026-04-08 21:21:17",{"title":3655,"description":47},{"loc":3863},"f6260e0e26516379","Learning Data","https:\u002F\u002Funknown","summaries\u002Ffixing-ml-pipelines-for-databricks-constraints-summary",[76,77,79],"Databricks free workspaces block public DBFS, continuous triggers, and large models—use Unity Catalog volumes, micro-batch streaming, vector_to_array for probs, and top-50k user subsets to ship reliably.",[79],"DDKxQzHNGJWdH7cF4yYPqtK0OMYTkcQDCmvx-XltOl0",{"id":3876,"title":3877,"ai":3878,"body":3883,"categories":3931,"created_at":54,"date_modified":54,"description":47,"extension":55,"faq":54,"featured":56,"kicker_label":54,"meta":3932,"navigation":64,"path":3950,"published_at":3951,"question":54,"scraped_at":3952,"seo":3953,"sitemap":3954,"source_id":3955,"source_name":3956,"source_type":72,"source_url":3957,"stem":3958,"tags":3959,"thumbnail_url":54,"tldr":3960,"tweet":54,"unknown_tags":3961,"__hash__":3962},"summaries\u002Fsummaries\u002F56100a2f235e4ed4-production-ml-pipelines-with-zenml-custom-material-summary.md","Production ML Pipelines with ZenML: Custom Materializers & HPO",{"provider":7,"model":8,"input_tokens":3879,"output_tokens":3880,"processing_time_ms":3881,"cost_usd":3882},9247,2138,40785,0.0028959,{"type":14,"value":3884,"toc":3925},[3885,3889,3892,3896,3903,3907,3918,3922],[17,3886,3888],{"id":3887},"custom-materializers-enable-metadata-rich-data-handling","Custom Materializers Enable Metadata-Rich Data Handling",[22,3890,3891],{},"Define DatasetBundle to encapsulate X, y, feature_names, and stats from sklearn's load_breast_cancer (569 samples, 30 features). Pair it with DatasetBundleMaterializer inheriting BaseMaterializer: save() stores X.npy, y.npy, and meta.json with feature_names\u002Fstats; load() reconstructs from files; extract_metadata() computes n_samples, n_features, class_distribution (e.g., {0: 357, 1: 212}). This auto-logs queryable metadata to artifacts, ensuring domain objects serialize seamlessly without pickling issues, while supporting ZenML's reproducibility.",[17,3893,3895],{"id":3894},"modular-steps-log-hyperparameters-and-metrics-at-every-stage","Modular Steps Log Hyperparameters and Metrics at Every Stage",[22,3897,3898,3899,3902],{},"Use @step(enable_cache=True) for load_data() returning Annotated",[3687,3900,3901],{},"DatasetBundle, \"raw_dataset\"",". split_and_scale() performs stratified train_test_split (default test_size=0.2), StandardScaler fit\u002Ftransform, logs train_size\u002Ftest_size via log_metadata(). train_candidate() supports model_type=\"random_forest\"|\"gradient_boosting\"|\"logistic\" with n_estimators=100, max_depth=5 defaults, fits on X_train\u002Fy_train, logs model_type\u002Fhyperparameters. evaluate_candidate() computes accuracy, f1, roc_auc on X_test\u002Fy_test (using predict_proba if available), logs all metrics with label. These steps cache outputs, track lineage, and expose metadata for debugging\u002Fproduction monitoring.",[17,3904,3906],{"id":3905},"fan-out-hpo-and-fan-in-selection-promote-best-model","Fan-Out HPO and Fan-In Selection Promote Best Model",[22,3908,3909,3910,3913,3914,3917],{},"SEARCH_SPACE defines 4 configs: {\"model_type\": \"random_forest\", \"n_estimators\": 50\u002F200, \"max_depth\": 3\u002F7}, {\"gradient_boosting\": 100\u002F3}, {\"logistic\":1\u002F1}. @pipeline(model=PRODUCTION_MODEL) training_pipeline() fans out: load_data → split_and_scale → loop over train_candidate(id=f\"train_",[26,3911,3912],{"i":47},"\") and evaluate_candidate(id=f\"eval","\", label=f\"{type}(n={n},d={d})\"). Fan-in via select_best(): picks max ROC AUC index, logs winning_metrics\u002Fchosen_candidate to model metadata, returns production_model to versioned breast_cancer_classifier (tags=",[3687,3915,3916],{},"\"tutorial\",\"advanced\"","). Generates 8 step runs (4 train+4 eval), automates promotion via Model control plane.",[17,3919,3921],{"id":3920},"client-api-ensures-inspection-caching-and-zero-recompute-reruns","Client API Ensures Inspection, Caching, and Zero-Recompute Reruns",[22,3923,3924],{},"Post-run, Client().get_pipeline_run() shows status, step counts (e.g., 9 steps), aggregated metadata. get_model_version(\"latest\") reveals version.number, linked artifacts, run_metadata (e.g., chosen_candidate). Reload prod_model = get_artifact_version(\"production_model\").load(), verify accuracy_score on stored X_test\u002Fy_test. raw_dataset metadata includes n_samples=569, n_features=30, class_distribution. Rerun hits cache (enable_cache=True), skips recompute. list_pipeline_runs(), list_model_versions(), list_artifact_versions() enable querying; full notebook at GitHub confirms 100% reproducibility without redundant work.",{"title":47,"searchDepth":48,"depth":48,"links":3926},[3927,3928,3929,3930],{"id":3887,"depth":48,"text":3888},{"id":3894,"depth":48,"text":3895},{"id":3905,"depth":48,"text":3906},{"id":3920,"depth":48,"text":3921},[53],{"content_references":3933,"triage":3948},[3934,3939,3944],{"type":3935,"title":3936,"url":3937,"context":3938},"tool","ZenML","https:\u002F\u002Fgithub.com\u002Fzenml-io\u002Fzenml","mentioned",{"type":3940,"title":3941,"url":3942,"context":3943},"other","zenml_advanced_end_to_end_pipeline_Marktechpost.ipynb","https:\u002F\u002Fgithub.com\u002FMarktechpost\u002FAI-Agents-Projects-Tutorials\u002Fblob\u002Fmain\u002FML%20Project%20Codes\u002Fzenml_advanced_end_to_end_pipeline_Marktechpost.ipynb","recommended",{"type":3945,"title":3946,"author":3947,"context":3938},"dataset","breast_cancer","sklearn.datasets",{"relevance":60,"novelty":61,"quality":61,"actionability":60,"composite":62,"reasoning":3949},"Category: AI Automation. The article provides a detailed guide on building production-grade ML pipelines using ZenML, addressing practical aspects like custom materializers and hyperparameter optimization, which are crucial for the target audience. It includes specific steps and code examples that the audience can directly implement in their projects.","\u002Fsummaries\u002F56100a2f235e4ed4-production-ml-pipelines-with-zenml-custom-material-summary","2026-05-04 22:11:37","2026-05-05 16:09:56",{"title":3877,"description":47},{"loc":3950},"56100a2f235e4ed4","MarkTechPost","https:\u002F\u002Fwww.marktechpost.com\u002F2026\u002F05\u002F04\u002Fhow-to-build-an-end-to-end-production-grade-machine-learning-pipeline-with-zenml-including-custom-materializers-metadata-tracking-and-hyperparameter-optimization\u002F","summaries\u002F56100a2f235e4ed4-production-ml-pipelines-with-zenml-custom-material-summary",[76,3683,77,78],"ZenML enables end-to-end ML pipelines with custom DatasetBundle materializers for metadata-rich serialization, fan-out over 4 hyperparameter configs for RandomForest\u002FGradientBoosting\u002FLogisticRegression, fan-in best-model selection by ROC AUC, full artifact tracking, and cache-driven reproducibility on breast cancer dataset.",[],"_jyeZef15FOC-726KyxSOjynaY54SFmoVQfVvb811WU",{"id":3964,"title":3965,"ai":3966,"body":3971,"categories":4000,"created_at":54,"date_modified":54,"description":47,"extension":55,"faq":54,"featured":56,"kicker_label":54,"meta":4001,"navigation":64,"path":4006,"published_at":4007,"question":54,"scraped_at":4008,"seo":4009,"sitemap":4010,"source_id":4011,"source_name":4012,"source_type":72,"source_url":4013,"stem":4014,"tags":4015,"thumbnail_url":54,"tldr":4016,"tweet":54,"unknown_tags":4017,"__hash__":4018},"summaries\u002Fsummaries\u002F2384d22f05952188-nmi-bias-favors-complex-clusters-over-insight-summary.md","NMI Bias Favors Complex Clusters Over Insight",{"provider":7,"model":8,"input_tokens":3967,"output_tokens":3968,"processing_time_ms":3969,"cost_usd":3970},3843,1618,27720,0.0015551,{"type":14,"value":3972,"toc":3996},[3973,3977,3980,3983,3987,3990,3993],[17,3974,3976],{"id":3975},"spotting-nmis-flaw-in-practice","Spotting NMI's Flaw in Practice",[22,3978,3979],{},"When evaluating clustering algorithms, NMI often gives higher scores to models that produce overly complex or over-segmented clusters, even if those clusters lack intuitive sense. This happens because NMI doesn't penalize unnecessary fragmentation enough, prioritizing mathematical alignment over practical insight. In a real clustering project, algorithms with counterintuitive outputs consistently outscored simpler, more meaningful ones—revealing how the metric can mislead developers into favoring flashy but flawed results.",[22,3981,3982],{},"To counter this, cross-check NMI with qualitative reviews of cluster coherence and alternative metrics like Adjusted Rand Index, which better penalize random over-segmentation. This ensures evaluations reflect real-world utility, not just normalized information overlap.",[17,3984,3986],{"id":3985},"consequences-for-ai-trust-and-deployment","Consequences for AI Trust and Deployment",[22,3988,3989],{},"NMI bias propagates errors across domains like medicine (e.g., patient grouping) and hiring (e.g., candidate categorization), where inflated scores lead to over-trusting underperforming models. It skews funding toward hyped algorithms, delays reliable deployments, and erodes confidence in AI outputs for high-stakes decisions.",[22,3991,3992],{},"Fix by combining NMI with domain-specific validation: visualize clusters, test stability under perturbations, and benchmark against baselines. This multi-metric approach grounds assessments in evidence, preventing bias from turning promising papers into production failures.",[22,3994,3995],{},"The content focuses on exposing the issue through anecdote but lacks deeper fixes or data—treat as a prompt to audit your own evals.",{"title":47,"searchDepth":48,"depth":48,"links":3997},[3998,3999],{"id":3975,"depth":48,"text":3976},{"id":3985,"depth":48,"text":3986},[53],{"content_references":4002,"triage":4003},[],{"relevance":61,"novelty":3701,"quality":3701,"actionability":3701,"composite":4004,"reasoning":4005},3.35,"Category: Data Science & Visualization. The article discusses the limitations of Normalized Mutual Information (NMI) in evaluating clustering algorithms, which is relevant to data science practitioners. It provides some actionable advice, such as cross-checking NMI with qualitative reviews and alternative metrics, but lacks depth in practical implementation.","\u002Fsummaries\u002F2384d22f05952188-nmi-bias-favors-complex-clusters-over-insight-summary","2026-05-08 14:11:00","2026-05-09 15:36:58",{"title":3965,"description":47},{"loc":4006},"2384d22f05952188","AI Simplified in Plain English","https:\u002F\u002Fmedium.com\u002Fai-simplified-in-plain-english\u002Fnmi-bias-exposed-1c76d6b366df?source=rss----f37ab7d4e76b---4","summaries\u002F2384d22f05952188-nmi-bias-favors-complex-clusters-over-insight-summary",[76,77],"Normalized Mutual Information (NMI) rewards over-segmentation and complexity in clustering, inflating scores for intuitively poor algorithms and distorting AI evaluations.",[],"bRK9QgU1fKpsZb9OQGPyAtwdso4nsYFrkag-CvgURVo",{"id":4020,"title":4021,"ai":4022,"body":4027,"categories":4064,"created_at":54,"date_modified":54,"description":47,"extension":55,"faq":54,"featured":56,"kicker_label":54,"meta":4065,"navigation":64,"path":4070,"published_at":4071,"question":54,"scraped_at":4072,"seo":4073,"sitemap":4074,"source_id":4075,"source_name":4076,"source_type":72,"source_url":4077,"stem":4078,"tags":4079,"thumbnail_url":54,"tldr":4080,"tweet":54,"unknown_tags":4081,"__hash__":4082},"summaries\u002Fsummaries\u002F896dc8bb5fa4ba77-balance-linear-simplicity-and-nonlinear-flexibilit-summary.md","Balance Linear Simplicity and Nonlinear Flexibility to Avoid Fit Failures",{"provider":7,"model":8,"input_tokens":4023,"output_tokens":4024,"processing_time_ms":4025,"cost_usd":4026},5426,1585,13524,0.00137085,{"type":14,"value":4028,"toc":4059},[4029,4033,4036,4039,4043,4046,4049,4053,4056],[17,4030,4032],{"id":4031},"decision-boundaries-reveal-model-fit-issues","Decision Boundaries Reveal Model Fit Issues",[22,4034,4035],{},"Decision boundaries separate classes in classification: lines in 2D, surfaces in 3D, hyperplanes in higher dimensions. Linear models (logistic regression, linear SVM) use straight boundaries, offering high interpretability but failing on nonlinear data like circles or spirals, causing underfitting—high bias, poor training and test performance. Nonlinear models (decision trees, random forests, kernel SVM, neural networks) create curved, flexible boundaries to capture complex patterns but risk overfitting by fitting noise, yielding high training accuracy yet poor test results due to high variance.",[22,4037,4038],{},"Underfitting happens when a simple linear boundary misses curved data structure, as in blue\u002Fred points separable only by curves. Overfitting occurs with 'snake-like' boundaries hugging every training point, memorizing quirks instead of patterns.",[17,4040,4042],{"id":4041},"bias-variance-tradeoff-guides-optimal-complexity","Bias-Variance Tradeoff Guides Optimal Complexity",[22,4044,4045],{},"Model performance follows a U-shaped curve: simple models have high bias (underfit), complex ones high variance (overfit). Learning curves diagnose: underfitting shows high, flat training\u002Fvalidation errors; overfitting shows low training error diverging from high validation error.",[22,4047,4048],{},"Linear models ensure generalization but underperform on real-world nonlinearity. Nonlinear flexibility models interactions but needs constraints. Goal: optimal complexity capturing structure without noise.",[17,4050,4052],{"id":4051},"practical-fixes-and-real-world-application","Practical Fixes and Real-World Application",[22,4054,4055],{},"Fix underfitting by switching to complex models, adding features, reducing regularization, or training longer. Combat overfitting with simpler models, L1\u002FL2 regularization, dropout, more data, augmentation, early stopping, or cross-validation.",[22,4057,4058],{},"In medical imaging (ultrasound\u002Fradiology), small datasets cause overfitting to patient noise over disease features—use augmentation, regularization, co-teaching. Key: prioritize consistent unseen data performance over training perfection.",{"title":47,"searchDepth":48,"depth":48,"links":4060},[4061,4062,4063],{"id":4031,"depth":48,"text":4032},{"id":4041,"depth":48,"text":4042},{"id":4051,"depth":48,"text":4052},[53],{"content_references":4066,"triage":4067},[],{"relevance":61,"novelty":3701,"quality":61,"actionability":61,"composite":4068,"reasoning":4069},3.8,"Category: Data Science & Visualization. The article discusses the bias-variance tradeoff and practical strategies for addressing underfitting and overfitting, which are critical for AI product builders. It provides actionable fixes like using regularization and data augmentation, making it relevant for developers looking to improve model performance.","\u002Fsummaries\u002F896dc8bb5fa4ba77-balance-linear-simplicity-and-nonlinear-flexibilit-summary","2026-05-07 16:03:54","2026-05-07 16:43:25",{"title":4021,"description":47},{"loc":4070},"896dc8bb5fa4ba77","Data and Beyond","https:\u002F\u002Fmedium.com\u002Fdata-and-beyond\u002Foverfitting-vs-underfitting-understanding-model-complexity-through-linear-and-nonlinear-decision-2a887e05f1f1?source=rss----b680b860beb1---4","summaries\u002F896dc8bb5fa4ba77-balance-linear-simplicity-and-nonlinear-flexibilit-summary",[76,77],"Linear models underfit nonlinear data with rigid straight boundaries; nonlinear models overfit by memorizing noise with wiggly curves. Fix via bias-variance tradeoff for optimal generalization.",[],"J7jr8xlmMvbF1MGWAEBXx4AgWnJAMUbqQUVh9oNTrlk"]