Prioritize Revenue Over Vanity Metrics with Full-Funnel Analysis

Raw clicks mislead—Test campaign's higher CTR (10.2% vs Control's 5.1%) attracts more traffic from fewer impressions, but mid-funnel reveals issues: lower view rate (30.8% vs 36.5%) and cart rate (47.4% vs 66.9%). It recovers with superior purchase rate (61.8% vs 45.7%), yielding more purchases despite drop-offs. Compute funnel rates as CTR = clicks/impressions, view_rate = view_content/clicks, etc., to spot bottlenecks like view-to-cart, where Test loses users due to potential landing page friction or ad-product mismatch. Statistical validation via proportions_ztest on purchases/clicks confirms significance (Z-stat 11.84, p~2.46e-32 <0.05), proving Test's edge isn't chance.

Trade Efficiency for Scale: Financial Metrics Guide Decisions

Test generates $781,850 revenue and $704,958 profit vs Control's $758,050 and $691,232, assuming AOV=$50 (revenue = purchases * 50, profit = revenue - spend). Yet Control wins efficiency: higher ROI (10.6 vs 9.3), lower CAC ($4.41 vs $4.92, spend/purchases). Break-even favors Control (~2.8 customers vs 3.3). Time trends show Test's inconsistent outperformance, stronger on weekends (60% purchase rate vs Control's 33% drop) and high-reach segments (67.8% vs 48.3%). Use groupby('campaign') on revenue/spend/profit for aggregates; ROI = (revenue - spend)/spend.

Scale Strategically: Scenarios Prove Test's Resilience

What-if scaling (20% spend up, 10% revenue up) projects Test at ~$767K profit vs Control's $753K. Scenario analysis (AOV multipliers: 0.95 pessimistic, 1.0 realistic, 1.1 optimistic) shows Test superior in all ($663K-$823K profit range vs Control's $629K-$781K). Recommendation: Scale Test for growth (max profit), retain Control for efficiency benchmark/hybrid. Fix mid-funnel first (view-to-cart bottleneck) via landing page tweaks; set guardrails like max CAC, min ROI; prioritize high-reach/weekend segments. Executive dashboard: bar plots of ROI/profit/CAC drive one-look decisions—Test justifies higher costs for total value.