Cron's Silent Failures Demand Better Orchestration

Cron provides one bit of feedback—exit zero or non-zero—leaving retries, overlaps, and data integrity to manual hacks. In a 15-line MLB stats fetch script run nightly at 2am, three failures emerge: (1) requests.raise_for_status() exits on 429 rate limits or timeouts without retry, causing stale data (e.g., 9 missed runs led to dropping a hot player); (2) fixed latest.json output creates races if runs overlap (slow fetch > schedule interval); (3) non-atomic write_text() corrupts files on mid-write crashes (OOM, signals). Patching with loops bloats code, loses state on crashes, and forces log spelunking for history. Outcome: unreliable data for decisions, no audit trail for "what ran at 3am Tuesday?"

Temporal eliminates this by separating orchestration (Workflows: deterministic, own when) from side effects (Activities: fetch/parse/write). State persists in Temporal's history, not process memory, ensuring completion despite reboots.

Workflows + Activities Deliver Crash-Proof Reliability

Define a StatsCollectionWorkflow that calls collect_stats activity with start_to_close_timeout=timedelta(minutes=10) and RetryPolicy(initial_interval=timedelta(seconds=3), backoff_coefficient=2.0, maximum_interval=timedelta(minutes=2), maximum_attempts=8). Retries survive worker crashes—e.g., die on attempt 3, resume at 4. Activity fetches MLB page (proxies optional via env vars for 429s/geo-blocks), extracts statsDatatable JSON via string search (needle='stats: {"statsDatatable"'), sanitizes HTML tags, picks current season row, and writes atomically: tmp file + replace() prevents partial JSON. Filename uses workflow_id__run_id.json (e.g., stats-manual-abc123__run456.json), enabling diffs across runs and eliminating races.

Sync activities (not async) suit blocking I/O like requests.get(timeout=60); they run in thread pools without blocking event loops. Workers scale horizontally, polling task_queue without touching scheduling.

Schedules and UI Provide Production-Grade Control

Schedule with cron_expressions=[cron], ScheduleOverlapPolicy.SKIP prevents overlaps—if a 12min run bleeds into a 15min schedule, next tick skips until free. Idempotent create/update: describe(), catch NOT_FOUND, then create_schedule or update. Local dev: temporal server start-dev, uv run temporal-cron-worker, uv run temporal-cron-schedule (default 15min cron).

UI at localhost:8233 shows timelines: inputs/outputs per attempt, retry details (e.g., 429 on #2, success #3), full event history (schedule, activity start/complete, results). Replaces stdout guessing with searchable audits—debug failures without logs.

Production: Use Temporal Cloud/self-host, add secrets/logging/metrics. Pairs with proxies (Bright Data) for flaky networks; Temporal owns retries/timeouts, proxy hardens paths. Pattern scales to work ingest jobs: same Workflow/Activity for more surface area.