Optimizing Context for Time-Series Foundation Models

GITCO (Gated Inference-Time Context Optimization) addresses a critical challenge in applying foundation models to structured time-series data: the inefficient or noisy utilization of historical context during inference. Unlike standard LLM approaches that process tokens sequentially, Time-Series Foundation Models (TSFMs) often struggle with long-range dependencies and the signal-to-noise ratio inherent in temporal datasets.

The Gating Mechanism

The core innovation of GITCO is a learned gating mechanism that acts as a filter for input context. By dynamically weighting the importance of historical data points at inference time, the model can suppress irrelevant noise and prioritize high-signal temporal features. This approach allows the model to maintain a more compact and relevant context window, which is essential for maintaining accuracy in complex forecasting and classification tasks where historical data can often be redundant or misleading.

Impact on Performance

By implementing this gated optimization, the authors demonstrate that TSFMs can achieve higher predictive accuracy without the computational overhead of processing the entire historical sequence. This method effectively bridges the gap between static pre-training and the dynamic requirements of real-world time-series forecasting. The approach is particularly relevant for practitioners working with structured data who need to balance model performance with the constraints of limited inference-time compute.