Google AI Releases TimesFM-3: A 330M Parameter Zero-Shot Foundation Model For Multivariate Time Series Forecasting
Google Research has launched TimesFM-3, a 330 million parameter time collection basis mannequin that forecasts a number of associated collection in a single ahead cross. Every TimesFM checkpoint via 2.5 was univariate: one collection, its personal historical past, nothing else. TimesFM-3 is pretrained natively for multivariate forecasting on greater than 1 trillion time factors, and accepts a number of targets, previous covariates, and past-future covariates with no task-specific fine-tuning. It takes the highest common rank amongst pretrained basis fashions on GIFT-Eval, fev-bench, and the TIME leaderboard, on each level and probabilistic metrics.
Is it deployable?
Partial, the TimesFM repository code is Apache-2.0, however the TimesFM 3.0 weights ship underneath timesfm-non-commercial-license-v1.0. They are restricted to non-commercial, non-production use. You can benchmark it right this moment. You can not ship it behind a manufacturing forecast API.
What modified
Every TimesFM launch via 2.5 was univariate. It forecast one collection from its personal historical past. Most actual forecasting issues aren’t formed that manner. Google’s example is ice cream gross sales, the place associated product gross sales, foot visitors, climate, promotions, and holidays all transfer the goal.
TimesFM-3 is pretrained natively for multivariate forecasting. It carries 330 million parameters and was pretrained on greater than 1 trillion time factors of actual and artificial collection. Three enter varieties work zero-shot, with no task-specific fine-tuning:
- Multiple targets forecast collectively, with level and quantile outputs for every.
- Past covariates, identified solely traditionally, reminiscent of previous foot visitors.
- Past-future covariates, whose future values are identified, reminiscent of a promotion calendar.
Architecture: patches, then two sorts of consideration
The spine stays a decoder-only transformer. Contiguous factors are grouped into patches of 32 steps, then normalized per collection in order that wildly totally different scales don’t dominate. Target and past-covariate tokens come from a single patch. Past-future covariate tokens use a lookahead trick: the present patch is concatenated with future patches, so the mannequin sees scheduled occasions earlier than they happen.
Tokens then enter a 2D grid and cross via two alternating consideration mechanisms:
- Causal temporal consideration runs horizontally. It is strictly causal and confined to earlier tokens inside the identical collection, which blocks leakage.
- Full variate consideration runs vertically. At a given time step, a token reads each different collection at that step, studying cross-series correlations.
One ahead cross as a substitute of many
Earlier TimesFM variations decoded one patch at a time. That provides latency, compute value, and compounding error. TimesFM-3 makes use of Contiguous Patch Masking, the training-time masking technique launched with TiRex. Masked placeholder tokens are appended for the entire horizon. Targets and previous covariates are masked there. Past-future covariates keep seen, so identified future alerts nonetheless attain the mannequin. The alternating consideration layers fill each masked horizon patch concurrently. Each goal receives 9 quantiles, the tenth via ninetieth percentile, at each horizon step.
Benchmarks
Google evaluated on GIFT-Eval, fev-bench, and the TIME leaderboard, in opposition to Chronos-2, the Toto 2.0 household, and TimesFM-2.5. Among pretrained basis fashions, TimesFM-3 takes the highest common rank on all three, for each level and probabilistic metrics. The package release notes file rank #1 general on fev-bench throughout 100 real-world duties, rank #1 general on TIME throughout 50 area datasets and 98 analysis duties, and rank #1 amongst basis fashions on GIFT-Eval.