How does a 102M-parameter transformer forecast multivariate time series?

I recently worked through the architecture of t0-alpha, a 101.6M-parameter foundation model for time-series forecasting.

The design choice I found most interesting is that it separates two kinds of reasoning:

• Time attention learns how each variable evolves across time.
• Group attention allows related variables to exchange information.

The rest of the architecture, briefly:

• inputs are split into patches of 32 time steps;
• each patch is embedded into a 512-dimensional representation;
• the model uses 24 transformer blocks: 16 time-attention and 8 group-attention;
• it uses time-aware rotary embeddings, RMSNorm and SwiGLU;
• it predicts nine quantiles for probabilistic forecasting;
• it supports a context window of up to 1,024 time steps.

Its reported aggregate CRPS on GIFT-Eval is 0.4941, roughly in the same range as TimesFM 2.5 and Chronos-2, despite having only around 102M parameters.

I wrote a visual, from-first-principles walkthrough here:

https://towardsdatascience.com/time-series-llms-explained-with-t0-alpha/

I would be interested in other views on two questions:

1. Does separating temporal attention from cross-variable attention provide a useful inductive bias?
2. Can smaller, specialised foundation models remain competitive with much larger forecasting models?

I am also running an iso-parameter GIFT-Eval comparison against rival foundation models and classical baselines, which I plan to write up next.