Kronos: A Foundation Model for the Language of Financial Markets

Reading guide

Problem and motivation

Financial market dynamics are predominantly captured via candlestick (K-line) data comprising Open, High, Low, Close prices, along with Volume and Amount ($D=6$, OHLCVA). While Time Series Foundation Models (TSFMs) have progressed in general forecasting, they frequently underperform specialized, non-pre-trained deep architectures (like iTransformer) on financial tasks. This underperformance stems from two factors:

1. Financial K-lines possess low signal-to-noise ratios, heavy tails, severe non-stationarity, and tight price–volume couplings that conflict with generic temporal inductive biases.
2. Mainstream TSFMs reserve less than 1% (often $<0.01\%$) of their training data for finance, leaving critical downstream quantitative objectives (such as volatility forecasting, return prediction, and synthetic data generation) underserved.

Kronos addresses this gap as a dedicated, open foundation model trained purely on financial market data.

Key ideas

• Discretize-and-Generate Language Modeling: Rather than predicting continuous floats with regression objectives that are fragile to outliers and flash crashes, Kronos quantizes continuous market observations into a discrete vocabulary, framing prediction as next-token generation.
• Binary Spherical Quantization (BSQ): Adapting hyperspherical projection ensures bounded expected reconstruction distortion ($\Vert u-\hat{u}\Vert <\sqrt{2}$), preserving angular information sensitive to fat-tailed microstructure shifts while capping outlier impact.
• Hierarchical Subtoken Decomposition: To avoid the memory overhead of a $2^{20}$ vocabulary, Kronos splits a $k=20$ bit code into $n=2$ equal subtokens ($k_c=k_f=10$, sub-vocabulary $2^{10}=1024$): a coarse subtoken capturing global structure and a fine subtoken encoding residual details.
• Pre-training at Scale: Pre-trained on an curated multi-asset corpus containing over 12 billion observations from 45 global exchanges spanning 7 temporal granularities (from 1-minute to weekly).

Method

Kronos operates in two distinct phases:

1. Instance-Based K-line Tokenization

Each continuous multivariate observation ${\mathbf{x}}_t\in {\mathbb{R}}^6$ is mapped to a discrete code $b_t=[b_t^c,b_t^f]$ where $b_t^c,b_t^f\in \{-1,1{\}}^{k/2}$. The tokenizer (a 3-layer Transformer encoder-decoder) is optimized via: $${\mathcal{L}}_{\text{tokenizer}}={\mathcal{L}}_{\text{coarse}}+{\mathcal{L}}_{\text{fine}}+\lambda {\mathcal{L}}_{\text{quant}}$$ where ${\mathcal{L}}_{\text{coarse}}=\mathbb{E}[\Vert \mathbf{x}-E_{\text{dec}}({\mathbf{b}}^c){\Vert }^2]$, ${\mathcal{L}}_{\text{fine}}=\mathbb{E}[\Vert \mathbf{x}-E_{\text{dec}}(\mathbf{b}){\Vert }^2]$, and ${\mathcal{L}}_{\text{quant}}$ is the BSQ commitment and spherical regularization loss.

2. Hierarchical Autoregressive Modeling

A decoder-only Transformer ($E_{\text{ar}}$) with causal attention and Rotary Position Embeddings (RoPE) processes historical fused token embeddings: $${\mathbf{v}}_i=W_{\text{fuse}}([e_c(b_i^c);e_f(b_i^f)])$$ Conditioned on hidden state ${\mathbf{h}}_t$, generation decomposes using the chain rule: $$p(b_t\mid {\mathbf{b}}_{<t})=p(b_t^c\mid {\mathbf{b}}_{<t})\cdot p(b_t^f\mid {\mathbf{b}}_{<t},b_t^c)$$

• Coarse head: $p(b_t^c\mid {\mathbf{b}}_{<t})=\text{softmax}(W_c{\mathbf{h}}_t)$.
• Fine head: A sampled prediction ${\hat{b}}_t^c$ updates the context via cross-attention: $${\mathbf{h}}_t^{\text{update}}=\text{CrossAttn}(q=e_c({\hat{b}}_t^c),k=v={\mathbf{h}}_t)$$ $$p(b_t^f\mid {\mathbf{b}}_{<t},b_t^c)=\text{softmax}(W_f{\mathbf{h}}_t^{\text{update}})$$ The total pre-training objective is the negative log-likelihood: $${\mathcal{L}}_{\text{ar}}=-{\mathbb{E}}_{\mathbf{b}\sim \mathcal{D}}\sum _{t=1}^T\left[\log p(b_t^c\mid {\mathbf{b}}_{<t})+\log p(b_t^f\mid {\mathbf{b}}_{<t},b_t^c)\right]$$

Results

Across three model variants (${\text{Kronos}}_{small}$ 24.7M, ${\text{Kronos}}_{base}$ 102.3M, ${\text{Kronos}}_{large}$ 499.2M):

• Price Series Forecasting: On 11 global asset test suites, ${\text{Kronos}}_{large}$ achieved average IC of 0.0440 and RankIC of 0.0267, outperforming the best zero-shot TSFM (${\text{Time-MOE}}_{base}$: RankIC 0.0138, a 93% gain) and best full-shot model (TimesNet: RankIC 0.0143, an 87% gain).
• Realized Volatility Forecasting: ${\text{Kronos}}_{large}$ attained an average MAE of 0.0370 and $R^2$ of 0.2624, outperforming specialized econometric baselines (GARCH MAE 0.0405, an approximate 9% error reduction) and TSFMs.
• Synthetic Generation: In discriminative classification score, ${\text{Kronos}}_{large}$ achieved 0.3048 (where 0.50 denotes indistinguishability), beating DiffusionTS (0.2504, a 22% relative improvement in fidelity). In Train-on-Synthetic, Test-on-Real (TSTR), synthetic sequences yielded higher downstream RankIC (0.0282 vs. 0.0102 for DiffusionTS).
• Investment Simulation (Chinese A-Shares): In backtests on the CSI 300 / CSI 800 universes with transaction costs (0.15%), ${\text{Kronos}}_{large}$ delivered an average Annualized Excess Return (AER) of 20.84% and Information Ratio (IR) of 1.6491, exceeding all 25 baselines.

Limitations and open questions

• Maximum Context Length: Capped at 512 tokens due to memory constraints; though multi-frequency alignments partially mitigate this, long-context window scaling remains unexplored.
• Inference Latency Trade-off: The 2-subtoken factorization requires two sequential forward passes per time step ($2\times$ latency compared to parallel prediction), and test-time Monte Carlo rollouts (e.g., $N=10$) multiply inference compute.
• Order Book / L2 Information: Pre-training is restricted to OHLCVA bars; integrating tick-level limit order books or news sentiment directly into token embeddings remains an open research path.