Speaking Numbers to LLMs: Multi-Wavelet Number Embeddings for Time Series Forecasting
Integrating continuous numerical data into Large Language Models (LLMs) remains a fundamental bottleneck for context-aware time series forecasting, as standard tokenization pipelines are optimized for discrete text rather than continuous scales. To bridge this gap, Defu Cao, Zijie Lei, Muyan Weng, Jiao Sun, and Yan Liu developed TempoWave, published at the International Joint Conference on Artificial Intelligence (IJCAI) 2026. This framework is designed specifically for machine learning engineers and researchers developing multimodal or context-enriched forecasting models, providing a plug-and-play temporal wavelet digit interface that redefines how LLMs process numerical inputs.
TempoWave operates by mapping individual scalar observations directly into digit-wise embeddings constructed from multi-wavelet, multi-scale coefficients. Instead of relying on traditional text tokenizers that often split numbers arbitrarily and disrupt spatial and numerical ordering, TempoWave overrides standard token representations directly within the embedding layer. This architecture captures both fine-grained, high-frequency local fluctuations and broad, low-frequency macro global structures in a mathematically rigorous, transformer-compatible format. Crucially, this multi-resolution representation preserves numerical formatting and digit identity while remaining robust to standard preprocessing operations like data normalization. Evaluated across five context-enriched forecasting benchmarks, TempoWave demonstrated consistent performance gains over conventional numeric tokenization schemes, setting a new state-of-the-art for LLM-based forecasters.
Going forward, this work establishes the numeric-linguistic interface as a primary performance bottleneck in multimodal LLMs and offers a viable pathway for aligning continuous physical measurements with discrete text reasoning. By decoupling numerical encoding from the limitations of subword tokenization, TempoWave enables more reliable integration of heterogeneous textual signals—such as financial news or weather reports—with raw numerical time series, promising a future of highly contextual and precise predictive modeling. Note that this analysis is based on the published abstract and metadata of the research paper.