Can I read TEAFormers: TEnsor-Augmented Transformers for Multi-Dimensional Time Series Forecasting on EtoBox?

TEAFormers: TEnsor-Augmented Transformers for Multi-Dimensional Time Series Forecasting by Kong, Linghang; Chen, Elynn; Chen, Yuzhou; Han, Yuefeng is a scholarly article available to read on EtoBox.

What is TEAFormers: TEnsor-Augmented Transformers for Multi-Dimensional Time Series Forecasting about?

Multi-dimensional time series data, such as matrix and tensor-variate time series, are increasingly prevalent in fields such as economics, finance, and climate science. Traditional Transformer models, though adept with sequential data, do not effectively preserve these multi-dimensional structures, as their internal operations in effect flatten multi-dimensional observations into vectors, thereby losing critical multi-dimensional relationships and patterns. To address this, we introduce the Tensor-Augmented Transformer (TEAFormer), a novel method that incorporates tensor expansion and compression within the Transformer framework to maintain and leverage the inherent multi-dimensional structures, thus reducing computational costs and improving prediction accuracy. The core feature of the TEAFormer, the Tensor-Augmentation (TEA) module, utilizes tensor expansion to enhance multi-view feature learning and tensor compression for efficient information aggregation and reduced computational load. The TEA module is not just a specific model architecture but a versatile component that is highly compatible with the attention mechanism and the encoder-decoder structure of Transformers, making

Author

Kong, Linghang; Chen, Elynn; Chen, Yuzhou; Han, Yuefeng

Published

2024

Language

EN