Can I read Integrating Multi-Modal Input Token Mixer Into Mamba-Based Decision Models: Decision MetaMamba on EtoBox?

Integrating Multi-Modal Input Token Mixer Into Mamba-Based Decision Models: Decision MetaMamba by Kim, Wall is a scholarly article available to read on EtoBox.

What is Integrating Multi-Modal Input Token Mixer Into Mamba-Based Decision Models: Decision MetaMamba about?

Sequence modeling with State Space models (SSMs) has demonstrated performance surpassing that of Transformers in various tasks, raising expectations for their potential to outperform the Decision Transformer and its enhanced variants in offline reinforcement learning (RL). However, decision models based on Mamba, a state-of-the-art SSM, failed to achieve superior performance compared to these enhanced Decision Transformers. We hypothesize that this limitation arises from information loss during the selective scanning phase. To address this, we propose the Decision MetaMamba (DMM), which augments Mamba with a token mixer in its input layer. This mixer explicitly accounts for the multimodal nature of offline RL inputs, comprising state, action, and return-to-go. The DMM demonstrates improved performance while significantly reducing parameter count compared to prior models. Notably, similar performance gains were achieved using a simple linear token mixer, emphasizing the importance of preserving information from proximate time steps rather than the specific design of the token mixer itself. This novel modification to Mamba's input layer represents a departure from conventional timest

Author

Kim, Wall

Published

2024

Language

EN