Can I read Multimodal Information Bottleneck for Deep Reinforcement Learning with Multiple Sensors on EtoBox?

Multimodal Information Bottleneck for Deep Reinforcement Learning with Multiple Sensors by You, Bang; Liu, Huaping is a scholarly article available to read on EtoBox.

What is Multimodal Information Bottleneck for Deep Reinforcement Learning with Multiple Sensors about?

Reinforcement learning has achieved promising results on robotic control tasks but struggles to leverage information effectively from multiple sensory modalities that differ in many characteristics. Recent works construct auxiliary losses based on reconstruction or mutual information to extract joint representations from multiple sensory inputs to improve the sample efficiency and performance of reinforcement learning algorithms. However, the representations learned by these methods could capture information irrelevant to learning a policy and may degrade the performance. We argue that compressing information in the learned joint representations about raw multimodal observations is helpful, and propose a multimodal information bottleneck model to learn task-relevant joint representations from egocentric images and proprioception. Our model compresses and retains the predictive information in multimodal observations for learning a compressed joint representation, which fuses complementary information from visual and proprioceptive feedback and meanwhile filters out task-irrelevant information in raw multimodal observations. We propose to minimize the upper bound of our multimodal in

Author

You, Bang; Liu, Huaping

Published

2024

Language

EN