Can I read Multimodal Audio-based Disease Prediction with Transformer-based Hierarchical Fusion Network on EtoBox?

Multimodal Audio-based Disease Prediction with Transformer-based Hierarchical Fusion Network by Cai, Jinjin; Wang, Ruiqi; Zhao, Dezhong; Yuan, Ziqin; McKenna, Victoria; Friedman, Aaron; Foot, Rachel; Storey, Susan; Boente, Ryan; Vhaduri, Sudip; Min, Byung-Cheol is a scholarly article available to read on EtoBox.

What is Multimodal Audio-based Disease Prediction with Transformer-based Hierarchical Fusion Network about?

Audio-based disease prediction is emerging as a promising supplement to traditional medical diagnosis methods, facilitating early, convenient, and non-invasive disease detection and prevention. Multimodal fusion, which integrates features from various domains within or across bio-acoustic modalities, has proven effective in enhancing diagnostic performance. However, most existing methods in the field employ unilateral fusion strategies that focus solely on either intra-modal or inter-modal fusion. This approach limits the full exploitation of the complementary nature of diverse acoustic feature domains and bio-acoustic modalities. Additionally, the inadequate and isolated exploration of latent dependencies within modality-specific and modality-shared spaces curtails their capacity to manage the inherent heterogeneity in multimodal data. To fill these gaps, we propose a transformer-based hierarchical fusion network designed for general multimodal audio-based disease prediction. Specifically, we seamlessly integrate intra-modal and inter-modal fusion in a hierarchical manner and proficiently encode the necessary intra-modal and inter-modal complementary correlations, respectively. Co

Author

Cai, Jinjin; Wang, Ruiqi; Zhao, Dezhong; Yuan, Ziqin; McKenna, Victoria; Friedman, Aaron; Foot, Rachel; Storey, Susan; Boente, Ryan; Vhaduri, Sudip; Min, Byung-Cheol

Published

2024

Language

EN