Can I read Synchronous Multi-modal Semantic Communication System with Packet-level Coding on EtoBox?

Synchronous Multi-modal Semantic Communication System with Packet-level Coding by Tian, Yun; Ying, Jingkai; Qin, Zhijin; Jin, Ye; Tao, Xiaoming is a scholarly article available to read on EtoBox.

What is Synchronous Multi-modal Semantic Communication System with Packet-level Coding about?

Although the semantic communication with joint semantic-channel coding design has shown promising performance in transmitting data of different modalities over physical layer channels, the synchronization and packet-level forward error correction of multimodal semantics have not been well studied. Due to the independent design of semantic encoders, synchronizing multimodal features in both the semantic and time domains is a challenging problem. In this paper, we take the facial video and speech transmission as an example and propose a Synchronous Multimodal Semantic Communication System (SyncSC) with Packet-Level Coding. To achieve semantic and time synchronization, 3D Morphable Mode (3DMM) coefficients and text are transmitted as semantics, and we propose a semantic codec that achieves similar quality of reconstruction and synchronization with lower bandwidth, compared to traditional methods. To protect semantic packets under the erasure channel, we propose a packet-Level Forward Error Correction (FEC) method, called PacSC, that maintains a certain visual quality performance even at high packet loss rates. Particularly, for text packets, a text packet loss concealment module, call

Author

Tian, Yun; Ying, Jingkai; Qin, Zhijin; Jin, Ye; Tao, Xiaoming

Published

2024

Language

EN