Can I read Breaking the Length Barrier: LLM-Enhanced CTR Prediction in Long Textual User Behaviors on EtoBox?

Breaking the Length Barrier: LLM-Enhanced CTR Prediction in Long Textual User Behaviors by Geng, Binzong; Huan, Zhaoxin; Zhang, Xiaolu; He, Yong; Zhang, Liang; Yuan, Fajie; Zhou, Jun; Mo, Linjian is a scholarly article available to read on EtoBox.

What is Breaking the Length Barrier: LLM-Enhanced CTR Prediction in Long Textual User Behaviors about?

With the rise of large language models (LLMs), recent works have leveraged LLMs to improve the performance of click-through rate (CTR) prediction. However, we argue that a critical obstacle remains in deploying LLMs for practical use: the efficiency of LLMs when processing long textual user behaviors. As user sequences grow longer, the current efficiency of LLMs is inadequate for training on billions of users and items. To break through the efficiency barrier of LLMs, we propose Behavior Aggregated Hierarchical Encoding (BAHE) to enhance the efficiency of LLM-based CTR modeling. Specifically, BAHE proposes a novel hierarchical architecture that decouples the encoding of user behaviors from inter-behavior interactions. Firstly, to prevent computational redundancy from repeated encoding of identical user behaviors, BAHE employs the LLM's pre-trained shallow layers to extract embeddings of the most granular, atomic user behaviors from extensive user sequences and stores them in the offline database. Subsequently, the deeper, trainable layers of the LLM facilitate intricate inter-behavior interactions, thereby generating comprehensive user embeddings. This separation allows the learnin

Author

Geng, Binzong; Huan, Zhaoxin; Zhang, Xiaolu; He, Yong; Zhang, Liang; Yuan, Fajie; Zhou, Jun; Mo, Linjian

Published

2024

Language

EN