Skip to content

Opening book details…

Can I read Hierarchical corpus encoder: Fusing generative retrieval and dense indices on EtoBox?

Hierarchical corpus encoder: Fusing generative retrieval and dense indices by Chen, Tongfei; Sharma, Ankita; Pauls, Adam; Van Durme, Benjamin is a scholarly article available to read on EtoBox.

What is Hierarchical corpus encoder: Fusing generative retrieval and dense indices about?

Generative retrieval employs sequence models for conditional generation of document IDs based on a query (DSI (Tay et al., 2022); NCI (Wang et al., 2022); inter alia). While this has led to improved performance in zero-shot retrieval, it is a challenge to support documents not seen during training. We identify the performance of generative retrieval lies in contrastive training between sibling nodes in a document hierarchy. This motivates our proposal, the hierarchical corpus encoder (HCE), which can be supported by traditional dense encoders. Our experiments show that HCE achieves superior results than generative retrieval models under both unsupervised zero-shot and supervised settings, while also allowing the easy addition and removal of documents to the index.

Author
Chen, Tongfei; Sharma, Ankita; Pauls, Adam; Van Durme, Benjamin
Published
2025
Language
EN

More by Chen, Tongfei; Sharma, Ankita; Pauls, Adam; Van Durme, Benjamin

Browse all works by Chen, Tongfei; Sharma, Ankita; Pauls, Adam; Van Durme, Benjamin