Can I read LIRA: A Learning-based Query-aware Partition Framework for Large-scale ANN Search on EtoBox?

LIRA: A Learning-based Query-aware Partition Framework for Large-scale ANN Search by Zeng, Ximu; Deng, Liwei; Chen, Penghao; Chen, Xu; Su, Han; Zheng, Kai is a scholarly article available to read on EtoBox.

What is LIRA: A Learning-based Query-aware Partition Framework for Large-scale ANN Search about?

Approximate nearest neighbor search is fundamental in information retrieval. Previous partition-based methods enhance search efficiency by probing partial partitions, yet they face two common issues. In the query phase, a common strategy is to probe partitions based on the distance ranks of a query to partition centroids, which inevitably probes irrelevant partitions as it ignores data distribution. In the partition construction phase, all partition-based methods face the boundary problem that separates a query's nearest neighbors to multiple partitions, resulting in a long-tailed kNN distribution and degrading the optimal nprobe (i.e., the number of probing partitions). To address this gap, we propose LIRA, a LearnIng-based queRy-aware pArtition framework. Specifically, we propose a probing model to directly probe the partitions containing the kNN of a query, which can reduce probing waste and allow for query-aware probing with nprobe individually. Moreover, we incorporate the probing model into a learning-based redundancy strategy to mitigate the adverse impact of the long-tailed kNN distribution on search efficiency. Extensive experiments on real-world vector datasets demonstrat

Author

Zeng, Ximu; Deng, Liwei; Chen, Penghao; Chen, Xu; Su, Han; Zheng, Kai

Published

2025

Language

EN