Can I read SelfExtend: Expanding LLM Contexts on EtoBox?
SelfExtend: Expanding LLM Contexts by shreyaditi is a document available to read on EtoBox.
What is SelfExtend: Expanding LLM Contexts about?
The document presents SelfExtend, a method designed to extend the context window of large language models (LLMs) without the need for fine-tuning, addressing the challenges posed by long input sequences. By utilizing bi-level attention mechanisms, grouped and neighbor attention, SelfExtend effectively maps unseen relative positions to known ones, allowing LLMs to maintain coherence over longer texts. Experimental results demonstrate that SelfExtend can outperform existing fine-tuning methods, highlighting t
- Author
- shreyaditi
- Language
- EN