Can I read Non-linear Pruning for LLMs on EtoBox?
Non-linear Pruning for LLMs by Nairouz Mrabah is a document available to read on EtoBox.
What is Non-linear Pruning for LLMs about?
This paper presents a novel pruning approach for Large Language Models (LLMs) that optimizes the approximation of the attention matrix, addressing the limitations of existing linear approximation methods. The authors provide theoretical guarantees for their Gradient Descent-based optimization method, demonstrating its effectiveness in maintaining model performance while significantly reducing computational costs compared to current state-of-the-art methods. This work lays a new theoretical foundation for pr
- Author
- Nairouz Mrabah
- Language
- EN