About this document
Forecasting Language Model Agent Performance by Avijit Paul is a document available to read on EtoBox.
The document evaluates six forecasting methods for predicting the capabilities of frontier language model agents, focusing on their performance on various benchmarks. A validated two-step approach predicts that by early 2026, non-specialized models will achieve a 54% success rate on SWE-Bench Verified, while state-of-the-art models will reach 87%. The study emphasizes the importance of accurate forecasting for societal preparedness as LMs become more autonomous.
- Author
- Avijit Paul
- Language
- EN