About this document
2025 Emnlp-Main 218 by prandavbhakta3112 is a document available to read on EtoBox.
The document introduces SolEval, the first repository-level benchmark for evaluating large language models (LLMs) in generating Solidity smart contracts, consisting of 1,507 samples from 28 real-world repositories. The evaluation of 16 LLMs revealed a performance gap, with the best model achieving only 26.29% Pass@10, highlighting the need for improvements in Solidity code generation. Fine-tuning on SolEval significantly enhanced performance, increasing Pass@5 from 16.67% to 58.33%, demonstrating the benchm
- Author
- prandavbhakta3112
- Language
- EN