Can I read Statically Contextualizing Large Language Models with Typed Holes on EtoBox?

Statically Contextualizing Large Language Models with Typed Holes by Blinn, Andrew; Li, Xiang; Kim, June Hyung; Omar, Cyrus is a scholarly article available to read on EtoBox.

What is Statically Contextualizing Large Language Models with Typed Holes about?

Large language models (LLMs) have reshaped the landscape of program synthesis. However, contemporary LLM-based code completion systems often hallucinate broken code because they lack appropriate context, particularly when working with definitions not in the training data nor near the cursor. This paper demonstrates that tight integration with the type and binding structure of a language, as exposed by its language server, can address this contextualization problem in a token-efficient manner. In short, we contend that AIs need IDEs, too! In particular, we integrate LLM code generation into the Hazel live program sketching environment. The Hazel Language Server identifies the type and typing context of the hole being filled, even in the presence of errors, ensuring that a meaningful program sketch is always available. This allows prompting with codebase-wide contextual information not lexically local to the cursor, nor necessarily in the same file, but that is likely to be semantically local to the developer's goal. Completions synthesized by the LLM are then iteratively refined via further dialog with the language server. To evaluate these techniques, we introduce MVUBench, a datas

Author

Blinn, Andrew; Li, Xiang; Kim, June Hyung; Omar, Cyrus

Published

2024

Language

EN