Can I read Evaluating the Performance of LLMs on Technical Language Processing tasks on EtoBox?

Evaluating the Performance of LLMs on Technical Language Processing tasks by Kernycky, Andrew; Coleman, David; Spence, Christopher; Das, Udayan is a scholarly article available to read on EtoBox.

What is Evaluating the Performance of LLMs on Technical Language Processing tasks about?

In this paper we present the results of an evaluation study of the perfor-mance of LLMs on Technical Language Processing tasks. Humans are often confronted with tasks in which they have to gather information from dispar-ate sources and require making sense of large bodies of text. These tasks can be significantly complex for humans and often require deep study including rereading portions of a text. Towards simplifying the task of gathering in-formation we evaluated LLMs with chat interfaces for their ability to provide answers to standard questions that a human can be expected to answer based on their reading of a body of text. The body of text under study is Title 47 of the United States Code of Federal Regulations (CFR) which describes regula-tions for commercial telecommunications as governed by the Federal Com-munications Commission (FCC). This has been a body of text of interest be-cause our larger research concerns the issue of making sense of information related to Wireless Spectrum Governance and usage in an automated manner to support Dynamic Spectrum Access. The information concerning this wireless spectrum domain is found in many disparate sources, with Title 47 of the

Author

Kernycky, Andrew; Coleman, David; Spence, Christopher; Das, Udayan

Published

2024

Language

EN