AUTHOREA
Log in Sign Up Browse Preprints
LOG IN SIGN UP
Jacobo Farray Rodríguez
Jacobo Farray Rodríguez

Public Documents 1
Grading open-ended questions using LLMs and RAG
Jacobo Farray Rodríguez
Antonio Jesús Fernández-García

Jacobo Farray Rodríguez

and 2 more

June 18, 2025
not-yet-known not-yet-known not-yet-known unknown Evaluating open-ended questions is a common and time-consuming task in the education environment. With the continuous and rapid advances in Natural Language Processing (NLP), we have large language models (LLMs) which have been trained with large datasets. The objective of this study is to evaluate the use of these LLMs using retrieval augmented generation (RAG) techniques for the numerical evaluation of open-ended questions with answers of approximately 250 words, evaluating the improvement that the use of the RAG technique brings to this task. For this purpose, a dataset composed of 351 questions evaluated by the teachers of 2 courses, and also the course materials, has been used. We found that, in general, using the RAG technique improves the numerical grades awarded by a LLM, achieving improvements in the MAE of up to 24%. It was also observed that generally the LLM tends to give high grades. Our study concludes with practical guidelines for integrating RAG models into educational settings.

| Powered by Authorea.com

  • Home