AUTHOREA
Log in Sign Up Browse Preprints
LOG IN SIGN UP
Chukwunwike Okafoeze
Chukwunwike Okafoeze

Public Documents 1
Analysing the potential solutions to LLM hallucinations in abstractive text summarisa...
Chukwunwike Okafoeze

Chukwunwike Okafoeze

and 2 more

March 17, 2025
This work explores the implementation of Retrieval-Augmented Generation (RAG) as a method to mitigate hallucinations in abstractive text summarisation using transformer-based BART and T5. The research uses the HaDeS dataset to evaluate how effectively RAG improves the quality and factuality of generated summaries. Quantitative analysis shows that summaries generated with RAG consistently outperform summaries without RAG on a variety of metrics like ROUGE, METEOR, BERTScore, and MoverScore. Interestingly, BART with RAG saw an improvement of 21% over its non-RAG counterpart, whereas T5 with RAG saw an improvement of 17.3%. The research also identifies that a combination of embedding retrieval management and model parameter adjustment is needed in order to get improved summary generation. As great as the success of RAG is, human evaluation of hallucinations in generated summaries finds that standard evaluation metrics are not the best measure for grading hallucinations in generated summaries. The outcomes highlight the effectiveness of RAG as a robust solution for improving abstractive summarisation compared to other approaches when being used alone, while also highlighting areas for future research, including the refinement of retrieval processes and the application of these techniques to larger and more diverse datasets including long text summarisation.

| Powered by Authorea.com

  • Home