loading page

LLMs for Science: Usage for Code Generation and Data Analysis
  • +1
  • Mohamed Nejjar,
  • Luca Zacharias,
  • Fabian Stiehle,
  • Ingo Weber
Mohamed Nejjar
Technical University of Munich
Author Profile
Luca Zacharias
Technical University of Munich
Author Profile
Fabian Stiehle
Technical University of Munich

Corresponding Author:fabian.stiehle@tum.de

Author Profile
Ingo Weber
Technical University of Munich
Author Profile

Abstract

Large language models (LLMs) have been touted to enable increased productivity in many areas of today’s work life. Scientific research as an area of work is no exception: the potential of LLM-based tools to assist in the daily work of scientists has become a highly discussed topic across disciplines. However, we are only at the very onset of this subject of study. It is still unclear how the potential of LLMs will materialise in research practice. With this study, we give first empirical evidence on the use of LLMs in the research process. We have investigated a set of use cases for LLM-based tools in scientific research, and conducted a first study to assess to which degree current tools are helpful. In this paper we report specifically on use cases related to software engineering; specifically, on generating application code and developing scripts for data analytics and visualisation. While we studied seemingly simple use cases, results across tools differ significantly. Our results highlight the promise of LLM-based tools in general, yet we also observe various issues, particularly regarding the integrity of the output these tools provide.
Submitted to Journal of Software: Evolution and Process
Submission Checks Completed
Assigned to Editor
Reviewer(s) Assigned
15 Jul 2024Review(s) Completed, Editorial Evaluation Pending
16 Jul 2024Editorial Decision: Revise Minor
17 Jul 20241st Revision Received
18 Jul 2024Submission Checks Completed
18 Jul 2024Assigned to Editor
18 Jul 2024Reviewer(s) Assigned
18 Jul 2024Review(s) Completed, Editorial Evaluation Pending
18 Jul 2024Editorial Decision: Accept