## Reference model

LLaMA-7B: We took the LLaMA-7B1 model from Meta and kept the same setup as our instructor-FINGPT-7b when reasoning. FinBERT: We got the FinBERT Model from the Hugging Face Model Hub. Before sentiment analysis, the raw data is preprocessed, including word segmentation and filling or truncating the text to fit the maximum input length of the model. After the pre-processing is complete, the data is fed into FinBERT for inference, and sentiment analysis results (positive, negative, or neutral) are obtained for each text input.
ChatGPT: The process for sentiment analysis using OpenAI’s ChatGPT API consists of four steps: API Setup: This involves setting up the OpenAI Python client as an interface to interact with the ChatGPT API. Data preparation: Adjust the data set for inference of the ChatGPT model using the instructions shown in Figure 1. API calls: Due to existing limitations, we use the GPT-3.5 API for requests. GPT-4.0 is currently not accessible programmatically and can only be interacted with through a Web interface. Response interpretation: The API’s response directly contains the emotion of the text. This direct emotion output simplifies the task of sentiment analysis.

## Evaluation and analysis

To evaluate the performance of our model, we tested it on a benchmark financial sentiment analysis dataset and compared the results with FinBERT. Key evaluation metrics include accuracy, which measures the proportion of correct predictions, and F1 scores, which are a harmonic average of accuracy and recall. Based on the evaluation results (see Table 1), our instructor-fine-tuned LLaMA-7B model consistently outperformed FinBERT and the original LLaMA-7B model in accuracy and F1 scores, particularly on all three datasets. Compared with the LLaMA-7B model without instruction fine-tuning, we significantly verify the effectiveness of the instruction fine-tuning method in financial sentiment analysis.
Table 3: Examples and results on the numerical sensitivity dataset