Ben Chen

and 1 more

This paper explores the role of Large Language Models (LLMs) in promoting sustainable behavior, specifically in overcoming procrastination. Despite widespread recognition of the need for sustainable behavior change, individuals often struggle to break free from entrenched, unsustainable habits. LLMs, such as OpenAI’s GPT-4, represent a significant breakthrough in artificial intelligence and are increasingly used in behavior change interventions. This study introduces ProactiMate, a chatbot built using Motivational Interviewing (MI) principles and a Chain of Models approach for prompt engineering, designed to help users combat procrastination. Our research compares four LLMs (GPT-3.5 Turbo, LLaMA-3.2, Qwen-2.5, and SmolLM-1.7B) for output influence on procrastination avoidance, and assesses the impact of hyperparameters (temperature and top-p values) on procrastination avoidance. The findings reveal that GPT-3.5 outperforms other models across various evaluation metrics, and higher temperature and top-p values lead to more effective procrastination avoidance from automatic evaluation. According to expert evaluations, Qwen-2.5 and GPT-3.5 Turbo demonstrated notable effectiveness in fostering user engagement and motivation for addressing procrastination, with GPT-3.5 Turbo particularly distinguished by its capacity to provide strategies that help maintain long-term motivation. And GPT’s output aligns well with both automatic evaluation metrics and human evaluation. The results provide insights into the most effective ways to use LLMs in chatbot design, offering solutions for future usability testing.