User Stories: Does ChatGPT Do It Better?

In agile software development, user stories play a central role in defining system requirements, fostering communication, and guiding development efforts. Despite their importance, they are often poorly written, exhibiting quality defects that hinder project outcomes and reduce team efficiency. Manual methods for creating user stories are time-consuming and prone to errors and inconsistencies. Advancements in Large Language Models (LLMs), such as ChatGPT, present a promising avenue for automating and improving this process. This research explores whether user stories generated by ChatGPT, using prompting techniques, achieve higher quality than those created manually by humans. User stories were assessed using the Quality User Story (QUS) framework. We conducted two empirical studies to address this. The first study compared manually created user stories with those generated by ChatGPT through free-form prompt. This study involved 30 participants and found no statistically significant difference between the two methods. The second study compared free-form prompt with meta-few-shot prompt, demonstrating that the latter outperformed both, achieving higher consistency and semantic quality with an efficiency calculated based on the success rate of 88.57%. These findings highlight the potential of LLMs with prompting techniques to enhance user story generation, offering a reliable and effective alternative to traditional methods.