TY - GEN
T1 - Applying Large Language Models to Issue Classification
AU - Aracena, Gabriel
AU - Luster, Kyle
AU - Santos, Fabio
AU - Steinmacher, Igor
AU - Gerosa, Marco A.
N1 - Publisher Copyright:
© 2024 ACM.
PY - 2024
Y1 - 2024
N2 - Effective prioritization of issue reports in software engineering helps to optimize resource allocation and information recovery. However, manual issue classification is laborious and lacks scalability. As an alternative, many open source software (OSS) projects employ automated processes for this task, yet this relies on substantial datasets for adequate training. This research investigates an automated approach to issue classification based on Generative Pre-Trained Transformers (GPT). By leveraging the capabilities of such models, we aim to develop a robust system for prioritizing issue reports accurately, mitigating the necessity for extensive training data while maintaining reliability. In our research, we have developed a GPT-based approach to label issues accurately with a reduced training dataset. By reducing reliance on massive data requirements and focusing on few-shot fine-Tuning, we found a more accessible and efficient solution for issue classification. Our model predicted issue labels in individual projects up to 93.2 \% in precision, 95 \% in recall, and 89.3 \% in F1-score.
AB - Effective prioritization of issue reports in software engineering helps to optimize resource allocation and information recovery. However, manual issue classification is laborious and lacks scalability. As an alternative, many open source software (OSS) projects employ automated processes for this task, yet this relies on substantial datasets for adequate training. This research investigates an automated approach to issue classification based on Generative Pre-Trained Transformers (GPT). By leveraging the capabilities of such models, we aim to develop a robust system for prioritizing issue reports accurately, mitigating the necessity for extensive training data while maintaining reliability. In our research, we have developed a GPT-based approach to label issues accurately with a reduced training dataset. By reducing reliance on massive data requirements and focusing on few-shot fine-Tuning, we found a more accessible and efficient solution for issue classification. Our model predicted issue labels in individual projects up to 93.2 \% in precision, 95 \% in recall, and 89.3 \% in F1-score.
KW - Empirical Study
KW - Issue Report Classification
KW - Labeling
KW - Large Language Model
KW - Multi-class Classification
KW - Natural Language Processing
KW - Software Engineering
UR - http://www.scopus.com/inward/record.url?scp=85203836249&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85203836249&partnerID=8YFLogxK
U2 - 10.1145/3643787.3648043
DO - 10.1145/3643787.3648043
M3 - Conference contribution
AN - SCOPUS:85203836249
T3 - Proceedings - 2024 ACM/IEEE International Workshop on NL-Based Software Engineering, NLBSE 2024
SP - 57
EP - 60
BT - Proceedings - 2024 ACM/IEEE International Workshop on NL-Based Software Engineering, NLBSE 2024
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 3rd ACM/IEEE International Workshop on NL-Based Software Engineering, NLBSE 2024
Y2 - 20 April 2024
ER -