Assessing Diversity in Creating Seed Set for Snowballing Search for Systematic Literature Review in Software Engineering

Background: Systematic literature reviews (SLRs) require robust search strategies to ensure comprehensive coverage. Although database searches have traditionally been the primary method, snowballing has emerged as an effective alternative strategy in software engineering research. However, the success of snowballing heavily depends on the initial seed set's composition, particularly regarding diversity across authors, publication years, and venues. Objective: This study investigates how different diversity characteristics in seed set creation influence snowballing performance and effectiveness in identifying relevant literature. Method: We conducted replication studies of two existing SLRs, comparing their conventional seed set creation approaches with our diversity-driven methodology, where we systematically incorporated diversity characteristics into constructing the seed sets. Results: Our diversity-based approach demonstrated substantial improvements, with a precision of 0.019 (compared to 0.006 in the original), a relative recall of 0.97 (versus 0.921), and an F-measure of 0.0372 (improving from 0.0119). Conclusions: The empirical evidence suggests that incorporating diversity criteria in seed set creation enhances snowballing efficacy while maintaining comprehensive coverage of relevant literature. This approach offers a systematic and effective method for conducting snowballbased literature reviews in software engineering research.