What to Expect from Code Review Bots on GitHub? A Survey with OSS Maintainers

Mairieli Wessel, Alexander Serebrenik, Igor Wiese, Igor Steinmacher, Marco A. Gerosa

Research output: Chapter in Book/Report/Conference proceedingConference contribution

18 Scopus citations


Software bots are used by Open Source Software (OSS) projects to streamline the code review process. Interfacing between developers and automated services, code review bots report continuous integration failures, code quality checks, and code coverage. However, the impact of such bots on maintenance tasks is still neglected. In this paper, we study how project maintainers experience code review bots. We surveyed 127 maintainers and asked about their expectations and perception of changes incurred by code review bots. Our findings reveal that the most frequent expectations include enhancing the feedback bots provide to developers, reducing the maintenance burden for developers, and enforcing code coverage. While maintainers report that bots satisfied their expectations, they also perceived unexpected effects, such as communication noise and newcomers' dropout. Based on these results, we provide a series of implications for bot developers, as well as insights for future research.

Original languageEnglish (US)
Title of host publicationProceedings - 34th Brazilian Symposium on Software Engineering, SBES 2020
PublisherAssociation for Computing Machinery
Number of pages6
ISBN (Electronic)9781450387538
StatePublished - Oct 21 2020
Event34th Brazilian Symposium on Software Engineering, SBES 2020 - Virtual, Online, Brazil
Duration: Oct 21 2020Oct 23 2020

Publication series

NameACM International Conference Proceeding Series


Conference34th Brazilian Symposium on Software Engineering, SBES 2020
CityVirtual, Online


  • code review
  • open source software
  • pull-based model
  • software bots

ASJC Scopus subject areas

  • Software
  • Human-Computer Interaction
  • Computer Vision and Pattern Recognition
  • Computer Networks and Communications


Dive into the research topics of 'What to Expect from Code Review Bots on GitHub? A Survey with OSS Maintainers'. Together they form a unique fingerprint.

Cite this