Several open source software (OSS) communities promote and participate in initiatives such as summers of code to foster contributions and attract new developers. However, little is known about how successful these initiatives are. As a case study, we analyzed Google Summer of Code (GSoC), which is a three-month program that fosters students' participation in OSS projects. We found that 82% of the studied OSS projects merged at least one students' commit in codebase. When only newcomers are considered, ∼54% of OSS projects merged at least one commit. We also found that ∼23% of newcomers started contributing to GSoC projects before knowing they would be accepted. We also did not find statistical difference between newcomers and students with prior participation in the projects regarding retention time after GSoC, except for 2015 edition. Using survival analysis, we found that ∼40% of students kept contributing longer than a month, while ∼15% contributed longer than a year. OSS communities can take advantage of our results to balance the trade-offs involved in joining this kind of program and to set expectations about how much contribution to expect and for how long students engage.