[Background:] Experimentation in Software Engineering (ESE) has increased in the last several years as a way to provide evidence by using statistical techniques. These techniques contribute to an auditable and reliable body of knowledge towards evolving a given topic. Therefore, teaching ESE becomes an essential task while disseminating and establishing an experimental culture to both academia and industry. [Aims:] In this paper we seek to understand how ESE has been taught including contents, materials used, strategies employed, and evaluation method applied. We also provide a research agenda on the subject. [Method:] We conducted a web-based questionnaire survey with 31 instructors who teach ESE. [Results:] We see several aspects, such as: ESE is mainly taught in an exclusive course; most of the materials used to teach ESE concepts are from third parties and some of them themselves; most people do not use any type of license; the core materials used to define the content of the courses are papers and books; learning management systems are most often used to share materials with students; key learning practices used are active learning, project-based learning, and problem-based learning; and most instructors assess their students with experimental projects and seminars. Based on such results, we provide and discuss a research agenda to improve teaching of ESE. [Conclusions:] This survey provides results towards planning a research agenda to improve teaching of ESE, thus benefiting instructors, researchers, and practitioners.