A randomized trial comparing mail versus in-office distribution of the CAHPS clinician and group survey

Objective. To assess the effect of survey distribution protocol (mail versus handout) on data quality and measurement of patient care experiences. Data Sources/Study Setting. Multisite randomized trial of survey distribution protocols. Analytic sample included 2,477 patients of 15 clinicians at three practice sites in New York State. Data Collection/Extraction Methods. Mail and handout distribution modes were alternated weekly at each site for 6 weeks. Principal Findings. Handout protocols yielded an incomplete distribution rate (74 percent) and lower overall response rates (40 percent versus 58 percent) compared with mail. Handout distribution rates decreased over time and resulted in more favorable survey scores compared with mailed surveys. There were significant mode-physician interaction effects, indicating that data cannot simply be pooled and adjusted for mode. Conclusions. In-office survey distribution has the potential to bias measurement and comparison of physicians and sites on patient care experiences. Incomplete distribution rates observed in-office, together with between-office differences in distribution rates and declining rates over time suggest staff may be burdened by the process and selective in their choice of patients. Further testing with a larger physician and site sample is important to definitively establish the potential role for in-office distribution in obtaining reliable, valid assessment of patient care experiences.