Reliability of a computerized scoring routine for an open-ended task

This article asserts the value of open-ended responses for CALL lessons and language tests. Results are presented from a study in which students' notes and recall protocols of computerized reading passages were scored by both people and a computer program. The reliability of human scores was calculated using coefficient alpha; the reliability of human and computer scores was computed using Pearson's product-moment correlation coefficient. Results indicated that the computer program scored reliably with the people, and in much less time.