This manual describes the process by which researchers collected, scored, and analyzed the thousands of teacher assignments and pieces of student work collected from a sample of Chicago public schools participating in Chicago Annenberg Challenge-funded networks. After providing a brief background of this work, researchers highlight three main aspects of the process: (1) Collection and management of the data; (2) assignment of scores to the data; and (3) statistical analyses of the data.
In March 1996, the UChicago Consortium proposed to evaluate the Chicago Annenberg Challenge’s (CAC) newly funded reform efforts in the Chicago public schools. The evaluation aimed to understand how schools involved in CAC efforts developed as organizations and, in turn, how these developments benefited students. The study aimed to look not just at student outcomes based on standardized test or survey data but to deepen knowledge on student learning by examining collected samples of assignments given and the work students produced in response. By fall 1996, the Chicago Annenberg Research Project (CARP) at the UChicago Consortium was funded and collection of assignments and student work had begun. The initial funding would allow data collection through 1999-2000. However in 2000, the Chicago Annenberg Challenge provided an additional grant to CARP that allowed expanded research with an additional enlarged sample of assignments and student work collected in 2000-01.
Download the Annenberg Scoring Rubrics from 1998
In this manual, we describe the process by which we collected, scored, and analyzed the thousands of teacher assignments and pieces of student work collected from our sample of Chicago public schools participating in Chicago Annenberg Challenge-funded networks. After providing a brief background of this work, we highlight three main aspects of the process: (a) collection and management of the data, (b) assignment of scores to the data, and (c) statistical analyses of the data. In addition, we offer some insights on the overall management of this large project and make observations on lessons we have learned that may prove useful the next time researchers undertake this type of work.
