See how a team of program evaluation and data specialists from a regional education service uses Remark Office OMR to make sure that all of their tests, surveys, and evaluations are completed and analyzed in their quick time constraints.
The Capital Area Intermediate Unit (CAIU) Program and Grant Evaluation Services (PAGES) provides a variety of program evaluation and data services for educational agencies. Their services include surveys, workshop evaluations, and assessment data collection.
CAIU PAGES uses Remark Office OMR for a variety of projects including surveys, assessments, and workshop evaluations. Their two most prevalent uses of Remark Office OMR are for surveys and assessments. The most variety occurs with surveys but the greatest volume of scanning occurs with the assessments.
Collecting and analyzing data from a variety of assessments, surveys, and evaluations usually under very tight deadlines and specific reporting requirements. For one current grant evaluation (Title IIB), CAIU PAGES conducts a large scale student science assessment twice a year and also a pre/post assessment of a teacher summer academy for school districts in Appalachia Intermediate Unit 8 in Altoona, PA (www.iu08.org). CAIU PAGES has been the independent evaluator for two different three-year grant cycles.
Twice a year, CAIU PAGES conducts a large scale student science assessment. Depending on the year, they have about 1,700 answer sheets per test administration. CAIU PAGES created a plain paper answer sheet in MS Word®. The assessments can have 20, 40 or 50 questions depending on grade level. For ease of large scale copying, one paper document was created but three different Remark Office OMR scanning templates were created. When they receive the completed assessments, they select the correct template based on the grade level prior to scanning.
The scanning goes very quickly and any difficulties are typically student or participant error. With Remark Office OMR, CAIU PAGES is able to scan and analyze the 1,700 answer sheets within 2-3 days, even with interruptions, for the pre-test and post-test assessment forms. Susan says, “I can’t imagine how we would be able to complete this without software that would scan the answers, grade them, and allow us to run basic statistical information on the results.”
Another piece of this project is a pre/post teacher assessment for a two week summer academy. About 70 teachers are given an assessment on their first day. They spend the next two weeks attending classes and enhancing their skills. On the last day of the two week academy they are given the post assessment. The CAIU team members create customized answer sheets in MS Word to match each test. They travel onsite with a laptop and portable Fujitsu® scanner. In about two hours they scan all the assessments and provide a preliminary report. The preliminary reports are exported and inserted into PowerPoint slides to show the pre- and post-assessment comparison and overview. Final reports are created later and included in the grant reporting documents.
For both parts of this project, Susan uses the Condensed Item Analysis and Test Statistics Reports in Remark Quick Stats®, the built-in analysis component. The Condensed Item Analysis Report shows each question, the answer choices, the frequency of each answer choice and corresponding percent, and a bar chart of the frequency. The Test Statistics Report provides basic statistics on the test as a whole and is useful for gaining a quick look at performance and basic statistical analysis. Susan especially likes the statistics for test reliability that are included, such as the Kuder-Richardson and Coefficient Alpha statistics. These reliability statistics are required for grant reporting requirements. Multiple reports are generated from the assessments with various sorting options, such as by school or grade. Reports are provided to Appalachia IU 8, who shares this with the district. The report information is eventually transferred to an online form provided by the US Department of Education for grant reporting requirements.
CAIU loves the image deskew feature in Remark Office OMR and Remark’s customer support team. Susan says, “The persistence of the support staff to find an answer to the problem and the quick response time beats out every other customer service helpline I have dealt with (even with other unrelated products).”
The CAIU is one of 29 regional educational service agencies in Pennsylvania. CAIU serves Cumberland, Dauphin, Perry, and Northern York Counties in south central PA. CAIU focuses our efforts on our 24 member school districts, 2 area vocational technical schools, over 60 non-public schools, several charter schools and cyber charter schools. In addition, CAIU operates the Capital Area School for the Arts (CASA), Capital Area Adventures (CAA), and the Capital Region Partnership for Career Development (PCD). CAIU also works very closely with partners in higher education.
Several statewide projects are administered by the CAIU, including Classrooms for the Future, Distinguished Educator, eStrategic Plan, and the PA Academic Standards Revision Project.