What did they try?

Much of the discussion on teacher quality has been dominated by the emphasis on the development and implementation of standards. To build on its leading role in the movement toward a greater use of standards in education, the Maryland State Department of Education used funding from a PT3 grant to develop Maryland Teacher Technology Standards (MTTS). These standards were developed using a consortium of stakeholders in teacher education. In addition, this consortium developed performance tasks that would provide departments of teacher education and professional development schools with specific activities that would help their students become proficient in these standards.

In MTTS IV, Assessment for Administration and Instruction, teacher candidates use technology to analyze problems and develop data-driven solutions to support student learning and to accomplish school improvement goals. The performance task developed for this standard focused on providing teacher candidates with student level data for analysis in determining the appropriate instructional approaches to meet instructional and school goals.

As the evaluators for this project, ORC Macro conducted an evaluation of the pilot of the performance task developed for MTTS IV. All performance tasks were piloted during previous years of the grant, but MTTS IV was piloted multiple times because of the greater complexity of the task (analysis, using technology for analysis, manipulating data, etc.) and the need to focus on the requirements of the No Child Left Behind Act. The attached presentation (created for the SITE conference in Orlando, Florida, 2006) focuses on the framework used to develop performance tasks and offers a dynamic process for developing and evaluating performance tasks using pilots. Additionally, it provides some of the results of the pilot and a discussion of plans for further piloting of the task.

What worked and why?

• Teacher candidates found the performance task a useful exercise.
• Instructors found the activities designed for the performance task manageable and engaging for students.
• Based on their self-ratings, teacher candidates improved significantly on all proficiency indicators for MTTS IV.

What didn't work and why?

• Based on the results from the scoring tool, fewer than half of the teacher candidates were assessed as being "proficient" after completing the task.
• Based on feedback from the instructors, the dataset received was too large and complex and needed to be simplified for the activities required as part of the task.
• Based on their self-rating, teacher candidates did not have all requisite knowledge, skills, and abilities needed for successful implementation of the task. This indicated that when teacher education programs or instructors include the performance task in a course, a teacher candidate's prior exposure to data analysis will aid that candidate in becoming proficient in the standard.

What will they do next?

• The MTTS IV Performance Task will be modified to reflect the feedback from the instructors, the teacher candidates, and the evaluator, and will be piloted again.
• The performance task will be modified to provide teacher candidates with instructions that provide more guidance on successfully completing the activities and products in the performance task that will be assessed.
• The scoring tool will be reviewed to assure that it aligns with the performance task activities and indicators.
• The task will be piloted by instructors that had not been previously involved in drafting and/or piloting the task in order to get a fresh perspective on the implementation of the performance task.

See attached PowerPoint presentation for additional details.