Evaluation

Evaluation typically refers to the means by which the effectiveness or utility of a course or of components within a course are analyzed. We often think of evaluation as something that happens at the end of the course. This idea of “evaluation as autopsy” led John Sener to ask, “instead of waiting until the patient has died, why not identify and address potential problems before the course is over?” [1] In reality, evaluation should be happening at every stage of the design process. As designers, we should invite feedback on our initial plans, on our design blueprint, and our development work long before students ever take the course for the first time. Even then, we want to invite their feedback on the pilot version of the course.

As designers, we want to create evaluation strategies that:

1. Help us improve the quality of the course,
2. Aid in student growth and learning,
3. Support faculty professional development,
4. Equip our educational organization to become a learning organization,
5. Result in communication that is valuable to stakeholders.

1. Quality Improvement

We also want to invite feedback from our peers as part of our design process. We also want to invite student feedback that enables us to create a plan for improving and revising the course. The learners’ perspective on the course is valuable as we consider ways to improve it for future offerings. Running a pilot of a new course is a valuable way to seek feedback from learners while the course is still in development.

2. Student Growth and Learning

Our end-of-course surveys should include questions about the course, but ideally they should also ask students to evaluate themselves as learners. A well-crafted evaluation strategy can lead learners to take responsibility for their own learning.

3. Professional Development

We also want to design evaluation strategies that enable formative feedback that supports teacher growth and development. When we invite students to give feedback to the teacher, we want to invite their contributions in a manner that is helpful for the teacher to continue to grow in her craft.

4. Organizational Learning

Evaluation is also useful for organizational learning. Learning organizations are always seeking new ways to grow and improve. Getting the right feedback at crucial points is an important part of the process. Evaluation can be a catalyst for growth and improvement if the organization is willing to receive feedback, internalize it, and learn from it. The learning then becomes part of the organizational memory and has an impact on future decisions.

5. Communication with Stakeholders

Organizations have a responsibility to communicate with key stakeholders who are invested in the organization. Feedback from evaluations provides content that needs to be shared with the appropriate parties. The level of detail that is needed will vary by stakeholder group (board members, faculty, students, alumni, etc.). Providing that feedback regularly and synthesizing it at the appropriate level is an important part of an organization’s overall communication strategy.

One Final Suggestion

As a starting point for designing an evaluation strategy, we would invite you to examine a Course Design Review scorecard published by the State University of New York. If you are a Horizon partner, you can access the scorecard via Horizon’s Moodle Resources. If you are not, you can download a copy of this document via the SUNY website.

This scorecard is a helpful place to begin thinking about how you might evaluate your course design. You will see that the points of evaluation are organized around 6 design areas:

1. Course Overview and Information
2. Course Technology and Tools
3. Design and Layout
4. Content and Activities
5. Interaction
6. Assessment and Feedback

Secondly, this scorecard is helpful in that a website has been developed to help explain each item in the scorecard. You can either click on each point of evaluation in the scorecard itself, which will open the accompanying website page, or once on the State University of New York website, you can simply scroll point by point by clicking on the forward arrow.