Evaluation of a Blended Course

The information on this page is intended to be a resource for the evaluation of a blended learning course at the postsecondary level.

evaluation.png

Designing, developing & implementing a blended course is an iterative process where evaluation (self-reflection, formal and information feedback from stakeholders, etc.) continually drives the redesign and redevelopment of course components for subsequent implementations.

Why evaluate?

Evaluation is essential for the success of a blended course, because it indicates if the course activities are leading to the intended outcomes, provides feedback related to areas of strength and improvement for the course and can inform decisions about future implementations. In this case, evaluation isn’t necessarily about the general effectiveness of blended learning, for which there is already a growing body of research. Instead, it focuses on a specific blended learning initiative within the context of a particular institution, course, and student body.

Evaluation also provides information to all stakeholders about the effectiveness and impact of a blended learning initiative. This includes students, staff (instructors and other faculty), institutions, government, industry, and general public.

Finally, given that evaluation of teaching at the University of Alberta should be multifaceted and may include “input from administrators, peers, self, undergraduate and graduate students, and alumni” (GFC Section 111.3), the evaluation of a blended course may also provide evidence of teaching excellence when an instructor is being considered for tenure or promotion.

When should an evaluation take place?
Evaluation can be conducted throughout a course, and instructors can make small tweaks, additions, and deletions based on student and faculty feedback. It may be advisable to conduct a more thorough evaluation after a course is completed, so that students can offer their perspective of the course in it’s entirety, and the instructor can use feedback to make modifications prior to the next implementation of the course.
What is being evaluated?

Evaluation can focus on the original issues and motivation behind a blended learning initiative, as well as the goals and objectives for a specific blended course. It can determine how a specific course, and perhaps other courses within an institutional blended learning initiative, succeeded in meeting original goals, such as to:

  • improve student learning
  • improve student retention
  • be on the cutting edge of pedagogical research and strategies
  • ease enrolment issues
  • promote increased enrolment
  • ease classroom space issues
  • increase convenience for students who commute to campus

Evaluation can also reveal the success of the framework of course goals and objectives and subsequent face-to-face and online elements (activities, tasks, etc.) upon which a blended course is built.

How should the evaluation happen?

Basis for Comparison
Most instructors will not have the resources (time or money) to conduct full-fledged research, nor is this necessary. However, there should be an effort to minimize bias and subjectivity as much as possible. Having a basis for comparison, such as intended vs. actual outcomes, or past vs. present experiences, can provide results that are reasonably valid and reliable. For example, a blended course could be compared to an almost identical unblended course (whether in the past or concurrent) in terms of academic level, content, activities, student demographics, etc.

Evaluation Metrics
There are a number of metrics that can provide insight into the effectiveness of a blended course. These metrics may include:

  • Student performance on formative and summative assessments
  • Student proficiency with skills
  • Student course completion rate
  • Student attendance to the face-to-face portion
  • Student engagement with the online portion
  • Student perceptions of the course experience (design, implementation)
  • Instructor perceptions of the course experience (design, implementation)
  • Institutional outcomes from the course (e.g. ease of large enrolment issues, increased enrolment because of pedagogical innovation, etc.)

Evaluation Instruments
In order for evaluation instruments to provide information that is meaningful and useful, they must be carefully chosen, constructed, and implemented. Some tools provide quantitative information, while others provide qualitative information . Potential tools include:

  • Pre-course test and post-course test
  • Standardized testing (within and beyond an institution)
  • Pre-survey and post-survey (questionnaire)
  • Observation
  • Focus group discussion
  • Interview, semi-structured or structured
  • Informal written feedback

Click here to see the instrument that was used to evaluate our different blended learning projects.

Important Considerations
  • Some evaluation will require ethics review and approval, prior to the being implemented. For more information on conducting research in class with students, contact CTL.
  • The privacy, security, and confidentiality of personal information should always be maintained. For more information on privacy and security, or to obtain training, please contact the Information and Privacy Office.