Does blended learning work?
I’ll bet many of you have asked or been asked that question. Regardless of our role in implementing blended learning, whether we are considering it as an instructional model, implementing it directly with students, supporting its implementation in a state, region, district or school, or anywhere in between, we all want to know whether or not it is improving teaching and learning. Why measure? We all need to know what works.
Does blended learning work for my students, teachers, and community?
This is probably the more important version of the question for you. Unless you are a researcher, my guess is you don’t feel particularly well-equipped to answer this question yourself. In fact, those of us who need that answer the most in order to do our jobs probably feel like we have the least chance of unlocking it. I regularly hear from states and school districts across the country who are eager to measure their blended learning initiatives, because they want to better understand what is and isn’t working, but they simply don’t know where to begin.
The truth is that measuring blended learning in classrooms and schools can not only help answer the broad question about effectiveness, but can also unearth specific, actionable insights for improving your implementation, and ultimately transform teaching and learning for your students. If transformative teaching and learning are a high priority for you, then measurement needs to be, too.
A district guide to blended learning measurement
But how should you measure? This five-step guide from The Learning Accelerator will tell you how districts can and should measure their own blended learning initiatives. It focuses on the teaching and learning that occurs within a blended model at the classroom and school levels so that your results are useful to you and your own needs while also contributing to a general understanding of blended learning’s effectiveness.
The steps included here can be carried out by principals, district administrators, instructional or other coaches, and teachers — anyone who is interested in the blended model at the classroom and school level. Of course, if you also have access to measurement, research or evaluation expertise in your district or school, we encourage you to take advantage of those resources and include them in your measurement planning process. Educators can and should contribute to measurement.
This guide is the result of several conversations with states, school districts and others who are supporting the implementation of blended models across the nation. I expect that as more of you read and use the guide, you will have ideas on how to improve it even further, so please share your thoughts and reactions with me.
My hope is that the guide will demystify measurement and motivate you to dig into the data your blended learning initiative is producing. I encourage you to use it to advance your measurement of blended learning and improve the instructional experience in your classrooms, all while increasing the field’s understanding of blended learning.
Here is a preview of the five steps I suggest you take when considering measuring your blended learning effort; there is much more detail and explanation of each in the guide itself:
1. Research and Evaluation: Two Sides of the Same Coin
Table 1 of the guide outlines the differences and similarities between “research” and “evaluation.” However, dwelling on these labels may be causing more angst than necessary. The measurement labels that we use can be confusing and unnecessarily obstructive, preventing us from making the important decision, which is selecting the measurement approach and design that matches the questions you want to answer and the resources you have.
2. When to Measure
As outlined in a theory of change which we have developed for blended learning, the guide explains how different metrics should be measured at different times, beginning with activities, before progressing to process outputs, and finally outcomes. Measuring out of sequence (or expecting to see changes in outcomes and impacts before understanding how well you are implementing) confuses rather than clarifies, especially because educational outcomes take time to occur.
3. What to Measure
In education today, when most people think about “measurement,” they automatically think about “impact.” As was implied in the “when to measure” step above, we should, in fact, be measuring process metrics, such as activities and outputs, in addition to outcomes and impacts. In other words, it is important to measure what we are doing in order to understand how well we are doing. Further, all of these (activities, outputs, outcomes and impacts) should be closely aligned with our goals for the initiative, and the problem we were initially trying to solve through the initiative.
4. Whom to Measure
The participants and those potentially affected by blended learning are more than our students. Our teachers and other educators also stand to gain or lose when we shift to a blended model. Depending on your goals for blending, other community members (e.g., families) should be included in your measurement plan as well.
5. How to Measure
Finally, when determining how to measure, it is important to consider the reliability and validity of the measures we choose. Remember, the data we collect are only as good as the measures we use to collect them, and the insights we gain are only as good as the data that informed them.
Saro Mohammed is a Partner at The Learning Accelerator (TLA). She has a decade of experience in education research and external evaluations of programs implemented in public, private, and nonprofit settings. Saro leads TLA’s work on measuring impact and evaluating the implementation of blended learning initiatives. She holds a Ph.D. in educational psychology from The University of Texas at Austin and a Bachelor of Science in brain and cognitive sciences from the Massachusetts Institute of Technology.
