Contrast

MVLRI Research in Review: K-12 Online Program Evaluation, Quality, and Policy

Published on September 29, 2020
Written By: 

Kristen DeBrulerMichigan Virtual Learning Research Institute

|

Christa GreenMichigan Virtual Learning Research Institute

While there is widespread agreement about the value of online courses, quality remains a significant concern. Educational stakeholders largely agree that there should be clear expectations and accountability for online programs and course providers however practice of this remains fragmented and inconsistent.

Suggested Citation

DeBruler, K. & Green, C. (2020). MVLRI research in review: K-12 online program evaluation, quality, and policy. Michigan Virtual. https://michiganvirtual.org/research/publications/mvlri-research-in-review:-k-12-online-program-evaluation,-quality,-and-policy/

Introduction

Since its creation in 2013 through 2020, the Michigan Virtual Learning Research Institute (MVLRI) at Michigan Virtual published approximately 20 research blogs and 75 research reports. This total does not represent everything published by MVLRI but rather only those publications including original research on K-12 blended and online learning. The nearly 100 resources represent research conducted internally by MVLRI staff, research conducted by partners at universities, colleges, and educational organizations, and covers a vast range of topics including, but not limited to, K-12 online best practices, online student motivation, K-12 blended teaching and professional development, and K-12 special populations. 

This body of work is extensive, and while there is tremendous value in each individual publication, there is also value in how that work fits with other similar research and the narrative that emerges from the collective understanding. Toward this end, MVLRI sought to identify, review, and synthesize the original research published in the past 6 years. Again, not every blog or report published via the MVLRI.org website was included, only those containing original research. 

Out of the synthesis of resources, 10 main themes emerged. Each theme is presented individually in the interest of brevity. A full reference list is provided at the end of this document noting the resources that contributed to this report. 

Methods

Resources for inclusion in the synthesis were identified through the MVLRI.org website in the “Publications” and “Blogs” sections. All published blogs and reports were assessed to determine if they included original research. Those that did were included for synthesis. Once the approximately 100 resources containing original research were identified, each blog or report was reviewed and given up to three keyword tags. The following fields were also completed for each of the 100 resources: what we already know about the topic of research, what the resource adds, and implications for policy and practice. Resources were then thematically grouped and keywords were refined and combined. For example, K-12 online program evaluation and quality was combined with K-12 online program policy because although distinct, the themes were related and spoke to many of the same concepts and conclusions. 

Once the 10 thematic categories were identified, the resources within that category were reviewed again, both for accuracy in interpretation and to determine its relationship to other resources in the same category. Out of this process, the core findings and practical implications were identified. What is presented below is the synthesized understanding from the original research included. Because of the process, not every finding of every resource could be included, rather resources were reviewed to form a broad understanding of each theme and to identify what MVLRI has contributed and learned in the 6 years since it was formed.

K-12 Online Program Evaluation, Quality, and Policy Core Findings

  • Evaluation and approval of K-12 online and blended programs and courses vary widely by state, but there are five common areas: level of evaluation, approval requirements, geographic reach, delivery model, and evaluation and approval procedures.
  • Educational stakeholders were largely in agreement that there should be clear expectations of accountability for everyone involved in online K-12 education. 
  • K-12 online providers reported a number of effective input practices to assure quality; however, they still reported significant challenges related to learner outcomes. 
  • Michigan adults felt that it was important for older K-12 learners to have the option to enroll in online courses.
  • Michigan adults viewed K-12 online learning as important to prepare learners for college and future success. 
  • Educational stakeholders were using K-12 online learning but had concerns about quality and overall were more focused on blended learning. 

K-12 Online Program Evaluation and Quality

Barbour, Clark, DeBruler, and Bruno (2014) found that there was no consistent national model of evaluation or approval of K-12 online or blended programs or courses. That is to say that there was no singular process that states or school districts used in either approving K-12 online or blended courses or programs, or evaluating existing programs or courses. As public education is largely left up to individual states, there is also no consistent national model encouraged by the federal government. Even with the considerable variability between states in approval and evaluation processes, five consistent areas for consideration were identified: 

  1. Level of evaluation and approval. 
  2. Approval requirement. 
  3. Geographic reach. 
  4. Delivery model. 
  5. Evaluation and approval procedures (Barbour, et al., 2014). 

There was also widespread agreement among Michigan educational stakeholders that there should be clear expectations of accountability for everyone involved: 

  1. Clear communications of expectations and awareness. 
  2. Addition of learner support structures including on-site mentors. 
  3. Specific guidelines for teacher of record and mentor of record role. 
  4. Changes to state reporting guidelines including statewide reporting and performance. 
  5. Updates to course catalog including learner and parent reporting and feedback.
  6. Additional Requirement of Online Course Syllabi under Section 21f.
  7. Pilot the use of Michigan interim assessments for online courses as end of course (EOC) exams. 
  8. Updates to course review process (Archambault, Kennedy, Freidhoff, Bruno, DeBruler, & Stimson, 2015). 

In a case study with K-12 online providers in Michigan, Clark (2016) found that effective internal evaluation practices seemed to converge on three main areas: learners, courses, and teachers. For learners, online providers developed and implemented personalized learning plans, used a variety of learning modalities and instructional support strategies, and took advantage of local mentor support. For courses, online providers utilized standards-aligned course reviews, planned curriculum development cycles, and targeted the use of adaptive release in online courses. For teachers, online providers created professional development groups and communities, and teacher start-of-course checklists. These same schools also noted some challenges, which did not converge as cleanly as the effective practices, but could be encapsulated by the following five concepts: 

  1. Rapid program growth and expanding capacity while maintaining quality.
  2. Low learner mastery and/or overall course performance leading to learners not earning credit towards graduation. 
  3. Variability in local support. 
  4. Variability in course format and rigor. 
  5. Funding special education services. 

K-12 Online Policy

For years, Michigan adults have consistently viewed online learning as important and expect it to continue to grow. Over 70% of respondents of two Public Sector Consultant surveys since 2014 (Michigan residents) felt that it is somewhat or very important for middle and high school learners to have the option of enrolling in an online course at their local school district. (Public Sector Consultants, 2017; Public Sector Consultants, 2019)

Additionally, Michigan college learners saw value in taking online courses at the K-12 level as a way to prepare for college. This sentiment was shared by Michigan adults who viewed knowing how to learn online as critical to future success (Public Sector Consultants, 2017; Public Sector Consultants, 2019). 

Nearly three-quarters of Michigan adults thought that high school learners should take at least one online course prior to graduation, a 10% increase from 2014 (Public Sector Consultants, 2017; Public Sector Consultants, 2019). In a survey of secondary school principals in 2013, 87% had learners enrolled in an online course (Michigan Virtual University, 2013). 

From 2014 to 2019, a majority of high school learners shifted from saying that a few or none of their online courses (62%) used a learning management system (LMS) to responding that all or most (54%) of their courses used an LMS (Public Sector Consultants, 2017; Public Sector Consultants, 2019). 

Educational stakeholders reported that standards alignment, quality, and rigor were among the most important considerations in selecting online providers. Once selected, satisfaction with online courses varied greatly by vendor (Michigan Virtual University, 2013). While online learning was happening at a majority of secondary schools in Michigan in 2013, most schools were focused on solutions for blended learning. 

K-12 Online Program Evaluation, Quality, and Policy Practical Implications and Actionable Resources

  • Based on the consistent findings of the Michigan statewide surveys, there appeared to be an appetite in the general public for high-quality online learning options. These options should be available; however, schools were finding it difficult to ensure high-quality courses for their students. Michigan Virtual was already responding to this appetite by offering high-quality courses and providing free online program reviews. Michigan Virtual course developers were trained by Quality Matters to assure that course content, learning activities and materials, and assessments were aligned with learning objectives and state or national standards. Further, developed courses also underwent third-party reviews with Quality Matters and were revised until a Quality Matters certification was achieved. Michigan Virtual program reviews were offered free to Michigan K-12 schools with supplemental online learning programs (i.e. not full-time cyber schools). The reviews allowed districts to reflect on what was working in their programs and ways to increase effectiveness. 
  • Michigan adults perceived online learning as important for K-12 learners to prepare them for college and felt that learners should have the option to enroll in online courses. Educational stakeholders also valued online learning; however, they reported serious concerns about quality and were more focused on blended learning which incorporates digital and internet learning but is not exclusively online. 
  • Educational stakeholders agreed that there should be accountability for online learning. Based on the limited research conducted by Michigan Virtual there did not seem to be a national, or even widely used model for approval and evaluation of K-12 online programs and blended programs. Given the concerns from educational stakeholders and low performance associated with some online learning programs, perhaps there needed to be more rigorous review and regular evaluation of online programs. 

References

Archambault, L., Kennedy, K., Freidhoff, J. R., Bruno, J., DeBruler, K., & Stimson, R. (2015). Accountability in K-12 online learning course access programs: Stakeholder recommendations for policy and practice. Michigan Virtual University. https://michiganvirtual.org/research/publications/accountability-in-k-12-online-learning-course-access-programs-stakeholder-recommendations-for-policy-and-practice/ 

Barbour, M. K., Clark, T., DeBruler, K., & Bruno, J. A. (2014). Evaluation and approval constructs for online and blended courses and providers. Michigan Virtual University. https://michiganvirtual.org/research/publications/michigan-virtual-report-examines-online-learning-policies-and-practices/ 

Clark, T. (2016). Quality assurance in K-12 online learning programs: Michigan case studies. Michigan Virtual University. https://michiganvirtual.org/research/publications/quality-assurance-in-k-12-online-learning-programs-michigan-case-studies/ 

Michigan Virtual University. (2013). Michigan association for secondary school principals online learning survey report. Michigan Virtual University. https://michiganvirtual.org/research/publications/michigan-association-for-secondary-school-principals-survey/  

Public Sector Consultants (2017). Public awareness and views of K-12 online learning in Michigan 2017. Michigan Virtual University. https://michiganvirtual.org/research/publications/public-awareness-and-views-of-k-12-online-learning-in-michigan-2017/ 

Public Sector Consultants (2019). Public awareness and views of K-12 online learning in Michigan 2019. Michigan Virtual University. https://michiganvirtual.org/research/publications/public-awareness-and-views-of-k-12-online-learning-in-michigan-2019/  

Table of Contents