MVLRI Research in Review: K-12 Online Best Practices

Other Reports in the MVLRI Research in Review Series 

Introduction

Since its creation in 2013 through 2020, the Michigan Virtual Learning Research Institute (MVLRI) at Michigan Virtual published approximately 20 research blogs and 75 research reports. This total does not represent everything published by MVLRI but rather only those publications including original research on K-12 blended and online learning. The nearly 100 resources represent research conducted internally by MVLRI staff, research conducted by partners at universities, colleges, and educational organizations, and covers a vast range of topics including, but not limited to, K-12 online best practices, online student motivation, K-12 blended teaching and professional development, and K-12 special populations. 

This body of work is extensive, and while there is tremendous value in each individual publication, there is also value in how that work fits with other similar research and the narrative that emerges from the collective understanding. Toward this end, MVLRI sought to identify, review, and synthesize the original research published in the past 6 years. Again, not every blog or report published via the MVLRI.org website was included, only those containing original research. 

Out of the synthesis of resources, 10 main themes emerged. Each theme is presented individually in the interest of brevity. A full reference list is provided at the end of this document noting the resources that contributed to this report. 

Methods

Resources for inclusion in the synthesis were identified through the MVLRI.org website in the “Publications” and “Blogs” sections. All published blogs and reports were assessed to determine if they included original research. Those that did were included for synthesis. Once the approximately 100 resources containing original research were identified, each blog or report was reviewed and given up to three keyword tags. The following fields were also completed for each of the 100 resources: what we already know about the topic of research, what the resource adds, and implications for policy and practice. Resources were then thematically grouped and keywords were refined and combined. For example, K-12 online program evaluation and quality was combined with K-12 online program policy because although distinct, the themes were related and spoke to many of the same concepts and conclusions. 

Once the 10 thematic categories were identified, the resources within that category were reviewed again, both for accuracy in interpretation and to determine its relationship to other resources in the same category. Out of this process, the core findings and practical implications were identified. What is presented below is the synthesized understanding from the original research included. Because of the process, not every finding of every resource could be included, rather resources were reviewed to form a broad understanding of each theme and to identify what MVLRI has contributed and learned in the 6 years since it was formed. 

K-12 Online Best Practices Core Findings

• Timing of registration for K-12 online courses does not seem to predict learner success. 
• Online K-12 learners who demonstrate linear progress throughout the semester comprise the highest performing learning trajectory group.
• Time spent in an online course does not always equate to positive outcomes as significant time spent near the end of a course can be indicative of little progress in the early stages of the course. 
• Project-based assignments and text-resources contributed positively to learner success in K-12 online English courses, low knowledge activities detracted from success. 
• Online K-12 learners tend to focus more heavily on auto-graded course items when in fact teacher-graded items contributed more positively to learner success. 
• The relationship between online K-12 class size and learner success is typically reverse-U shaped; however, the degree to which class size contributes to learner success varies by content area. 
• Teacher-learner communication is important to online K-12 learner satisfaction and success, and learners largely prefer LMS embedded messaging tools. 

Registration Timing 

There tends to be an assumption that learners who enroll later in the registration window are less likely to succeed in their course for any number of reasons, such as motivations for taking the course and ability to pre-plan and enroll early. However, data collected from an online K-12 learning management system (LMS) do not support the assumption that learners who register late for a course have less chance for success. Instead, enrolling in the second week of a course, learners have an equal probability of passing or failing said course (Ranzolin, 2015). 

Learning Trajectories

Growth mixture modeling (GMM) found several different learning trajectory profiles in online K-12 mathematics courses. The first profile was that of nearly linear, on-pace progression, constituting approximately three-quarters of enrollments. This profile was more common in advanced placement (AP) courses likely because students in such courses move through them in a cohort model, whereas most other online courses are self-paced (Kwon, 2018). 

The second was a profile marked by a steep increase in learner scores near the end of the semester (Kwon & DeBruler, 2019). Success with this profile required learners to earn a significant portion of course points in the final month(s) of the course, which was largely unattainable by a majority of learners (Kwon & DeBruler, 2019). Additionally, learners enrolled in foundation courses such as Algebra and Geometry, and who indicated enrollment reasons of credit recovery or personal learning preference were more likely to show unpromising learning trajectories (Kwon, 2018). 

The third profile was one of hardly any progress over the semester; the learners did not show any “bursts” of activity nor did they consistently proceed through their course. Learners who ultimately withdrew from their courses showed this pattern of unpromising growth. The number of possible course points earned by these learners grew very little from the start to near the end of the semester (Kwon, 2017e).

A final profile was marked by strong early achievement (Kwon & DeBruler, 2019). High performing groups, those with the greatest probability of success in their online course, tend to demonstrate more robust, linear progress from the beginning to the end of the semester (Kwon & DeBruler, 2019). This group, however, is not always representative of the class overall, rather the largest group featured a spike in time invested in the course during a particular time (like the final weeks of a course) (Kwon, 2017c). This strategy was more common in the fall and spring semesters than in the summer semester (Kwon, 2017d). 

Because of this and findings from other studies, there does not appear to be a clear relationship between greater amount of time spent in course and/or multiple peaks of time in course and success. The strategy of little involvement with a final surge is a common one, and one that is successful in some content areas and courses but not others (Kwon, 2017c). 

Course Design

A study conducted of Michigan Virtual English language and literature (ELL) courses found that project-based assignments were beneficial for learners and that text-based learning resources, such as teacher guides, helped learners achieve better learning outcomes (Zheng, 2018). Also important to learner success was learner autonomy in their learning through things such as promoting discussion, exchanging feedback, and encouraging learners’ sense of audience and authorship.  

Conversely, low-level knowledge activities, such as remembering, had a negative impact on learning outcomes (Zheng, 2018). 

Still specific to ELL courses, there were more auto-graded course items (individual questions on an auto-graded quiz) than teacher-graded; however, the teacher-graded items accounted for a larger percentage of course points (Lin, 2019). The proportion of auto-graded work varied considerably between courses. 

Learners attempted to earn a higher percentage of auto-graded course points, yet they actually earned a higher percentage of course points from teacher-graded work. This is because teachers awarded a higher percentage of points attempted (Lin, 2019). Further analysis indicated that neither the percentages nor the point totals earned from auto-graded coursework affected learners’ pass rates (Lin, 2019). If learners are struggling with their overall workload, they may want to concentrate more of their efforts on coursework that is teacher-graded rather than on auto-graded coursework as they are more likely to do (Lin, 2019). 

Class Size

The relationship between class size and learners’ final course grade is a reverse-U shape for math, social science, and “other” subject areas (arts, etc.). For these content areas, as class size increases, so does final course grade to a certain point (peak class size of 38 for mathematics; 42 for social science; about 35 for “other”). For foreign language and science, final grades increased as class size increased to the peak (15 for foreign language; 35 for science) and then decreased; however, this relationship was not statistically significant. Class size in English language and literature (ELL) did not have an impact on learner final grade (Lin, Bae, & Zhang, 2019). 

Teacher-Learner-Content Interactions

Teacher-learner interaction and communication are important in K-12 online courses, so much so that learning outcomes and overall course satisfaction are significantly related to teacher-initiated communications such as a welcome message, course announcements, and feedback (Kwon, 2019a; Lin, Zheng, & Zhang, 2016). Further, learners who perceived communication as the best part of online learning were more likely to engage with course content and be satisfied with the course overall (Kwon, 2019a). Zhang & Lin (2020) did not find a relationship between teacher-learner interactions and learners’ learning satisfaction; however, this is contradictory to several previous studies and may be an area where more specifically focused, empirical research is needed. In the Zhang & Lin (2019) study, only learner-content interactions were predictive of learner learning satisfaction. 

Of the various mediums for teacher-learner communication, K-12 online learners preferred messaging tools embedded within the LMS. Learners who took better advantage of the message tools (as evidenced through more out-going messages) were more successful in their courses (Kwon, 2019a). This was particularly true for mathematics and AP courses (Kwon, 2019a). 

Zhang & Lin (2019) also found that for teachers, pedagogically focused behaviors were a significant positive predictor of learner satisfaction, whereas managerial behaviors were a significant negative predictor. Their results suggest that improvements in learner-content interaction may help to increase learner satisfaction, and an increase in teacher-learner interaction may also benefit learner satisfaction (Lin, Zheng, & Zhang, 2016). 

K-12 Online Best Practices Practical Implications and Actionable Resources

• Reason for enrollment, not timing, seem to impact learners’ probability of success, so long as learners are within the registration window, and presumably have ample time to complete their course, they should be permitted to enroll. It may, however, be helpful to flag learners who indicated enrollment reasons of “credit recovery” or “personal learning preference” for more careful monitoring during the early weeks of the course. We know that learners who show little progression early in the semester are unlikely to make significant progress and are likely to withdraw from the course. It is recommended to have established early monitoring procedures for monitoring and following up with these learners. 
• The most successful group of online learners are those who demonstrate consistent, on-pace progress through their courses. Because of this, online programs and course providers should continue to provide, and strongly encourage students to follow, pacing guides. They should not require that learners strictly adhere to the pacing guides (with the exception of AP courses and the final end of course deadline) without conducting research into the benefits and possible unintended consequences of such a requirement. This may be an area of interesting research in that the “any pace” concept on online learning may not actually be one that is beneficial to learner success. 
• Autograded items like quizzes offer learners an opportunity to practice and solidify understanding of course concepts; they also relieve some of the burden of assessment from online teachers. However, learners tend to over-invest their time and effort in auto-graded items for which they are awarded less points and which contribute little to the overall course progress and progression. When designing new courses, or updating courses online programs and course providers should be cognizant of learners over-reliance on autograded items and use them in targeted and specific ways as we also know that project-based assignments led to better course outcomes than memorization tasks. Additionally, when developing interventions for learners who are behind in their courses, it may be more advantageous for learners to focus their time and effort on instructor-graded items, and less on auto-graded. This research should be expanded out beyond ELL courses to determine the true impact of these assessments for other content areas. 
• There seems to be a complicated relationship between the number of learners in an online course and learner outcomes. While Michigan Virtual has some data on optimal class size, it is recommended that more research on this topic is conducted to more clearly determine the nature and strength of this relationship before developing any specific policies. 
• It is clear from research sponsored by Michigan Virtual, and intuitive to those in education, that communication between learners and teachers is important to most learners. Learners reported a preference for messaging tools embedded in the LMS. Online programs and course providers should continue to encourage learners to communicate with teachers through the LMS messaging tool. Online learners who communicated more with their teachers reported higher course satisfaction; however, course satisfaction is unlikely to increase broadly by requiring communication of all learners.
• Online learners were more satisfied with their online courses when their teachers engaged in pedagogical behaviors, rather than managerial ones. Reducing the amount of managerial tasks that online teachers must complete likely frees up time for teachers to work closely with learners, which in turn increases learner satisfaction. Online programs and course providers should continue to work closely with course designers and instructional staff to decrease the amount of managerial work of online teachers. 

References 

Kwon, J. B. (2017c). Course engagement patterns in mathematics and non-mathematics courses. Michigan Virtual University. https://michiganvirtual.org/research/publications/course-engagement-patterns-in-mathematics-and-non-mathematics-courses/ 

Kwon, J. B. (2017d). Exploring patterns of time investment using time-series clustering analysis. Michigan Virtual University. https://michiganvirtual.org/research/publications/exploring-patterns-of-time-investment-in-courses-using-time-series-clustering-analysis/  

Kwon, J. B. (2017e). Growth modeling with LMS data: Data preparation, plotting, and screening. Michigan Virtual University. https://michiganvirtual.org/research/publications/growth-modeling-with-lms-data-data-preparation-plotting-and-screening/ 

Kwon, J. B. (2018). Learning trajectories in online mathematics courses. Michigan Virtual University. https://michiganvirtual.org/research/publications/learning-trajectories-in-online-mathematics-courses/ 

Kwon, J. B. (2019a). Communicative interactions with teachers in K-12 online courses: From the student perspective. Michigan Virtual University. https://michiganvirtual.org/research/publications/communicative-interactions-with-teachers-in-k-12-online-courses-from-the-student-perspective/ 

Kwon, J. B. & DeBruler, K. (2019, September 26). Pacing Guide for Success in Online Mathematics Courses. Michigan Virtual. https://michiganvirtual.org/blog/pacing-guide-for-success-in-online-mathematics-courses/  

Lin, C. H. (2019). Auto-grading versus instructor grading in online english courses. Michigan Virtual University. https://michiganvirtual.org/research/publications/auto-grading-versus-instructor-grading-in-online-english-courses/ 

Lin, C. H., Bae, J., & Zhang, Y. (2019). Online self-paced high-school class size and student achievement. Educational Technology Research and Development, 67, 317- 336. https://doi.org/10.1007/s11423-018-9614-x  

Lin, C. H., Zheng, B. & Zhang, Y. (2016). Interactions and learning outcomes in online language courses: Online interactions and learning outcomes. British Journal of Educational Technology. 48(3). https://doi.org/10.1111/bjet.12457 

Ranzolin, D. (2015, April 1). To serve and subsist: Reflections on finding the ideal registration window. Michigan Virtual. https://michiganvirtual.org/blog/to-serve-and-subsist-reflections-on-finding-the-ideal-registration-window/ 

Zhang, Y. & Lin, C. H. (2019). Motivational profiles and their correlates among students in virtual school foreign language courses. British Journal of Educational Technology, 51(2). https://doi.org/10.1111/bjet.12871 

Zheng, B. (2018). Exploring the impact of student-, instructor-, and course-level factors on student learning in online English language and literature courses. Michigan Virtual University. https://michiganvirtual.org/research/publications/exploring-the-impact-of-student-instructor-and-course-level-factors-on-student-learning-in-online-english-language-and-literature-courses/