It seems like everything about our lives can be quantified and made into a chart these days. Data collection is easier than ever as software systems and mobile apps record our activities and track our progress, with a range of applications including health, fitness, marketing, and sales. Online learning is no exception.
The 2014 Horizon Report identifies learning analytics is as an emerging technology with “one year or less” as the estimated time to widespread adoption. In other words, it’s here. What can we expect from learning analytics in online higher education?
Gathering information about students and materials is just the beginning. Learning management systems (LMS) and other technologies are already collecting data and generating charts in the background via user-friendly dashboard tools. JISC describes learning analytics as “a way for institutions to use the rafts of data they routinely collect and hold for more than reporting – to gain a better understanding of what they are doing.”
Courseware firm Acrobatiq provides a three-category approach that describes analytics and identifies possible applications in online higher education:
- Institutional: “Tracking learners through their educational lifecycle, from enrollment to graduation,” includes information such as student demographics, use of support services, and degree completion rates.
- Engagement: This category includes “track[ing] student activity within the course environment, which is typically the learning management system (LMS).” Knowing how frequently a student logged in to a course and on what days and times can be helpful for monitoring progress and participation, as well as use of specific materials.
- Learning: What did the learners actually learn in a course or program? This type of analytics “measures changes in a student’s knowledge and skill level, with respect to specific curriculum.” Analytics in this category can also inform decisions about instructional design and course revisions.
There are many possible goals of implementing an analytics program. Gaining a better understanding of what we are doing allows us to make better decisions about a variety of concerns in online education, such as enhancing design and navigation, personalizing learning environments, predicting student performance, and providing timely support to learners.
How does the knowledge gained from analytics lead to change? Data-driven projects that guide technology selection, course revisions, and student support can be found at institutions nationwide. The following initiatives are examples of what’s taking place – they are not only working toward better student experiences, but also often sharing their results with the rest of us.
The Course Signals program used at Purdue University “detects early warning signs and provides interventions to students who may not be performing to the best of their abilities before they reach a critical point.” According to a recent article from NPR.org, Course Signals “has been shown to increase the number of students earning A’s and B’s and lower the number of D’s and F’s.”
The system allows instructors to closely monitor individual student progress, and send specific feedback as needed during the academic term. This may be particularly helpful for those teaching courses with hundreds of students, and other situations in which one-on-one interaction with each student is a challenge.
Predictive Analysis Reporting (PAR) Framework
PAR is a collaborative effort involving 16 member institutions (and more than 1.6 million students) with the goals of “identifying points of student loss and [finding] effective practices that improve student retention.” One of the resulting products is the development of a student success matrix [PDF], which identifies a list of factors related to successful learner connection, entry, progress, and completion of online courses at different types of institutions.
An ongoing project originated as part of the WICHE Cooperative for Educational Technologies, PAR recently announced plans to move the initiative forward as an independent non-profit organization later this year.
Learning Management Systems
You don’t need to launch a new initiative to find out what’s happening in your classes. LMS tracking functions have become quite sophisticated in recent years, and you may be surprised at the data already available. Blackboard and Canvas are just two of the systems actively adding analytics functions at the course level. Explore your school’s LMS for information related to identifying patterns and problems in your courses. Here are a few of the features you might find:
- Page views: The total number of pages in the course that were viewed by day, and by how many students. Are some days more popular for logging in? There may be specific pages or modules that get more “hits” during the term, and these may or may not be the ones you would expect.
- Assignment status: How many students have submitted each assignment? Analytics functions often provide totals as well as details about the percentage of submissions that were on time or late. Are there specific weeks in which the majority of assignments are turned in after the deadline? Review those modules with pace in mind and check those assignment instructions for clarity.
- Student statistics: In addition to the course-level options listed above, you can also view each student’s progress in terms of page views and turn ins, as well as, of course, grade average. Are students with higher levels of participation (i.e., page views, posts) also the ones with higher grades? Are the students with more late turn the ones with lower grades?
Students also provide helpful data in the form of course evaluation feedback. Patterns found in their critiques and suggestions should be part of your overall analytics effort.
There’s great power in learning analytics and the potential for these tools to help us get better at what we are doing. However, adding analytic initiatives won’t be a cure for all that ails online education. There are a number of growing concerns among educators about how analytics are being used. Discussion of these challenges adds to the larger conversation about the improvement of online learning.
- Privacy issues: Who owns the data being collected about students and our courses? What specifically is being documented and who has access to it? An interview with two academic technology experts published last year by The International Journal of the First Year in Higher Education identifies ethical, moral, and privacy concerns. Continued, open debate is encouraged, as there are no easy answers or responses to these emerging questions.
- Interpretation skills: There’s a lot to learn from looking at data analyses, but like diagnosing our own illnesses via medical websites, there’s a point at which a trained professional should be consulted. Skills related to analytic methods and interpretation of the results need to be developed before making data-based decisions.
- Limited focus: Learning analytics approaches address quantifiable concepts and result in reportable numbers, such as retention rates and grades. But, as educator John Warner points out, there are other goals for students in higher education, including the development of decision-making skills, learning from failure, and becoming self-sufficient. This kind of achievement is harder to measure, and a focus on data can distract from efforts made in these areas.
- Making predictions: EdTech Magazine presents concerns about the prediction capabilities of analytics systems, including Course Signals. There is a “fear of … a self-fulfilling prophecy.” Algorithms based on a student’s past course work, GPA, etc. may raise flags and impact how students view their own “chances” of doing well in a course, before they even get started.
Explore the EDUCAUSE Library and the Society for Leaning Analytics Research for additional resources and information. You can also connect with your institution’s academic technologies unit, internal research office, and faculty development group for details about the data collected from your students and in your classes, as well as ways you might be able to get involved in initiatives to share your ideas and concerns.
Source: Inside Online Learning Blog