Two Years of Research on HarvardX and MITx
Today, my colleagues and I from the HarvardX and MITx research teams released a new report: HarvardX and MITx: Two Years of Open Online Courses Fall 2012-Summer 2014. In it we present key findings from the analysis of over 60 courses conducted in the past two years.
Here are my thoughts on the top three insights from the paper.
1) The Unclear Consequences of Linear Growth
First, growth of MOOC participants is linear.
For the 790 days encompassed in the report, new participants entered at a rate of about 1,300 per day. Of course, HarvardX and MITx are not subscription services, these are courses, so people leave at the same rate as well. Linear growth in registration may mean something like a relatively stable active daily user based.
Linear growth raises a series of confounding questions about the future of the effort. Facebook in its first few years went from 1MM, to 5.5MM, to 12MM, to 50MM users, from December of ’04 to October of ’07. If we were seeing exponential growth like that, HarvardX would be pulling together a business development team and every year would be the year of the MOOC. If we saw zero or declining growth, they’d be selling the furniture and closing up shop. But what kind of contribution to the universities and to society can HarvardX and MITx make with 1,300 new people per day? The answers to that are less clear.
Linear growth explains two phenomena that you see across the MOOC space. First, the platform providers are still adding new universities to their consortium. Since even Harvard and MIT can’t get to exponential growth, the only way to go beyond linear growth is to try to grow the partner base. It’s possible that there are some universities out there with MOOC courses and programs that are continuing to grow (probably in computer science, data science, and related fields), but if most are plodding along, then the only way for the platforms to have greater than linear growth is to bring in more institutions into the consortiums. One could imagine that if the core universities had been exploding in growth, the consortia would have elected to maintain greater selectivity and exclusiveness in the consortia.
Second, there is a strong turn inwards in many university MOOC programs, Harvard and MIT included, to examine how digital learning resources created for MOOCs can improve residential education. MIT, in particular, has make incredible strides along these lines: 83% of MIT undergraduates are taking a course where a substantial component of the course is hosted on the MITx platform. My undergraduates at MIT write papers every year about their observations of an MIT class; in a typical year maybe 4 or 5 of 25 papers will be about technology; this year, over half made some reference to the impact of MITx and other technologies on classes. This has always been part of the mission of the MOOC venture, but it has come into greater emphasis, in part because it looks like if MOOCs are going to herald the end of college, they are going to do it very, very slowly.
If HarvardX and MITx are approaching a steady state of daily learners, who should those learners be and how should these institutions serve them? What does the “expanding access to learning opportunities” part of the mission of HarvardX, MITx, and edX look like under these conditions? That future is yet to be written.
2) From courses to sequences
At Harvard, MIT, and other universities, we’re starting to have enough courses to see sequences of courses emerge.
Some of these sequences are developed by design–the 15 month, 10 module ChinaX course stands out–but we’re beginning to see how students progress through multiple courses over time. This opens up new avenues for assessing courses: to what extent does participation and success in one course then predict registration, participation, and success in a subsequent course. Could one marker of a successful course be that it propels students to do well in future courses, within the same domain or beyond?
Transfer–the ability to take learning in one setting and apply it elsewhere–is the holy grail of education outcomes. Having students progress through sequences and clusters of courses offers the opportunity to develop new measures of these kinds of outcomes.
3) Many early MOOC studies are holding up well.
One somewhat terrifying part of releasing these larger reports is that we have been testing our earlier findings against larger datasets. Many of the early studies were drawn from small samples of courses. For instance, I did a study on MOOC learner intention that included nine courses, and we re-ran those findings across 35 courses both from Harvard and MIT. Among survey respondents in the initial sample, about 56% of students intended to complete a course. In the larger sample, it was 57%. Phew! We’ve seen this in a number of contexts now, that the patterns of participation and behavior that we found in the earliest courses in most contexts, seem similar to what we are finding now.
It’s great that folks are doing these kinds of replications, and the fact that they are coming up with similar findings suggest that some of the early patterns that we found in MOOC student behavior appear to be durable, at least through these first two years.
We conclude the report with these words:
The HarvardX and MITx MOOCs have called many longstanding features of residential courses into question: the rigidity of the semester schedule, the lengthy periods between assignment submission and feedback, and the rarity of sharing syllabi, lecture notes, lesson plans, assessments, and rubrics across faculty teaching common courses, to name only a few examples. Even if MOOCs are not a flawless answer to these questions, they deserve credit for helping to clarify issues worth addressing and inspiring campus-wide conversations about the future of learning at our institutions.
The diverse contributions of MOOCs have been obscured by a common expectation that MOOC certification would grow exponentially in number and value, especially among those without access to conventional higher education. That they have not yet delivered on this promise should not blind us to their other benefits. As we progress through the third year of HarvardX and MITx, with courses, experiments, participation, and residential innovations continuing to proliferate, the diverse audiences and stakeholders are increasingly well-defined. The opportunity we face is one of prioritization, not in terms of choosing some audiences and stakeholders over others, but in terms of choosing strategies that capitalize on synergies among these audiences and stakeholders.
Participation, exploration, certification, teacher enrollment, and residential enhancement are less independent dials to turn up and down but interacting ingredients in a recipe for online and residential advances in learning. From the baseline data we have shown in this report, we hope to continue to facilitate good cooking.
I’d be excited to see that the MOOC project facilitates better cooking at Harvard and MIT, but I hope the focus doesn’t entirely shift inward. I hope that both institutions continue to engage in thoughtful dialogue and deliberation about what kinds of recipes and meals they might share with the rest of the world. The challenge for both institutions is to continue to revise and articulate the future of the “access” mission of HarvardX and MITx.