I'm very pleased to announce that the first report from my research project, the Distributed Collaborative Learning Communities project, is published in this month's issue of Educational Researcher, the flagship journal of the American Educational Research Association. The article can be found through this direct link or at this landing page and is titled, "The State of Wiki Usage in U.S. K-12 Schools: Leveraging Web 2.0 Data Warehouses to Assess Quality and Equality in U.S. K-12 Schools." I'm very grateful to the Hewlett Foundation's Open Educational Resources initiative for financial support, and to my co-authors Richard Murnane and John Willett.
The article is written for a general audience of education researchers, and while I think it's highly readable (so far as scholarly articles go), many folks may prefer a summaryof the article. This post, therefore, is particularly targeted at classroom teachers and other school educators to share a bit about our findings and suggestions for using social media in classrooms. First, I explain what we did, and then I give some advice for wiki using educators. If you want to skip to the advice, Click Here.
(This post is also available as a white paper: The State of Wiki Usage: A Summary for Educators)
Our Research Questions
The use of social media in K-12 settings has grown at a tremendous rate. According to a very well conducted 2009 federal study (the Fast Response System Survey), 40% of teachers report using blogs or wikis in their classroom and approximately 20% of teachers require their students to contribute to blogs or wikis. Given the recent history of these tools, that is remarkable growth.
Teacher self-reports on surveys, however, don't tell us anything about what they are doing with those tools, and that was our mission. We asked two kinds of questions: "What kinds of learning opportunities do students have with wikis?" and "Are those learning opportunities equitably distributed among schools serving difference socioeconomic populations?" To put these more simply, are wikis any good and do only certain kids get the good ones?
What We Did
To answer these questions, we first developed a tool to measure wiki quality, called the Wiki Quality Instrument. We used three strategies to develop this instrument (described thoroughly here).
- We spent about a year asking teachers and students what they thought high quality work on wikis looked like.
- We randomly sampled thousands of wikis from a population of nearly 200,000 wikis, and we looked at what kinds of things happen on U.S., K-12 wikis.
- We thoroughly reviewed the research literature on measuring quality and learning in online learning environments.
Teachers told us that they used wikis for four reasons: 1) to help with class logistics, 2) to develop collaboration skills, 3) to deepen and display student understanding, and 4) to learn to use technology to communicate. These ideas cohere very well with the framework of 21st Century Skills, so our Wiki Quality Instrument (WQI) measures the degree to which wikis provide opportunities for students to develop 21st century skills such as expert thinking, complex communication, and new media literacy. The WQI consists of 24 questions about things students might do on a wiki-- questions like "Do students copyedit each other's work?" or "Do students embed multimedia?' The questions are in five categories: Information Consumption, Participation, Expert Thinking, Complex Communication, and New Media Literacy, and the whole instrument can be found here.
Having created a great new instrument to measure wiki quality, we needed wikis to measure. PBworks.com helped us (for free) get access to all 179,851 publicly-viewable, education-related wikis hosted on PBWorks from the creation of the company in 2005 through August of 2008 (this is 70% of all ed wikis, another 30% were private). These wikis are used in every grade level from Pre-K through Grad school, in every subject area, in dozens of countries around the world, and for multiple purposes: from posting syllabi to creating portfolios to maintaining mini-topical encyclopedias.
We took a 1% random sample of these wikis, so the wikis we analyzed were representative of the population as a whole (they are certainly representative of viewable PBworks wikis, and we argue in the paper that they are probably representative of classroom wikis in general). We then separated out all wikis used in higher ed, in other countries, in private schools, and in unidentifiable places, and we were left with a subsample of 255 wikis used in U.S., K-12 public schools. We could examine the entire edit history of these wikis to measure their quality, and since we knew what school they came from, we could also gather school level socioeconomic status data from the Common Core of Data (It's fun! Check out your own school!).
We then measured wiki quality in all of these wikis at multiple time points, at days 7, 14, 30, 60, 100, and 400. This allowed us to represent wiki quality as a trajectory, rather than a single value. We used statistical models to estimate the expected wiki quality trajectory for the whole population.
What We Learned
Where are wikis used?
We found wikis are used throughout the academic subject areas and in all grade levels.
This diverse activity occurred throughout the K–12 sector. Of our 255 public school wikis, 25% supported instruction in Grades K–5, 28% in Grades 6–8, and 52% in Grades 9–12 (the sum of these percentages exceeds 100% because some wikis supported multiple grades). Wikis were used not just in computer classes; they supported instruction throughout the curriculum. We found that 34% of wikis supported English/language arts instruction, 13% supported social studies, 18% supported science, 13% supported math, 14% supported computer science, and 26% supported another subject or no subject.
This coheres with findings from the 2009 FRSS survey.
How does wiki quality develop?
Wiki quality typically grows very quickly early on, and then quality growth levels. In other words, whatever behaviors you see emerge in the first two weeks, tend to be the only behaviors that ever happen on the wiki.
Look at the table below of wiki quality trajectories. Remember that each line represents the number of opportunities that we identified on each wiki for 21st century skill development (out of our list of 24 behaviors). Each box in the matrix contains the measures for a single wiki. Note that nearly every line is straight, and the others curve up rapidly early on and then flatten out (you should also find one exception). One way to describe this pattern to say that great wikis are born rather than made. By day 14 or so, we can reliably predict a wiki's quality.
What is the distribution of wiki quality in the U.S.?
The table below shows the distribution of wiki quality in U.S. schools. We summarize wiki quality by using the Day 14 composite wiki quality score. We break wikis into four categories. About 40% of wikis serve no useful purpose for students. Most of these are "trial wikis," which a teacher creates but does nothing with. About a third of wikis are "Teacher-Centered Content Delivery Devices." These are teacher websites, where teachers disseminate content--syllabi, facts, links, and so forth--to students. About a quarter of wikis are individual student assignments. Rather than write a paper or keep a portfolio in a folder, individual students complete these tasks on a wikis. About 1% of wikis are "Collaborative, Multimedia Performances of Understanding." These are the wikis that are what ed-tech enthusiasts imagine when they talk about the potential of wikis. They are pretty inspiring . There aren't very many of them.
In general, our findings cohere with 30 years of educational technology research. There are a handful of teachers who make remarkable use of new technologies, but for the most part, when teachers adopt new technologies, they use them to extend existing practices rather than to develop innovative practices.
How does wiki usage differ by school level socioeconomic status?
Too much. Wikis created in more affluent schools are significantly more likely to provide opportunities for students to develop 21st century skills compared to wikis created in schools serving low-income families. Wikis also persist longer in affluent schools. Wikis in Title I eligible schools (40% of students eligible for free lunches) typically last 6 days, compared to 30 days in wikis in non-Title I schools. Not only that, but these figures only quantify between-school differences. While interviewing teachers, we found striking within-school inequities as well. Teachers often told us that they primarily use wikis, or do more sophisticated stuff with wikis, with their AP and Honors students, who we know are disproportionately white or Asian and affluent.
What Should Educators Learn from this Study?
This is a descriptive study of wiki usage in U.S., K-12 schools. It tells us a lot about what is happening. It is not designed to provide specific, actionable advice in the way that a randomized experiment is designed to figure out "what works." But, we live in a world where everyday educators have to make real decisions, so this is my best effort at translating our findings into suggestions. I should emphasize that these are my interpretations of what teachers should consider based on these data; these are not "proven practices" from our findings. You might say that these are all suggestions for action research projects that educators can take on in their own schools and classrooms.
1) Clarify your learning goals and enshrine these goals in rubrics and assessment criteria.
Teachers told us that they used wikis to develop collaboration skills, technology skills, and critical thinking skills. But most wikis are just content delivery devices. Why the disconnect between goals and reality?
There are lots of reasons, but one we can control has to do with rubrics. We looked at lots of teacher's rubrics, and they tended to check whether students had 5 pages, 4 paragraphs, 3 links, 2 images, and a partridge in a pear tree. Basically, teachers want deeper learning, but they evaluate whether students are following directions. Following directions is a useful skill, but nobody told us it is why they created a wiki. Teachers should be sure that their assessment criteria align with their most important goals. (I don't recommend any particular set of rubrics online. I haven't found ones I think are really great. Make your own.)
2) Carefully structure the development of early norms.
High quality wikis tend to start at high levels of quality. My hunch is that the early norms that teachers set determine the behaviors that follow on wikis. If you want really deep collaboration to occur on your wikis, establish that very early on as a norm. Do exercises that force students to discuss ideas, copyedit, and substantively edit each others work. I believe the first few days and weeks are crucial.
3) Work with a buddy.
Many teachers do wiki projects in highly isolated environments. Find a teacher-- in your school, through Twitter or classroom20.com or somewhere--to be your thought partner in using wikis in the classroom.
4) Focus on between-school inequality rather than within-school inequality.
We met some remarkable teachers doing awesome work with wikis with students in urban schools. When you asked these teachers about inequality in schools, they focused on between-school inequalities. "Since kids in the suburbs get to do this stuff, I need to give my kids these opportunities, even if access is difficult." Teachers in low-income schools who shy away from technology tend to focus on within-school inequalities. "Not all my kids have access at home, so it's not fair to make them use wikis." I have talked about this at length in another post, but basically, I think we need to coach teachers to focus on between-school inequalities.
5) If you are a school leader, provide professional support
Lots of schools spend 6 or 7 figure sums on hardware and infrastructure, and then 4 or 5 figure sums on professional development and support. If you want innovative uses of new media throughout your school (not just in isolated pockets), don't expect that providing hardware is sufficient. Thirty years of ed-tech research have discredited that idea. We're going to need to provide extensive support to educators to help them figure out how to develop new lessons, new curricula, and new pedagogy that realize the terrific potential of new media in schools.
If you have questions or responses, please feel free to leave something in the comments. If you have specific criticisms of the methods, advice or findings, you might consider reading the full article before posting.