Extended statement on #massiveteaching (part II)

(This post is meant to be an account what happened in the Coursera course Massive Teaching: New skills required. I suggest reading the previous post first. This honors my commitment to trying to get the truth to my students)

Why the course?

I do believe that MOOCs offer many opportunities. Beyond providing educational material at a very large scale, they might for instance also help strengthen democracy or enable new discoveries. I have submitted a grant proposal along those lines. Since utopian ideas often get compromised and ultimately shaped by commercial interests, I started in May 2013 to take a closer look at contracts between MOOC providers and educational institutions. Coursera was a natural choice for my focus: my university has partnered with them, they use closed source software (which increases lock-in), and they are the leading MOOC provider right now.

What I read really bothered me, for a wide array of reasons. While the utopian vision of MOOCs (free-education-for-all) is compelling, these contracts reflect a very disturbing approach to that goal. In my opinion these contracts will have large societal implications, should they become the norm: disappearance of academic freedom, of the free agency of students, and complete disappearance of any form of privacy when learning. When I tried to communicate this around me, I encountered a clear lack of information on the subject. Getting past the utopian vision took time. Once there, many responded that my concerns were valid and worth discussing. In any case, this approach was ineffective: MOOC partnership decisions are often made by university administrations, based on factors other than those that drove my concerns.

I decided that the best way to proceed was to teach a MOOC on the topic. The most natural place to do so was of course Coursera, and it also seemed natural to address the course to professors, i.e. individuals who might be able to affect decisions at their own institutions or in their local communities. It would also allow me to reach out to the more "typical" Coursera students and explain to them the contracts that are constructed around their learning. Finally, I thought it would be interesting to see the capacities at Coursera to reflect on their own practices.

I understand the paradox of teaching on Coursera about concerns relating to teaching on Coursera, but made the calculation that the benefits outweigh the costs in this case.

Timeline of preparation and structure of the course

The course was approved by Coursera mid-April. The Quality Assurance Protocol at Coursera requires the first two weeks of material to be uploaded ahead of the beginning of the course. The third week (and later) can be uploaded at the last minute.

In consequence, I structured the course in the following way:

  • week 1: pedagogy
  • week 2: technology and copyright
  • week 3: business model, science fiction (utopia/dystopia), unresolved questions

In other words, the first two weeks were conceived to provide background for the third week, which would be quite critical of MOOC contracts. I didn't hide from Coursera or the students that I would be critical, and used the flexibility offered by Coursera not to provide the third week material ahead of time. There were several reasons for this:

  • The MOOC world changes fast. In the three weeks before the course started, two comprehensive reports came out on MOOCs. I also attended two conferences on MOOCs, and was co-organiser of one.
  • I knew there was a risk that Coursera would shut down the course, either directly or indirectly. Coursera did drag their feet more and more in the lead up to the course. I attribute that to the realisation of the extent of what I wanted to say, but they could easily chalk it to other reasons.

I conceived simple videos to illustrate concepts (like an illustrated glossary), but did not mean for this to constitute the whole course. The videos would be accompanied by readings and forum interactions. Of course, I recognised that students could still desire different levels of interactions, and I was fine with this diversity. Also, this course was meant to be very responsive to evolutions in the MOOC world in its later runs, which explains why the videos are relatively condensed and interchangeable.

Because of the openness of the topic, I tried as much as possible not to give the impression of a "full course". This meant that it did not deliver either a Statement of Accomplishment or a Verified Certificate. Due to the Coursera contracts, I was still required to deliver a numerical grade at the end (which couldn't be 0% for all or 100% for all). I tentatively settled for a bland peer-feedback assessment, as I thought this would be more useful than a multiple choice quiz.

In addition, I prepared some "experiments" through the course forums. The term is very confusing: these were not scientific or social experiments, but clearly advertised and explained attempts to talk with the students, get the students to engage with the content and interact with each other and advance the state-of-the-art in collaborative tools in MOOCs. Unfortunately I have now lost electronic access to the course material itself, so I cannot relay the exact wording used to explain this to the students.

First week of the course

The first week of the course went according to me quite well, although it was exhausting: I interacted and welcomed dozens of students on the forum, orientated them, etc.

In addition, I participated with the students in the "experiments" that I had prepared:

  • I had defined a process to manage the forums based on their preferences (like Area 51 on Stack Exchange), but this had to be scrapped (lack of interest) and turned instead into a listing of desired features in forums;
  • One student initiated a wide survey of MOOC students by MOOC students, that I encouraged;
  • I decided to attempt to build a badging system (Open Badges is an infrastructure that allows the cryptographically secure attribution of rewards for contributions and achievements, however small or big). This was done in response to Coursera's requirement for a grade, and because I thought it would be very effective at shifting power towards the students/professors relationship rather than the universities/MOOC platforms. In my mind these badges would be based around the forums, and aim to make the final outcome more meaningful to each individual student (via student-defined badges granted through peer-feedback and certified by the instructor). This could have been achieved in this iteration or more realistically in a later iteration of the course, based on interactions already happening in the first run. In any case, this required looking a bit deeper into the kind of data collected on students (or at least the fraction of that data that was accessible to me as an instructor). To give an example, I would need to be able to see how an upvote was encoded, and whether there would be a way to retrieve useful information from that. I asked Coursera for an anonymized database dump taken before the course properly started, with some forums seeded with test data (generated by me and another instructor account). When I asked to share parts of this dump with the students (even in a sanitized version with synthetic data), in the interest of transparency, it was refused. Throughout Coursera dragged their feet, which only encouraged me to dig deeper: I was also forbidden to share the supporting documentation PDFs, and my request to share those refusal emails with the students was never answered.

During the first week, some students also started Facebook and Google Plus groups associated to the course, unprompted.

From the start of the course, I also used confusion as a teaching tool, by sometimes acting a bit randomly but in non obtrusive ways. This was done to expose hidden assumptions, and incite reactions, which in a MOOC would be numerous and were likely to be contradictory and diverse. I saw this as conducive to questioning of the instructor himself (i.e. me, an individual having signed contracts unknown to the students) and therefore to exploratory learning, especially with my intended audience of professors. I realise this is quite bold, but I merely saw this as an extreme transposition to the web of the Socratic method. Since this concerned purely my teaching methods and did not involve any research component, I did not make a formal announcement about this, but was overt at the start, for instance by opening a thread with title "?". This lead one student to answer 42, then another to give an equation. Other students wrote poems.

I would love to debate the value of this technique, particularly when the goal is partly to push students to turn a critical eye towards an opaque and complex legal and technical construction such as Coursera, under various restrictions on the content. Bear also in mind that students were at any point free to leave. Some certainly did but many stayed. I was also hoping that the combinatorics of the peer-feedback exercise at the end would tie loose ends at scale and cement the efficacy of the technique (since it is my suspicion that a comment such as "I am still confused by..." is more likely to elicit constructive responses from peers than "I assert this and that...").

Second week of the course

Social aspects of learning are currently completely unstructured and relatively weak on MOOC platforms, so a natural next step for them is to build some form of social network for students. Whatever form it takes (decentralised around each course, for instance), it will require extensive research unlikely to be done by the university partners since it will be core to the MOOC platforms. Concerning research practices, Facebook and Coursera have very similar, open-ended, Terms of Use.

During the first weekend of the course (June 29th?), the Facebook Emotion study made news: Facebook had manipulated the newsfeeds of many users, trying to selectively induce either happiness or sadness. The news coverage was extremely confusing: many academics were outraged at the lack of IRB approval, while others were unsurprised at these commercial practices.

Having read Coursera Terms of Use, I knew right away that similar abuses could take place there, and had many reasons to think not enough safeguards were in place at Coursera either. I very quickly saw the pernicious threat that Coursera's business model and practices represent to the thick legitimacy of instructors, researchers and universities, and ultimately to society.

Over and above this, I knew that data collected on the Coursera platform has no expiration date, can be replayed at will and that it had not been welcoming of my own transparency effort. In that sense, there was a more immediate concern, towards my students.

Unlike all the material I had prepared for week 3, which was researched and based on documents available to the general public (i.e. not derived from the Quality Assurance process, the Coursera Partners' Portal or private communication), any comment of mine linking ethics of experimenting at Coursera to ethics of experimenting at Facebook would have to be speculative at the time. Making this speculation public would expose me to the risk of a legal challenge. Continuing the course would put more ethical responsibility on me since I could not be fully transparent with the students, as I had been thus far. To add to all of this, all the unresolved ethical questions led me to question the wisdom of implementing the badging experimentation within Coursera itself.

I started questioning the ethics of delivering a course in those conditions. I could not escape thinking along the lines of nested Stanford Experiment (Should I promote some students to Community TA? What are the other 599 courses doing?). Whatever I could think of doing I somehow could find a darker side as well, associating it to one of the studies that I had read in preparing the course. I became confused, but at a much deeper level: under all these adverse conditions, I should probably have decided to stop the course, even if doing so exposed me to legal risk. Instead, I pressed on, and resolved to deliver the rest of the course via Twitter and Youtube instead (while the willing students would support each other through the Facebook and Google Plus groups as well). I made no decision about the third week peer-feedback exam. Of course, in doing all this, I also confused the students to this much deeper level.

For somewhere between 24 and 48 hours, I improvised and set out to prepare material on Twitter to support my upcoming explanations. Some of it was also intentionally confounding, likely to be misunderstood by anyone who was not involved in the course, but easily explainable with the proper context (Coursera repeatedly ignored requests to engage with me in a more public way within the course). In my (poor) judgement, confusion could be used to expose hidden assumptions through reactions of other parties. For instance, I knew this might lead Coursera to stop the course, which was fine with me.

Another "subtext" of my tweets was the risk of corrupt personalisation (June 26th) in teaching, originating from algorithmic culture. I thought it would be interesting to show this to students, centered around the World Cup taking place at the same time (Twitter, for the duration of the World Cup, offered the opportunity to do your own A/B testing by selecting which team you wanted to support).

Yet another plan was to continue the course as a Twitter based game, that could involve participants external to the course as well. Setting it up as a game would allow me to imply things without properly saying them, diffusing some of the legal risk away from myself.

I realise these are many options, but they did not need all to work and stick. And I knew I would be more free to explain them once the course finished.

When ready, on Wednesday, I removed all content and forums, except for one forum. I also pinned one student post with an encouragement to fellow students to take ownership of their learning and join the Google Plus group associated to the course (in fact, this post is what triggered me to do this at that exact time). I removed video content because I wanted to encourage students who only watch video to consult the forums and notice that something was happening beyond business as usual. At the time I still felt very wary of the final peer-feedback exercise, and did not want these students to suddenly feel cheated. I intended to explain my actions on the very limited space I had left on the Coursera forums, at the best I could within my legal constraints.

As the atmosphere had evolved between Monday and Wednesday, it was clear to me that students would be critical of switching from Coursera to another web platform. Again, this was fine as I could point them to the numerous Terms of Use I had read in preparation of the course, and show them contradictions in their own reasonings about privacy, and overreliance on sales pitches. I had already started highlighting the social network aspect of Coursera on Sunday or Monday, and ultimately wanted students to treat all those options on an equal footing, and make critical choices.

Both Twitter and Youtube afford advantages that were required for me to be able to continue with the course content originally planned (hyperlinking to specific second or Tweet), while not requiring any login to consume passively.

Fallout

At this stage (Wednesday), I was given a 24 hour deadline by Coursera to reinstate the content on their servers. This request arrived at 12:30 AM my time. When I woke up, I asked my university for ethical guidance through one channel. Before I could consult with my university, by 11AM, I was removed as an instructor and Coursera engineers started reinstating content in a suspiciously selective fashion, reeking of sanctioned censorship (part of the contracts is that the instructor has to sign away rights to modify the material).

Once my Coursera instructor rights were removed, in the interest of transparency, I immediately explained to the students what was going on through an etherpad document (for increased interactivity, still feeling under legal threat). Agreeing to a student request, I encouraged participants in that etherpad chat to share the document with the whole class. I also posted on the course forums and at the time I could not see that all my posts were being systematically deleted. I still do not know by whom.

Confusingly, some students still received e-mail notifications of my messages, and they started to suspect I was deleting them myself. In parallel, there was increasing speculation on the blogosphere that I was performing some form of social experiment. New students were not allowed to sign up, so newcomers had little access to information about what had happened in the course itself. This fed an increased paranoia of some students against me.

Despite all these misunderstandings, I was forbidden to clarify the situation with my students or the press. Later, I learned of misleading and false accusations made by Coursera to my university, and that Coursera used browsing behaviour information to support some of their claims. I also learned that Coursera had issued legal threats against my university and was told that my own legal situation was precarious. Coursera temporarily suspended their agreement with my university. My assessment is that Coursera has so far successfully manipulated the media, my university and the students to damage my credibility and introduce doubts about my integrity.

Summary

I regret the breakdown of trust that occurred during the preparation of the course between Coursera and me, since this left me with no good option when an external event (the Facebook Emotion experiment) disrupted my course plan. I think Coursera's actions after I removed the content, such as deleting my messages or misleading journalists and my university, caused undue stress to the students.

I can fully understand the perspective of students who would feel cheated or manipulated in some way since so much information was hidden away from them. I tried my best to convey that a lot was going on behind the scenes (and clearly said so many times, referring explicitly to contracts and denied requests for data transparency).

In the end, it feels like I have missed a chance to raise this debate and others more constructively. This is unfortunate since there are many more issues and opportunities of MOOCs I would have wanted to discuss in the third week:

Of course, I missed a chance, but any of the other 103 Coursera partners, of which 25 or so are European or Swiss, have the option of starting a course on the same topics. I think it is very much needed: the shared interest of students and academics is to discuss these issues directly together at scale, without channeling this interaction through an intermediate corporation with obvious monetary interest. It is not clear that that many students are actually aware of any of those issues, highlighting the need for an urgent intervention on the topic.

(I realise I have not answered all the questions or concerns that people might have about the course. The comment section is below, feel free to use it!)