A crisis in assessment
The huge expansion of education broadcast methods has provoked a crisis in assessment, with educators taking polarized positions. We review some of these conflicts and show how online collaborative grading with Crowdmark can unify groups of opposing stakeholders, add rigor to massive online courses, and generate a stream of data to make better evidence-based credit transfer decisions.
Machine versus traditional methods in the evaluation of essays
On the one hand, pioneers of massive open online course technology are promising a digital panacea in which robots read and assess essays using artificial intelligence. On the other hand, some traditional writing instructors are petitioning policy makers to take positions against computerized evaluation of essays.
There is a serious issue here: How can a small team of instructors possibly evaluate one hundred thousand essays fairly and without compromising standards? This issue is not restricted to essays. The same challenges arise when considering mathematical proofs, engineering design projects, reports on experiments and many other learning activities. Online collaborative grading amplifies—to arbitrary scale—human capacity to evaluate student work. By allowing for the creation and management of large teams of skilled humans, Crowdmark facilitates an expansion of traditional methods for evaluating student work. Teams of collaborating graders can generate rich training data sets to improve machine learning methods. Through Crowdmark, a new paradigm is emerging in which machine tools can be developed and deployed to help rather than replace human graders.
Rigor versus expediency in the evaluation of student work
The 2012 version of massive online courses looks a lot like Sesame Street with multiple choice questions superimposed over the video lectures. The synchronous feedback from these quick checks promotes engagement and generates short-term confirmation of learning progress. However, these tiny quizzes are not enough. Faculty, like those who opposed California’s SB 520 MOOC Bill, are defenders and pioneers of fields of knowledge. They will resist expanding credit opportunities through online courses until the time when methods for evaluating student performance in those courses meet a stringent standard. Similarly, universities will not offer credit for the completion of online courses until student performance in those courses can be assessed at a level consistent with that of students in their bricks-and-mortar classrooms.
The new paradigm generated by Crowdmark allows universities to offer courses with the same rigorous, paper-based handwritten assessments all evaluated by skilled graders as in traditional courses. Moreover, Crowdmark generates a digital portfolio of student work that can be evaluated by multiple graders to ensure consistency. A recent white paper outlines an effective business model exploiting online collaborative grading to deliver rigorous first-year university courses at 3.5% of the current cost.
Portfolio-based credit transfer arbitration
Credit transfer arbitration is crucial to both the expansion of access to higher learning and to the generation of new approaches to shorten the time to degree and save costs in the higher education system. Current methods involve a consideration of the grade earned in and the syllabus of a course. This is flimsy evidence upon which to base a credit transfer decision. Students transferring into a new program on the basis of prerequisite courses taken elsewhere may find themselves inadequately prepared for their course of study. Moreover, universities often resist transfers from community colleges based on a real or perceived difference in the level of performance required of students. These curricular misalignments obstruct or delay degree completion for some students. The proliferation of new online programs of study will add new pressure to the credit transfer system.
Better credit transfer decisions can be made based on better evidence. Crowdmark captures an entire archive of student work together with the associated feedback and scores. Students seeking transfer credit who present their Crowdmark portfolio provide registrars and other administrators with a rich set of data allowing them to make more informed program plans for transfer students. This streamlining, in turn, will allow universities to achieve better and faster rates of degree completion.