This had been a sticking point in much of the Connectivism debate, so I was really interested in how Rory McGreal and team are addressing the challenge of assessment and accreditation in an open world in a formal way (recordings here of Rory’s presentation). The project (called OER University) objectives were to survey the existing methods of assessment and accreditation across the world, assess scalable approaches and document the lessons learnt in order to provision a framework for learners who are taking the OER. As the OERU website states, “(t)he OER university concept aims to create a parallel learning universe based solely on OER to augment and add value to the formal education sector”.
So here is why the problem is complex. By definition, if the learning process is open and the technologies and content on the basis of which the learning experiences are based are also open, this poses several challenges to assessing the process. It is not a controlled institutional environment and in many cases it is only a partial learning environment, suited to content (as of now) that doesn’t have supporting physical dependencies like LABs or for traditional classroom type group collaborative activities – i.e. more suited for digital self-sufficient learning processes. The other problem is formalizing a process that is inherently complex and chaotic. Furthermore, it requires resources, a challenge that the open movement has succeeded in surmounting in the creation process, but critically, not in the support process. There is also the problem of the digital, an affordance that the bulk of the global learning population has limited (if at all) access to.
The OER-U Model
The OER-U project focuses on a subset of these challenges. Open Collaboration (for learning) is constituted by four things: open curriculum, open design and development, open pedagogy and open student support. Educational Institutions are then playing the role of a service provider – open assessment services, open credential services and open community support. This is backed by the OER support infrastructure that comprises open business models, open ICT infrastructure and open student administration (a new term for me). The use cases range from OER based free self-directed learning to consuming fee-based services. Cost is a major concern and therefore two important things need to be pursued vigorously – use of technology to make the assessment process more efficient (automatic marking?) and pushing down the cost of any type of assessment (teachers in India get 2$ or lower per exam script, which in itself provides a great business case). Obviously quality assurance, a weapon of the establishment, finds its way into the project as a critical component while accrediting institutions.
The model paradox
The acceptability of the model is being promoted in comparison to the existing models. For example, for open assessment services to be credible, they “must be strictly equivalent to that for mainstream students” and therefore must involve a fee. My argument is that the world over, traditional assessment mechanisms (as also learning) have been criticised for their inability to assess students accurately and for their inability to provide a job-ready candidate. Even different components of the same system don’t really trust each other’s credentials (witness entrance testing and qualification tests like GRE and TOEFL which also serve an important need of matching limited institutional resources to most deserving students). In this, and many other senses, the OER-U model promotes a hybrid inside-the-box model, not an out-of-the-box open model.
Fearing the establishment
Part of the reason for this approach is fear. This is a new realization for me. It is the fear, as Rory remarked, that if this model takes off, the establishment (traditional universities) will mount a heavy offense (backlash) with quality as their main argument. As I think about it more, this fear has many other dimensions as well that OER proponents must contend with – acceptability by employers, acceptability by students, lack of government support and so on. And our thinking is often shaped by these fears.
These fears are real. But I would also like to point out the opportunity to do something new. World over, demand is exponentially outstripping available educational infrastructure and resources. We have the opportunity to seriously consider the emergence of other systems which are scalable, cost effective and open. We have the opportunity to master our fears. These other systems need not to be thought of in conflicting terms (either these or the traditional system), but as options to help a fast growing, often disadvantaged population to navigate around the systemic problems that plague the traditional systems.
For example, the open software / open source movement succeeded because it empowered software users. The term OER University conflicts, if there is a parallel intended, with that vision. It is like saying Microsoft should have open source initiatives (which it has), rather than saying that Microsoft should be open source. We have to think of models that empower our learners, our teachers and our employers rather than help traditional universities extend their reach as consumers and ultimate approvers of a new system. Of course, it doesn’t help that I am an incurable optimist, even as I write this!
Services in the model
So, apart from Open Assessment services, there are also Open Credentials Services. Again, by making participating institutions, and there could be other entities or individuals who should be included in scope, have to comply with having credible local accreditation, the model perpetuates the hybrid in-the-box approach. It lays the claim, further, to be a nodal organization for assessments, with any student being able to join, with any student being able to use open content and open support (from perhaps participating institutions and academic volunteers worldwide). The skew is then on assessment rather than learning and by definition, then, it differs from a traditional university in these terms because it removes formal teaching from the mix. And assessments in this way involves many other challenges – proctoring and security, for example.
So OER-U is like a melting pot which brings students (learners), teachers, academic volunteers, institutions all together, but takes ownership over assessments. It is not a new model in that sense either. There are assessment and accreditation bodies across the world.
Stephen pointed out an inherent conflict because resources for learning are intimately connected with assessment design and development. This would lead to other formalizations such as a process for certifying additional resources (not necessarily) open and even creating a cadre of certified open support resources. Rory’s response, in the context of the university, was that it was a way surely of generate an alternate revenue stream, but was open to a more distributed approach, with this being one way that students could choose from. Also, he talked about quality in the context of teaching to the test and also talked of evaluating competency based models for assessments.
So what are possible directions for change?
In my mind, and since this is Change11, the change could come from various sides. The most important factors influencing change will neither be OER or technology, nor will change come from private players. Rather they will come from either the government or from an entire distributed movement that is able to capture mindshare and shape future systems.
Why I say government is because any new initiative whose outcomes are not clearly visible (except in the vision) and which entails a single window (democratic) decision-making in the face of risk and uncertainty, quite clearly has to be backed the government (our combined resources). The government is critical because it is responsible for creating the traditional system and the onus of change rests with it. By definition of democratic systems, the onus of convincing government lies within us. Of course there are challenges to make governments think and feel this vision, but it is not an impossible task.
Why I say movement, is equally obvious. The more we can take control of our own futures, the more the chances of change actually happening will improve. Obviously too, there are challenges with this approach, and inherent contradictions, but isn’t that we are all already trying to achieve through our individual and collective efforts? Perhaps it needs to be more organized and focused.
An important feature of this change will be innovation in assessments in the OER context. Dave pointed to the need for students to be able to pass the test which is especially important professionally, so it would be important to design tests differently for different needs (for example, early learning assessment vs. late learning assessment). Some suggestions in the chat room also pointed to an absence in thinking of assessment types that we have been talking about in the past – like those covered in LAK11 (Learning Analytics), peer assessments, competency models, data trails and critical literacies. Obviously, when we are talking of open design of curriculum and content and learning environments, we must also focus on open design of and new forms of assessment.