Thread by @Dr_Joe_McIntyre

#onlinelearning #ERL in the Chaos of of #COVID19

Thread 3: Substituting Exams as Final Assessments

OK, it has been a while - but #pandemicademia...

In this thread I will outline some thoughts on final assessments, and developing responses to the loss of traditional exams

Firstly, you will note the change of nomenclature.

Over the course of the last month, I have been increasingly convinced of the importance of describing what we are doing as ** 'Emergency Remote Learning' **('ERL'), and not as Online Learning.

Because (1) what we are doing is not normal - we don't have the time, resources, experience or capacity to do this 'properly' - we cant lose sight of the emergency context; and (2) out of deference to the excellent OL people and to differentiate the ambitions

I had originally planned to do a thread on specific ERL techniques (it may yet happen...) but the current pressing concern for many of us is the development of appropriate assessment techniques to replace the traditional sit down exam.

The worst thing we can do right now is to try and REPLICATE a sit-down exam. This leads to the dangerous and inherently controversial rabbit hole of proctoring software:

theguardian.com/australia-news…

This approach is flawed, because it elevates TRADITION over all other relevant values that should inform the decision. These including issues of:

▪️ equity;
▪️ privacy;
▪️ integrity;
▪️ pedagogy;
▪️ health & well-being;
▪️ logistics;
▪️ workload;
▪️ costs.

In assessment design in the ERL #Covid19 context, will require compromises will to be made to strike in balancing these of potentially conflicting objectives. That design should be driven more by pedagogical purpose rather than tradition.

The Traditional Exam

The traditional exam has a number of objectives and advantages, some of which may be important to maintain in a COVID-19 driven ERL environment, and other less vital/less possible. Design should be driven by these ideas rather than a given model

The following tables seeks to unpack the underlying values that have traditionally justified the reliance on exams as a form of assessment - and make some comment about their replicability

1. Pedagogical Advantages

2. Process Advantages

3. Assessment Advantages

In my opinion, the process considerations of replication pose a major challenge to the adoption of a straight ‘online exam’ approach – and while there may be possible solutions, it is difficult to justify their use given that pedagogical advantages may be met in other ways.

In particular, issues of equity and lack of standardisation of conditions make it improper to set timed-exams under current conditions.

Doing so now will only lead to unnecessary stress, anxiety and future appeals.

Remember this is the fundamental framing:

My preference is that every School present staff with a small number of regulated options to substitute for the exam, depending upon which pedagogical purpose the coordinator wishes to pursue.

The following are a couple of possible models, with comments one related values

It is then up to the coordinator to select which option they wish - for example if decision-making under time pressure is the skill they want to test: Option 1. Depth of analysis: Option 2.

There is no RIGHT answer here - we are striking a balance between competing interests.

However there is a WRONG process. We should not say 'we had an exam how can we recreate an exam'.

Rather, we must ask 'why did we have an exam' - what where they advantages and justifications in its use.

We should then ask 'which objectives remain valid', 'which are impeded', 'which are impossible' and 'which are critical'

These PROCESS questions can help inform the design of our substituting assessment - which may or may not look somewhat like an exam

NOTE 1: There will be an number of policy restrictions in each Uni that may need to be negotiated in this process (eg workload, feedback, timelines). I can't address these here, other than to say the best tool you have is a clear rational for choices

NOTE 2: Remember all that this is an ERL context. Nothing is normal. Everyone remains stressed. The Heirarchy of Needs remains valid. Do your best!

Thanks and Stay Safe