Thread by @mikamckinnon

Academic Twitter is having a slapfest re: cheating on online exams.

For ~20yrs, I’ve participated in pedagogical research on how to effectively assess student learning.

ProTip:
It’s impossible to block all forms of cheating, but it’s impossible to cheat on a good assessment.

It is really easy to write a bad exam.

Passing by elimination & luck.
Passing by brute memorization or regurgitation without comprehension.
Answers revealed by question grammar.
Inherent assumptions confounding what’s actually tested.
Misaligned objectives.

So. many. ways.

I got nervous before giving my students their midterm yesterday.

Why?
Because I wanted it to be a Good Test where they walked away feeling like they showed me what they learned. A good test leaves you proud (or at least content it’s an accurate reflection), not frustrated.

Sometimes limited resources mean trade-offs, but no-note time-limited exams are rarely a good reflection of student learning OR match for how they’ll later use the content in a professional capacity.

They’re usually a test of short-term memory & handwriting speed.

Oops.

For online exams:

Any “let me Google that for you” question is bad.

Anything tightly time-constrained is testing how fast students think, not what they know.

Anything that repeats between students isn’t taking adequate advantage of the technology.

So how can you cheat?

Q: Multiple submission to brute-force the answer?
A: ...your learning platform sux & should be ashamed of itself.

Q: Formatting errors?
A: ...your learning platform sux etc.

Q: But I give students a second chance!
A: Good practice! Unless it’s elimination & luck.

Let’s assume your objective is for students to learn, and that you’d be delighted if everyone scored 100%.

Hint: This should be your objective.

Q: But I grade on a curve!
A: Then you‘ve got unclear learning outcomes, are crap at teaching, or are an evil jackass. Fix it.

If you want your students to succeed, what do you do? What does a test look like? How do you help them prepare for it?

What even is the purpose of an exam?

What EXACTLY are you assessing? How do you assess that, and ONLY that?

It takes a lot of thinking to write a good exam.

ProTip: Clearly-stated learning objectives are utterly vital to successful education.

Novices need scaffolding to become experts. You need to TELL students what you want to learn so they can fit pieces together.

Aside: Objectives also make class prep so much easier.

Writing good learning objectives (per course & per lesson) is HARD.

Good objectives are specific, concise, & measurable, with each containing behavior, conditions, & criteria.

More info: blogs.ubc.ca/lled3602015/fi…

Once you have objectives, your assessments should test those objectives and ONLY those objectives.

Note none of the suggested verb stems are “Memorize.”

Note how many verb stems require critical thinking that you can’t cheat around by asking google or sneaking a peek at notes.

Open secret: Most students learn more during test prep than any other time.

So why not harness that?

Give them a structured study guide skeleton (intro) or flat-out open note (advanced). They’ll learn SO MUCH during prep.

Bonus: Accurately reflects real-world conditions.

But the best tests don’t just assess students; they help students learn during the test.

How?

Hint: Best way to learn is to teach.

Try running group exams after individual exams. They’ll catch each other’s misconceptions & provide immediate feedback.

I’m teaching intro geo at 2 universities this year with wildly different student populations, learning objectives, teaching support, & instructor freedom.

Here’s how assessment plays out:

Uni 1:
Challenge: International pop makes it easy to accidentally test English fluency.

Challenge: Little-to-no STEM in majors/professions alters fundamental point of class into more “appreciation” than “execution”

Exam: study guide skeleton + questions on practice of science

Uni 2:
Challenge: Enormous class size

Challenge: Must consistently fulfill professional licensing requirements

Exams: Closed-note multiple choice (sigh) drawn directly from objectives. Individual exams immediately followed by group exams.

Q: How do I know if I created a Good Test?
A: Is every question drawn from the learning objectives?

Analyze how your students get questions wrong. Patterns may indicate common misconceptions or poor phrasing. Their performance is partially a test of your teaching.

I’ve been a good student & a bad one. I’ve run out of time or finished disturbingly early. I’ve spent hours studying the “wrong” thing or resorted to brute-force memorization.

I’ve failed & aced tests.

I was only frustrated with grades that didn’t reflect what I learned.

Q: If everyone gets 100%, doesn’t that mean my class is too easy?
A: Maybe, or it could mean you’re a kick-ass teacher with awesomely motivated students.

Pre-test/post-test pairs will give you insight into what students walk in already knowing vs what they learn from class.

Q: But I don’t have the time or the resources or support or even the skill to create Good Assessments...
A: It’s ok. None of us do.

Simple ideas can be really hard. Try something, analyze, iterate, & improve. Even if it’s just “less bad.” Teaching is a work in progress.