,
34 tweets,
7 min read
Read on Twitter
As I sit here grading finals, I realized I could instead be on Twitter giving advice about*writing* exams for all the new AP's out there!
It's a lot harder than it seems to write a good exam.
It's a lot harder than it seems to write a good exam.
1// You need to figure out what you are testing. As obvious as it's seems.... If you are a sequence course, you need to make sure you are testing students on their preparation for the next class in the sequence.
For a topics course, you decide. Be clear.
For a topics course, you decide. Be clear.
2// What matters is the distribution, not the average. Your "50 point average" is not impressive, it shows you don't know what your students know.
Include 10%-ish F-bombs (most students will get near perfect scores) and 10%-ish A-makers (few students will get perfect scores).
Include 10%-ish F-bombs (most students will get near perfect scores) and 10%-ish A-makers (few students will get perfect scores).
3// Provide your past exams for study.
Why? Whether Coursehero, or a student society, or athletes, there is a group of students with copies of your past exams. Accept it and equal the playing field.
Why? Whether Coursehero, or a student society, or athletes, there is a group of students with copies of your past exams. Accept it and equal the playing field.
4// Figuring out optimal length is hard. People say things like "1 point a minute" which is helpful except...you set the points as well. 😂
You figure it out through experience.
You figure it out through experience.
5// Students may interpret questions in a way you didn't anticipate. If they worked it correctly given *their* interpretation: give them the points.
You wrote an ambiguous question, they aren't the experts in the field, or mind readers.
You wrote an ambiguous question, they aren't the experts in the field, or mind readers.
6// My favorite "tricks"?
I allows students to write "I don't know" as an answer for 2 points on any question (range is 5 to 15 points). No hedging: it's all or nothing.
My benefit: speeds up grading, & no guilt when giving a student 0 points.
Their benefit: denial is harder.
I allows students to write "I don't know" as an answer for 2 points on any question (range is 5 to 15 points). No hedging: it's all or nothing.
My benefit: speeds up grading, & no guilt when giving a student 0 points.
Their benefit: denial is harder.
7//
(Advanced) Design the midterm to have a diagnostic element, esp. in principles/intermediate/toolkit classes. Is it math they are struggling with? Short answer? Graphs? This well help students identify and focus their attention, esp. if the problem is not the course material.
(Advanced) Design the midterm to have a diagnostic element, esp. in principles/intermediate/toolkit classes. Is it math they are struggling with? Short answer? Graphs? This well help students identify and focus their attention, esp. if the problem is not the course material.
8// In my upper division courses I would divide the material and have each student write question+answer in one assigned segment as a open-note/book/collaborative take home. (10% of exam grade).
They then submitted their question+answer to me, which I posted (ungraded).
They then submitted their question+answer to me, which I posted (ungraded).
9// I then blended the questions, flipped things around, etc. Grading the "take home" was a pain, but writing the in class exam was a joy (and interesting!).
I would let them use the class prior to the exam as a review session, where they discussed the problem they submitted.
I would let them use the class prior to the exam as a review session, where they discussed the problem they submitted.
10// Don't make your exams about the time constraint. I was guilty of this originally: students had to work fast to get my exam done.
I now aim for 3/4 of the exam time to give students the chance to review their answers, or mess up and recover on a question.
I now aim for 3/4 of the exam time to give students the chance to review their answers, or mess up and recover on a question.
11// I *do* include time bleeders. (solving for surplus/calculating elasticity/equilibrium, etc).
If you know what you're doing, it's fast. If you don't, it's slow. It may cost you the ability to go back and review the exam, but you should still have time to finish.
If you know what you're doing, it's fast. If you don't, it's slow. It may cost you the ability to go back and review the exam, but you should still have time to finish.
12// The exam is not the place for the material you've never asked about before. Restrict yourself to concepts you've covered on homeworks, in class problems/discussions/practice exams.
13// Why? Students have limited time to dedicate to your course: others exams, jobs, personal lives (good events or bad). Even those that care may not have time to study the minutae. Remember: you are only trying to test their knowledge & understanding, not their time constraint.
14// This does mean you have to assign your homework/classes with the exam in mind. That is part of "course design". All elements of your course should combine to help students obtain mastery of the material that matters for your class.
15// WRITE THE ANSWER AND GRADING KEY BEFORE GIVING THE EXAM.
You will need to edit the GK once you start grading, but it will help make sure (1) you have included the required information and made the problem doable.
(2) How the question contributes to your distribution.
You will need to edit the GK once you start grading, but it will help make sure (1) you have included the required information and made the problem doable.
(2) How the question contributes to your distribution.
Seriously. Do this. I failed a prelim because the question was not solvable so grading based on "how well you show understanding". You're selecting for students who think like you: Not the point if you're a teacher.
(50% failed the prelim, & obviously I eventually passed).
(50% failed the prelim, & obviously I eventually passed).
16// "Buying" answers.
This is something I've done for more math-intensive courses. Students can ask me to write/correct their exam answer during the exam but it will cost them points. (Use a red pen so it's really clear you've written something).
This is something I've done for more math-intensive courses. Students can ask me to write/correct their exam answer during the exam but it will cost them points. (Use a red pen so it's really clear you've written something).
17// This is helpful for exams where the questions ladder: for example "(a) solve for equilibrium. (b) In the equilibrium you found, what happens if money supply is increased." I could provide the answers for (a) (locks out their ability to get points for it), maybe just the FOC.
18// "Assessment design" is a very useful phrase to keep in mind w/exams.
You are assessing the students mastery of the material. It is not about your ego. It's not about establishing a reputation for a difficult class.
You are assessing the students mastery of the material. It is not about your ego. It's not about establishing a reputation for a difficult class.
19// Multiple choice (MC) questions? Students get training in MC exams due to the SAT/ACT and I'm concerned that I can't write questions that don't implicitly benefit the students who could afford that training.
20// I value giving students in my class the opportunity to interpret the question in a way I didn't intend. I prefer to keep the opportunity to give partial/full credit in that case.
So: I don't use MC. But I've been convinced that a good MC exam is possible *if* well designed.
So: I don't use MC. But I've been convinced that a good MC exam is possible *if* well designed.
21// The problem is that most MC exams are designed, first & foremost, to reduce grading burden. Most are not well designed to assess students knowledge.
Do not assume the MC included in the textbook exam guide are well designed either! Textbooks are rarely written by teachers.
Do not assume the MC included in the textbook exam guide are well designed either! Textbooks are rarely written by teachers.
22// Good exam design means you're not accidently discriminating. Avoiding binding time constraints, and providing equal access to the relevant study materials will get you most of the way there.
23// Some things I do DURING the exam. I bring in my laptop and project the time on the overhead screen (I think I use clocktab?). This helps students manage their time of they don't wear watches (many of them wear smart watches, which I make them remove).
24// Keep an eye in the air quality in your classroom. Many campuses will turn off air circulation during exams because "classes aren't in session". PPM above 1000 is associated with decreased cognitive function.
25// I proctor my own exams, and walk around when I do: it makes it easier for students to engage because it can be more discrete when they ask for help: not everyone sees me charging down the aisle to help them.
26// I also tell students they should feel free to ask me any question, and *I* will decide whether or not I can answer it fairly.
This helps students don't want to seem "pushy" or engage in actions that could even vaguely appear like they are asking for an unfair advantage.
This helps students don't want to seem "pushy" or engage in actions that could even vaguely appear like they are asking for an unfair advantage.
26// I write the answers to common questions on the board. That way students can refer to it if they were "in a groove" and my announcement didn't stick.
I'll send a picture of the board to accessibility services halfway through the exam so that all students have the same info.
I'll send a picture of the board to accessibility services halfway through the exam so that all students have the same info.
27// Insider tip: If you're new to teaching in economics, check out the physics discipline's teaching resources. They face some of the same technical contraints we do, but as a discipline have been far more proactive in optimizing teaching methodology.
It's foreshadowing your student's reactions if they get the question wrong.
I also teach my students the F-bomb & A-maker terms in context of my exam design. I've found it helps students who get anxious/obsessive because "the question seems too easy".
I also teach my students the F-bomb & A-maker terms in context of my exam design. I've found it helps students who get anxious/obsessive because "the question seems too easy".
28// I do the following on my 1st midterms for 1st-year-1st-semester heavy courses (Principles of Micro).
The 2nd-to-last exam q asks whether they think their score is above, below, or approximately average. The last q asks them to guess their score (margin of 5) w/o these 2 qs.
The 2nd-to-last exam q asks whether they think their score is above, below, or approximately average. The last q asks them to guess their score (margin of 5) w/o these 2 qs.
29// Administratively, it's a mild pain because you have to grade it after you've totaled. Putting it on homework is a feasible alternative if you have one due before returning the exam.
The question *must* have points associated with it to avoid false bravado/modesty answer.
The question *must* have points associated with it to avoid false bravado/modesty answer.
30// Why do I do this?
First years actually are pretty good at knowing where they fall in the distribution (below/at/above) but are wildly optimistic about the score associated with that rank ("Average, score of 88"). The q-combo helps them contextualize/process their exam score.
First years actually are pretty good at knowing where they fall in the distribution (below/at/above) but are wildly optimistic about the score associated with that rank ("Average, score of 88"). The q-combo helps them contextualize/process their exam score.
