1. A thread for my fellow college and university instructors, though high school teachers and students may be interested as well. Amidst the pandemic we're all trying to adapt to giving and taking exams online.

The "simple" solution is to keep writing exams as we always have...
2. ...and use the multifarious tools of digital surveillance to reproduce the conditions of a supervised classroom exam.

But think for a moment about what this does. First:


Software for online exams can require a student to scan their whole room before testing.
3. Not everyone wants to reveal their home conditions. I know students who sit in their cars in a McDonald's parking lot, using the restaurant wifi, because they don't have their own wifi at home or, in some cases, a place to call home at all.
4. These packages allow an instructor to watch a student from up close, over a webcam, for the entire duration of an exam without the student's knowledge. The potential creep factor on that goes right off the scale.
5. They can penalize a student who interrupts their test-taking activity to turn away, to speak to someone else, to leave the room — assuming that the student is cheating, when more likely they are attending to child or parent or grandparent.
6. They can monitor all internet traffic, take screenshots of the student's desktop, track the mouse around the screen, compare the face on the webcam with a scanned ID, even use the webcam for eye-tracking to make sure the student does not look in an unsanctioned direction.
7. Supposedly they don't access pre-existing files on the computer unless those files are opened during the exam. Supposedly.

But these are proprietary technologies, not open source code, so who knows what they do.

You're asking students to install spyware, plain and simple.

Using one of these surveillance programs sends a signal to every student in your class that they can't be trusted. And then showing a lack of trust signals a lack of respect. I won't go into detail, but carefully consider the cost of that.

While I have not verified the authenticity of these instructions for a math exam, they illustrate the level of detail that students would to deal with when taking an exam on the higher security settings of these programs.


I've seen very little from the providers of this software about false positive rates. In my 20 years of teaching, I have accused students of cheating—but I've only done so in the face overwhelming evidence.
11. Are you going to trust a software package that claims a student had unauthorized programs running? Think about the array of operating systems and background applications—virus checkers, backup software, indexing apps, etc. Would you do so w/o knowing the false positive rate?
12. What if the software says the students eyes left the screen too often? Do you trust the face recognition and eye-tracking to have gotten that right? Has it been tested on all ethnic groups? Or, like all too much of facial recognition, was it trained mostly on white faces?

Just about every ill effect of these exam surveillance packages falls harder on students of color, first-generation students, and students in other protected classes.

I'll explore just a few of the ways. Some tie in to the factors above.
14. Lower SES students may be less likely to have access to spaces (1) that they are comfortable allowing others to see and (2) where they can take the test in privacy without interruption—which may be necessary to avoid being accused of cheating.
15. They may also be less likely to be running the up-to-date hardware and operating systems on which I suspect these surveillance programs will have been most thoroughly tested. Again, that puts these students at higher risk of being falsely accused.
16. Non-traditional students (among others) may not have the luxury, during a pandemic, of setting aside all family responsibilities to avoid all interaction with others and to keep eyes solely on the allowed part of the computer screen. Again, putting them at higher risk.
17. Signaling that students are not trusted may have a very different impact based on the student's background. A student who has always assumed he will go to college and feels that he belongs there may react differently from a first-gen student questioning her place in academia.
18. The cost of falsely accusing a student of cheating may be very different as well. Someone with my privileged background might be able to shrug it off and say "man, that software is bullsh*t." Someone used to feeling like others question their right and ability to be there...
19 ...will have a very different response. Even when the instructor does not directly accuse a student of cheating, but simply notes deviations from required protocol ("you got up during the exam", "you had background apps running"), this can be traumatic.
20. In college I dealt with an anxiety disorder. Even if the teacher waved it off, having my test flagged for potential cheating by a software algorithm would have sent me into a spiral that could have ruined my mental health for a quarter, or longer.
21. In short, the false positives that these systems generate will amplify feelings of not belonging and non-inclusion on the part of precisely the students that we most want to be supporting in their academic careers.
22. I see electronic surveillance for testing as a fundamental affront to the privacy and dignity of our students.
23. At the same time, it is likely to disproportionately label students of color and other protected classes as cheaters—and when such accusations are made, they will disproportionate harm to students in those groups.
24. If this past June you made a commitment to antiracism, consider whether there are racist consequences of electronic surveillance software for exam-taking, along the lines of those that I've laid out above — and then act accordingly.
25. --- (deep breath) ---
26. So what can we do instead?

As I mentioned in the first tweet of this thread, writing our tests as before and using these software packages is the "simple" solution.

We don't have to do things that way.

We can take it upon ourselves to do the heavy lifting needed...
27. ...to provide fair and accurate assessment without disrespecting our students or denying their dignity.

In the process, we may be able to improve both the quality of our assessment and the education value of the assessment experience.
28. We can transition from testing students at lower levels of Bloom's taxonomy (whatever you may think of its specifics, it captures the general notion I'm trying to convey), to testing their ability to analyze, evaluate, and create.
29. Cheating — or, more positively, open books and open use of search engines and in the internet — is easy for exams that test your ability to remember facts and definitions, and probably useful for tests that get at understanding and application.
30. That's fine. With a few very specific exceptions, if pre-pandemic we were primarily testing our students ability to remember, we were failing at our job — higher education should be about so much more and assessing only at that level does our students a deep injustice.
31. Consider shifting the balance of courses away from examinations and toward other kinds of work. This can change to nature of the evaluation you are doing for the better, and will be much appreciated to the many very good students who suffer from test anxiety.
32. When you do give exams, write them to be open book, open note, open internet. It's not *easy*, but it's not that hard either once you get the hang of it. I find that this automatically pushes my test questions up onto the higher levels of Bloom's taxonomy.
33. When you are asking your students to *evaluate* a scenario that you have developed yourself specifically for the exam, the internet will not provide answers—though it may support the process of deriving answers in a very positive way.
34. When you ask your students to *create* their own scenario, or anecdote, or example, or narrative, or problem, or proof, or analysis, or argument, based on an innovative prompt, google isn't going to have the answers but it may help them create deeper, better answers.
35. You can think about it kind of like improvisational theatre. How do you know the troupe isn't running a rehearsed skit? They took the words "duck", "walnut", and "steam shovel" from the audience and crafted their story around those.

Writing test questions can be similar.
36. For example, on in-person exams I sometimes ask my students to write their own exam question that they think I should have asked, and provide an answer key.

If you do that with open internet, they can just search for one.
37. But if you choose an event in yesterday's newspaper and specify that their question needs to relate to that, while bringing in something else that you specify, they are the ones who have to do the creating. The internet becomes a resource, not a source.
38. These are the skills we should be evaluating our students on anyway.

Once they leave school, they'll have books, notes, internet...and they'll be asked to analyze, evaluate, and create.

Let's use this pandemic as a chance to restructure our own practices accordingly.
39. Of course all of this stuff is hard. I'm not getting it right yet, and a lot of the latter part of the thread is based on things I've learned the hard way, by screwing up. We don't have to get it perfect the first time around.
40. This year is also a good year for lenience, patience, and empathy.

If your test ends up too easy, so be it. This fall, the world is too hard and a bit of balance would do everyone some good.

Thank you for reading this far, and good luck!


• • •

Missing some Tweet in this thread? You can try to force a refresh

Keep Current with Carl T. Bergstrom

Carl T. Bergstrom Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!


Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @CT_Bergstrom

1 Nov
Trump's pandemic advisor Scott Atlas touts his own appearance on the Russian media outlet RT.

The piece begins with anti-American propaganda: "With two days to go before an election in a country which has been militarily involved in 98% of all countries on earth....." Image
"...having invaded 84 of the UN's 193 countries."

That is the intro to Scott's segment.

So now we've literally got the White House making guest appearances in Russian propaganda flicks. Not tricked, Rudy-style — willingly, and boasting about it on twitter.
People are saying "that's not propaganda, that's data."


True or not (I haven't checked), it's propaganda, because it's *utterly* irrelevant to the story at hand.

It's amazing fast y'all switched from cosplaying Red Dawn to carrying water for Mother Russia.
Read 5 tweets
30 Oct
How are you planning to avoid COVID as we head into winter?

Jade eggs that generate abnormal lithospheric long-wavelength magnetic anomalies?

Coughing into your telephone for an AI?

Eating cucumber and cabbage?

Have faith in Scott Atlas?

"The discovery of the chiral-induced spin selectivity ef-fect suggests that a resonant external magneticfield could alter the spin state of electrons in biogenic moleculesand result in the magnetic catalysis of aberrant molecules and disease."
"AI techniques can produce a free, non-invasive, real-time, any-time, instantly distributable, large-scale COVID-19 asymptomatic screening tool"

Read 5 tweets
29 Oct
This is an attractive visualization about COVID transmission risks. But I'm puzzled about something. With risks this high, how could we have R0 around 2.5 or 3?

Lots of people suggesting that the answer lies in the dispersion factor k.

That is *absolutely* not what the article suggests. In the image below, the claim is only consistent with a very high k value.
If the answer lies in the dispersion factor, it would *possible* that the large number of people shown in the data graphics would be infected, but with high probability, zero or at least very few would be in each of these scenarios.
Read 9 tweets
27 Oct
This was probably my best twitter thread ever.
To be clear, no such book exists. The point was that before and during his psychic break, Philip K Dick anticipated much of the insanity of the present day. We see it in Man in the High Castle, The Valis Trilogy, The Simulcra, The Penultimate Truth, Flow My Tears, everywhere.
Why is it a sequel to Man In The High Castle? I don’t want to put any spoilers here, but for those who have read it, think about Juliana’s epiphany at the end of the book.
Read 4 tweets
25 Oct
The vastly underappreciated Ronald Englefield quotes a psychology text: "The significant condition for play seems to be a metabolic state which is conducive to a high level of activity when there are no stimuli with which to contend leading to the serious business of living."
"In other words", Englefield explains, "animals tend to play when they feel lively and have nothing better to do."

This example follows an equally brutal dismantling of John Dewey—less than a full page into Englefield's _Critique of Pure Verbiage_.

A new paper from Zachariah Brown, @EricAnicich, and @AdamGalinsky asks why people use this sort of bullshit jargon.
Read 4 tweets
23 Oct
Closing schools for 6 weeks is one thing; closing them for a year is another entirely. We need to prioritize reopening schools, especially K-5, over just about everything else. Given age-specific disease dynamics, this should be possible, esp. w/ testing.

Since the comments are filling up the predictable responses, I'm not saying that we should immediately throw open the doors without taking precautions.

I *am* saying that getting K-5 open is something that with concerted investment and ingenuity we should be able to do.
Please re-read my original post.

I claimed that opening K-5 should be a priority.

I did not claim that opening K-5 tomorrow without additional precautions would be safe or even advisable.
Read 8 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!