Thread by @benadida

1/ I've had the chance to carefully read the study by @umbernhard, @jhalderm, et al on Ballot Marking Devices, and I have some thoughts.

jhalderm.com/pub/papers/bmd…

2/ first, this is an extremely well done and carefully crafted study and paper. I'm blown away at the effort it must have required to set up and run. Outside of running a study during a real election, I can't imagine a better approach.

Hats off to the whole research team!

3/ also, my interpretation may be rosier than some, as I have come to believe, through many wonderful interactions with election officials, that Ballot Marking Devices are useful and possibly necessary for complex US elections and in serving voters with disabilities.

4/ so my take is that we must try hard to secure BMDs, since many millions of voters will lose the chance to vote secretly if we don't, and some elections in large counties would be a mess.

5/ So, to the paper. I'm finding the public reporting on this study to be quite a bit more negative and doomsday than I believe is warranted. Let's look at the data in the paper.

6/ With voters told they were participating in a usability (not security) study of a fake election, with a mediocre small-font ballot design (not the fault of the study, they were mimicking an existing vendor), 14% detected a malicious change in the top 5 contests.

7/ That's certainly not enough to cry victory, of course, but compared to many pessimistic claims that voters never check their ballots, it's a lot better.

8/ The study then tried a number of poll-worker interventions to increase verification. The most successful intervention yielded 85.7% *detection* rate across *all contests* -- which is quite high -- I would have expected such numbers only for top contests at best.

9/ here's why 85% is so good. It means that for every vote an evil BMD flips without detection, it needs to attempt to flip 7, and 6 of those will be noticed by voters, corrected, and possibly lead to complaints. Even with tight margins, that's a lot of complaints.

10/ Thus, the paper suggests that a good detection level would be 80% on a fairly tight election. So... Does that mean it's solved? Just do that intervention?

11/ Not quite. The intervention in question is verbal prompting *plus* a prescribed slate of choices, which means voters were handed a piece of paper they referred to when voting and could use to compare their BMD-printed paper ballot against.

12/ So, how to interpret this? The most pessimistic way is how most of the press and @jhalderm have described this: they ignore this intervention because, in a real-world scenario, few voters bring in a filled out slate of candidates to the polls.

13/ on the flip side, the overly optimistic way is to point out that, in a real election, voters have a true intent of whom they want to vote for, unlike in this study's fake election, and so the slate is a way to recreate that intention.

14/ I don't believe the overly optimistic interpretation, because a written slate is stronger than a real-election intent. But I also don't believe the overly pessimistic interpretation, which assumes no difference between a no-stakes fake election and a real election.

15/ the truth is likely somewhere in between. Where exactly, no one knows yet. Still, it doesn't seem crazy to me to think one might shoot for 50% verification when the stakes are real, the ballot is well designed, prompts are improved, and we focus on top contests.

16/ but here's the real reason why I am more optimistic than press reports: the study kept track of both whether voters *looked* at their ballot and whether they *noticed* errors. And it indicates that when voters are prompted to check *and look*, many do find errors.

17/ So we are far from declaring victory, but we also need to stop saying that voters can't verify their ballots. They can. Even without a slate. They need to be nudged / given the opportunity to actually look. When they look, they find errors. Even in elections with zero stakes.

18/ so let's do that. Let's figure out the myriad ways we can design a voter flow that will prompt them to look. Maybe it's a poll worker message that says "sometimes the printer makes a mistake, make sure you check your ballot."

19/ or maybe it's a dedicated space for voters to check their ballot before casting it in the ballot box.

20/ After all, when it comes to other parts of the election chain of custody, there is a ton of science and checkpoints to make sure ballots are accounted for properly before, during, and after the election. The same careful design will be necessary for BMDs.

21/ We don't yet know how to build an amazing BMD. But we now have important clues. So let's pursue them. Because, in the real world, millions of voters need BMDs, millions more prefer them, and large counties with vote centers find them necessary. So let's make them secure.