This is a very good question about Examplar, so I'm highlighting it and responding. Also a good reminder about something hard-core CSists often forget: tools are never divorced from pedagogy. There's always at least an implicit one (literally the tool's "operating system"). »
There is a very real reason to be concerned about the "gamification" element here. Some students are indeed going to obsess about "closing both circles". (The left one they always should. The right one is the problem case. But both _could_ take hours.) »
Examplar currently offers no feedback on failing wheats/chaffs. It easily can. Past versions of auto-grading student examples (in batch mode, by us, w/out tool for students) would give "filenames" of failing wheats/passing chaffs. So easy to imagine incremental output here. »
Let's also remember: this is a student who *has the wrong problem spec*. Trade-off between "blissfully submitting wrong solution and being frustrated at grade" and "struggling to get the problem definition right up front". »
The one who doesn't care about the former likely doesn't much about the latter. But if having the tool facing them forces them to care more and they end up turning in better work in return for more time … I consider that a win. But most of all: »
I've told students to stop trying to get all chaffs after about "15 minutes". What I really expect is one of two things to happen, both of which I consider very good: »
1. They carefully re-read the problem statement. Maybe they were sloppy. Maybe there were careful, but glossed over a subtle/side/seemingly thrown-away remark. Now they're hunting for it and appreciate its significance. Now they get the problem spec more fully. Or; »
2. They go to TA hours and say, "This damn thing is wrong. I've tried median as left of center and right of center and neither works. Here are my examples, and both fail wheat, and the highlight shows both of them. Median should be in the list. WTF?" »
I think that's an amazing conversation to have happen, and we've never had a particularly good way of enabling it before. If Examplar enables that, I would be over the moon. «»
