Thread by @ShriramKMurthi

1/ This was a fun question with lots of interesting answers, though in the end Ian was pretty disappointed by the replies. But I thought it would be interesting to relate the history of how to language levels of @racketlang came to be. ↵

https://twitter.com/IanArawjo/status/1522024072506445825

2/ It started with a pretty user-centric method, which was observing what students were doing. A LOT. We didn't set up cams, didn't log keys (this was 1995). We (Matthew, Robby, I) were all TAs, and even when not, spent a lot of time walking around in labs.

3/ We would then debrief with each other. We noticed that students were running into certain very particular kinds of frustrations, and that these had nothing to do with the problem, but rather the language. For instance:

4/ If you write

(define (F l)
(cond
[(empty? l) …]
[(cons? l) …]))

and pass a non-list, F would return `void` (by falling through the cond). This would manifest in some later computation. Students had never heard of void, much less know where it came from.

5/ Initially, of course, we would just TEACH the language's semantics. Look, here's what a void is, here's what Scheme does, and so on. Students picked up on it quickly enough. But the thing that niggled at us was we couldn't explain WHY it had to be that way.

6/ I think you will see a lot of parallels here to what happens every day in programming pedagogy. With any language, there are a whole bunch of warts that students run into that are not relevant to learning the *subject* (as opposed to the language). And we deflect around them.

7/ The difference was, we realized, we didn't need to make excuses. We controlled the (then) DrScheme/PLT Scheme implementation. What if we just … CHANGE it to NOT do the dumb and annoying and difficult-to-justify thing?

8/ As we looked more closely, we found more and more things that we didn't like because they were tripping up students. We eventually realized that we were effectively defining a different programming language: one that was a student-appropriate sub-language of Scheme.

9/ But why stop there? We noted that when you use a BOOK, the language you use grows with the chapters. Why don't IDEs reflect that? This was the same time the AP CS Board was engaging in fantasies about "subsets" of C++…that were unenforced. If it's not enforced, it's not real.

10/ Notice the subtle but very significant shift in methodology. We'd gone from user-driven bottom-up observations to authoritative, top-down decisions based on a very deep expertise in (a) semantics, (b) program reasoning, (c) curriculum and (d) implementation. Fordian/Jobsian…

11/ From there we rolled out a set of language levels that reflected our curriculum design. The abstraction into #lang, new linguistic principles, programmability, etc., all came later. We first addressed the immediate needs of our students.

12/ How did we know it "worked"? We just went back and did a lot more lurking and helping in labs. (I TAed about 5x the requirement of my PhD program.) Lots of anecdotal observations led to lots of adjustments, year after year. (Ditto error messages!)

13/ None of us knew the first thing about HCI methods. If we'd written a paper we'd have been laughed at. (We wrote a paper for SIGCSE and it was soundly rejected.) But here's the thing:

14/ We didn't set out to write papers. We set out to make our students' lives better. And we could tell that their experience was improving year after year. That was our "evaluation". And honestly, I am just fine with that.

15/ In retrospect, this idea was so obviously good that several people have cloned it, giving it their own names, sometimes not even properly citing the original. But you know what? That's how you know you won. (-:

And there's a coda:

16/ When I interviewed at Maryland, one of my 1-on-1s was with @bederson. I mentioned language levels and he said something like "Oh, you do HCI?" I recoiled in terror: no, I said, I didn't know a darn thing about HCI! I told him what we'd done (namely, the above). …

17/ With extraordinary (and probably undeserved) grace, Ben said, "Oh no, that's okay, I'd consider that HCI!" I think that was super generous of him. And I think this highlights research leaders: they care about progress rather than gatekeeping about methods. •