Thread by @rose_e_wang

How can we algorithmically figure out what our model doesn’t know, and then construct datasets to improve it?
We tackle this question in “Know thy student: Interactive learning with Gaussian processes” at #ICLR2022 @cells2societies workshop.
Paper: openreview.net/pdf?id=rpGGNrM…
[1/N]

We cast this problem as a teacher-student setup where the teacher must first interact to diagnose 🧪the student (the model), before teaching 👩‍🏫(constructing the training dataset).
[2/N]

Eg. in an offline reinforcement learning setting, the student must navigate to the goal (green). The teacher determines states (yellow) the student has explored and accomplishes this task. The teacher can then construct demonstrations from states (orange) the student fails.
[3/N]

The algorithm is pretty simple:
1. Diagnose 🧪: Infer what the student doesn’t know (eg. w/ Gaussian process) by probing & receiving feedback from student (eg. did student accomplish task from state?)
2. Teach 👩‍🏫: Use posterior to construct a dataset (eg. demonstrations)
[4/N]

Note: The teacher can’t exhaustively probe the student esp. if (a) the state space is huge [this would take too long!], and (b) with limited communication [eg. if the student were a human, they would get exhausted!]

[5/N]

Lately, I’ve been super excited about teacher-student settings & thinking about how we can enable machines to (one day) reliably interact & _teach_ humans! If you’re interested in this direction, let's chat at @cells2societies poster session Fri April 29 8:15-9:05am PT!
[6/N]

Paper: openreview.net/pdf?id=rpGGNrM… w/ the amazing @mike_h_wu & Noah Goodman

I’m grateful to many for sharing their insight, including my co-authors, Xiaojin Zhu (through his machine teaching works), @jayelmnop @DorsaSadigh @EmmaBrunskill @andyshih_ @willieneis @polynoamial
[N/N]