Thread by @svlevine

Data-driven design is a lot like offline RL. Want to design a drug molecule, protein, or robot? Offline model-based optimization (MBO) tackles this, and our new algorithm, conservative objective models (COMs) provides a simple approach: arxiv.org/abs/2107.06882

A thread:

The basic setup: say you have prior experimental data D={(x,y)} (e.g., drugs you've tested). How to use it to get the best drug? Well, you could train a neural net f(x) = y, then pick the best x. This is a *very* bad idea, because you'll just get an adversarial example!

This is very important: lots of recent work shows how to train really good predictive models in biology, chemistry, etc. (e.g., AlphaFold), but using these for design runs into this adversarial example problem. This is actually very similar to problems we see in offline RL!

Just like in offline RL, the problem is distributional shift: the new designs we want to evaluate might be out of distribution for our training data. So we can borrow some ideas from conservative Q-learning (CQL) to "robustify" our models, thus making them suitable for design.

Our method, conservative objective models (COMs), is very similar to adversarial training: just find "adversarial" designs that the model thinks are good, and make them look bad, while making the designs in the data look good to counterbalance this.

Just like CQL, we can guarantee that for a suitable regularizer weight, COMs gets us a lower bound on the true value of a design, providing a principled solution. It's also simple to use and implement.

Website: sites.google.com/berkeley.edu/c…
ICML talk:

Great collaboration w/ Brandon Trabucco, @aviral_kumar2, @YoungGeng. For more offline MBO papers, also check out these from our lab:
arxiv.org/abs/1912.13464
arxiv.org/abs/2102.07970

And these great papers from @jlistgarten's group:
arxiv.org/abs/1901.10060
arxiv.org/abs/2006.08052

We'll be presenting COMs at #ICML2021:

Thu Jul 22, 6:40am-6:45am PT
icml.cc/virtual/2021/s…

Poster: Thu Jul 22, 6am - 8 am PT

Or just watch the talk in advance here: