Thread by @yisongyue

Seems like a great time to plug AI/ML research at @Caltech!

A thread below 👇

1/N

https://twitter.com/hardmaru/status/1298114780763316227

We studied how to personalize the (implanted) neuromodulation of paralyzed subjects to help them stand. Led to novel bandit & Bayesian opt algs that learn via subjective preferences while exploring safely. Work by @YananSui

arxiv.org/abs/1806.07555
arxiv.org/abs/1705.00253

2/N

We built upon this work to address personalized gait optimization of exoskeletons.

arxiv.org/abs/1909.12316
(@icra2020 Best Paper Award)

Project: roams.caltech.edu



3/N

We studied how to properly blend learning & physics to achieve state-of-the-art in agile drone control. Our first result was the Neural Lander. Work led by
@GuanyaShi.

arxiv.org/abs/1811.08027

caltech.edu/about/news/neu…

4/N

We extended Neural Lander to arrive at Neural Swarm, to model close-proximity disturbances such as downwash. By learning to model such disturbances, we can design controllers that can do tight agile flight formations.

arxiv.org/abs/2003.02992

caltech.edu/about/news/mac…

5/N

The same press release describes work on fast controller design for cheap compute, called GLAS. After all, you can't put giant neural nets on small drones. Work lead by @BenRiviere2.

arxiv.org/abs/2002.11807

caltech.edu/about/news/mac…

6/N

We are very interested in "learning to optimize", and when using learning can actually outperform existing solvers, e.g., in real wall-clock time. That led to our discovery of learning-augmented large neighborhood search that can outperform Gurobi.

arxiv.org/abs/2004.00422

7/N

We've recently begun studying how to integrate "learning to optimize" ideas into the space navigation systems (e.g., those used on the Mars Rovers). Amazing collaboration with @NASAJPL that will hopefully lead to deep imitation learning on Mars on day :)

8/N

Our work on robotics led to studying learning for safety-critical control. Novel theory that blends learning & control. Work led by @anqi_liu33 & @Yashwanth_Nakka & others.

arxiv.org/abs/1906.05819
arxiv.org/abs/2005.04374
arxiv.org/abs/1912.10099



9/N

Our interests in blending learning & models goes beyond physics to other forms of symbolic logic. We are part of a multi-university NSF Expeditions team on "Neurosymbolic Learning":
neurosymbolic.org

Relevant papers:
arxiv.org/abs/2007.12101
arxiv.org/abs/1907.05431

10/N

We are very excited about #AI4Science at @Caltech. One area that has seen a lot of traction recently is behavior modeling... turns out there are many ways a lab animal can bite another one!

people.vision.caltech.edu/~eeyjolfs/beha…

Work led by Pietro Perona:
authors.library.caltech.edu/cgi/exportshel…

11/N

My interest in behavior modeling started with sports analytics. One interesting recent result is fine-grained controllable generation, trained on NBA games.

https://twitter.com/yisongyue/status/1285332075248103424

12/N

We've also developed many other sports analytics methods as well, a few that were field deployed. Work led by @HoangMinhLe.

Sports Illustrated:
si.com/media/2016/06/…

Papers:
arxiv.org/abs/1703.03121
yisongyue.com/publications/i…
yisongyue.com/publications/s…
yisongyue.com/publications/c…

13/N

Back to #AI4Science, I'm super excited about AI for Experiment Design, i.e., Bayesian Optimization for Science. Some cool collaborations already with @francesarnold & Harry Atwater's groups.

arxiv.org/abs/1811.07707
arxiv.org/abs/1811.00755
yisongyue.com/publications/a…

14/N

@Caltech researchers have started studying how to use modern neural architectures for real science applications. Here's a cool result by @ZvxyWu & @KevinKaichuang from @francesarnold's group:

https://twitter.com/ZvxyWu/status/1281687506124062721

15/N

Here's another result from @sarah_reisman's group on predicting substrate-specific organic reaction conditions:

arxiv.org/abs/2007.04275

16/N

@Caltech is also at the forefront of transforming seismology & geophysics using deep learning, mainly led by @zross_. Here are some recent papers of his:

arxiv.org/abs/1809.02880
arxiv.org/abs/1901.03467
arxiv.org/abs/2002.02040
arxiv.org/abs/1907.00496

17/N

We also love working at the interplay between learning & neuroscience. For example, check out this work by @jxbz that relates a novel multiplicative weights optimizer for deep learning w/ representations studied in neuroscience.

https://twitter.com/jxbz/status/1282391445857611778

18/N

Some research is also becoming commercialized, such as OrbNet created by @EntosAI (co-founded by @tfmiller3).

https://twitter.com/tfmiller3/status/1283928561166905344

19/N

We're starting new projects at the intersection of learning & smart sensing & imaging (w/ @klbouman). From blending learning w/ physical models to policy learning for adaptive sensing.

Here's a recent result by @_sun_he (postdoc w/ Katie):
cosense.cms.caltech.edu

20/N

Pietro Perona's group is very interested in capturing the long tail of categories in real-world images. For instance, @sarameghanbeery has done great work in designing vision algorithms for wildlife monitoring:

https://twitter.com/sarameghanbeery/status/1276576534552166400

21/N

Finally (& still my fav), novel theory w/ real-world motivations!

Online Optimization & Competitive Control:
arxiv.org/abs/2002.05318

Robust Regression for Safe Exploration:
arxiv.org/abs/1906.05819

Program Learning as Constrained Mirror Descent:
arxiv.org/abs/1907.05431

22/N

What's the common attribute? A relentless focus on the fundamentals coupled with rich cross-disciplinary dialogues. That's how you can identify new high-impact directions that are overlooked by others.

23/23