Thread by @Mitchnw

Excited to share our blog on ~Discovering Neural Wirings~ expanding on recent work with Ali Farhadi & @morastegari to appear at #NeurIPS2019 (see thread below for more info)!

Blog: mitchellnw.github.io/blog/2019/dnw/
Preprint: arxiv.org/abs/1906.00586
Code: github.com/allenai/dnw

(1/4)

@morastegari Two cool takeaways:
1) In _some_ ways, the problem of NAS and sparse neural network learning are really two sides of the same coin. As NAS becomes more fine grained, finding a good architecture is akin to finding a sparse subnetwork of the complete graph.

(2/4)

@morastegari 2) It is still possible to play the initialization lottery (see arxiv.org/abs/1803.03635) with a combinatorial number of subnetworks even when the model is sparse.

(3/n)

@morastegari Extremely grateful for the chance to work on this project with countless incredible people, extending beyond the fantastic coauthors. As just one example check out the amazing interactive visualizations on the blog by Alvaro Herrasti!

(4/n)

@morastegari This work culminates an incredible year in the PYI program at @allen_ai with @RoozbehMottaghi. I cannot recommend this program highly enough (reach out w/ any questions). Across all teams and positions, @allen_ai is filled with amazing & supportive people.

(5/n)

@morastegari @allen_ai @RoozbehMottaghi (somewhat sidetracked)
As the AI/ML research space grows increasingly competitive, the importance of programs where prior research is not a pre-requisite must be emphasized!

(6/6)