Thread by @sylvaingigan

Let me try to turn my tutorial into a tweet storm ! bear with me (1/47)

disclaimer #1 : lots of slides borrowed from a lot of people
disclaimer #2 : it's a tutorial but by no means a comprehensive review.

First : Machine Learning is around us everyone, but in optics, if you didn't notice, you've not been paying attention : it is everywhere! (2/47)

Let us first clarify a few terms for the layman. (3/47)

in the 80's artificial intelligence was essentially expert decision systems, based on a set of user-defined rules. (4/47)

while nowadays, it is all about (automatic) data analysis. A lot of data. (5/47)

I start by detailing a very standard dataset, that I will use throughout my talk. (6/47)

just having a look at the dataset in a more abstract/ mathematical way as a structured set of points in high-dimensional space (7/47)

and give an example of a pretty standard (and inefficient) classifier, illustrating the computational complexity of the task (8/47)

and suggesting that an alternative, inspired by nature, can help us think of new ways to analyse data : our brain. (9/47)

The first tentative to mimic the brain's approach to computing, the perceptron, and its hardware implementation. Yes, there is a lot of wires. The analogy with the brain is actually pretty loose (10/47)

Still the approach can be refined and complexified, here the example of the multilayer perceptron. (11/47)

an example of classification with the MLP, shamelessly stolen from the internet (12/47)

a specific type of matrix multiplication, convolutions, are particularly useful for image analysis. (13/47)

my 1-slide explanation of the principle of backpropagation. 😬😬😬(14/47)

a peek into the vertiginous growth of datasets. MNIST is VERY small. (15/47)

some considerations on deep learning, the bigger the better! (16/47)

if you thought Moore law was fast, think twice! (17/47)

and maybe you didn't know, but hey, Moore's law is officially dead, anyway ! (18/47)

Considering the two previous slides, we have a tiny problem in front of us! (19/47)

current hardware architectures are essentially ill-adapted to deep learning, and to ANN in general, because of the Von Neumann bottleneck. GPU, TPU, etc... improve upon this, but it's just a patch, not a solution. It allows us to buy some time. (20/47)

A very nice read: the hardware lottery! why hardware limits progress in algorithms, and nice versa (21/47)

other approaches, analog and hardwiring the NN structure, have emerged: this is the field of neuromorphic computing. (22/47)

Meanwhile Optical Computing has been explored in the last decades, with some nice successes ... (23/47)

(mentionning a nice review on the topic) (24/47)

...while optics for communications has become absolutely essential, optical computing has seen a "winter" during nearly two decades. (25/47)

The recent surge in deep learning has however changed the paradigm, and optics is coming back as a VERY appealing alternative for neuromorphic computing ! This is covered by two recent reviews in Nature and Nature Photonics (I participated in one) (26/47)

If you want to take a break and have a coffee, now's a good time ! (27/47)

a direct implementation of matrix multiplication, the crossbar array, has recently been shown in optics, leveraging phase change materials and the progresses in integrated photonics. (28/47)

An important advantage of optics is the frequency multiplexing advantage : you can pass multiple information in the same pipe! Frequency combs proposes up to 1000s of lasers in parallel! (29/47)

Another way to build reconfigurable matrix multiplications is to use cascaded mach-zenhder interferometers (30/47)

it is relatively bulky (mm^2 to cm^2 on chips, for a few tens of modes at most) but is fab-compatible and can be very fast. (31/47)

The complementary approach is to use free space, which naturally perfom multiple simple matrix multiplications over many many modes in parallel. (32/47)

The "Stanford matrix vector multiplier" is one such implementation, where arbitrary matrix multiplication can be performed. 1D emitters and detectors "arrays" allows relatively rapid input-output modulation and detection. (33/47)

a recent demonstration from @GordonWetzstein demonstrates the ability to perform multiple convolution in parallel for image classification. (34/47)

Thanks to spatial light modulators (here DMD) convolutions and input can be made reconfigurable, and inference can be performed in principle at several kHz rates over millions of spatial modes. (35/47)

stacking multiple masks, one can achieve more complex transformations. This are deep "diffractive" networks, that are not "deep" as in "deep learning" but more "dense" single layer neural networks. (36/47)

A nice recent result by @peterlmcmahon demonstrate that these matrix multiplication can be done with very few photons/energy, down to less than a (detected) photon per MAC (multiply and accumulate, the basic operation in matrix multiplication). (37/47)

Optical non-linearities are HARD. fully optical multilayers with non-linear intermediary function are elusive. An elegant alternative has been recently proposed using non-linearities in multimode fibers by @u_tegin and coworkers. Not a silver bullet yet, but very promising! (38

a few slides on my own work at @labo_lkb within @comedialkb, and in the startup I co-founded, @LightOnIO on exploiting multiple scattering of light. (39/47)

We have shown that a complex medium perform a large scale, fixed, random matrix multiplication. it is actually a very interesting operation for signal processing and machine learning! (40/47)

two out of a zillion examples on how to use random projections : for dimensionality reduction, and for the "kernel trick". Note the number of citations of these two seminal papers! (41/47)

Why is it ridiculously interesting to perform these operations optically ? check @LightOnIO if you want to know more! (42/47)

Just mentioning two recent works from the @comedialkb on other computing applications (Ising model and some quantum circuits) (43/47)

my take home messages ! (44/47)

and some perspectives and considerations for the field to move forward! (45/47)

cannot finish without thanking my team, coworkers, collaborators.... (46/47)

and thank you for following! feedbacks / questions / comments / more than welcome! OUT AND OVER ((47/47)

I am still happy to give the talk live if anyone is interested :-) I will definitely improve and develop it in the future!

And funders of course!

200 like and 60 RT (and counting) I certainly rarely had such a large audience for my live nor online talks ! Interesting outcome of this experiment!