Cyril Diagne Profile picture
Artist, Designer, Programmer - AI+HCI - in residency @googlearts - making https://t.co/2zjmuvk2MR - previously: head of @ecal_mid, co-founder of @lab212
17 May 20
Proto 6/10: Copy printed text to desktop with AR+ML

Code: github.com/cyrildiagne/ar…
Book: Neurones, les intelligences simulées, Frédéric Migayroux & al (Editions Hyx 2018 @centrepompidou)

#ML #AR #AI #AIUX #ARCore #ARKit #WebXR

Technical Insights: ↓
The magic here is to use ARCore + AugmentedImages rather than SIFT.
Phone gets a new desktop screenshot on touch and adds it to ARCore (< 100ms).
Tracking is crazy fast & precise.

Interesting alternative to touch screen for interactive installations!
The text detection is performed on device with @Firebase #MLKit. Super fast, good accuracy and cross platform.
Read 6 tweets
3 May 20
4/10 - Cut & paste your surroundings to Photoshop

Code: github.com/cyrildiagne/ar…

Book: @HOLOmagazine
Garment: SS17 by @thekarentopacio
Type: Sainte Colombe by @MinetYoann @ProductionType
Technical Insights: ↓

#ML #AR #AI #AIUX #Adobe #Photoshop
The secret sauce here is BASNet (Qin et al, CVPR 2019) for salient object detection and background removal.

The accuracy and range of this model are stunning and there are many nice use cases so I packaged it as a micro-service / docker image: github.com/cyrildiagne/ba…
And again, the OpenCV SIFT trick to find where the phone is pointing at the screen.

I also packaged it as a small python library: github.com/cyrildiagne/sc…

Send a camera image + a screenshot and you get accurate x, y screen coordinates!
Read 5 tweets
10 Feb 19
The coarse knob of #StyleGAN really is mesmerizing. Esp. how it adds/replaces some features to match the artistic style (collars, hats..etc)
On random interpolation, the truncated latent space also looks surprisingly continuous & morphologically coherent.
(Last one) with the fine knobs (range 6-16). Looks much more like a traditional style transfer which keeps the face features & pose relatively intact.
Read 4 tweets