Share this page!

Cyril Diagne Profile picture
Twitter logo
11 Apr, 3 tweets, 6 min read
Using Drag&Drop in #AR with @TensorFlow and #OpenCV to connect @zachlieberman prints to their video counterparts.

#AIUX #WebXR #MixedReality #AugmentedReality #TensorFlowJS #MachineLearning #Interactive #openframeworks
The trick is to use OpenCV (SIFT + findHomography) to find where the camera is looking at the screen
Also works as an #AR party trick - thanks to my wife & @IronMaiden (turn the sound on!) 🤟🙃🤟

#StayHome #AIUX #WebXR #MixedReality #AR #AugmentedReality #TensorFlowJS #MachineLearning #Interactive #openframeworks

• • •

Missing some Tweet in this thread? You can try to force a refresh
　

Keep Current with Cyril Diagne

Cyril Diagne Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @cyrildiagne

Cyril Diagne Profile picture
22 Oct
Aaand here it is..!!! 😱

After months of hard work with @jblanchefr, @ClipDropApp beta (AR Copy Paste) is now publicly available on #Android, #iOS, #macOS, and #Windows

🔥 clipdrop.co 🔥

Here's a thread of what you can already do with it ↓ 1/n

#ML #AR #AI
You can capture directly from your desktop with a screenshot and drag & drop to any other app (my favorite feature 🤩)

It works from/to virtually any application & websites: @figmadesign, @Photoshop, @Canva, @Pitch, @Powerpoint, @GoogleDocs...etc!

#ML #Augmented #DesktopOS
You can extract anything: objects, people, drawings, and text. The quality of the salient object detection, background removal, and text detection is now quite incredible 😳😲🤯
Read 8 tweets
Cyril Diagne Profile picture
17 May
Proto 6/10: Copy printed text to desktop with AR+ML

Code: github.com/cyrildiagne/ar…
Book: Neurones, les intelligences simulées, Frédéric Migayroux & al (Editions Hyx 2018 @centrepompidou)

#ML #AR #AI #AIUX #ARCore #ARKit #WebXR

Technical Insights: ↓
The magic here is to use ARCore + AugmentedImages rather than SIFT.
Phone gets a new desktop screenshot on touch and adds it to ARCore (< 100ms).
Tracking is crazy fast & precise.

Interesting alternative to touch screen for interactive installations!
The text detection is performed on device with @Firebase #MLKit. Super fast, good accuracy and cross platform.
Read 6 tweets
Cyril Diagne Profile picture
3 May
4/10 - Cut & paste your surroundings to Photoshop

Code: github.com/cyrildiagne/ar…

Book: @HOLOmagazine
Garment: SS17 by @thekarentopacio
Type: Sainte Colombe by @MinetYoann @ProductionType
Technical Insights: ↓

#ML #AR #AI #AIUX #Adobe #Photoshop
The secret sauce here is BASNet (Qin et al, CVPR 2019) for salient object detection and background removal.

The accuracy and range of this model are stunning and there are many nice use cases so I packaged it as a micro-service / docker image: github.com/cyrildiagne/ba… Image
And again, the OpenCV SIFT trick to find where the phone is pointing at the screen.

I also packaged it as a small python library: github.com/cyrildiagne/sc…

Send a camera image + a screenshot and you get accurate x, y screen coordinates! Image
Read 6 tweets
Cyril Diagne Profile picture
10 Feb 19
The coarse knob of #StyleGAN really is mesmerizing. Esp. how it adds/replaces some features to match the artistic style (collars, hats..etc)
On random interpolation, the truncated latent space also looks surprisingly continuous & morphologically coherent.
(Last one) with the fine knobs (range 6-16). Looks much more like a traditional style transfer which keeps the face features & pose relatively intact.
Read 4 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!

Like this author's thread?

Get email notifications when new thread is out from this Author!

Check out other threads from this Author!

Read all threads by @cyrildiagne