Luciano Lucas Profile picture
Sep 30, 2020 17 tweets 27 min read Read on X
Excited to introduce #3DRCAN for denoising, super resolution and expansion microscopy. Super proud of this colab with Hari Shroff, Jiji Chen et al.

@NIBIBgov @arpcomplex

Preprint: biorxiv.org/content/10.110…

Code: github.com/AiviaCommunity…

#microscopy #aimicroscopy #aivia

1/17
"Three-dimensional residual channel attention networks denoise and sharpen fluorescence microscopy image volumes"

J Chen, H Sasaki, H Lai, Y Su, J Liu, Y Wu, A Zhovmer, C Combs, I Rey-Suarez, H Chang, C Huang, X Li, M Guo, S Nizambad, A Upadhyaya, J Lee, L Lucas, H Shroff.
2/17
We are particularly interested in characterizing the limits of #deeplearning based approaches to image restoration. You will find several experiments comparing several best-in-class approaches: #3DRCAN, #CARE, #SRResNet and #ESRGAN.

#microscopy #aimicroscopy #iSIM

3/17
Firstly, we looked at how #3DRCAN and other #DL based approaches behave when denoising iSIM images depicting a range of organelles, including: Actin, ER, Golgi, Lysosome, Microtubule, Mitochondria.

#microscopy #aimicroscopy #cellbio #deeplearning #superresolution

4/17
We were also interested in finding out the lowest level of illumination (and lowest SNR) for which #3DRCAN could still "work", i.e. create a good quality restoration. Imaging done on an iSIM.

#iSIM #microscopy #aimicroscopy #cellbio #deeplearning #bioimageanalysis #aivia

5/17
Now that we had nicely restored images (good performance in terms of SSIM and PSNR), we detected and analyzed mitochondria. This was nearly impossible when using the raw/input data.

#microscopy #aimicroscopy #cellbio #deeplearning #bioimageanalysis #aivia #3DRCAN

6/17
We then applied the trained #3DRCAN to live cell imaging data (4D). As a result we could study fusion and fission (as imaged by iSIM) events in mitochondria over long time lapses.

#microscopy #aimicroscopy #cellbio #deeplearning #bioimageanalysis

7/17
Secondly, we wanted to determine if #3DRCAN could be used for other types of super resolution microscopy - we used paired Confocal<->STED 3D data sets to train our models. Imaging done with a Leica SP8. @LeicaMicro

#STED #superresolution #aimicroscopy #microscopy

8/17
The trained #3DRCAN was applied to raw confocal data (incl. nuclear pores, microtubules and DNA), greatly improving (~2-fold) the spatial resolution. Imaging done with a Leica SP8. @LeicaMicro

#STED #superresolution #aimicroscopy #microscopy

9/17
Ultimately, we wanted to perform fast 4D confocal imaging and "convert" the raw data to STED-like images. So, we applied the trained #3DRCAN to live cell 3D (confocal) data sets. See 👇

#STED #superresolution #aimicroscopy #microscopy

10/17
Lastly, we tested whether it was possible to further increase the resolution of iSIM (itself a #superresolution microscopy modality). To do this, we used expansion microscopy to generate the GT we needed to train #3DRCAN for this task.

#iSIM #ExM #aimicroscopy #microscopy 11/17
#3DRCAN was shown to improve both the lateral and axial resolution of iSIM images. Moreover, the improvement in resolution was superior to deconvolving the raw images.

#iSIM #ExM #superresolution #aimicroscopy #Microscopy 12/17
As with the other key experiments, we successfully tested out the trained #3DRCAN on live cell 3D data sets. In this case we imaged mitochondria and microtubules using iSIM.

#iSIM #ExM #superresolution #aimicroscopy #Microscopy 13/17
Disclaimer: this being a twitter thread, a lot of the details are missing - you are most welcome to review the pre-print. Your suggestions and input are welcome.

15/17
You can train your own #3DRCAN models and/or test out the ones we have described here. See github.com/AiviaCommunity… There are also some test images there. Let us know if you have any questions and/or suggestions.

#python #opensource #openaccess #OpenScience #aimicroscopy

16/17
Note to Aivia users: in the very short term all the models described in the #3DRCAN preprint will be available in #Aivia, #AiviaWeb and #AiviaCloud.

17/17

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Luciano Lucas

Luciano Lucas Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @lucianoaglucas

May 15, 2020
#ML powered image analysis is a great way to gain insight when all you have is RI variations - lable free. Quyen (linkedin.com/in/qatran/) and Luca (linkedin.com/in/luca-clario/) are doing a great job discussing the technology behind #Aivia and @NanoLiveLtd. Hosted by @MEDIASystemLab ImageImageImage
Now a live demo! Comparing "threshold based" segmentation methods vs #pixelclassifier (machine learning) based segmentation. There are pros and cons. For more details of the #pixelclassifier see: drvtechnologies.com/post/ascb-embo… Image
Read 5 tweets
Apr 15, 2020
The #OSABiophotonics20 (20-23 of April) congress has been moved to an 100% online format due to #covid19. This event is packed with great (microscopy and image analysis) presentations and is FREE to attend! Register here: osa.org/en-us/meetings… 1/3
#aimicroscopy #biophotonics Image
My talk "AI for Microscopy Applications|The Era of Implementation", highlights 2+ years of R&D (colab w/ Hari Shroff et al.), and is influenced by @kaifulee @EricTopol @loicaroyer @martweig @florianjug @TheGeneMyers @uschmidt83 @PavelTomancak @HenriquesLab #OSABiophotonics20 2/3 Image
KUDOS to @OpticalSociety @OSAPublishing and organizers for quickly moving the event to "online" and for making it free to participate. Also, I thank the organizers for the invite to present our work - MUCH appreciated. 3/3
#communityimpact #OpenScience #aimicroscopy #microscopy
Read 4 tweets
Mar 25, 2020
We will start to roll out #AiviaWeb at no cost to use, tomorrow. All #aivia functionality accessible via a web browser. This is part of our #covid19 mitigation plan - more actions soon so you can continue to be productive while #wfh. #bioimageanalyis #aiviacommunity #microscopy
To get early access to #AiviaWeb please sign in/up. See drvtechnologies.com
Once you are signed up we will reach out to you with all the details including usage and booking guidelines. A webpage dedicated to our covid19 / wfh mitigation plan will be up very soon too. Many more resources are in the pipeline.
Read 6 tweets
Mar 21, 2020
The Seattle metro region and the whole #PNW are being severely affected by the #COVID19outbreak. In >1 month we went from "there is a small cluster of cases in a nursing home" to "we need to lock down the whole region to avoid worst case scenarios" (not happened yet) 1/13 ImageImageImageImage
The (social) media frenzy is impressive. Even Facebook and others are worried their servers / data centers may not be able to cope with demand for long (cnn.com/2020/03/18/tec…). We have used social media for good - checking in on friends and family all over the world. 2/13
While this is a challenging time, there are many positive things to be thankful for. Personally, I am enjoying having more time with my young boys (4 and 6 yo) and wife. the boys are curious and scared about the whole story - but 3/13
Read 14 tweets
Dec 6, 2019
1/ At #ASCBEMBO19 we will present five posters and a tech talk on #aimicroscopy and #CLEM. Details here: drvtechnologies.com/ascbembo2019

Many thanks to our collaborators Hari Shroff, Jiji Chen, @huzhao4, Rachel Wong, Chris Combs, @3i_inc.

#microscopy #imageanalysis #Aivia #ASCBEMBO2019 Image
2/ P35/B36 Universal EM Connectomic analyses by Deep Learning Powered App‐matching Image Conversion

drvtechnologies.com/post/ascb-embo…

With Prof. Rachel Wong and @Sallywing

#ASCBEMBO19 #AImicroscopy #Aivia #AiviaCloud #imageanalysis #microscopy #CellBiology #machinelearning #drawacell Image
Read 8 tweets
Jun 4, 2019
#ELMI2019 lightning talks... Go! Image
Image
Image
Read 19 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Don't want to be a Premium member but still want to support us?

Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal

Or Donate anonymously using crypto!

Ethereum

0xfe58350B80634f60Fa6Dc149a72b4DFbc17D341E copy

Bitcoin

3ATGMxNzCUFzxpMCHL5sWSt4DVtS8UqXpi copy

Thank you for your support!

Follow Us!

:(