Discover and read the best of Twitter Threads about #ml

Most recents (24)

#MLOps standard industry best practices” don’t apply to most #ML teams’ reality.


Those who write and share best practices are doing ML at a hyper scale.

Those who read and re-share them are doing ML at a reasonable scale.
Companies like Google, Netflix, Uber, and Airbnb are doing an awesome job for the community by sharing their blogs, white papers, and open-sourcing their tools.

But whatever they do, it is shaped (and biased) by THEIR MLOps problems.

Most companies don’t have their problems.
They would love to have their problems, but they don’t.

They operate on a smaller scale & have different (& other) challenges.

And they are the biggest part of the ML industry.

They want to know what’s the best way to do MLOps at their scale, with their resources & limitations
Read 6 tweets
Last week @OpenAI released ChatGPT - a Large Language AI Model that interacts with users in a natural conversational way. The chatbot is able to answer complex questions, even in highly technically demanding categories.

It is also able to answer the follow up question, backtrack on wrong assumptions, and provide other detailed resources, including code fragments.

Most people in tech consider this to be the greatest technological advancement of the year. Many of us consider it even more epochal, perhaps one of the biggest turning points in history.

Read 7 tweets
We want to pretrain🤞
Instead we finetune🚮😔
Could we collaborate?🤗

ColD Fusion:
🔄Recycle finetuning to multitask
➡️evolve pretrained models forever

On 35 datasets
+2% improvement over RoBERTa
+7% in few shot settings

#NLProc #MachinLearning #NLP #ML #modelRecyclying
We all wish to improve pretraining
If only we had unlimited compute and data...
Together we have!

We propose a way to recycle finetuning
and transform it into multitask learning!

@Shachar_Don @VenezianElad @colinraffel @noamslonim @YoavKatz73 me
How to perform multitasking, by simply uploading models?

Collaborative Descent (ColD) Fusion is simple:
Start from a pretrained model
Let contributors finetune on it, and share their models
Fuse the models to get a new better model
Take the improved model as the new best model
Read 9 tweets
PyTorch 2.0 is out! This major release upgrade brings about many new features, but the main improvements are under the hood.

1/6 Image
The three main principles behind PyTorch

1. High-Performance eager execution
2. Pythonic internals
3. Good abstractions for Distributed, Autodiff, Data loading, Accelerators, etc.

PyTorch 2.0 is fully backward compatible with the previous versions of PyTorch.

The main new feature is torch.compile, "a feature that pushes PyTorch performance to new heights and starts the move for parts of PyTorch from C++ back into Python."

Read 6 tweets
Our recent paper "Exposing the Limitations of Molecular Machine Learning with Activity Cliffs" is now online!…

@korney34 @fra_grisoni @molecularML #AI #ML #drugdiscovery #Chemistry

By going to the limits of molecular property prediction ... 👇
with activity cliffs -- molecules that have highly similar structures, but large differences in bioactivity -- we see that deep learning still lacks behind of traditional machine learning for bioactivity prediction.
We showed that evaluating your models performance activity cliffs is crucial for prospective applications, especially in low data scenarios. To do this we developed a cool tool called MoleculeACE…
Read 3 tweets
Today we announce a first in #TeamOneFist history - #cyber striking an operational #Russian #AI/#ML (#MachineLearning) model, in addition to a #power #grid #SCADA/#ICS!
This is Op.Neutrino, an electrical counterattack against #SPB, #Russia, and now, it's story is here. 1/4
At 17:00 local time, we assumed control over an @EnstoGroup #grid #automation #controller belonging to the DK Port substation. Timing was chosen to match peak usage hours. In addition to controlling power supply, it was supplying data for Rosenergo's FLISR fault #algorithm 2/4
From the controller, we successfully fed bad data into the FLISR #ArtificialIntelligence model, via the connected sensors. Then, we nuked it!
Every attack against #Ukraine will be avenged, every #RU #data model will be corrupted! 🇺🇦☢️👊3/4 #UkraineWillWin #cybersecurity #infosec
Read 4 tweets
What is #TechBio

Everyone has heard of Biotech. These are companies that develop technology around develop drugs. Its a very biology driven endeavor. Its about looking at the science, coming up with ideas and then testing them in the lab to find ones that work.
1/ Where Biotech is driven by Biology, #TechBio is driven by technology. Its about using tools like Computer Aided Design, Computer Modeling, Artificial Intelligence #AI, and Machine Learning #ML to design drugs. Its a technology first type of drug design.
2/ I have heard of it referred to as the Bionic Scientist. I love that concept as it truly represents what TechBio is all about. Its about using tools of technology to enable Scientists to better develop drugs. Lets look at the statistics of drug development.
Read 18 tweets
🌟Abs 0001 Botson, et al.
MIRROR RCT: adding MTX to pegloticase for gout
👉52 wk results: response 60% vs 31% pbo

*response: SU<6mg/dl for >=80% of wks 48-52
Abs 0001 cont'd

👉Also: complete tophi resolution 54% MTX vs 31% pbo at 52 wks

Read 11 tweets
♻️ Introducing CircularNet, a set of data efficient models created to support the way we manage and recycle materials across the waste management ecosystem.

Learn how CircularNet uses TensorFlow's Mask R-CNN algorithm to change the future of recycling ➡️ Model identifying the produ...
Once data is collected at Material Recovery Facilities,

✅ Annotation files get converted into COCO JSON
✅ Incorrect labels, errors and corrupt images are removed to ensure smooth training.
✅ Final file gets converted to the TF record format A material recovery facilit...
The Mask RCNN is then trained using the Model Garden Repository.

✅ Hyper parameter optimization is performed by changing image size, batch TensorFlow Model Garden logo
Read 5 tweets
Interested in reconstructing computational dynamics from neural data using RNNs?

Here we review dynamical systems (DS) concepts, recent #ML/ #AI methods for recovering DS from data, evaluation, interpretation, analysis, & applic. in #Neuroscience:…
A 🧵... Image
Some important take homes:
1) To formally constitute a true state space or reconstructed DS, some math. conditions need to be met. PCA and other dim. reduc. tools often won’t give you a state space in the DS sense (may even destroy it). Just training an RNN on data may not either Image
2) For DS reconstruction (DSR), a trained RNN should be able to reproduce *invariant geometrical and temporal properties* of the underlying system when run on its own. Image
Read 12 tweets
The Chrome team is cutting support for the superior JPEG-XL codec in its browser — even before they enabled it!

The decision was made in secret under the direction of a single person who has conflicts of interests, and promotes the inferior AVIF alternative.
AVIF is based on VP10 codec, like a successor to WEBP which is based on the VP8 codec. Google owns & controls VP10, so has interests in promoting it instead of superior alternatives.

This means there'll be ~50% more energy used, and thus carbon, for internet bandwidth. 🙄
Most of our internal pipelines for data processing in #ML are based on JPEG-XL. For instance, the FFHQ dataset with "visually lossless" compression takes now 84Gb instead of 995Gb. Transfers faster, loads faster, trains faster.
Read 9 tweets
Many #ML startups face challenges growing at scale and managing its infrastructure.

FinTech startup, @digits was able to mitigate these challenges with the early deployment of TensorFlow Extended.

Learn more ➡️ Digits and TensorFlow logo
💡No. 1: Standardization

With TFX, @digits can increase code reusability and ramp up projects faster than ever before.

⚡Its continuous integration system automatically deploys its #ML models and tracks all changes in its Git repository. System components using Ten...
💡 No. 2: Growth

📈 With minimal code changes, @digits can easily swap out its Apache Beam configuration when its datasets grow and require more processing capabilities. Various Beam Runner systems...
Read 5 tweets
In NVIDIA's new paper on #Diffusion Models, they show how more denoisers (for each stage) and more embeddings (text, image) helps with quality!

TL;DR: If you buy more GPUs, you get correct spelling too. #AI #ML
With so many different labs rushing to research and deploy this kind of technology, this will quickly turn into a race for more efficiency as different providers compete on costs too.
The paper is a bit evasive on the dataset (LAION?) — I presume for legal reasons. But the good news is that it's "only" 1B text/image pairs... although they are highly filtered.

IMHO there's much more room to improve quality with the current datasets.
Read 5 tweets
🤯Tercer y última parte de ERRORES QUE DAN MIEDO en #DataScience 🎃

☠️ERRORES mortales que incluso los expertos cometen⚰️…

Continúa leyendo, si te atreves...👻
#HorrorStats #HappyHalloween #DataAnalytics #Halloween #FelizLunes #dataviz #RStats #Python #ML
🚫No realizar una investigación reproducible💀

“Every analysis you do on a dataset will have to be redone 10-15 times before publication. Plan accordingly” Trevor A.Branch

No crear un informe replicable, reproducible y reutilizable sí que DA MIEDO

#HorrorStats #HappyHalloween
🚫No seleccionar la prueba de hipótesis o el modelo de regresión correcto para tu objetivo🎃

¿Cuáles son las hipótesis? ¿Cómo son las muestras? ¿Qué tipo de prueba/modelo elegir? ¿Una cola o dos colas? ¿Qué hacer si mis datos no cumplen los supuestos? BOOO!! 👻

#HorrorStats #ML
Read 12 tweets
📊"Una imagen vale más que mil palabras", o que mil datos. Los gráficos cuentan la historia de los datos, nos ayudan a guiar, interpretar y comunicar😉
Cuidado con estos #HorrorStats
#HappyHalloween #Halloween #FelizDomingo #HalloweenEnds
🚫1. Elegir el gráfico incorrecto💀

Cada gráfico tiene sus propios casos de uso. ¿Tiene sentido representar el crédito € de una tarjeta con un gráfico de sectores? 🤌

#HorrorStats #HappyHalloween~ #trickortreat #DataScience #dataviz #DataScience #data
¿Qué gráfico utilizar?👇
🚫2. Manipular los ejes del gráfico💀

👉Distorsionar la escala, truncarla u omitir líneas de base es un error, intencionado o no.🤦🏻‍♀️

¿Quieres más ejemplos?👇

#HorrorStats #HappyHalloween~ #trickortreat #DataScience #dataviz #RStats #Python #DataVisualization #Stats #Analytics
Read 7 tweets
@TheRealDrDre2 @ElliotMurphy91 @GaryMarcus @EvelinaLeivada @john_e_laird It is a great paper for two reasons.
1. It is a systematic analysis of what DALL-E does, its strengths and weaknesses. I find that a lot of #LLM 'papers' are descriptions of the engineering with very little analysis.
@TheRealDrDre2 @ElliotMurphy91 @GaryMarcus @EvelinaLeivada @john_e_laird 2. It highlights what a language model will need when it is used for communication between intelligent agents.

Language for communication is a different computational problem than language for information retrieval. Most modern #NLP is the latter.
@TheRealDrDre2 @ElliotMurphy91 @GaryMarcus @EvelinaLeivada @john_e_laird We @PARCinc are exploring the former - We use the Common Model of Cognition #CMC as an agent architecture and instantiate language comprehension in a complex agent.
Read 6 tweets
Carolyn Calfee Clinical and Biological phenotypes of ARDS
- what do they have in common?

ARDS : subgrouping since the begining
- sepsis vs. non sepsis
- hyper vs. hypoinflamm
- reactive vs uninflamed
#ventilation #ards #phenotypes #LIVES2022
Are clinical phenotypes biologically distinct?
looking at Trauma vs. Non trauma

ICAM-1 , SP-D, vWF, sTNFr-1 are different.
What about in "Direct" vs. "indirect"
or "Diffuse" vs "focal" -- sRAGE comes up again.
#ventilation #ARDS #LIVES2022
Read 13 tweets
NEXT : @AriErcole
Association is not necessarily a causation
RCTs are thought of as "gold standard" for a good reason.
"Randomisation" eliminates influences of confounders.
- allows "causality" inference.
@AriErcole RCTs require relatively little prior knowledge.
we try to "by having a inclusion criteria"
🧐 we can only control what we know

RCTs have limitations - dont really imply causality absolutely.

@ESICM #datascience #ai #ml #icudata #RCT
Read 12 tweets
Now Harm-Jan De Grooth "Understanding Bayesian Analysis"

- more specifically "Bayesian Trial analysis"
@ESICM #datascience #ai #ml #icudata #bayesian
@ESICM Trials are now using Bayesian analysis or at least in secondary analysis.

But also NEJM IL-6 in covid - purely bayesian analysis.
Read 22 tweets
NEXT Adaptive and Platform trial designs by @Lennie333 #datascience #ai #ml #icudata
#LIVES2022 #ventilation
we assume "large effect sizes" in ICU trials
@Lennie333 @ESICM this is because otherwise we will need large "n" and long time for trials.

e.g., anti-hypertensives - you dont want to test one drug at one dose. you want to test a range of doses and a range of duration.

third-thing : we struggle to find a end-point especially in critical care
@Lennie333 @ESICM huge effort doing RCT but we use "mortality" yes/no as a very binary endpoint. For patient, length of stay, quality of life after d/c important end points beyond survival.

in RCT, we cant learn whilst the trials are still running. in classic RCT.
Read 20 tweets
NEXT:Interfacing ICU data Nicolas Bennet
- Nicolas was very pleased to hear Chris Sauer(earlier speaker) advocating use of 2 -data set at least
@ESICM #criticalcare #ai #ml #icudatasets
showcasing this R dataset

starting with
> library(ricu)
> lact (loaded all lactate data)
@ESICM NEVER THOUGHT THAT I would be tweeting code in an ICU conference :)) @ESICM #datascience :P #datascience #LIVES2022
Read 16 tweets
NEXT Inventory and Comparison of ICU datasets by Christopher SAUER

"Why talk on differences in ICU databases?"
Ans: becuase data is "CORE"
@ESICM #ml #ai #databases #datascience #LIVES2022
@ESICM Merit of publicly available ICU databases
- no randomzined evidence exists for most clinical situations
-data and pt level insights incredibly useful.
-local epidemiology and treatment difers
-real world data sets help deliver optimal treatment policies.
#DataScience #LIVES2022
@ESICM 1st publicly available dataset MIMIC-3 in 2016,
Beth Israel Deaconess Medical Centre, Boston,MA
>70,000 icu stays, 2008 to 2019
now also includes chest x-rays, emergency room data
- large, community developed Github repo.
Read 16 tweets
Day 2. Starting on pitfalls in leveraging EHR by Stephanie HYLAND @ESICM #criticalcare #ehr #datascience #ai #LIVES2022
This problem is mainly for ML engineers who may not have talked to domain expert or clinicians / end users.
Pitfall 1 : sampling bias
"whos included in the analysis"
"who in your EHR"?
- e.g., - COVID prediction dataset where missing all blood tests were removed, but this missingess has a meaning. Thus not generalisable.
e.g., yesterday I mentioned about females < 6% of sample popn
Read 10 tweets
Decision trees based Machine Learning models are some of the best performant algorithms in eras of predictive capability, especially on small and heterogenous datasets.

They also provide an unparalleled level of interpretability compared to all other non-linear algorithms. However, they are very hard to optimize on Von Neumann architecture machines due to their non-uniform memory access patterns.

In groundbreaking work published in Nature Communications a team of researchers has shown that analog content addressable memory (CAM) devices with in-memory calculation can dramatically accelerate tree-based model inference, as much as 10**3 over the conventional approaches 3/4
Read 4 tweets

Related hashtags

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3.00/month or $30.00/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!