• TensorFlow Lite models now work with TensorFlow.js (train once, deploy twice)
• Google's on-device machine learning page tailors ML guides for your smaller device needs
• TF Lite model maker library helps you train on-device models faster
• TensorFlow Hub gets a facelift, plus, now you can try pretrained models before you buy them (jk the models are free)
• TensorFlow Cloud library helps you scale up your smaller experiments to cloud-scale in a few lines of code (e.g. Google Colab -> 8 GPUs)
• Google Cloud's AI Platform gets renamed to Vertex AI and now Google Cloud's one-stop-shop for your ML needs (think data storage, feature storage, model training, model deployment etc)
• To go along with Vertex AI is a new MLOps White Paper piecing together everything ML
• The new TensorFlow forum! Now there's a town square to meet and talk with TensorFlow developers from around the world
• The People and AI Guidebook 2.0 helps you design ML-powered applications by thinking about things like: "explain the benefit, not the technology"
And from the rest of the internet:
• Next-generation pose detection with MoveNet and TensorFlow.js (17 body keypoints @ up to 51 FPS in the browser of an iPhone 12!!!)
• Datasets & code are on arXiv ala @paperswithcode (find the data and code associated with ML papers)
• @facebookai's wav2vec-U (unsupervised) speech recognition model performs equivalent to state of the art 2 years ago without *any* labelled data (previous model used ~1000 hours)
• What is active learning? by @roboflow - doing practical ML? You'll want active learning
• Reproducible Deep Learning by @s_scardapane - Ever tried to build a reproducible deep learning model? It's harder than you think. Not to worry, Simone's course goes through steps to help you do so.
• The Rise of @huggingface by @marksaroufim - an outstanding take on how ML companies like HuggingFace and @weights_biases have built incredible value by creating community around their product offerings.
Far out...
As usual a massive month on tour for the world of ML!
New video: Tracking every item in my house with video using Google Gemini 🎥 -> 🛋️
I call it "KeepTrack" 😎
Input: 10-minute casual walk around video.
Output: Structured database w/ 70+ items.
Cost: ~$0.07 w/ caching, ~$0.10 w/o caching.
Full details... 🧵
TL;DR
Gemini features which make this possible:
1. Video processing. 2. Long context windows (video data = ~300 tokens per second, 10 minute video = 165,000 tokens). 3. Context caching (process inputs once, inference for 4x cheaper).
Prices are with Gemini 1.5 Flash.
1. Video Processing/Prompting
Intuition: Show *and* tell.
Technical: Video = 4 modalities in one: vision, audio, time, text (Gemini can read text in video frames).
Instead of writing down every item in my house, I just walked through pointing at things and talking about them.
Gemini tracked everything I said/saw almost flawlessly (it missed a few things due to 1 FPS sampling but this will get better).
Doing this via text/photos alone would've taken much longer.
There are many more fields/problems where video input unlocks a whole new range of possibilities.
If machine learning projects were a relationship...
Data collecting and processing is the dating phase, fun, chaotic, up and down, tormenting and carefree, seeing if you're a good fit.
Modelling is the wedding day, takes forever to plan, over before you know it.
People using your model is the honeymoon.
Then comes the data drift.
Your data changes like the person you thought you married, maybe they're getting fat (distribution changes) or they're finding it hard to love you (your data features are no longer ideal).
So you bring in data monitoring, model evaluation (marriage counselling) and pull all the tricks.
Your marriage counsellor tells you to go back to what got you started.
The fun dates (collecting data), talking for hours learning about each other (processing data).
1. 🤔 Problems - some of the main use cases for ML. 2. ♻️ Process - what does a solution look like? 3. 🛠 Tools - how can you build your solution? 4. 🧮 Math - ML is applied mathematics, what kind? 5. 📚 Resources - where to learn the above.
3/ Although very colorful, at first glance, the map can be very intimidating.
So there's a video walkthrough to go along with it:
We start with a high level overview which answers questions like "what is machine learning good for?"