The Real-World ML guy | Learn to build real-world ML apps at https://t.co/xWr8Hm8zI5
3 subscribed
Jul 3 โข 11 tweets โข 3 min read
Most of my ML model prototypes never reached production ๐ตโ๐ซ
Until I changed my mindset ๐ง โโโ
๐ฌ Model-first mindset
A model-first mindset is what Kaggle competitions and most online courses are about.
Your ONLY focus is to build the best possible mapping between a set of input features, and a target metric
And in real-world ML this is often not the best approach...
Jul 2 โข 13 tweets โข 4 min read
How do you build
> ๐ฟ๐ฒ๐ฎ๐น-๐๐ถ๐บ๐ฒ ML systems โก
> at ๐๐ฐ๐ฎ๐น๐ฒ ๐๏ธ
> ๐๐ถ๐๐ต๐ผ๐๐ ๐ฏ๐๐ฟ๐ป๐ถ๐ป๐ด ๐ฐ๐ฎ๐๐ต ๐ธ?
Letโs say you work as an ML engineer at a fintech startup, whose flagship product is a mobile app for online payments.
A critical problem you need to tackle from day 0 is the automatic detection of fraudulent transactions.
Jul 1 โข 10 tweets โข 4 min read
ML Project Idea ๐ก
Let's predict taxi demand in NYC in the next 60 minutes ๐โ
Business problem ๐ผ
Let's create a predictive model to forecast the number of taxi rides that will happen in Manhattan (New York City)
- in the next hour
- for each taxi zone (e.g. Zone 113 "Lower Manhattan)
Let's do it in 6 steps โ
Jun 25 โข 6 tweets โข 2 min read
ML Project Idea ๐ก
Let's predict flight delays ๐ฌ โ
Here is a full example, with source code, to learn how to build a complete ML app that predicts flight delays for Stockholm Arlanda airport.
Let's build an LLM agent in Python, step-by-step โ๐งต
Why agents ๐คโ
Because Large Language Models alone are not enough to accurately answer complex tasks that require
-> External information that was not present in the training dataset used to fit the LLM paramaters
or
-> Many reasoning steps
Jun 12 โข 6 tweets โข 2 min read
ML Project Idea ๐ก
Let's predict flight delays ๐ฌ โ
Here is a full example, with source code, to learn how to build a complete ML app that predicts flight delays for Stockholm Arlanda airport.
Let's predict air quality โ
Here is a full example, with source code, to learn how to build a complete ML app that predicts air quality in different European cities.
Here is a project you can build ๐ฉ๐ฝโ๐ป๐จโ๐ปโโโ
Reading blog posts about multi-billion-parameter Language Models is very cool.
However, building real-world NLP products from these models is where the real business value is. And this is what companies look for in the job market.
So, here is a PRO project you can build โ
Jun 7 โข 11 tweets โข 3 min read
Tired of training lots of Machine Learning models, and not getting better results? ๐ตโ๐ซ
This is how you solve this ๐ง โ
A Machine Learning model is the output of a 3-step workflow where you:
1 โ Fetch raw data, for example from an external database.
2 โ Process the data into a tabular format, so you have N features and 1 target.
3 โ Train ML models (e.g. XGBoost) and tune hyper-parameters.
Jun 6 โข 14 tweets โข 4 min read
The one skill every professional data scientist must have? ๐ค
Don't look for it in online courses, it's not there.
Read the thread below and find out โโโ
In the real world, data science projects start from a business problem.
They are born to move a key business metric (KPI):
And you, as a data scientist, need to
1 โ Understand the business problem
2 โ Transform the business problem into a data science problem
3 โ Solve it
May 17 โข 16 tweets โข 4 min read
Let's build an LLM agent in Python, step-by-step โ๐งต
Why agents ๐คโ
Because Large Language Models alone are not enough to accurately answer complex tasks that require
-> External information that was not present in the training dataset used to fit the LLM paramaters
or
-> Many reasoning steps
May 15 โข 12 tweets โข 3 min read
How do you test your ML model before deploying it?
3 strategies to help you ๐ง โ
A better offline metric does NOT mean a better model, because
โ An offline metric (e.g test ROC) is *just* a proxy for the actual business metric you care about (e.g money lost in fraudulent transactions)
โ The ML model is just a small bit of the whole ML system in production
May 9 โข 9 tweets โข 3 min read
3 years ago I struggled to build ML products.
Then I discovered this โ
Unless you are a researcher in academia, and your goal is to publish a paper, you cannot just focus on the ML model you wanna train.
You need to think further down the line and think of the business problem you are trying to solve.
This is the "product-first" mindset.
May 8 โข 13 tweets โข 3 min read
Let's build an AI Coding assistant with Llama3 โ๐งต๐ฆ
Step 1. Download llama3 with Ollama ๐ฆ
Ollama is an open-source tool to run Large Language Models locally, that you can download for free from here.
A hands-on tutorial in 10 steps ๐ฉ๐ฝโ๐ป๐จโ๐ปโโโ
#1 Create your project folder and cd into it
Apr 4 โข 10 tweets โข 3 min read
XGBoost is one of the most effective algorithms for time-series prediction.
But, you need to prepare your data carefully.
These are the steps to transform raw data into supervised ML data for time-series prediction โ
Example
Imagine you work at a ride-sharing app company in NYC as an ML engineer.
You want to help the operations team allocate the fleet of drivers optimally each hour of the day.
The end goal is to maximize revenue
Mar 14 โข 6 tweets โข 2 min read
ML Project Idea ๐ก
Let's find your Twin Celebrity ๐๐ธ โ
Here is a full example, with source code, to learn how to build a complete ML app that finds your Twin Celebrity
Training an ML model inside a Jupyter notebook is something every data scientist knows ๐๏ธ
But do you know how to create a real-world ML service that makes a difference for the company you work for? ๐
If the answer is NO, this thread is for you ๐ค๐งตโ
So, what is the difference between model training and ML service? ๐ค
An ML service is a sequence of processing and storage steps that takes in raw data and outputs predictions that are used by the business to make smarter decisions.
Model training is just one of those steps.
Feb 15 โข 10 tweets โข 4 min read
ML Project Idea ๐ก
Let's predict taxi demand in NYC in the next 60 minutes ๐โ
Business problem ๐ผ
Let's create a predictive model to forecast the number of taxi rides that will happen in Manhattan (New York City)
- in the next hour
- for each taxi zone (e.g. Zone 113 "Lower Manhattan)