ML Project Idea 💡

Let's predict air quality ↓ Here is a full example, with source code, to learn how to build a complete ML app that predicts air quality in different European cities.

Clone the code, modify it, and deploy it!
github.com/logicalclocks/…

Are you a data scientist using CSV files to store your data?

What if I told you there is a better way?

Can you imagine a

-> lighter 🦋
-> faster 🏎️
-> cheaper 💸

file format to save your datasets?

Read this thread so you don't need to imagine anymore 👇🏾 Do not get me wrong. I love CSVs.

You can open them with any text editor, inspect them and share them with others.

They have become the standard file format for datasets in the AI/ML community.

However, they have a little problem...

3 years ago I struggled to build ML products.

Then I discovered this ↓ Unless you are a researcher in academia, and your goal is to publish a paper, you cannot just focus on the ML model you wanna train.

You need to think further down the line and think of the business problem you are trying to solve.

This is the "product-first" mindset.

Let's design an ML system to predict crypto prices, step-by-step ↓🧵 The problem

We want to build a real-time API that serves in real-time short-term predictions on crypto prices.

For example
To predict the price of Ethereum (ETH) in the next 10 seconds.

One skill every ML engineer has to master ↓ 𝗠𝗟 𝗦𝘆𝘀𝘁𝗲𝗺 𝗱𝗲𝘀𝗶𝗴𝗻

Yes. And do you know why?

Because good ML system design hasn't changed at all in the last 5 years.

And it won't.

Every aspiring data scientist I talk to is overwhelmed by the colossal amount of online courses to choose from 🤯

My solution to this problem ↓ Learning is about connecting the dots.

However, it feels like there are too many dots to connect when learning data science.

Too many courses...
Too many blog posts...
Too many technologies...

Solution: You need to change the way you learn.

Time-series are used everywhere

→ At Uber to optimize fleet efficiency
→ At Amazon to forecast inventory levels
→ At every hedge fund to project asset prices.

Still, there is a lack of ML engineers who can build real-world time-series products.

So here is your chance ↓ Let's build a *complete* ML service that forecasts taxi rides in NYC, similar to what Uber does to forecast demand.

The 3 ingredients we need are;

- a dataset
- a Python library to build a good predictive model
- a deployment strategy

For example ↓

3 years ago I struggled to land my first freelance ML engineering contract.

Then I discovered this ↓ Building one professional real-world ML project is the best way to stand out from the crowd, and land an ML job.

Here is what I did, 𝘀𝘁𝗲𝗽-𝗯𝘆-𝘀𝘁𝗲𝗽 👩‍💻👨🏽‍💻↓

Let's build an AI Coding assistant with Llama3 ↓🧵🦙 Step 1. Download llama3 with Ollama 🦙

Ollama is an open-source tool to run Large Language Models locally, that you can download for free from here.

ollama.com/download

Let's design an ML system to predict crypto prices, step-by-step ↓🧵 The problem

We want to build a real-time API that serves in real-time short-term predictions on crypto prices.

For example
To predict the price of Ethereum (ETH) in the next 10 seconds.

Let's build a real-time ML system to predict short-term prices
↓↓↓🧵 The problem

We want to build a real-time API that serves in real-time short-term predictions on crypto prices.

For example
To predict the price of Ethereum (ETH) in the next 10 seconds.

Most of my ML model prototypes never reached production 😵‍💫

Until I changed my mindset 🧠↓↓↓ 🔬 Model-first mindset

A model-first mindset is what Kaggle competitions and most online courses are about.

Your ONLY focus is to build the best possible mapping between a set of input features, and a target metric

And in real-world ML this is often not the best approach...

How do you build
> 𝗿𝗲𝗮𝗹-𝘁𝗶𝗺𝗲 ML systems ⚡
> at 𝘀𝗰𝗮𝗹𝗲 🎛️
> 𝘄𝗶𝘁𝗵𝗼𝘂𝘁 𝗯𝘂𝗿𝗻𝗶𝗻𝗴 𝗰𝗮𝘀𝗵 💸?

🧵↓ 𝗧𝗵𝗲 𝗽𝗿𝗼𝗯𝗹𝗲𝗺 🤔

Let’s say you work as an ML engineer at a fintech startup, whose flagship product is a mobile app for online payments.

A critical problem you need to tackle from day 0 is the automatic detection of fraudulent transactions.

ML Project Idea 💡

Let's predict taxi demand in NYC in the next 60 minutes 🚕↓ Business problem 💼

Let's create a predictive model to forecast the number of taxi rides that will happen in Manhattan (New York City)

- in the next hour
- for each taxi zone (e.g. Zone 113 "Lower Manhattan)

Let's do it in 6 steps ↓

ML Project Idea 💡

Let's predict flight delays 🛬 ↓ Here is a full example, with source code, to learn how to build a complete ML app that predicts flight delays for Stockholm Arlanda airport.

Clone the code, modify it, and deploy it!
github.com/SebastianoMene…

Let's build an LLM agent in Python, step-by-step ↓🧵 Why agents 🤖❓

Because Large Language Models alone are not enough to accurately answer complex tasks that require

-> External information that was not present in the training dataset used to fit the LLM paramaters
or
-> Many reasoning steps

ML Project Idea 💡

Let's predict flight delays 🛬 ↓ Here is a full example, with source code, to learn how to build a complete ML app that predicts flight delays for Stockholm Arlanda airport.

Clone the code, modify it, and deploy it!
github.com/SebastianoMene…

ML Project Idea 💡

Let's predict air quality ↓ Here is a full example, with source code, to learn how to build a complete ML app that predicts air quality in different European cities.

Clone the code, modify it, and deploy it!
github.com/logicalclocks/…

Wanna learn time-series forecasting? 📈

No more reading blog posts.
It is time to forecast for real 😎

Here is a project you can build 👩‍💻🧑🏽‍💻↓ Business problem 💼

Let's create a predictive model to forecast the number of taxi rides that will happen in Manhattan (New York City)

- per hour (e.g. tomorrow between 5 PM and 6 PM), and
- per zone (e.g. Zone 113 "Lower Manhattan)

in the following 3 days.

Wanna become an NLP engineer?

Stop taking online courses.
Build something instead 🏗️

Here is a project you can build 👩🏽‍💻👨‍💻↓↓↓ Reading blog posts about multi-billion-parameter Language Models is very cool.

However, building real-world NLP products from these models is where the real business value is. And this is what companies look for in the job market.

So, here is a PRO project you can build ↓

Tired of training lots of Machine Learning models, and not getting better results? 😵‍💫

This is how you solve this 🧠↓ A Machine Learning model is the output of a 3-step workflow where you:

1 → Fetch raw data, for example from an external database.

2 → Process the data into a tabular format, so you have N features and 1 target.

3 → Train ML models (e.g. XGBoost) and tune hyper-parameters.