1/ 🧵 Getting started with data science and machine learning.
- the first step is to know what data science and machine learning mean and is this field worth getting into?
- if you ask me, I would say yes. the world is data-centric.
2/ in every industry, data is useful and will be for a long long time.
- if you are a developer, it will be easier for you to get into data science
- but it also means you have to work twice as hard since you are already working
3/ - if you are a fresher or a student, it's much easier for you to spend at least 4 hours a day learning something new
- now a lot of people ask where to start from?
- step 0 will be learning to code.
💥 Did you know that there are problems other than MNIST and iris that you can solve (or try to solve) to learn deep learning and computer vision? Here is a list of my favourite Kaggle competitions to learn deep learning and computer vision from ⬇️ 1/13
🚀 If you are starting with machine learning / deep learning and get a new dataset to work on, either on kaggle or in real-world or just for fun. There are a few things you must always take care of to squeeze the most out of your model and make it awesome: ⬇️⬇️⬇️
1/6
🔹 Look at the data carefully. Do EDA.
🔹 Look at the targets. See how they are distributed and what kind of problem this is.
🔹 Choose the right metric to evaluate your models
2/6
🔹 Split the data into folds. You can use this for cross validation or for hold out based validation
🔹 Build a first basic model. This is going to be your baseline.
🔹 Now try to improve on the baseline by adding new features
3/6
Not surprised that none of the nay-sayers were not able to respond. That's what happens when you start accusing and abusing someone without understanding the context.
So, people who called me names here is a test for you. You need to use python.
- You have 100k CSVs in a folder.
- Read all files in the folder
- Combine them in a single CSV
- Save the combined file for feature engineering using pandas
- All files share the same header
1/4
where do I find 100k CSVs in a folder? Well, in many scenarios and real-life situations. I have made it easy for you: github.com/abhishekkrthak…
Those who called me names must use pandas.
Those who are willing to learn, scroll below.
2/4
Using pandas, in a simple way, took 120 seconds to do this for me. Using pure python took 5.5 seconds, using pypy took 3.8 seconds. That's why it's important to learn the basics too.
After that, ill use pandas for feature engineering. You don't need a bazooka to kill a fly
3/4