🚀 If you are starting with machine learning / deep learning and get a new dataset to work on, either on kaggle or in real-world or just for fun. There are a few things you must always take care of to squeeze the most out of your model and make it awesome: ⬇️⬇️⬇️
1/6

🔹 Look at the data carefully. Do EDA.
🔹 Look at the targets. See how they are distributed and what kind of problem this is.
🔹 Choose the right metric to evaluate your models
2/6

🔹 Split the data into folds. You can use this for cross validation or for hold out based validation
🔹 Build a first basic model. This is going to be your baseline.
🔹 Now try to improve on the baseline by adding new features
3/6

Read 6 tweets

abhishek

@abhi1thakur

8 Feb

https://twitter.com/abhi1thakur/status/1358674182121418752

Not surprised that none of the nay-sayers were not able to respond. That's what happens when you start accusing and abusing someone without understanding the context.

https://twitter.com/abhi1thakur/status/1358674182121418752

Here are some solutions in this thread 🔽🔽🔽 1/7

Here is a solution using pandas. Time taken: 191.77s
2/7

Here is another solution using pandas. Time taken: 188.21s
3/ 7

Read 7 tweets

abhishek

@abhi1thakur

8 Feb

So, people who called me names here is a test for you. You need to use python.

- You have 100k CSVs in a folder.
- Read all files in the folder
- Combine them in a single CSV
- Save the combined file for feature engineering using pandas
- All files share the same header
1/4

where do I find 100k CSVs in a folder? Well, in many scenarios and real-life situations. I have made it easy for you: github.com/abhishekkrthak…

Those who called me names must use pandas.
Those who are willing to learn, scroll below.

2/4

Using pandas, in a simple way, took 120 seconds to do this for me. Using pure python took 5.5 seconds, using pypy took 3.8 seconds. That's why it's important to learn the basics too.

After that, ill use pandas for feature engineering. You don't need a bazooka to kill a fly

3/4

Read 5 tweets

abhishek

@abhi1thakur

7 Feb

Stacking in machine learning 🔽 1/4

- Divide the training data into folds.
- Train a bunch of models: M1, M2.....Mn.
- Create full training predictions (using out of fold training) and test predictions using all these models. 2/4

- Till here it is Level – 1 (L1).
- Use the fold predictions from these models as features to another model.
This is now a Level – 2 (L2) model.
- Use the same folds as before to train this L2 model. 3/4

Read 4 tweets

abhishek

@abhi1thakur

7 Feb

Have you had troubles or having troubles arranging your machine learning projects? This thread should give you some idea on how to arrange machine learning / deep learning projects. See the folder structure: 1/6 🔽

input/: This folder consists of all the input files and data for your machine learning project. If you are working on NLP projects, you can keep your embeddings here. If you are working on image projects, all images go to a subfolder inside this folder. 2/6

src/: We will keep all the python scripts associated with the project here. If I talk about a python script, i.e. any *.py file, it is stored in the src folder. 3/6

Read 6 tweets

Share this page!

abhishek

Try unrolling a thread yourself!

More from @abhi1thakur

abhishek

abhishek

abhishek

abhishek

abhishek

abhishek

Did Thread Reader help you today?

Like this author's thread?