Santiago Profile picture
22 Apr, 14 tweets, 4 min read
A 13-tweet introduction to one of the most basic structures used in machine learning: a tensor.

Understanding how tensors work is fundamental. They aren't complex but working with them may get confusing if you don't understand all the pieces.

Let's solve that today.

↓ 1/13
Three primary attributes define a tensor:

• Rank: Number of axes.
• Shape: Number of dimensions per axis.
• Data type: Type of data contained in it.

↓ 2/13
The rank of a tensor refers to the tensor's number of axes.

Examples:

• Rank of a matrix is 2.
• Rank of a vector is 1.
• Rank of a scalar is 0.

↓ 3/13
The shape of a tensor describes the number of dimensions along each axis.

Examples:

• () scalar
• (2,) vector
• (3, 2) matrix
• (3, 2, 5) 3D tensor

↓ 4/13
The data type of a tensor refers to the type of data contained in it.

For example, here are some of the supported data types that we could store in a tensor:

• float32
• float64
• uint8
• int32
• int64

↓ 5/13
A scalar—or 0D tensor—has rank 0 and contains a single number. These are also called "0-dimensional tensors."

The attached image shows how to construct a 0D tensor using NumPy. Notice its shape and its rank (.ndim attribute.)

↓ 6/13
A vector—or 1D tensor—has rank 1 and represents an array of numbers.

The attached image shows a vector with shape (4, ). Notice how its rank (.ndim attribute) is 1.

↓ 7/13
A matrix—or 2D tensor—has rank 2 and represents an array of vectors. The two axes of a matrix are usually referred to as "rows" and "columns."

The attached image shows a matrix with shape (3, 4).

↓ 8/13
You can obtain higher-dimensional tensors (3D, 4D, etc.) by packing lower-dimensional tensors in an array.

For example, packing a 2D tensor in an array gives you a 3D tensor. Packing this one in another array gives you a 4D tensor, and so on.

↓ 9/13
Some common tensor representations in practice:

• Vectors - 1D: (features)
• Sequences - 2D: (timesteps, features)
• Images - 3D: (height, width, channels)
• Videos - 4D: (frames, height, width, channels)

↓ 10/13
Machine learning algorithms deal with a subset of data at a time. We call these "batches."

When using a batch, the tensor's first axis is reserved for the size of the batch. This adds another axis to the representations we saw before.

↓ 11/13
For example, if your handling 2D tensors (matrices), a batch of them will have a total of 3 dimensions:

(samples, rows, columns)

Notice how the first axis is the number of matrices that you have in the batch.

↓ 12/13
Following the same logic, a batch of images can be represented as a 4D tensor and a batch of videos as a 5D tensor:

• (samples, height, width, channels)
• (samples, frames, height, width, channels)

↓ 13/13
If you found this thread helpful, follow me @svpino for weekly posts touching on machine learning and how to use it to build real-life systems.

It’s all about building value, and that’s way more fun if we do it together.

• • •

Missing some Tweet in this thread? You can try to force a refresh
 

Keep Current with Santiago

Santiago Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @svpino

23 Apr
I've been teaching people how to start with machine learning for more than a year now.

This is a thread with what I've learned and some advice if you are looking to start.

↓ 1/14
People enjoy the process of getting ready to start something new.

Like the first time they go to the gym: they buy supplements, shorts, shoes, and a new headband.

Unfortunately, this is not enough.

↓ 2/14
Most people never stop preparing.

They keep collecting books, tutorials, the "best" videos and ask the same "how to start" questions.

The only thing they forget: taking a concrete step forward.

↓ 3/14
Read 14 tweets
21 Apr
700 people have watched "How To Get Started With Machine Learning." 86 have rated it.

Let's celebrate!

• You can buy the course today for $7.
• $0 if you don't like it.
• Back to $15 tomorrow.

gum.co/kBjbC/only7

If you can't afford it, keep reading:

For every copy I sell today, I'll give away one for free.

To apply for the free copy, reply below with why you think this course will help you.

I'll prioritize the best stories I read.

If you want to support my content, like/retweet this thread, so more people see it.
So far, 8 copies sold, and 8 free copies shared.

Thanks for the continuous support! It helps tremendously!
Read 8 tweets
21 Apr
Creating a good machine learning model is really sexy. That's what's different and where everyone focuses all of their attention.

But machine learning is much more than that.

A thread with a few thoughts about the real job.

1/9
Machine learning engineers spend a lot of time designing and training new models, but this is just a small fraction of their job.

2/9
In reality, dealing with data and operationalizing models is much more time-consuming and sometimes even harder and more involved than creating the models in the first place.

3/9
Read 10 tweets
20 Apr
The backbone of my end-to-end machine learning setup:

• A 48-page Field Notes
• Python
• NumPy, Pandas, Matplotlib, OpenCV
• Scikit-Learn, XGBoost
• TensorFlow
• Google Colab, Jupyter, VSCode
• Docker, Flask
• AWS SageMaker
I personally don't use C/C++.

That doesn't mean it's not useful. I know plenty of people in the industry that rely on C/C++ to do their work.

It just means that I personally haven't needed it.

There are a lot of satellite tools that I have to use depending on the project. Kinesis, Airflow, SQS... the list is endless.

I just tried to list the core of what I need, and it rarely varies.

Read 5 tweets
20 Apr
Yesterday, @PrasoonPratham posted a step-by-step guide to solve the Titanic challenge on Kaggle.

I thought it'd be fun to engineer some features that can help build an even better model.

Here are some ideas worth considering.

↓ 1/10
Attached you can find the original set of input variables that come with the data to solve the problem.

We are going to transform some of these into features that should help our model produce better results.

This is what Feature Engineering is all about.

↓ 2/13
Keep in mind that these are just hypotheses that you'll have to try and validate.

Some of these suggestions might not improve the results or could even make the model perform worse.

This is an exercise to try and think creatively about the data we are getting.

↓ 3/13
Read 15 tweets
19 Apr
Is 10 twice as worse as 5? Sometimes it is, but sometimes it's even worse.

This is the question I always ask myself when deciding how to penalize my models.

Read on for more details and a couple of examples:

↓ 1/11
When we are training a machine learning model, we need to compute how different our predictions are from the expected results.

For example, if we predict a house's price as $150,000, but the correct answer is $200,000, our "error" is $50,000.

↓ 2/11
There are multiple ways we can compute this error, but two common choices are:

• RMSE — Root Mean Squared Error
• MAE — Mean Absolute Error

Both of these have different properties that will shine depending on the problem you want to solve.

↓ 3/11
Read 12 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!