Santiago Profile picture
2 Feb, 7 tweets, 2 min read
Here is a full Python 🐍 implementation of a neural network from scratch in less than 20 lines of code!

It shows how it can learn 5 logic functions. (But it's powerful enough to learn much more.)

An excellent exercise in learning how feedforward and backpropagation work!
A quick rundown of the code:

▫️ X β†’ input
▫️ layer β†’ hidden layer
▫️ output β†’ output layer
▫️ W1 β†’ set of weights between X and layer
▫️ W2 β†’ set of weights between layer and output
▫️ error β†’ how far is our prediction after every epoch
I'm using a sigmoid as the activation function. You will recognize it through this formula:

sigmoid(x) = 1 / 1 + exp(-x)

It would have been nicer to extract it as a separate function, but then the code wouldn't be as compact πŸ˜‰
Within the loop, we first update the value of both layers. This is "forward propagation."

Then we compute the error.

Then we update the value of the weights (starting with the last set.) This is "backpropagation."
If you'd like to play with the code, here is the link to it:

gist.github.com/svpino/e54ff03…
There are a couple of reasons you want to use sigmoid on the last layer:

1. You need to constraint the weights from growing too much. The sigmoid will do just that.

2. You want your output to be between 0 and 1.

Try to remove it and print the output.

Here are some (hand-wavy) notes about the backpropagation step.

Hopefully, these explain how W2 and W1 are computed.

β€’ β€’ β€’

Missing some Tweet in this thread? You can try to force a refresh
γ€€

Keep Current with Santiago

Santiago Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @svpino

4 Feb
I built a model to predict whether you'll be involved in a crash next time you get in a car.

And it's 99% accurate!

Allow me to show you...πŸ‘‡ Image
Here is the model:

πŸ‘‡ Image
The National Safety Council reports that the odds of being in a car crash in the United States are 1 in 102.

That's a probability of 0.98% of being involved in a crash.

Therefore, my silly model is accurate 99% of the time!

See? I wasn't joking before.

πŸ‘‡
Read 21 tweets
2 Feb
For the past few months, I've been trying to improve the quality of the content I publish.

There are a couple of ways I'm measuring this:

▫️ Efficiency
▫️ Engagement

Efficiency is about how many impressions and followers I get for every tweet I post.

πŸ‘‡ Image
I've gone from posting 3,126 tweets back in August down to 949 tweets last month.

I've cut a lot of the noise!

During the same period, I've doubled my impressions (up to 14.4M last month,) and I'm now converting 5.38 followers for every tweet (up from 2.52.)

πŸ‘‡
The second way I'm watching the quality of the content I'm posting is through the engagement rate.

This has gone down quite a bit since August (almost cut in half!)

As impressions increase, the more pressure I have to put engaging content out there.

πŸ‘‡ Image
Read 7 tweets
1 Feb
Time spent developing better datasets is usually more productive than squeezing the algorithms that process them.
One thing to keep in mind is that "better datasets" is not equivalent to "more data."

Regardless of your ability to collect the data, properly pre-processing it will usually give you a very good bang for your buck.

Hopefully, credit is given for the ultimate predictive ability of the solution.

A machine learning system is not just a model. There are a lot of pieces that need to work together.

Read 5 tweets
1 Feb
Here is a simple example of a machine learning model.

I put it together a long time ago, and it was very helpful! I sliced it apart a thousand times until things started to make sense.

It's TensorFlow and Keras.

If you are starting out, this may be a good puzzle to solve.
The goal of this model is to learn to multiply one-digit numbers.

The dataset has two values (the ones we want to multiply.) That's why the input shape is 2D.

The input shape represents the input layer of our model. It connects to the first hidden layer: a 4-unit Dense layer.

Then you get another 4-unit Dense layer.

Read 4 tweets
31 Jan
What's your favorite machine learning book?
This edition of the book does indeed uses TensorFlow 2.0 (notice the top right corner of the picture.)

Read 4 tweets
31 Jan
"Hands-On Machine Learning..." is β€”without a doubtβ€” my favorite machine learning book.

It's not only a great reference, but it's the type of book that you can easily read cover to cover!

If you want to start from a solid foundation, look no further.

πŸ’° amzn.to/2KPuRAo
The book assumes that you have some experience with:

▫️ Python programming
▫️ NumPy
▫️ Pandas
▫️ Matplotlib

For a deeper dive, it expects you to have a reasonable understanding of calculus, linear algebra, probabilities, and statistics.
The book is organized into two parts:

▫️ The Fundamentals of Machine Learning
▫️ Neural Networks and Deep Learning

Here is the outline of what's covered: πŸ‘‡
Read 9 tweets

Did Thread Reader help you today?

Support us! We are indie developers!


This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!