Thread by @svpino

You want to build a face recognition system for your office, but getting many pictures from your coworkers is not a choice.

Also, having to retrain the model for every new employee seems like a burden.

How do we solve this?

Grab your ☕️ and let's do the thing!👇

To solve a standard classification problem, you collect many images representing the different classes you want to classify.

You label the images and train a classification model.

This is all good, but sometimes getting a lot of images is not an option.

[2 / 13]

A face recognition system is one example: getting many images for every person we want to support is impractical.

Another example is a signature verification system: we want a model capable of verifying a signature even when we didn't train it.

[3 / 13]

Clearly, we can't solve these problems with a standard classification model, so here is a different solution:

▫️ A Siamese Neural Network.

Let's talk a little bit more about this beauty 😎

[4 / 13]

The trunk of a Siamese Network contains two identical subnetworks (I call these "twins.")

These subnetworks are exactly the same: a mirror of each other.

The attached picture is my attempt to draw this thing.

[5 / 13]

The goal of the twin subnetworks is to produce a feature vector for each one of the given images.

Then we have a layer that will compute the distance from one feature vector to the other.

Let's say the distance is zero. What does that mean?

[6 / 13]

If the distance is zero, both image 1 and image 2 must be the same!

A shorter distance means that both images have more features in common.

A larger distance means that both images have fewer features in common.

Makes sense, right?

[7 / 13]

Let's get back to our face recognition system.

We could take a live picture of a person and compare it with a saved picture (a reference image.)

Our Siamese network will determine how far away both images are. If they are close, we have a match!

Beautiful, right?

[8 / 13]

Here is the main takeaway of this story:

👉 Our Siamese network is not learning to classify the input image into specific classes.

Instead, it is learning a similarity function that we can use to compare any two images.

[9 / 13]

This means that we don't need to retrain the network to add more faces (or signatures in the case of signature verification.)

If we learn a good similarity function, we can compare any two domain images to determine whether they are the same.

[10 / 13]

It may be evident at this point, but a massive advantage of Siamese Networks is that we don't need many images to solve the problem:

One-shot Learning gets away with a single image!

[11 / 13]

I'm currently doing a lot of work with Siamese networks, and the results so far have been fantastic!

I'll write more about that as I collect more information. Taking these things out of papers and into production systems is trickier than what may seem.

[12 / 13]

Speaking of tricky things, if you are looking for some deep dives into machine learning and how to make it work in real life, follow me.

This stuff is fun, but it's even better if we do it together!

[13 / 13]

🦕

Great question! No, you don't necessarily need multiple pictures of the same person. Remember: you aren't trying to classify a person. You are trying to create a similarity function.

You train the network using pairs of images (A, A, 1) and (A, B, 0).

https://twitter.com/JurieHorneman/status/1365699961313239041?s=20

(A, A, 1) represents a pair with two pictures that match. The one is your "y" (1 because they match.)

(A, B, 0) represents a pair with two pictures that don't match. In this case, "y" is 0.

This is a good question, and I personally don't know whether there's any research on this area.

An idea would be with a binary search approach:

* Is the subject white? Search that pile.
* Does he has a bear? Search that other pile.

What do you think?

https://twitter.com/abdulsami/status/1365718742810976256?s=20