I like building stuff. Cyber security for AI - VP of Engineering @FR0NTIER_X. Used to program self-driving cars at BMW.
7 subscribed
Jul 5, 2022 โข 20 tweets โข 5 min read
Zero-Knowledge Proofs 0๏ธโฃ๐
How can I prove to you that I know a secret, without revealing any information about the secret itself?
This is called a zero-knowledge proof and it is a super interesting area of cryptography! But how does it work?
Thread ๐งต
Let's start with an example
Peggie and Victor travel between cities A and B. There are two paths - a long path and a short path. The problem is that there is a gate on the short path for which you need a password.
We will be dealing with an ML model to detect traffic lights for a self-driving car ๐ค๐
Traffic lights are small so you will have much more parts of the image that are not traffic lights.
Furthermore, yellow lights ๐ก are much rarer than green ๐ข or red ๐ด.
Mar 22, 2022 โข 16 tweets โข 7 min read
Machine Learning Explained ๐จโ๐ซ
PCA
Principal Component Analysis is a commonly used method for dimensionality reduction.
It's a good example of how fairly complex math can have an intuitive explanation and be easy to use in practice.
Let's start from the application of PCA ๐
Dimensionality Reduction
This is one of the common uses of PCA in machine learning.
Imagine you want to predict house prices. You get a large table of many houses and different features for them like size, number of rooms, location, age, etc.
Some features seem correlated ๐
Mar 18, 2022 โข 8 tweets โข 4 min read
s this formula difficult? ๐ค
This is the formula for Gradient Descent with Momentum as presented in Wikipedia.
It may look intimidating at first, but I promise you that by the end of this thread it will be easy to understand!
Let's break it down! The basis is this simple formula describing an iterative optimization method.
We have some weights (parameters) and we iteratively update them in some way to reach a goal
Iterative methods are used when we cannot compute the solution directly
Mar 16, 2022 โข 20 tweets โข 4 min read
Machine Learning Formulas Explained ๐จโ๐ซ
For regression problems you can use one of several loss functions:
โช๏ธ MSE
โช๏ธ MAE
โช๏ธ Huber loss
But which one is best? When should you prefer one instead of the other?
Thread ๐งต
Let's first quickly recap what each of the loss functions does. After that, we can compare them and see the differences based on some examples.
๐
Mar 11, 2022 โข 16 tweets โข 5 min read
Machine Learning in the Real World ๐ง ๐ค
ML for real-world applications is much more than designing fancy networks and fine-tuning parameters.
In fact, you will spend most of your time curating a good dataset.
We need to represent the real world as accurately as possible. If some situations are underrepresented we are introducing Sampling Bias.
Sampling Bias is nasty because we'll have high test accuracy, but our model will perform badly when deployed.
๐
Mar 8, 2022 โข 13 tweets โข 5 min read
Machine Learning Formulas Explained ๐จโ๐ซ
This is the Huber loss - another complicated-looking formula...
Yet again, if you break it down and understand the individual, it becomes really easy.
Let me show you ๐
Background
The Huber loss is a loss function that is similar to the Mean Squared Error (MSE) but it is designed to be more robust to outliers.
MSE suffers from the problem that if there is a small number of severe outliers they can dominate the whole loss
How does it work? ๐
Mar 4, 2022 โข 13 tweets โข 5 min read
Machine Learning Formulas Explained! ๐จโ๐ซ
This is the formula for the Binary Cross Entropy Loss. It is commonly used for binary classification problems.
It may look super confusing, but I promise you that it is actually quite simple!
Let's go step by step ๐
#RepostFriday
The Cross-Entropy Loss function is one of the most used losses for classification problems. It tells us how well a machine learning model classifies a dataset compared to the ground truth labels.
The Binary Cross-Entropy Loss is a special case when we have only 2 classes.
๐
Mar 3, 2022 โข 11 tweets โข 7 min read
When machine learning met crypto art... they fell in love โค๏ธ
The Decentralized Autonomous Artist (DAA) is a concept that is uniquely enabled by these technologies.
Meet my favorite DAA - Botto.
Let me tell you how it works ๐
Botto uses a popular technique to create images - VQGAN+CLIP
In simple terms, it uses a neural network model generating images (VQCAN) guided by the powerful CLIP model which can relate images to text.
This method can create stunning visuals from a simple text prompt!
๐
Feb 25, 2022 โข 20 tweets โข 7 min read
There are two problems with ROC curves
โ They don't work for imbalanced datasets
โ They don't work for object detection problems
So what do we do to evaluate our machine learning models properly in these cases?
We use a Precision-Recall curve.
Thread ๐
#RepostFriday
Last week I wrote another detailed thread on ROC curves. I recommend that you read it first if you don't know what they are.
Is your machine learning model performing well? What about in 6 months? ๐ค
If you are wondering why I'm asking this, you need to learn about ๐ฐ๐ผ๐ป๐ฐ๐ฒ๐ฝ๐ ๐ฑ๐ฟ๐ถ๐ณ๐ and ๐ฑ๐ฎ๐๐ฎ ๐ฑ๐ฟ๐ถ๐ณ๐.
Let me explain this to you using two real world examples.
Thread ๐
Imagine you are developing a model for a self-driving car to detect other vehicles at night.
Well, this is not too difficult, since vehicles have two red tail lights and it is easy to get a lot of data. You model works great!
But then... ๐
Feb 22, 2022 โข 9 tweets โข 5 min read
Math is not very important when you are using a machine learning method to solve your problem.
Everybody that disagrees, should study the 92-page appendix of the Self-normalizing networks (SNN) paper, before using
torch.nn.SELU.
And the core idea of SNN is actually simple ๐
SNNs use an activation function called Scaled Exponential Linear Unit (SELU) that is pretty simple to define.
It has the advantage that the activations converge to zero mean and unit variance, which allows training of deeper networks and employing strong regularization.
๐
Feb 21, 2022 โข 10 tweets โข 3 min read
This is like an NFT in the physical world
This is a special edition BMW 8 series painted by the famous artist Jeff Koons. A limited-edition of 99 with a price of $350K - about $200K more than the regular M850i.
If you think about it, you'll see many similarities with NFTs
๐
Artificially scarce
BMW can surely produce (mint ๐ ) more than 99 cars with this paint. The collection size is limited artificially in order to make it more exclusive.
Same as most NFT collections - they create artificial scarcity.
๐
Feb 18, 2022 โข 19 tweets โข 6 min read
Did you ever want to learn how to read ROC curves? ๐๐ค
This is something you will encounter a lot when analyzing the performance of machine learning models.
ROC stands for Receiver Operating Characteristic but just forget about it. This is a military term from the 1940s and doesn't make much sense today.
Think about these curves as True Positive Rate vs. False Positive Rate plots.
Now, let's dive in ๐
Feb 17, 2022 โข 12 tweets โข 2 min read
It sucks if your ML model can't achieve good performance, but it is even worse if you don't know it!
Sometimes you follow all the best practices and your experiments show your model performing very well, but it fails when deployed.
A thread about Sampling Bias ๐
There is a lot of information about rules you need to follow when evaluating your machine learning model:
โช๏ธ Balance your dataset
โช๏ธ Use the right metric
โช๏ธ Use high-quality labels
โช๏ธ Split your training and test data
โช๏ธ Perform cross-validation
But this may not be enough ๐
Jan 18, 2022 โข 15 tweets โข 4 min read
The Internet is already decentralized, why do we need web3? ๐ค
This is a common critique of web3. However, decentralization on its own is not always enough - sometimes we need to agree on a set of facts.
Blockchains give us a consensus mechanism for that!
Thread ๐งต
1/12
The Internet is built of servers that communicate using open protocols like HTTP, SMTP, WebRTC etc. Everybody can set up a server and participate. It is decentralized!
However, if two servers distribute contradicting information, how do you know which one is right?
2/12
Jan 18, 2022 โข 9 tweets โข 4 min read
How decentralized is web3 really?
While there is a lot of hype around web3, NFTs, and decentralized apps (dApps), there is also a lot of criticism. Today, I'll focus on the critique that web3 is actually too centralized.
Let's try to have an honest discussion ๐
These are the main arguments I see regularly. Please add more in the comments.
1๏ธโฃ The Internet is already decentralized
2๏ธโฃ It is inefficient
3๏ธโฃ Everything can be implemented better using a centralized approach
4๏ธโฃ Important services are centralized
๐
Jan 17, 2022 โข 7 tweets โข 5 min read
How many parameters do you need in your neural network to solve any problem? ๐ค
GPT-3 has 175 billion, MT-NLG has 530 billion and Wu Dao has 1.75 trillion.
But the truth is you only need 1 parameter. No, not 1 billion. Just a single parameter!
Let me explain ๐
Yes, of course, I'm trolling you, but only a little bit ๐
I want to show you this very cool work by @ranlot75 about how to fit an arbitrary dataset with a single parameter and the following function
You think you know what is an NFT? Well, think again...
You are doing it wrong if you think about NFTs as pixelated images of punks, toads, or apes. It is not about the JPEG!
A better mental model for thinking about NFTs ๐
Forget the images for now. Owning an NFT means that your wallet address is listed as the owner of a specific digital asset on the blockchain.
Digital assets are organized in collections and an NFT is one specific piece of this collection.
Let's look at an example ๐
Dec 21, 2021 โข 5 tweets โข 3 min read
Things are getting more and more interesting for AI-generated images! ๐จ
GLIDE is a new model by @OpenAI that can generate images guided by a text prompt. It is based on a diffusion model instead of the more widely used GAN models.
Some details ๐ @OpenAI GLIDE also has the interesting ability to perform inpainting allowing for some interesting usages.