Share this page!

Luiz GUStavo Profile picture
Twitter logo
27 Mar, 13 tweets, 4 min read
If you are looking for something to learn during the weekend,

How about on-device Machine Learning?

You'll only need some understanding of ML and some of Mobile development.

Let me give you all the pointers in FAQ style:

[reading: 5.84 min]

1/12🧵
"Which tools will I need to start?"



2/12🧵
"Ok, how can a ML model run on a phone?"



3/12🧵
"Where can I find models to play with?"

TensorFlow Hub has 1000's of models for you to take a look at!
You can start here: tfhub.dev/s?deployment-f…

#TFHub

4/12🧵
"How do I convert Machine Learning models to run on-device?"



5/12🧵
"Ok, can you guide me through a full example?"

Yes, of course: tensorflow.page.link/tflite-codelab

6/12🧵
"The inference is not fast enough for what I want, any tips?"

Sure, let's talk about Model optimizations:

7/12🧵
"My phone has this cool ML specific chip, how do I take advantage of that?"

One easy gain in performance is using the specific hardware on your phone like the GPU, DSP, Neural Engine.

Like this:

8/12🧵
"Where can I find more sample code?"

The official repo has great samples: github.com/tensorflow/exa…

9/12🧵
"Where can I find samples from the community?"

That's a great question!
This repo is awesome and has a lot of great resources: github.com/margaretmz/awe…

@margaretmz

10/12🧵
"Any book suggestions?"

Of course!!
@lmoroney book is excellent and has some chapters about TFLite



11/12🧵
Do you have more questions? feel free to leave a comment!

Help me reach as many people as possible with this content:
- share with your friends, maybe this can also help them!
- share your work too!! What did you build? Let me know!

12/12🧵
And of course, I can't recommend enough the main book about the subject!



13/13🧵

• • •

Missing some Tweet in this thread? You can try to force a refresh
　

Keep Current with Luiz GUStavo

Luiz GUStavo Profile picture

Stay in touch and get notified when new unrolls are available from this author!

Read all threads

This Thread may be Removed Anytime!

PDF

Twitter may remove this content at anytime! Save it as PDF for later use!

Try unrolling a thread yourself!

how to unroll video
  1. Follow @ThreadReaderApp to mention us!

  2. From a Twitter thread mention us with a keyword "unroll"
@threadreaderapp unroll

Practice here first or read more on our help page!

More from @gusthema

Luiz GUStavo Profile picture
25 Mar
To make Apps with Magical User Experiences, you need to get all the performance possible from the hardware.

From the on-device ML perspective, you can achieve that using the TFLite Delegates.

They enable you to access the power of HW acceleration.

1/6🧵
Your phone's CPU is usually very fast but as a multi-purpose processor it's not optimized for the heavy math that ML needs

Like on their big brothers (servers 🤓) phones have also more specialized chips more suitable for ML, the most popular being the GPUs

2/6🧵
Another popular accelerator is the Qualcomm Hexagon DSP that has shown 75% reduction in power consumption.

On the Apple side, you can use the Core ML delegate to access the Neural Engine processor on newer i[Phones|Pads] and that can give huge boosts in performance!

3/6🧵
Read 6 tweets
Luiz GUStavo Profile picture
24 Mar
When we want to deploy a ML models on-device you may need to optimize them.

A model with bigger accuracy might also be bigger in size and also use more memory and slower

Do you need real time inference?

Let's take a look on how to optimize your model

[3.14 minutes]

1/6🧵
TFLite have the model optimization toolkit: tensorflow.org/model_optimiza…
To help you with this very important task.

Among the techniques are: Quantization and Pruning

2/6🧵
Quantization works by reducing the precision of the numbers used to represent a model's parameters, which by default are float32.

This results in a smaller model and faster computation.

More info here: blog.tensorflow.org/2020/04/quanti…

3/6🧵
Read 6 tweets
Luiz GUStavo Profile picture
23 Mar
When you have your TensorFlow Model and want to use it on a mobile device, you'll need to convert it to the TFLite format.

This process can be done in two ways:
- Using the Python API
- Using a command line tool

Let's look into some more details….

1/6🧵
Why do we need to convert?

The TFLite is an optimized format (Flatbuffer) for faster loading
To keep the framework lite and fast, all the Operations are optimized for mobile execution but not all TF operations are available

The available ops: tensorflow.org/lite/guide/ops…

2/6🧵
How to convert the model?

The Python API to convert a model is straightforward and it's the recommended method of conversion

You can, during the conversion, apply some optimizations, like post training quantization to reduce your model size and latency.

3/6🧵
Read 6 tweets
Luiz GUStavo Profile picture
22 Mar
Usually we imagine Machine Learning models running on expensive servers with lots of memory and resources

A change that is enabling complete new types of apps is executing ML models on the edge, like on phones and microcontrollers

Let's find out why that matters

[2min]

1/6🧵
Why would you want to run a ML model on a phone?

Lower Latency: if you want to get inference on real time, running a model over a cloud API is not going to give a good user experience.

Running locally is much better and potentially much faster ⚡️

2/6🧵
When running ML models on-device, your app will be able to keep working even without network connectivity

For example, if your app translates text, it will still work in another country, when you need most and it won't use your hard earned money on roaming fees!

3/6🧵
Read 6 tweets
Luiz GUStavo Profile picture
1 Mar
Let's start with some theory

I've been working with ML on the Audio domain and at first I couldn't understand much but as I kept reading I managed to figure out some things.

Let me share some of the basic theory with you:

[10 minutes read]

1/n🧵
Sound is a vibration that propagates as an acoustic wave.

It has some properties:
- Frequency
- Amplitude
- Speed
- Direction

For us, Frequency and Amplitude are the important features.

en.wikipedia.org/wiki/Sound#Sou…

2/n🧵
An important aspect is that Sounds are a mixture of their component Sinusoidal waves(follow a sine curve) of different frequencies

From the equation below:
- A is amplitude
- f is frequency
- t is time

The code replicates the formula and compose a third wave from 2 other

3/n🧵
Read 15 tweets
Luiz GUStavo Profile picture
19 Jan
For developers, a good debugger and profiler are fundamental tools for their productivity.

On the ML world, TensorBoard can help you with that by enabling:

- Visualizing metrics, model, histograms of weights or biases
- Displaying images, text and audio data
- Profiling

1/5🧵
You can load TensorBoard directly on Colab using a magic word to load the extension and another one to load the tool.

The nice part is that this does not require installing anything on your computer.

2/5🧵
To visualize your training data, you'll need to create a callback and use it on the fit method.

The callback just needs the directory where the log will be written.

3/5🧵
Read 5 tweets

Did Thread Reader help you today?

Support us! We are indie developers!

This site is made by just two indie developers on a laptop doing marketing, support and development! Read more about the story.

Become a Premium Member ($3/month or $30/year) and get exclusive features!

Become Premium

Too expensive? Make a small donation by buying us coffee ($5) or help with server cost ($10)

Donate via Paypal Become our Patreon

Thank you for your support!

Follow Us on Twitter!

Like this author's thread?

Get emails when @gusthema has new unrolls!

Check out other threads from @gusthema!

Read all threads by @gusthema