To make Apps with Magical User Experiences, you need to get all the performance possible from the hardware.
From the on-device ML perspective, you can achieve that using the TFLite Delegates.
They enable you to access the power of HW acceleration.
1/6🧵
From the on-device ML perspective, you can achieve that using the TFLite Delegates.
They enable you to access the power of HW acceleration.
1/6🧵
Your phone's CPU is usually very fast but as a multi-purpose processor it's not optimized for the heavy math that ML needs
Like on their big brothers (servers 🤓) phones have also more specialized chips more suitable for ML, the most popular being the GPUs
2/6🧵
Like on their big brothers (servers 🤓) phones have also more specialized chips more suitable for ML, the most popular being the GPUs
2/6🧵
Another popular accelerator is the Qualcomm Hexagon DSP that has shown 75% reduction in power consumption.
On the Apple side, you can use the Core ML delegate to access the Neural Engine processor on newer i[Phones|Pads] and that can give huge boosts in performance!
3/6🧵
On the Apple side, you can use the Core ML delegate to access the Neural Engine processor on newer i[Phones|Pads] and that can give huge boosts in performance!
3/6🧵
On Android, you can also use the NNAPI delegate on newer OS versions (API Level 27) and this will enable access to the phones GPU, DSP or NPU, depending on what's available.
tensorflow.org/lite/performan…
4/6🧵
tensorflow.org/lite/performan…
4/6🧵
To use any of the delegates is just a configuration when creating the interpreter on the app.
tensorflow.org/lite/performan…
5/6🧵
tensorflow.org/lite/performan…
5/6🧵
Using Delegates can be challenging and bring some complexity to your code but they also enable very good speed ups!
They are a key component to get the best performance of ML on-device.
6/6🧵
They are a key component to get the best performance of ML on-device.
6/6🧵
• • •
Missing some Tweet in this thread? You can try to
force a refresh
