Thread by @rohanpaul_ai

Did you know how TensorFlow can run on a single mobile device as well as on an entire data center? Read this thread

1/n

#TensorFlow #DataScience #DeepLearning #MachineLearning #ComputerVision #100DaysOfMLCode #Python #DataScientist #Statistics #programming #Data

2/n
Google has designed TensorFlow such that it is capable of dividing a large model graph whenever needed.

#TensorFlow #DataScience #DeepLearning #MachineLearning #ComputerVision #100DaysOfMLCode #Python #DataScientist #Statistics #programming #Data #Math #Stat #AI

3/n
It assigns special SEND and RECV nodes whenever a graph is divided between multiple devices (CPUs or GPUs).

#TensorFlow #DataScience #DeepLearning #MachineLearning #ComputerVision #100DaysOfMLCode #Python #DataScientist #Statistics #programming #Data #Math #Stat #AI

4/n
It has internal heuristics which suggest when to divide a graph.

#TensorFlow #DataScience #DeepLearning #MachineLearning #ComputerVision #100DaysOfMLCode #Python #DataScientist #Statistics #programming #Data #Math #Stat #AI

5/n
Thus, TensorFlow can run on a single device, multiple devices on the same machine, and on multiple devices on multiple machines.

#TensorFlow #DataScience #DeepLearning #MachineLearning #ComputerVision #100DaysOfMLCode #Python #DataScientist #Statistics #programming #Data

6/n

`tf.distribute.Strategy` is integrated to `tf.keras`, so when `model.fit` is used with `tf.distribute.Strategy` instance and then using `strategy.scope()` for your model allows to create distributed variables.
#TensorFlow #DataScience #DeepLearning

7/n
This allows it to equally divide your input data on your devices.
You can use `tf.distribute.Strategy` with very few changes to your code, because the underlying components of TensorFlow have been changed to become strategy-aware.

#TensorFlow #DataScience #DeepLearning

8/n
This includes variables, layers, models, optimizers, metrics, summaries, and checkpoints.

#TensorFlow #DataScience #DeepLearning #MachineLearning #ComputerVision #100DaysOfMLCode #Python #DataScientist #Statistics #programming #Data

9/n

Follow me - @rohanpaul_ai for regular tips and techniques on Deep Learning/Computer Vision / TensorFlow / Pytorch

#TensorFlow #DataScience #DeepLearning #MachineLearning #ComputerVision #100DaysOfMLCode #Python #DataScientist #Statistics #programming #Data