Thread by @TheSequenceAI

Model serving = processes of operationalizing a machine learning model for production.

OR what most normal software developers call ‘deployment’. Read more about it.

Thread⬇️
thesequence.substack.com/p/edge12-the-c…

Model serving goes a bit beyond deployment, given the unique nature of the lifecycle of ML programs.

ML models operate in a circular lifecycle, where phases such as training and optimization are continuously repeated.
2/⬇️

Some of the most important aspects of any model serving pipeline:
+API interface
+real-time vs. batch execution
+versioning
+A/B testing
+scalability
3/⬇️

Model serving solutions try to create a consistent framework that abstracts the core capabilities needed to run ML models in production.

For instance, the architecture for models executed in real-time is fundamentally different from ones that are executed in batch modes.
4/⬇️

"TensorFlow-Serving: Flexible, High-Performance ML Serving" by @JeremiahHarmsen, @FangweiLi, @sukritiramesh, Christopher Olston, Noah Fiedel, Kiril Gorovoy, Li Lao, Vinu Rajashekhar, Jordan Soyke

outlined the architecture of a serving pipeline for @TensorFlow models
5/⬇️

TensorFlow Serving = the first mainstream model serving architecture in machine learning frameworks

Link: arxiv.org/abs/1712.06139

6/⬇️

TheSequence Edge covers:
+ML concept you should learn
+Review of an impactful research paper
+New ML framework or platform and how you can use it
thesequence.substack.com/subscribe
7/7