,
8 tweets,
2 min read
Read on Twitter
A tremendously useful explainer about symbolic APIs (Sequential + Functional API) and Model subclassing in TF 2.0, by @random_forests: medium.com/tensorflow/wha…
Symbolic APIs are APIs to build graphs of layers. Their strong points are that:
- They match how we think about our networks (NNs are always visualized as graphs of layers in textbooks & papers)
- They run extensive static checks during model construction, like a compiler would
- They match how we think about our networks (NNs are always visualized as graphs of layers in textbooks & papers)
- They run extensive static checks during model construction, like a compiler would
This gives you the guarantee that any model that you can build, will run. The only form of debugging you'd have to do at runtime would be convergence-related. The UX of these APIs is highly intuitive and productive
Meanwhile, the subclassing API has the look and feel of objected-oriented Numpy development. It's ideal if you're doing anything that cannot easily be expressed as a graph of layers, and you feel comfortable with software engineering best practices and large Python projects.
It will involve execution-time debugging, more code, and will expose a greater error surface, but at the same time it will give you greater flexibility to express unconventional architectures.
Importantly, in TF 2.0, both of these styles are available and are fully interoperable. You can mix and match models defined with either style. At the end of the day, everything is a Model!
That way, you are free to pick the most appropriate API for the task at hand.
That way, you are free to pick the most appropriate API for the task at hand.
In general I expect ~90-95% of use cases to be covered by the Functional API. The Model subclassing API targets deep learning researchers specifically (about 5% of use cases).
I think it's great that we don't silo researchers and everyone else into completely separate frameworks. It's all one API, that enables a spectrum of workflows, from really easy (Sequential) to advanced (Functional) to fully flexible and hackable (Model subclassing)
