Thread by @martin_casado

[New Post] The economics of AI / ML companies are poorly understood.

In this post, @BornsteinMatt and I cover some of the core challenges and how they are being tackled by top teams in the industry.

a16z.com/2020/08/12/tam…

Highlights below 👇

1/ Unlike traditional software, margins, scalability, and defensibility for AI companies are usually a function of the problem -- not the technology. And often the problems are ugly.

2/ Many AI problems show long-tailed distributions of input data (meaning most inputs happen infrequently), and supervised learning is not well equipped to handle it.

3/ If the long tail is distributed amongst customers, that can result in especially high variable cost (e.g. data labeling + transformation and frequent retraining per customer) and negatively impact margins.

4/ In some cases, it can even create diseconomies of scale. As @Hassan_Sawaf put it, “At some point you need 10x more data to get 2x improvement”

5/ We covered these challenges in more detail in a previous post a16z.com/2020/02/16/the…

6/ We spoke with a bunch of experts and received great guidance on how to tackle the heavy tail problem. To do so, you need to understand the distribution of the problem you’re solving. Given that …

7/ Easy mode: if you have a well-bounded problem, you may not need ML. Or you can start simple and upgrade to bigger models only when the problem demands it (logistic regression and random forests are popular for a reason)

8/ Harder: if you have a global long tail problem -- i.e., similar distribution for all customers -- you can optimize, narrow, and reframe...

9/ ...and explore a growing technique called componetizing, where a single problem is broken into smaller pieces

10/ Really hard: if you have a local long tail problem -- every customer looks different -- meta models and transfer learning may be able to help… although we didn’t find many examples of them being used successfully at scale.

11/ ...and trunk models are a fascinating new approach that feel like an emerging API standard for machine learning -- though according to Shubhos Sengupta we are still in “pre-POSIX days” 🙂

12/ Finally, operations are really important -- e.g., write better data pipelines, build an edge case engine, run your own infra, etc.

13/ THANK YOU to @Aman_Naimat, @vitalygordon, @Hassan_Sawaf, @zaydenam, @alex_holub, @evanrsparks, @mitultiwari, @spiantino and others whose twitter handles we couldn’t figure out :)

Full post : a16z.com/2020/08/12/tam…