1) Even more important than the rising cost of AI driven by increasing model sizes and the associated compute is the fact that the quantity of relevant data drives AI quality.
nytimes.com/2019/09/26/tec…
nytimes.com/2019/09/26/tec…
2) A 10x increase in the quantity of relevant data used for training generally doubles the quality of the AI per multiple Google and Microsoft research papers.
3) And very few co's have the massive, growing unique/relevant/specific datasets necessary to compete at the frontier of AI. Might be the most powerful scale based barrier to entry I have ever seen in any industry in that it is impossible(ish) to overcome and ever growing.
4) i.e. If someone was willing to spend hundreds of billions they could replicate any network of physical stores, create a competitor to any brand, replicate the credit card networks 😱😀. There is no amount of spending today that would let a new entrant to search truly compete
5) Beyond breakups – which would be awesome for most mega cap tech stocks – I wonder if the most effective way to regulate them would simply be to require them (and maybe all co's) to open source all of their data in anonymized form with a 1-2ish year lag.
6) Net, net the high cost of AI isn’t worrisome to me in a world awash in venture funding and abundant cloud computing capacity, but the fact that only a few companies have the requisite datasets is problematic as a citizen, although fantastic as a shareholder.
7) BTW – I am aware of and believe I understand(ish) all the caveats around data specificity and all the research going into creating more efficient algorithms – but don’t think it changes any of my points and transfer learning seems to be getting better all the time
