Some thoughts on GPT-4 in Medicine, which is a watershed development

1. This is a massive step forward and our team will be publishing more thoughts.

2. Contextual world knowledge is a massive upgrade for medicine, where hypothesis generation often involves complex reasoning. 3. GPT-4 still struggles w/ omission and other similar issues and these problems still need to be solve for deployment into clinical workflows.

4. This is why we are big believers in human-in-the-loop workflows and constructions on top of GPT-4 to achieve reliability and safety.

LLMs are highly non-deterministic, which presents a huge design challenge: we've never built non-deterministic software, it's always been based off of branching logic and restricted options (hence: the GUI) even classic ml systems don't meet this bar. they have a restricted set of outputs and a limited state space. the whole idea behind conversational systems like slot filling is to reduce the dimensionality of the problem.

the software can only go down a discrete number of paths.

One of the big gripes about LLMs is sounding overconfident.

Thus, I suspect the next big thing akin to Chain-of-Thought prompting will be knowledge certainty prompting.

Some examples: Here you ask GPT-3 what day it is. Now add instructions to consider its own limitations and the model refuses to answer.

Here's my take on the rise of new mental health companies (Ginger, Lyra, Spring, Modern, etc.). These companies have raised a ton of money and are getting traction in a large market.

The question is: what now? who wins? Mental Health is a hot button issue right now. That's driving massive adoption from employers looking to have an answer. Here's the problem: mental health is *expensive.* The least it can cost to impact someone's mental health is $600 (6 sessions * $100/session).

1/ A thread on the future + the pandemic:

As @Dale_W_Harrison notes, it looks like the vaccine may prevent serious illness, but not transmission. We'll probably need to continue social distancing + masks until 90% of the population is vaccinated.

dalewharrison.substack.com/p/results-from… 2/ Now read The Vaccine War by @Noahpinion: noahpinion.substack.com/p/the-vaccine-…

The virus is mutating. If we can't stop it, we're going to have to keep vaccinating ourselves to these mutations. This means we're going to need a wartime economy style vaccination program for a while.

1/ Saw this tweet from @DavidSacks and it prompted a bunch of thoughts.

There are are two types of co's that are facing a reckoning.

1. Tech-enabled w/o proficiency in cost attribution ("The WeWork/Uber effect")
2. Pre-PM Fit ("The YC effect")

https://twitter.com/DavidSacks/status/1188883554190520322

2/ The "WeWork effect" proposes a real challenge for tech-enabled businesses where the unit economics and leverage from technology is typically unclear until you actually build the business, which often takes capital and time.

1/ Building on @pmarca’s “Software is eating the world” paradigm: It occurs to me that many early software co’s ate the world by virtualizing a formerly analog thing. Netflix w/ DVD's, Salesforce w/ rolodexes/CRM, other SaaS companies for things like billing, HR, collaboration. 2/ As the low hanging, fully digital fruit was picked, the next generation of companies took hybrid models: software eating industries that needed a physical components but not deep vertical integration or ownership. Uber, Airbnb, etc.