What are some domains of knowledge where big language models will be impactful?
Maybe domains with vast, messy stores of content that few humans master. E.g.
1. All US laws+regulations
2. Biological details of every beetle (>1M species)
3. All code in 787 (14M lines)
Maybe domains with vast, messy stores of content that few humans master. E.g.
1. All US laws+regulations
2. Biological details of every beetle (>1M species)
3. All code in 787 (14M lines)
4. Function of all genes in all genomes (20k in humans)
5. Obscure human languages (Akkadian)
6. For a big company, what's the standard operating procedure for every staff role.
5. Obscure human languages (Akkadian)
6. For a big company, what's the standard operating procedure for every staff role.
Let’s say there’s N items of interconnected knowledge in a domain. Even if humans can understand any *one* item better than a GPT-3-like model, the model can provide value by understanding N>100,000 items modestly well.
Moreover, the GPT-3-like model will have not just >100,000 specialized facts but the standard GPT-3 general knowledge (which is shallow but superhumanly broad). See the impressive DeepMind Gopher result on Hendrycks' MMLU. arxiv.org/abs/2112.11446
Postscript: I think big language models will also be useful to augment web search or Google Scholar Search by being able to read/understand text much better than traditional search. This applies to all academic fields (while above I focus on particular narrow domains)
• • •
Missing some Tweet in this thread? You can try to
force a refresh
