Thread by @IntuitMachine

What is wrong with knowledge representations that it has barely moved the needle in machine understanding? @danbri

Intuitively, KR should be useful in that it diagrammatically records how concepts relate to other concepts. Yet for a reason that is not apparent, it isn't very useful in parsing out new understandings of the concepts in its graph. Where did we go wrong here?

Perhaps it's because knowledge graphs are noun-centric and not verb-centric. Reality is verb-centric. To get an intuition about this, watch this explanation of the open-world game Nethack:

Wolfram @stephen_wolfram has a nuanced take on language understanding. For the narrow domain of question and answering, his symbolic system should be able to derive the answers to known questions. wolfram.com/language/princ…

Wolfram Language is an interesting framing. Are AIs essentially advanced programming languages? Does it make sense that our existing programming languages are decoupled from knowledge about the world?

Ben Goertzel in his roadmap for AGI describes a system that is also based on rewrite rules. Is this the problem with KBs in that they are declarative and not imperative? Is knowledge in general intelligence stored in imperative form?

https://twitter.com/IntuitMachine/status/1409960961272913924

What does it mean for knowledge to be stored in interpretative form and how is this form more useful than declarative forms? The problem with declarative forms is that information as to its interpretation is made implicit. It is not encoded anywhere in the knowledge graph.

There exists a continuum of what is made implicit in the serialization of imperative information. For example, machine language is more detailed than code written in a high-level programming language. It is the interpreters and compilers that translate code into exact actions.

The information held in KGs is rigid in the sense that the concepts are interconnected in rigid predefined connections. This rigidity does not accommodate the flexibility of thought required by general intelligence

Contrast this with the fuzzy nature of a language model like GPT-3. Why is it that we cannot generate interesting content from a KG as compared to GPT-3? Why does GPT-3 capture knowledge in a more human-like manner than that of a manually encoded KG?

To be fair, GPT-3 knows nothing about this world. But it knows almost everything that is useful about English syntax. Entangled in English usage are fragments of semantic relations that GPT-3 is very proficient in mimicking.

What then is the solution? Simple, train GPT-3 on this yet to described imperative knowledge encoding.