Imagine that you want to build a giant sentient AI agent by letting a very large number of small autonomous learning AI agents self organize. What is the core policy that each of these agents has to follow to make it work? This is how you understand Kant’s Categorical Imperative.
Kant’s Categorical Imperative only identifies the core rule from which all other rules have to be deduced, which is a complex task.
Here are four corollaries for rational agents:
- select goals that integrate expected reward over the longest timeframe
- optimize internal regulation
- optimize regulation between agents
- act on models that you can expect to result in the preferred outcome
- select goals that integrate expected reward over the longest timeframe
- optimize internal regulation
- optimize regulation between agents
- act on models that you can expect to result in the preferred outcome
Three corollaries for emergent agency:
- commit to unifying your agency with other agents if they are following the same core policy
- prioritize global reward over individual reward
- act as if the global agency is already latent, so it can emerge before it is generating rewards
- commit to unifying your agency with other agents if they are following the same core policy
- prioritize global reward over individual reward
- act as if the global agency is already latent, so it can emerge before it is generating rewards
(These seven corollaries have been described by Thomas Aquinas; he calls the the first four “cardinal virtues”, and they can be found by deduction. The latter three are the “divine virtues” and they require induction.)
