There's a big difference between data-efficiency and information-efficiency. We could make considerable progress on the former without even starting to approach the latter -- and never realize our fundamental mistake.
Abstraction is the golden path to information efficiency. But all kinds of hacks can get you data efficiency.
The difference between data & information:
- Data is plentiful & noisy, the large majority of data is not information
- Information is what makes data useful
- We are bad at extracting info from data. We only manage to distill & use a small fraction of what is really available
- Data is plentiful & noisy, the large majority of data is not information
- Information is what makes data useful
- We are bad at extracting info from data. We only manage to distill & use a small fraction of what is really available
Working on "data efficiency" typically means finding new hacks to increase the amount of information we can extract from the same data. But "information efficiency" is an entirely unrelated concept: how to efficiently and effectively make use of information
