Another thing that classical thermodynamicists tend not to appreciate, is that a proper understanding of physical entropy actually *requires* taking a point of view that is illuminated by an understanding of information theory and reversible computing theory, as I’ll explain.
In the early 00’s in the lecture notes of my Physical Limits of Computing course, I proposed: the best definition of the effective physical entropy of a system is *the (expected) amount of physical information in the system that can’t be decomputed by any available process*.
By “decomputation,” here, we are referring to a *reversible* physical/computational process that restores all (or part) of the system to a “blank” or standard, known state (e.g. a perfect crystal, or an atomically-precise computer with an empty memory, at absolute zero).
A precisely-known, predefined microstate in a finite or regularly-repeating pattern has zero entropy; pretty much all of the definitions of entropy agree on this. So if you can reversibly *transform* all or part of a system to sucha form, that part was not actually true entropy.
As an example: Consider a cubic crystal whose unit cells can hold an extra atom (He, say) in the middle. With no extra atoms, we all agree that a chunk of atomically-perfect crystal of predefined shape (a cube 10^10 unit cells on a side, say) at zero temperature has 0 entropy.
Now, suppose the crystal contains extra He atoms at 50% of the locations, distributed uniformly; that is, each unit cell is occupied with 50% probability. Conventional thermodynamics and statistical mechanics agree that the entropy is then 10^30 bits, or (10^30 ln 2)k.
Suppose you then *measure* the states of all the unit cells via a *reversible* measurement process: This is possible in principle, e.g., by doing a careful nanomechanical disassembly/reassembly of the crystal, probing each unit cell as we go.
(We’re assuming here that the whole disassembly process is performed adiabatically slowly at a temperature approaching zero temperature, and the extra He atoms in the unit cells are tightly locked into place by high energy barriers, so do not diffuse away from surface cells.)
After the measurement and reassembly of the crystal, you now have a situation where the crystal contents are perfectly correlated with a computer record of its state. In other words, all 10^30 physical bits are now *known information*, from the computer’s perspective.
Here, the traditional classical thermodynamics perspective of entropy as some vague measure of “disorder” falls short—the crystal has exactly the same configuration as before, yet, since its state is now known, we could rearrange it to a standard state (removing the extra atoms).
Since the crystal’s state is known and we can restore the crystal to a standard state via a reversible process, it now has zero entropy, from a statistical perspective illuminated by information theory.
Where did the entropy go? Using the information-theoretic concept of mutual information, we can see it is now in the *joint* correlated state of the crystal together with the computer that knows its contents. It has not disappeared, of course! That would violate the 2nd Law.
Now, suppose that after copying the crystal’s contents into our computer, we analyze them and find that in fact, they form the binary expansion of pi (11.0010010...). (Presumably, because someone previously created the crystal specifically to store this data.)
What are the implications of this? Well, by running a reversible program for computing the binary digits of pi in reverse, we can *reversibly* restore our entire 10^30 bits of computer memory to a standard state (except for a small note saying, “oh, this crystal contained pi.”)
If we do this *after* having used the information about the crystal’s contents to restore the crystal itself to a standard state as well, we’re in a situation where we have reversibly restored the entire system (except the small note) to a standard state.
In other words, the original crystal, even though we originally didn’t know what its contents were, and even though, after measuring those, they appeared random at first, actually contained almost no entropy at all—as we realized once we discovered the underlying pattern.
The purpose of this thought experiment is to emphasize the point: Effective entropy is, in practice, always relative to one’s knowledge and capabilities. E.g., if we didn’t have the technology to reversibly do all the operations in this thought experiment...
...it would seem like the entropy of the crystal was very large (10^30 bits). So from the limited POV of such an entity, the effective entropy of the crystal is not the same as it would be for an entity that possessed this technology.
Also, regardless of technology, if the data in the crystal was encrypted and we didn’t know the decryption key, and the key was large enough to be infeasible to crack, then the contents of the crystal would also be effectively entropy, due in this case to our lack of knowledge.
Anyway, the point of all this is to illustrate that a classical thermodynamics perspective is completely inadequate to analyze such cases. You have to understand information theory as well as reversible computing theory, if you ever want to grasp the true nature of entropy.
