Well, here's the problem: Linux is Unix, and Unix is older than even CP/M. There are design decisions from farther back affecting it today!
Bell Labs bought the PDP-11/20 with a RK11 controller (supporting up to EIGHT hard drives), and initially fitted it with two RK05 hard drives.
Each of these drives had a MASSIVE 2.5 megabytes of storage!
It had directories /bin and /usr at least, and /usr was USER DIRECTORIES!
And when they upgraded to two drives on the PDP-11, they thought: hey, let's split this up logically: disk 1 is root, disk 2 is users!
Until they ran out of space on disk 1.
RK05 hard drives were expensive, so just adding a third drive would be a pain... and /usr wasn't very full...
(Because if /bin/mount got moved to /usr/bin/mount, they wouldn't be able to mount disk 2)
Other than that, the only guideline for placing files was "where it'd fit"