OK, so there's a physical problem with our eyes: We move them in short fast bursts called "saccades", right? very quick, synchronized movements.
The only problem is: they go all blurry and useless during this
1. make the vision go black during movement. (Some VR games do this!)
2. just keep showing the last thing we saw prior to movement
Your brain does neither of these options, really.
then when you finish your saccade, it shows you what you now see at the new position. and then it pretends it can time travel.
And because your brain is not a computer with a consistent clock, this shit works.
Look away (with just your eyes, not your head), then look back to the second hand.
It'll seem like it takes longer than a second to move, then resumes moving as normal.
and the answer is simple! your brain has EVEN MORE UGLY HACKS on top of this to avoid you seeing that
But I bet if you shift your attention to your peripheral vision right now, it's in color.
They actually see the ball as bigger because they need to hit it. their vision is exaggerating it to make it easier to see!
So if you want an accurate visual representation of what things look like? Use a camera. Not your eyes.
You're blind during those moments. But you still think you are seeing.
So you damage a small spot of your retina and your visual system covers it up.
eventually you accumulate so much damage that your visual system simply cannot manage hiding it all and your vision rapidly degrades.
There was an experiment back in 1890 where someone wore glasses made with mirrors in them to flip their vision.
After about 8 days, they could see just fine with them on. Their vision system had started "flipping" the image.
It only took them a few hours to get back to normal after taking these glasses off, though.
We don't, but only because our visual system has had our whole life to adapt to this.
(Really, it has "vampires", while still being hard-SF)