Thread by @kane

I’m seeing articles on how camera white balance and #nofilter are surprisingly hard concepts. A thread:

theatlantic.com/technology/arc…

1/ the human brain is really good at “normalizing” colors in a scene given context clues. ie the spectral wavelength that hits your eye may be remapped you a different color in your perception. See Adelson’s illusion:

2/ the challenge is that photos remove viewers from spatial and illumination context. You ever look at an unflattering picture and think “why am I so blue/yellow/red/etc?” Lights IRL are tinted, and your brain normalizes it for you. The camera records the actual tint though.

3/ enter White Balance. Camera designers *know* that light is tinted, and our brains don’t normalize tints in photos the way they do w actual light. White Balance tells cameras what the environment’s light is like, so it can remove tints like your brain does IRL

4/ here’s the challenge - how do you define the scene’s lighting to the camera? If you want to be precise, you use physics: degrees kelvin of a theoretical black body that radiates light in that spectrum. But of course, no one actually thinks about light like that (or very few)

5/ so of course, camera designers make a usability design decision to map black body temperatures to scenes that where those spectrums dominate. This is an imprecise, subjective decision:

6/ of course, for smartphone cameras try to make things even simpler and try to only “auto white balance” - the software will do its best to judge a scene’s illumination, and normalize it the way your brain might

7/ like all automatic systems, edge conditions can break the system - and “the sky is covered in ash and smoke and the sun is but a dim red orb” is probably not a scene that’s programmed into phones’ auto white balance system. So it tries its best 🤷‍♂️

8/ here’s why photos are hard. “Why not just show objective scene?” Our brains do more than just normalize tint - we have tons of heuristics for illuminance & flux (“brightness”), tint, hue, etc. So when we’re shown what a digital system truly “sees”, we think “thats so bland”

9/ our perception is painted by all kinds of biases, neurological shortcuts, and straight-up guesses (your peripheral vision is black and white, your brain paints in color - ask me how we know). We want our photos to match our perception! So camera designers have to guess

10/ So what’s a true “no filter?” Is it the objective spectral wavelengths in a scene? Bc our brains go “no way, that’s much too dull”. Is it Apple’s camera heuristic? Google’s? Nikon’s?

11/ “it it’s so hard to do digitally what about analog processes like film?” - film is even HARDER, because the chemistry has its own weird idiosyncrasies in how it renders light which can change w age or temperature 🤪

12/ and it’s not even like if we all agreed to use boring-ass sterile spectral industrial sensor photos, we’d be on the same page. The dress that went viral is an example of different people’s brains “normalizing” the same set of information in completely different ways!

13/ if you want to get really metaphysical for a second, what is the “real” color of a scene anyways? We have design digital cameras to be twice as sensitive to green because our weird meat jelly eyes are ~2x as sensitive to green as red or blue...

14/ and don't even get me started on displaying color once we've recorded it. What screens can display, what we can print, what we record analog, what we can record digital, and what we perceive are all DIFFERENT

15/ anyways, I took a job a while ago as an imaging technician thinking it’d be chill but it really made me a little loopy about color. Plz respect the poor camera engineers that have to try to package this into a point-and-click feature.

https://twitter.com/kane/status/1295971542791155712

16/ PS I’d love to see research on if a lot of these challenges hold in VR, or if the increased immersion can trigger different visual heuristics in our brains 🌚