This image gets posted a lot lately, and not everyone knows what it means.
It's a reference to “survivor bias”: a statistical problem in which a sample is non-representative because some elements have been eliminated before the sample was taken. Here's a brief explainer.
The story: You're Britain. It's WW2. Your planes are getting shot down. You want to reinforce them with armor. But you can't armor the whole plane (for weight among other reasons).
What parts of the plane do you prioritize for armor?
Your researchers collect data on where your planes are getting shot. Whenever a plane returns from a mission, they note where they found bullet holes. This diagram shows all the holes that were found across many missions.
Given this data, where do you put the armor?
The naive answer is: where the holes were found.
At this point, an astute observer, or one who cross-checks data with intuition, might notice that this does *not* include the engines or cockpit, which seem like pretty sensitive parts of the plane—more so than, say, the wingtips.
Why don't holes in the engine, for instance, show up on this diagram?
Go back to how the diagram was made, specifically the *process* that was used to generate the sample:
“Whenever a plane *returns from a mission*…”
Planes that get shot in the engine *don't return from their mission*. They go down.
The sample is not *representative* of the true distribution of bullet holes. It is a biased sample, because it only includes the survivors. Hence, “survivor bias”, a form of selection bias.
In this case, not only is the sample not representative, it's actually *inversely* correlated with what we want to decide.
The diagram doesn't show where planes get shot—it shows where planes get shot *and still survive*.
In other words, the diagram shows specifically where we *don't* need to put armor (because planes are surviving without it).
The naive answer, then, was *exactly backwards*. We want to armor where we're *not* finding the holes.
Note: This concept/meme is starting to get popular enough that people are tossing it around pretty casually. Often when I see the image, I don't think the concept actually applies.
In particular, simply trying to learn from success is not, in and of itself, survivor bias!
It was created by Wikipedia user “McGeddon” in 2016. They put it on the page for “Survivorship bias”, and I guess everyone's been taking it from there: commons.wikimedia.org/wiki/File:Surv…
Based on the link in the article, it *seems* to be from his talk “Good vs. Great Design” at @aneventapart Boston 2007? But the link is broken, and the only slides I can find for the talk don't contain this image.
And here's his blog post where he actually looks at the original Wald paper (which, incidentally, does not contain this or any other diagram): counting.substack.com/p/its-that-ds-…
• • •
Missing some Tweet in this thread? You can try to
force a refresh
If “low-hanging fruit” or “ideas getting harder to find” was the main factor in the rate of technological progress, then the fastest progress would have been in the Stone Age.
Ideas were *very easy to find* in the Stone Age! There was *so much* low-hanging fruit!
Instead, the pattern we see is the opposite: progress accelerates over time. (Note that the chart below is *already on a log scale*)
Clearly, there is some positive factor that more than makes up for ideas getting harder to find / low-hanging fruit getting picked.
“Ideas getting harder to find” is ambiguous, let me clarify.
In the econ literature it refers to a specific phenomenon, which is that it takes exponentially increasing R&D investment to sustain exponential growth. This is basically all the low-hanging fruit getting picked.
• The AI can do better at the goal if it can upgrade itself
• It will fail at the goal if it is shut down or destroyed (“you can’t get the coffee if you’re dead”)
• Less obviously, it will fail if anyone ever *modifies* its goals
There is an AI doom argument that goes, in essence:
1. Sufficiently advanced AI will be smarter than us 2. Anything smarter than us, we cannot control 3. Having something in the world that we cannot control would be bad
∴ Sufficiently advanced AI would be bad. QED
One counter is to deny (1), eg: AI will never be that smart; intelligence is multi-dimensional and it doesn't make sense to compare them; super-human intelligence is so far in the future that we shouldn't worry about it; etc
This is becoming less popular recently as AI advances.
Another counter is to deny (2): we can build superintelligent systems, but have them be our tools or servants.
This is probably most popular among techno-optimists.
1. So dangerous that no one can use it safely 2. Safe if used carefully, dangerous otherwise 3. Safe if used normally, dangerous in malicious hands 4. So safe that even bad actors cannot cause harm
Important to know which you are talking about.
Arguably:
Level 1 should be banned
Level 2 requires licensing/insurance schemes
Level 3 requires security against bad actors
Level 4 is ideal!
(All of this is a bit oversimplified but hopefully useful)
“Optimal Policies Tend to Seek Power” supposedly gives a theoretical basis for power-seeking behavior from AI
But it seems to just analyze a toy model and show that if you head towards a larger part of the state space, you are more likely to optimize a random reward function?
The intro claims that “power-seeking tendencies arise not from anthropomorphism, but from certain graphical symmetries present in many MDPs [markov decision processes]”
But what is actually demonstrated seems much more trivial than that. What am I missing?