404//MEANINGSSYS.ONLINE

Biases

Survivorship Bias

why does success advice not work

ser-VY-ver-ship BY-us/səˈvaɪ.və.ʃɪp ˈbaɪ.əs/

Survivorship bias is drawing conclusions from the things that made it through a selection process while never seeing the things that didn't — so your evidence is systematically the wrong sample.

The reasoning is Abraham Wald's, from 1943. The label 'survivorship bias' is a later statistical coinage with no single credited author — Wald solved the problem before anyone named it.

// Where it sits

Survivorship bias
The whole sampleOnly what came back

The dangerous part is that the missing data leaves no trace of being missing.

What it really means

The military brought Abraham Wald a genuinely reasonable question in 1943: given a map of where returning aircraft had been hit, where should the extra armour go? The bullet holes clustered on the wings and fuselage. The obvious answer was to armour the wings and fuselage.

Wald's answer was to armour the places with no holes. The data covered planes that came back. A plane hit in the engine did not come back, and so contributed no dot to the map. The blank regions weren't areas that never got shot — they were areas where getting shot was fatal, and the silence was the finding.

Worth correcting the version that circulates online: Wald did not glance at a diagram and deliver a quip. Across eight memoranda he built a mathematical method for estimating, component by component, the probability that a hit there would down the aircraft — reconstructing the full population from the survivors. The famous red-dotted bomber picture is a modern illustration, not his.

What makes survivorship bias so durable is that the missing data leaves no trace of being missing. Nothing on the map said engines untested. The sample looked complete, because absent evidence never announces itself — which is the same reason a shelf of founder memoirs looks like a study of what works.

Where it comes from

survivorshipborrowed from actuarial and finance usage

In finance, a 'survivorship-biased' index is one that quietly drops the funds that collapsed.

1943Abraham Wald, working at Columbia's Statistical Research Group, writes eight memoranda on aircraft vulnerability. The available data recorded where returning aircraft had been hit — a sample that excluded every plane shot down.
1980Wald's wartime memoranda, never published externally during the war, are reprinted by the Center for Naval Analyses. The method had by then been used on damage data from Korea and Vietnam.

Myths & misconceptions

Myth

The WWII bomber story is a nice legend that never happened.

Reality

It happened. Wald's eight memoranda were produced in 1943 for the National Defense Research Committee, and the full set, never published externally at the time, was reprinted by the Center for Naval Analyses in 1980. You can read it.

Myth

Wald looked at a diagram of bullet holes and quipped 'armour where the holes aren't'.

Reality

That's the punchline, not the work. Wald derived a mathematical method for estimating, part by part, the probability that a plane hit there would be lost — reasoning backwards from the damage on survivors to the damage on everything that flew. The famous red-dotted bomber diagram is a modern illustration, not his figure.

Compare & contrast

Survivorship bias vs selection bias

Survivorship bias is a subtype of selection bias — the specific case where the filter is survival or success. All survivorship bias is selection bias; not all selection bias involves anything surviving.

//Survivorship biasSelection bias
FilterSurvivalAny
ScopeNarrowBroad
Classic caseWald's aircraftAny unrepresentative sample
RelationshipA subtypeThe category

How it connects

  • Hindsight Biasboth make outcomes look more explicable than the evidence supports.
  • Optimism Biasa diet of visible winners feeds the belief that your own odds are better than average.

Tell it apart

Questions people ask

What is survivorship bias?
Studying only the winners. Five biographies of college dropouts who founded billion-dollar companies tell you nothing useful about dropping out, because the thousands who dropped out and failed never got a book deal.
What did Abraham Wald actually do?
In 1943 he worked from data on where returning aircraft had been hit, and produced a method for estimating the vulnerability of each part of a plane from that survivor-only data — the areas showing least damage among survivors being the ones most likely to be fatal. The familiar version, in which he personally stops engineers from armouring the bullet holes and names the engines, is a later embellishment.
Where does it show up in everyday life?
Reviews, mostly. A restaurant's page shows the customers who came back to post; the ones who had a bad night simply never returned and never wrote anything. A 4.6 average can be an average of people already inclined to like it.

Sources