The Fallacy of Averages
It seems somehow strange to me that - in this age at least - the difference between our comprehension of numbers and what they mean, and our comprehension of words and what they mean, seems so stark.
Although perhaps that’s simply my own lack of comprehension of all the ways that words trip us up.
But counter-intuitive number-related stories are everywhere if we look for them.
Freakonomics remains a truly astounding read.
The similar but football-themed Soccernomics/Why England Lose changed the way I thought about football in much the same way that Freakonomics busted open my assumptions about how property sales and drug rings worked (and many more things).
As someone with a background that somewhat qualifies them to understand these kinds of things (a BSc in Mathematics, for example), I find them particularly fascinating.
Why do our assumptions about what numbers mean sometimes vary SO much from what they actually mean? Why is it possible for there to be things in maths that feel intuitively incredibly clear but actually aren’t?
The shifts created when we have this kind of insight are often memorable. One of my favourite recent discoveries in this field is burned into my mind - I can remember the exact piece of muddy Warwickshire path I was walking along when its implications hit me.
That is, The Fallacy of Averages.
I learned about it from Matthew Syed’s book, Rebel Ideas: The Power of Thinking Differently. (Some of you will remember that Syed’s book Bounce had a paradigm-shifting impact on me relating to talent and practice.)
In Rebel Ideas, Syed tells a story something like this. That around 1950, the US Air Force was investiating a surprising number of crashes. They were stumped as to why this was until they reflected that the cockpit was designed to make it perfectly fit the average pilot. The perfect seat size, distance from controls to seat, all of these measurements to match perfectly the average of their 4,000 pilots.
It transpired that not a single one of the 4,000 pilots was the ‘average pilot’ among the many different measurements that had been used.
Once this was discovered, the cockpit was redesigned. Instead of the fixed ‘average’ positions, aspects of the cockpit were adjustable in order to allow the many pilots who were close to the average to fit the cockpit to them. To allow for the diversity among the pilots.
In Rebel Ideas, Syed’s second example of The Fallacy of Averages is our dietary health. The reason there are so many diets all seeming to have incredible impact for people whilst also contradicting each other (high carbs here, no carbs there etc) is… there is no average human response to food. In fact, our response to - say - sugary foods varies hugely from person to person.
In an example in the book, one person in an experiment’s blood sugar spiked more from iceberg lettuce than from a chocolate brownie. (We want to avoid blood sugar spikes, broadly, because they are associated with poor health outcomes over the long term and irritability in the short term.) This kind of story had me searching the internet for a way to learn more about my diet, and led me to using the personalised diet app Zoe for several months: to have a much better idea of what impact food had on me felt like something almost priceless.
But The Fallacy of Averages isn’t only in diets and cockpits, it is everywhere.
As a parent of young children, I can’t help look at all the advice from even well-meaning sources like the NHS and see how much based on averages they are. How - therefore - they are likely to be wide of the mark only always when it comes to my girls. How flawed it is to base my sense of success as a parent on that (whilst of course potentially providing useful guidance, as long as I remember that my daughter is unlikely to do any of the things they say at exactly the time or in exactly the way they describe.)
And yet, really this kind of advice is all that the rational parts of me have to go on.
The ‘average’ is the kind of blunt mechanism that I suspect no self-respecting statistician would ever use (which average even is it? Secondary school maths teaches us three different types of average!).
And yet for the lay-person, it seems to make complete sense.
Just like so many number-related assumptions seem to make sense.
Until we experiment with them, look deep into the data, behave like statistician or scientist, and realise that often they don’t.
—
This is the latest in a series of articles written using the 12-Minute Method: write for twelve minutes, proof read once with tiny edits and then post online.
Subscribe to this blog here
Read the archive here.
Buy the 12-Minute Method series of books, written 12 minutes a week over three years, here.