Tuesday, December 9, 2008

the World - Cliff's notes edition

We all know Statisctics can be manipulated to fit an agenda, but sometimes it is surprising how many ways this can be done. The thing to always keep in mind is:
Statistics are really just convenient summaries of some reality; like cliff's notes of the world.
Just like a Cliff's notes summary of a book, statistics leave out information for the purpose of ease of understanding. But also like Cliff's notes, they are not to be confused with the real thing. They are a pale imitation of reality - missing most of the information that makes the real world unpredictable and complex. Sit down this weekend and read the Cliff's notes version of your favorite classic novel, and I'm sure you will see how unfulfilling a summary can be.

The Top 10 ways that statistics lie ---
10 - Insufficient sample size - 20 people's opinions are just that, 20 people's opinions. This is not a summary of the world.

9 - Selection Bias - If your sample is not diverse, your answer will not be also. A good way to fudge. How about we call the NRA to see who likes guns?

8 - Correlation IS NOT Causation - Come-on people, the car is in the driveway and it is raining - really, the car didn't do it; I swear. And Really - the rain didn't put the car there, I just didn't drive to work that day.

7- Picking the sample period - Right now it is colder then it has been since noon. Boy that's cold, must be some global climatic deep freeze. Oh yeah, it's cold every night at this time, but it sounded better to compare it to noon. The NYSE is up infinity percent over the last 1000 years. Boo Yeah!

6- Picking the questions - Sometimes a survey picks their questions to get their answers. "When did you stop beating your wife?"

5- Scale - Those pesky graphs can show whatever you like if you mess with the scale.

4- Scope - Meaningless scope can obscure the reality rather than enlighten it. You'd be in the top 1% of incomes if you lived in Bangledesh. Who cares, your pay american rent.

3- Lying/Errors - OK, so sometimes an overzelous reporter downright lies and/or repeats bogus numbers with no basis. Happens all the time. Did you know that spinach is not really that high in iron? A math mistake happened in 1870 and was blindly reported for 67 years before being discovered. Who checks this stuff?

2- Bad math - The mean house price in my neighborhood is $850,000, the median price is $450,000. Math matters.

1- Stupid reporting - Often, REALLY OFTEN, the researches get it right, the reporters just don't read the report and report something stupid. REALLY OFTEN!

No comments: