The (In)utility of Statistical Analysis: Why Statistics Don't Matter
"There are three types of lies - lies, damn lies and statistics"...
The above statement is not new, it has been repeated time after time in various places and occasions; most folks would have heard of it, and those who don't are likely to agree that the statement is generally valid...
Unfortunately, few of us are willing to bite the bullet - and question these statistics - particularly politicians and journalists.
To politicians (of which I will return to later), the reason is obvious: Statistics can be manipulated and massaged to fit any form of political agenda.
To journalists, the reason is less-obvious, but no less appealing: Statistics can be used as headline news fodder to awe the average man into a conclusion that does not take into account the complexities of social phenomenon.
Now you may ask - isn't the duty of a newspaper to seek and inform the masses of the latest happenings? Why is there a need to go into detailed analysis...even so, there are experts interviewed who have given their take on the problem(s)...what else is there to know?
Which brings me to my next point: That journalists themselves do not actually realize that there is more than meets the eye - which is why statistics are used: Numerical precision replaces logic and a sense of critical observation.
Let's take a look at a case study:
In the May general elections, the PAP was voted back into power with a 66.6% total vote...according to PM Lee Hsien Loong, the percentage figure suggested "a strong mandate" (but of course, what else do you expect him to say, a weak mandate?)
The explanation by PAP folks was simple: By comparing these percentage with those in other developed countries, 66.6% is a strong vote. After all, if George Bush polled only slightly over 50%, that makes our mandate pretty good.
Whoa...hold on. Percentage-wise, it is definitely superior to the Bush's administration, but I don't remember our PM ever debating opposition leaders like CSJ, LTK or even CST on prime time national TV. In addition, I don't remember our local media critiquing our incumbent politicians with regards to social policies etc... come to think of it, I don't remember Bush ever calling Kerry "a liar" ... in fact, they shook hands after each televised debate, didn't they?
My point is then, if you want to make statistical comparisons, then you jolly well ensure that the variables are similar. As any good science student would tell you, in any experiment, one has to keep all conditions - except the tested property - unchanged. Otherwise, no meaningful conclusion can be made.
Secondly, and this is perhaps the more important point, that statistics (by themselves) cannot possibly allow us to come to any conclusion about the strength of public support. The reason is simple: There is a qualitative difference between quality and quantity.
As such, percentages and statistical reports are highly deceiving and do not take into account the complexity of human personality and complexities behind his choice. As such, whether the ruling party gets 66% or 46% does not prove whether it is doing a "good" job or not (and we should not delude ourselves to think that these figures do actually indicate anything). What these figures show, is simply, who the electorate casts his/her vote for.Period. Any attempt to extrapolate the vote into a qualitative measurement of political ability is simply shrouded in too much ambiguity.
To put it in layman terms, having 66.6% does not mean that the electorate necessarily agree that the PAP is doing a better job. If that's the case, than the 99.9% that Saddam Hussein received during the Iraqi elections would mean an amazing mandate... Whether a government does well or not is not measured by the level of support, but by other indicators.
So does that mean we should discard all forms of statistical reports? Of course not.
What it means is that as readers, we had better start to think the next time we are presented with a bunch of statistics (on whether Singaporeans are eating healthier or whether the job market is doing better). In any case, we should not swallow these figures whole sale, what is unsaid sometimes speaks louder than what is said.
The above statement is not new, it has been repeated time after time in various places and occasions; most folks would have heard of it, and those who don't are likely to agree that the statement is generally valid...
Unfortunately, few of us are willing to bite the bullet - and question these statistics - particularly politicians and journalists.
To politicians (of which I will return to later), the reason is obvious: Statistics can be manipulated and massaged to fit any form of political agenda.
To journalists, the reason is less-obvious, but no less appealing: Statistics can be used as headline news fodder to awe the average man into a conclusion that does not take into account the complexities of social phenomenon.
Now you may ask - isn't the duty of a newspaper to seek and inform the masses of the latest happenings? Why is there a need to go into detailed analysis...even so, there are experts interviewed who have given their take on the problem(s)...what else is there to know?
Which brings me to my next point: That journalists themselves do not actually realize that there is more than meets the eye - which is why statistics are used: Numerical precision replaces logic and a sense of critical observation.
Let's take a look at a case study:
In the May general elections, the PAP was voted back into power with a 66.6% total vote...according to PM Lee Hsien Loong, the percentage figure suggested "a strong mandate" (but of course, what else do you expect him to say, a weak mandate?)
The explanation by PAP folks was simple: By comparing these percentage with those in other developed countries, 66.6% is a strong vote. After all, if George Bush polled only slightly over 50%, that makes our mandate pretty good.
Whoa...hold on. Percentage-wise, it is definitely superior to the Bush's administration, but I don't remember our PM ever debating opposition leaders like CSJ, LTK or even CST on prime time national TV. In addition, I don't remember our local media critiquing our incumbent politicians with regards to social policies etc... come to think of it, I don't remember Bush ever calling Kerry "a liar" ... in fact, they shook hands after each televised debate, didn't they?
My point is then, if you want to make statistical comparisons, then you jolly well ensure that the variables are similar. As any good science student would tell you, in any experiment, one has to keep all conditions - except the tested property - unchanged. Otherwise, no meaningful conclusion can be made.
Secondly, and this is perhaps the more important point, that statistics (by themselves) cannot possibly allow us to come to any conclusion about the strength of public support. The reason is simple: There is a qualitative difference between quality and quantity.
As such, percentages and statistical reports are highly deceiving and do not take into account the complexity of human personality and complexities behind his choice. As such, whether the ruling party gets 66% or 46% does not prove whether it is doing a "good" job or not (and we should not delude ourselves to think that these figures do actually indicate anything). What these figures show, is simply, who the electorate casts his/her vote for.Period. Any attempt to extrapolate the vote into a qualitative measurement of political ability is simply shrouded in too much ambiguity.
To put it in layman terms, having 66.6% does not mean that the electorate necessarily agree that the PAP is doing a better job. If that's the case, than the 99.9% that Saddam Hussein received during the Iraqi elections would mean an amazing mandate... Whether a government does well or not is not measured by the level of support, but by other indicators.
So does that mean we should discard all forms of statistical reports? Of course not.
What it means is that as readers, we had better start to think the next time we are presented with a bunch of statistics (on whether Singaporeans are eating healthier or whether the job market is doing better). In any case, we should not swallow these figures whole sale, what is unsaid sometimes speaks louder than what is said.
posted by Benjamin Ho at
6:46 PM
2 Comments:
Well, what, then, is the use of statistics?
what does 66% tell us? and if we don't use statistics what do we use to measure the government's performance and the will of the people?
I wrote an econometrics spin about example of PAP 66.6% votes.
http://charissaishere.blogspot.com/2006/08/analysis-of-paps-666-votes.html
inspir3d, I think it might answer your questions.
Post a Comment
Subscribe to Post Comments [Atom]
<< Home