Maybe you’ve heard this one but read along because it has lessons for today.
Little Johnny has to do a project for school and, being a little boy, he decides to torture a frog in the name of science. He acquires a notebook, a sharp pencil, a knife, a measuring tape, and a frog. He also has the rudiments of the scientific method imperfectly acquired from class.
The experiment starts innocently enough. Johnny draws a line in the sand, places the frog behind it and the loudly stamps his foot just behind the frog. Startled, the animal jumps, and Johnny measures the distance recording in his notebook, “A frog with four legs jumps four feet.”
Good enough. Let’s call this the control part of the experiment.
Johnny then gets to work. He cuts off one of the frog’s legs and repeats the process, finally writing in his notebook, “A frog with 3 legs jumps 3 feet.”
That’s rather brutal but on the scale of things, many thousands of mice (and other animals) are routinely sacrificed annually in labs around the world. So, this experiment though conducted by an admittedly junior researcher should not be judged as unnecessarily cruel though subsequent events might give us all pause.
At any rate, Johnny cuts another leg off and repeats the experiment and writes in his notebook, “A frog with 2 legs jumps 2 feet.”
We know where this is going and subsequently our intrepid experimenter records the observation that a frog with one leg somehow jumps a whole foot.
Then the inevitable happens and he severs the last leg. The almost dead frog though wanting to get away from its tormentor does not have the ability and simply writhes on the ground as it expires. Little Johnny grabs his notebook and records that “A frog with no legs is deaf.”
When told well this joke gets a laugh because it follows many of the rules of comedy, especially creating cognitive dissonance. Most people hearing the joke for the first time laugh because they can’t fathom jumping from observations about the data, i.e. a frog with 4 legs jumps four feet, etc. and a conclusion from left field. How can cutting the legs off a frog affect its hearing? Can frogs even hear in the first place? Statisticians might feel better about the data if there were more data points, that is, more frogs. But it’s hard to consider compelling more such experimentation.
This is a great example of poor data interpretation. No peer reviewed science journal would ever dream of publishing the results of this experiment because the design and analysis of the data are flawed. The conclusions are not drawn from the experimental data, they’re off on a tangent which makes good comedy but terrible science.
This joke nevertheless serves a practical purpose in reminding the listener about how good science is done. We laugh at the joke because from our own lived experiences we know the conclusion is wrong though we might not have a concrete understanding of why.
In political circles though we might not be greeted by the same response. There we might encounter serious responses of the nature, that Johnny’s views are his and he has a right to them, though that misses the point. We all have our views on a variety of topics and right or wrong we can express them without fear of anything more than societal disapproval.
But in science, policy making and many other areas where we must have a common starting point for analysis, this kind of misinterpretation of data is simply not acceptable. As Sen. Daniel Patrick Moynihan famously observed, “Everyone is entitled to his own opinion, but not to his own facts.”
Some conclusions are just wrong and that’s good because disproving ideas is part of the scientific method. In science a hypothesis is a guess that needs experimentation to determine its validity. A theory, on the other hand, is a hypothesis that has been vetted by experiment though it is still subject to disproval by future experiments. That’s why experiment and correct data analysis are so important.
A scientific law is irrefutable and cannot be disproved, a theory is something we live comfortably with because a preponderance of evidence supports it. Thus, we have the Law of Gravity and the Theory of Evolution. It’s very hard to prove a scientific law but at the same time when we colloquially say something is “only a theory” what we are more likely declaring is that something is just a hypothesis or a hunch.
To be sure, many ideas that are scientific fact today began life as the opinions of researchers who collected data and tried, sometimes unsuccessfully, to share their discoveries with a skeptical world. In the late Renaissance people like Galileo were persecuted for beliefs based on data that strayed far from the accepted wisdom of the Church which was, itself, partially based on the teachings of Aristotle. All that gave way eventually to The Enlightenment a time of new learning and new beginnings when researchers of all stripes felt safe enough to share their newly acquired knowledge.
From The Enlightenment flow many of the ideas that we take for granted today such as modern economics and capitalism, representative democracy and especially, the scientific method.
Ironically all three are under some pressure today from people unschooled in the sciences or politics and who feel that, more than simply having an opinion, they have a right to a tortured interpretation of reality when it suits their need to validate their opinions.
Of course, nothing follows from this and we’ve gone from a time when fringe groups questioned the income tax or the safety of vaccinations to a present in which similar opinions pose a serious challenge to how our society is organized and operates. Unfortunately, and even dangerously, actively supporting such ideas leaves no other or better ideas in place to keep society running. If we suddenly deny the existence or need for an income tax how do we pay for operating the government, fight wars, or even patch potholes?
Back in 19th century Britain, then Prime Minister Benjamin Disraeli was at pains to improve the lot of the average Londoner. It was an age of poor sanitation, housing, and few labor laws so the lot of the average person was far from perfect. But Disraeli just knew in his gut that the problem wasn’t as bad as it seemed. So government did a radical thing for the times. It conducted a census to determine just how many poor people in London lived in deplorable conditions.
The story goes that answer came back much higher than Disraeli’s gut told him it should be. It should also be noted that statistical analysis which was then in its infancy and its utility was questionable in many minds. This confluence of Disraeli’s gut and public ambivalence about statistics is said to have caused Disraeli to utter the famous phrase, “There are three kinds of lies: Lies, damn lies, and statistics.”
Unfortunately, the story is apocryphal and Disraeli is not likely the source though the story was popularized in America by the humorist and author, Mark Twain who attributed it to Disraeli. Nevertheless, the phrase comes down to us as a way of casting doubt on using statistics as a way to clarify murky observations.
But “Lies, damn lies and statistics” is also a warning for our times. Data gathering and analysis may not be the perfect solution for understanding the world and it may leave the uninitiated with a cold feeling of alienation, but this approach is what we have until something better comes along. Without them we would have no basis for engagement on numerous fronts leading to chaos. Our species did that before, it was called the Middle Ages and we really don’t want to go back.