Tikalon Blog by Dev Gualtieri

Proofiness

November 19, 2010

It's really a tragedy. In my state, about half the students score below average on standardized reading tests. Obviously, the politicians and many parents are outraged. These statements should have provoked at least a chortle from my scientifically trained readers. Other people, however, wouldn't understand that this statement about student scores is nearly a tautology . In any sample, half the members fall below the median ; and, if the sample follows a normal distribution , then the median and the mean (a.k.a., "average") are identical.

Such matters are at the heart of the book, "Proofiness," by Charles Seife . The term, "Proofiness," is a play on Stephen Colbert's neologism, Truthiness , which describes "a 'truth' that a person claims to know intuitively 'from the gut' without regard to evidence, logic, intellectual examination, or facts."[1] Seife is an associate professor of journalism at New York University . He has an M.S. in journalism from Columbia University , but what sets him apart from other journalists is his A.B. in mathematics from Princeton University . Seife has an Erdos Number of four, having co-authored a paper with Frank Moss,[2] who had coauthored a paper with Robert Gilmore, who had co-authored a paper with Peter Salamon , who had co-authored a paper with Paul Erdos . Seife's put both of his degrees to good work by writing about mathematics and physics for the last fifteen years and authoring five books, the latest of which is "Proofiness."[3]

The premise explored by Seife is that when you assign a number to something, it attains a magical air of certitude for non-scientists. The opposite effect is usually true for scientists, since they would then have an absolute means of judging the falsifiability of a proposition. They have the tools to decide whether the number makes sense. In an interview for the New York Times , Seife said the following:[4]

"From school days, we are trained to treat numbers as platonic , perfect objects. They are the closest we get to absolute truth. Two plus two always equals four. Numbers in the abstract are pure, perfect creatures. The numbers we deal with in the real world are different... The numbers we create aren't perfect platonic ideals. They are mixed with falsehood, but we don't recognize that."

Seife writes that bad statistics are "toxic to democracy." He lists among the simple sins things he calls "fruit-packing." These are cherry-picking, comparing apples to oranges and apple-polishing. An example of the later is when advertisers use misleading graphics, such as those pseudo-scientific and unlabeled bar charts we see in television drug commercials. How much does eating oatmeal reduce serum cholesterol levels? A lot more when your bar chart starts from a very high non-zero origin.[5] Surveys may be accurately tabulated, but the results do not account for slanted questions. Close elections draw his worst scorn, when winners and losers are called on a margin of just a few votes. He suggests that the 2000 Gore-Bush Florida ballot should have been settled by drawing lots , since Florida law specifies that remedy when an election is too close to call.[6]

Looking for a "hanging chad," Florida 2000 Election

Scientists should watch their own house, especially when it comes to inferring causality from experimental data. He has a major problem with findings of the sort that wearing a certain color will have an effect on athletic or business performance (the "power necktie"); or deriving a formula that predicts happiness. These findings make nice reading in the Sunday supplement , but they don't make scientific sense. Seife coined the term "causuistry" for wrongly inferring causation. Correlation does not mean causality , but few non-scientists realize this. Even when causality is responsible for a correlation , it's often hard to specify what is the cause and what is the effect. If you have higher credit card debt, you're likely in worse health than others. This doesn't mean that you should keep your credit card balance low to stay healthy. It means that unhealthy people have more medical bills and higher debt.

It's not that proofiness is a recent phenomenon, brought on by television and the Internet. Seife points to US Senator Joseph McCarthy's Red Scare as a past example. McCarthy fired the public imagination when he held up some papers and proclaimed that it was a list of 205 communists working for the US State Department . Putting an exact number on this made it look real, but McCarthy had no idea how many there were, if any at all.

Regression analysis was always a useful tool for me when making sense out of experiments, and the regression idea is used to a useful extreme in the design of experiments methodology. Seife, however, heaps special scorn on regression analysis when it's applied outside the laboratory. It's used for predicting election results based on economic conditions. Your result seems to make sense, but without a valid linear model based on dependent and independent variables , it's really meaningless.

Proofiness

References: