Tikalon Header

Science as Data Compression

August 2, 2011

There's an old joke about timepieces. It's so old that it refers to analog timepieces, called wristwatches, that were once worn before everyone carried a cellphone with built-in clock. Not only that, the joke mentions an action, called "winding," that's probably unknown to the younger generation. The joke is understandable today only if it's rephrased for a digital clock, as follows.
Someone notices that a man's clock is flashing 12:00, and asks why he doesn't set it. He replies that the clock always ran fast, and this way it keeps better time. Perplexed, the other person asks how that could be. The man answers, "Before, the clock never showed the right time. Now it's correct twice each day."

I was reminded of this joke when I read something similar in the draft of a very interesting book by Daniel Burfoot posted on the arXiv Preprint Server.[1] Burfoot has a background in mathematics, physics and computer science. The book, entitled "Notes on a New Philosophy of Empirical Science," contains the following parable.

A physics professor is challenged by a student who claims that he can predict the outcome of physics experiments by communication with the spirit world. Not only that, he claims that he can make better predictions than she can using her Newtonian mechanics. As a test of their abilities, they agree to predict the time a projectile will remain in the air when launched by a spring device.

The professor minds her forces and angles, and she announces her prediction of two seconds to the class. The student communicates with the spirit world and writes his prediction on a piece of paper. The experiment is performed, and measurement indicates 2.134 seconds. The student claims victory, since his prediction, when revealed, is 1< t < 30. His prediction is correct, and the professor's is wrong by 0.132 seconds.

Daniel BurfootDaniel Burfoot, author of "Notes on a New Philosophy of Empirical Science", a draft of which is on the arXiv Preprint Server.[1]

The idea of Burfoot's book is that theories are compression algorithms. They reduce vast amounts of data to an equivalent, but smaller form. Armed with this idea, he considers a refined (but perhaps less efficient) version of the scientific method that proceeds as follows:[2]
• Assemble a large database of measurements of a phenomenon.

• Attempt to develop a theory of the phenomenon, or revise a previous theory.

• Instantiate the new theory as a compression program.

• Invoke the compressor on the database.

• The new theory bests a previous one if it reduces the data to a smaller file size when the length of the compressor program is included.

Every dataset will have at least one "zeroth-order" theory, which would be a Huffman coding of the data; but a real theory, of the Newtonian mechanics variety, will have a compression ratio many orders of magnitude better. The similarity of this approach to algorithmic information theory, as championed by Gregory Chaitin, is apparent.

Figure captionFrancis Bacon.

Bacon was an early proponent of what is now called the scientific method, although his techniques were more empirical than analytical. That's why he's called the father of empiricism, and not the father of the scientific method.

Bacon died of pneumonia while studying the preservation of meat by freezing (chicken, not bacon).

(Via Wikimedia Commons).

One advantage of Burfoot's approach is that Occam's Razor is an explicit feature. His main argument is that theories developed in this fashion are no different than those developed by physicists; that is, you have a bunch of data, and you reduce it to an equation.

One other feature of Burfoot's method is that it sidesteps the demarcation problem. This problem, which has been argued for more than a century, is where to draw the line between scientific problems and other problems. You just apply the method, and get your result. The process makes no distinction as to the suitability of the subject matter, and it can be applied to a wider range of data than the classical scientific method. Quoting from the arXiv abstract,[1]
"In this view a theory is scientific if it can be used to build a data compression program, and it is valuable if it can compress a standard benchmark database to a small size, taking into account the length of the compressor itself."

In a private communication, Burfoot emphasized that the compression approach is highly effective at debunking pseudoscience, since pseudoscientific theories cannot achieve compression. In the physics experiment of the professor and her student, we see that the professor's classical mechanics approach will compress the dataset of all projectile problems much better than the student's.

Burfoot also distinguishes the equation F = ma from the hypothesis that generates the equation. Purely mathematical statements cannot achieve compression, but the hypotheses that they encapsulate can.

One interesting idea, called the veridical simulation principle, is that a good decompressor will generate sensible data from a random input. The decompressor complement of a good compressor for English text will generate grammatical, meaningful English sentences when fed random data.

Returning to the topic of wristwatches, I haven't worn a wristwatch in about twenty-five years, but there was an interesting article written by an engineer about wristwatches when they were still popular.[3] He wore a wristwatch, and whenever he needed to know the time, he would look around to see whether there was another clock in the vicinity. He found that this was nearly always the case, so he concluded he didn't need a wristwatch. He also admitted that he still wore one.

References:

  1. Daniel Burfoot, "Notes on a New Philosophy of Empirical Science," arXiv Preprint Server, April 28, 2011
  2. Research page on Dan Burfoot's web site.
  3. You can read the sentence either way, as meaning when engineers were still popular, or when wristwatches were still popular. Sorry, no reference to the original article.