fugitive peace

A digression from my normal topic and a return to my geek days, when I wrote about science and technology.

My erstwhile colleague at The Economist (and one of the people who interviewed me for my first job there), Chris Anderson, now the editor of Wired magazine, has published an essay in the latest issue entitled “The End of Theory“. It doesn’t even come with the question mark that the authors of such audacious claims usually tack on to protect themselves.

In a sentence, it argues that the classic scientific model - come up with a theory,  then test it by experiment - is becoming obsolete with the advent of gigantic data sets (anything from the contents of a library to the genomes of all the species of microbes found in air) and extremely powerful computer networks that can crunch the data to look for patterns.

The purpose of a scientific theory is to allow you to make predictions about how things will behave - “in all cases where you have A and B, you will get C”. To be able to make those predictions you need to have an explanation, ie, a theory: “Because of XYZ, A and B will always produce C”. What Chris is saying is that in this new era we will cut out the middleman - the theory. Processing vast amounts of experimental data will find out what A and B lead to, and do so highly accurately, without your ever having to understand exactly why. (A good comparison is translation engines that “learn” to translate by comparing vast corpuses of texts that have been translated from one language to another, such as European Union documents. What emerges after the computer has been trained is a fiendishly complex, computer-generated algorithm that makes no sense whatsoever - there is no theory in it - but can translate extremely well.)

There are lots of comments running around the blogosphere about Chris’s idea; some think he’s a genius, others think he’s off his trolley. What I like is that Wired is simultaneously running a feature on its website entitled “5 Things Wired Pronounced Dead Prematurely“.