Friday, September 28, 2012

Open Eyes. Open Data

The business case for open data (government data in particular) is a hot topic at the moment. There are already several interesting and detailed discussions on the topic, including this thoughtful blog article from Jeni Tennison who has recently been appointed Technical Director of the Open Data Institute. But  in my opinion these discussions are not bold enough. I want to claim that open data is not just a promising new technology trend which has the potential to add value to our societies - instead, it plays an essential part in the very survival of our societies.

The "finance crisis" of the past few years has shown us the inherent vulnerabilities of both government and the private sector, and destroyed our complacent belief that our way of life will continue indefinitely. Suddenly we found ourselves in a "pathless wood".

The way back to prosperity is not yet clear. But one thing is clear - the solution is going to involve efficient and intelligent government. Yet in a beast as complex and distributed as governments can be, how is this intelligence going to be achieved? Paradoxically, I believe the answer can be seen in a claim about human consciousness, put forward by the Nobel Prize winning biologist and neuroscientist Francis Crick and his colleague Christof Koch.

Crick believes that consciousness (visual consciousness in particular) evolved in complex organisms by necessity, because their range of behavioral responses to their complex, dynamic environments could not be supported by simple hard-wired responses. Simple organisms like frogs can get by with un conscious reflexes which, for example, make it snap at any small prey-like objects. But more complex organisms with wider behavioral repertoires would need a proliferation of these dedicated responses to cope with a growing range of environmental contingencies. Clearly an inefficient arrangement. The answer according to Crick: "Better to produce a single but complex representation and make it available for a sufficient time to the parts of the brain that make a choice among many different but possible plans for action. This, in our view, is what seeing is about. As pointed out to us by Ramachandran and Hirstein (1997), it is sensible to have a single conscious interpretation of the visual scene, in order to eliminate hesitation .....  and to make this interpretation directly available, for a sufficient time, to the parts of the brain that contemplate and plan voluntary motor output, of one sort or another, including speech."

It is time for governments to become conscious. It is time to replace inefficient, isolated and informationally encapsulated response modules with an efficient organism in which information from one part can be used by another part in a seamless, timely and flexible manner. It is time for a single, open, interoperable format for data representation and exchange. It is time to evolve.

The "business case" for open government data is the same as the "business case" for consciousness: survival.