Your relationship to data and openness
Sunday, 23 May 2010
As an economist my view of data I use is generally some variation of “it’s just data” That is, data is just one input among many that I’ll use to create something. I didn’t think too much about data other than to be frustrated when acquiring the data necessary for a given project is more than a trivial matter.
I’ve worked with many people in several different fields and I was, to be honest, more than a little surprised to find that many people, even academics, will go to great lengths to guard their data from use by others. At first I was merely frustrated by those who keep their data behind walls; if there is one thing that hinders progress and innovation it is when information is kept behind a wall.
But, I’ve come to (I hope) a better understanding of why such walls are constructed at all. Economists are intensive users of data but rarely create it so we have a difficult understanding those who do collect/create data. Those who create data, by going out and measuring the weight of various wildlife, or interviewing people about their consumption habits likely do not see themselves as just data sources. Rather, they feel that they are creators of something of value and they feel a deep connection to their work. When someone comes along and asks them for data it is not just data to them, it is their creation.
As economists, and other social scientist, we have a duty to understand the connection that some have with their data and respect that. Of course, this is not to say that I’ve changed my tune and data should be kept behind walls or encrypted on disks in building basements. Data wants to be free and it is most useful when it is freely and easily accessed. I hope that when users of data, such as myself and other social scientists, understand that the creators of data can have a deep connection to their works, we will be able to increase data openness and availability.
Comments on this article can be sent to me by e-mail..
No. 1 — June 18th, 2010 at 10:10 am
Fantastic piece. I am a clinical epidemiologist and also develop electronic medical records. I run a surveillance in a public hospital in Kenya where i have developed the first local electronic medical record system that used at the point of care by doctors and in the laboratory. This system is generating good real time data and has led on other researchers who kind of looked down at the efforts to start pushing for open data sharing. I support open systems and sharing, but there has to be somebody to underwrite the initial costs of development and collection of data – something not common in developing countries where research is conducted in a predatory manner.
No. 2 — June 18th, 2010 at 9:59 pm
Thanks Moses!
I am sure that you (and even those who turned their noses at your efforts) will begin to see great things coming from the data you are collecting. It is a difficult thing to completely open up your data, especially when collection is costly, however, I still want to believe that opening up data will be mutually beneficial. Unfortunately, I think that in practice it is a more complicated matter.
I’ve added your blog RSS feed and I look forward to hearing more about your work as it unfolds.
As an aside, I believe it was in the book Superfreakonomics that the authors describe a story where doctors were perplexed by high birth mortality rates. It was only after digging into the data that it was found that at least part of the reason was that doctors who delivered babies after having just worked on a cadaver were more likely to have these infants die. It was then that hand washing was encouraged and they were able to reduce infant mortality in the hospital. There is actually a whole chapter in that book about the how data collection in hospitals was a driving force behind better health outcomes. If you haven’t already, it’s definitely worth a read.