Last week I had fun making a newspaper with Tom Taylor and Dan Catt. Tom and Dan have already written it up, along with Zach Beauvais at Nodalities. I spent most of my time trawling the data.gov.uk site finding data relevant at a postcode level. It was a great experience and I really enjoyed reconnecting with making things, having spent a long time writing about other people making things.
The intent was to make something that would give you a guide to a new local area, the context was something that a council might send out to you a few times a year. There is a huge amount of interesting information gathered by Government, but what we usually see is aggregated, averaged and sometimes skewed to tell a message. Taking the data and linking it directly to a postcode makes this data more impartial and meaningful, which leads me on to the gist of this post. Freeing up the postcode to census output area data as a freely available API, I'll explain.
Much of the data collected at a Government level is gathered at quite a fine grained level. The lowest level of this data collection is the output area. An output area is quite a small parcel of land, for my work it is essentially the canal basin behind my office. To see this, head over to the Neighbourhood Statistics website and type in N1 9XW on the lefthand panel, click on the More areas link and select the output area radio button. The Neighbourhood Statistics website discourages direct linking, so this direct link to N1 9XW might well fail. The ONS have a helpful guide to UK geography too.
However this Neighbourhood Statistics website is a gold mine of useful information. plus it offers a key conversion from postcode to output area. Land is hard to describe, for the patch of land your chair is sitting on there are many different levels of descriptor. Some of these are point based like latitude and longitude which allow placing on a map. Most of the descriptors for Government data are boundary based. The output area is one of these. It is the smallest parcel of land that the Census collects data for. Once you get the output area though you can then find out the higher area entities like which MP constituency or Health Trust you are inside. The data behind this is all available for free from the ONS, but for internal use only.
The whole process to figure out your MP goes something like this. Enter postcode, map to output area, from there determine which ward and thence which constituency you are living in. It gets more complex if you don't know your postcode or if it has changed recently. The Royal Mail PAF (postcode address file) is the key to managing postcodes, it links postcodes and addresses. The names of the areas of land are managed by a gazetteer which links names to boundaries and will often include older names for areas, eg the old counties of England no longer exist at an authority level, there are all now different types of council.
Coming back to the simple case of knowing your postcode, there are a lot of concerns about access to this data, including campaigns to free the PAF file, which got a pretty lukewarm response. The Royal Mail own the PAF file and the Ordnance Survey own the Boundary line data, so between them they seem to have the entire thing sown up, but the mapping of postcode to output area is the property of the ONS. It is available free of charge from them, go order a copy now. You can use it internally as much as you like.
While that is useful, what I'd like to see is the lovely service on the Neighbourhood Statistics website made available as a REST API. Give it a postcode and get back JSON listing the output areas and upwards it falls into. The data is all on the results pages for the site and as the site states Boundaries for Output Areas (OAs), Lower Layer Super Output Areas (LSOAs), Middle Layer Super Output Areas (MSOAs) and Travel to Work Areas (TTWAs) are available free of charge to all users.
Making it available as a postcode to Output Area API unlocks all the Government data that the data.gov.uk project intends to release. I feel that this should be the responsible attitude of a Government committed to an Open Data initiative. Releasing the Postcode to OA does not free maps from the OS, nor give address details which the PAF file covers. Yet it provides an immense value to online civic projects.