Recently in web and internet Category

Last week I had fun making a newspaper with Tom Taylor and Dan Catt. Tom and Dan have already written it up, along with Zach Beauvais at Nodalities. I spent most of my time trawling the data.gov.uk site finding data relevant at a postcode level. It was a great experience and I really enjoyed reconnecting with making things, having spent a long time writing about other people making things.

The intent was to make something that would give you a guide to a new local area, the context was something that a council might send out to you a few times a year. There is a huge amount of interesting information gathered by Government, but what we usually see is aggregated, averaged and sometimes skewed to tell a message. Taking the data and linking it directly to a postcode makes this data more impartial and meaningful, which leads me on to the gist of this post. Freeing up the postcode to census output area data as a freely available API, I'll explain.

Much of the data collected at a Government level is gathered at quite a fine grained level. The lowest level of this data collection is the output area. An output area is quite a small parcel of land, for my work it is essentially the canal basin behind my office. To see this, head over to the Neighbourhood Statistics website and type in N1 9XW on the lefthand panel, click on the More areas link and select the output area radio button. The Neighbourhood Statistics website discourages direct linking, so this direct link to N1 9XW might well fail. The ONS have a helpful guide to UK geography too.

However this Neighbourhood Statistics website is a gold mine of useful information. plus it offers a key conversion from postcode to output area. Land is hard to describe, for the patch of land your chair is sitting on there are many different levels of descriptor. Some of these are point based like latitude and longitude which allow placing on a map. Most of the descriptors for Government data are boundary based. The output area is one of these. It is the smallest parcel of land that the Census collects data for. Once you get the output area though you can then find out the higher area entities like which MP constituency or Health Trust you are inside. The data behind this is all available for free from the ONS, but for internal use only.

The whole process to figure out your MP goes something like this. Enter postcode, map to output area, from there determine which ward and thence which constituency you are living in. It gets more complex if you don't know your postcode or if it has changed recently. The Royal Mail PAF (postcode address file) is the key to managing postcodes, it links postcodes and addresses. The names of the areas of land are managed by a gazetteer which links names to boundaries and will often include older names for areas, eg the old counties of England no longer exist at an authority level, there are all now different types of council.

Coming back to the simple case of knowing your postcode, there are a lot of concerns about access to this data, including campaigns to free the PAF file, which got a pretty lukewarm response. The Royal Mail own the PAF file and the Ordnance Survey own the Boundary line data, so between them they seem to have the entire thing sown up, but the mapping of postcode to output area is the property of the ONS. It is available free of charge from them, go order a copy now. You can use it internally as much as you like.

While that is useful, what I'd like to see is the lovely service on the Neighbourhood Statistics website made available as a REST API. Give it a postcode and get back JSON listing the output areas and upwards it falls into. The data is all on the results pages for the site and as the site states Boundaries for Output Areas (OAs), Lower Layer Super Output Areas (LSOAs), Middle Layer Super Output Areas (MSOAs) and Travel to Work Areas (TTWAs) are available free of charge to all users.

Making it available as a postcode to Output Area API unlocks all the Government data that the data.gov.uk project intends to release. I feel that this should be the responsible attitude of a Government committed to an Open Data initiative. Releasing the Postcode to OA does not free maps from the OS, nor give address details which the PAF file covers. Yet it provides an immense value to online civic projects.

I've posted my presentation on website psychology to slideshare. It looks at cognitive psychology and how this should be shaping our choices in web development. It is best viewed with the notes on slideshare, but you can review the themes below.

| View

Tags:
social software, web development, psychology, barcamp, barcamplondon3, cognitive, experience design, flow, schema

The passing of time is something that web applications often miss. Many events that we model have a fixed progression from browse to purchase and ship, or edit to publish. However some of them have a state which is semi-permanent. I've been involved with a couple of these recently and both were around roles in software. One was for work and one was in discussion with friends. The details are not that vital, so I'll discuss the general case.

The basis issue is that we let people take a role on our software and then something structural changes. Usually outside the world of our software, but that means the person can not or should not fulfil that role anymore. An example is employment, a person might have a role on a site, as they are employed by a certain organisation. If their employment status changes, then two things need to happen.

They need to be able to change or have changed the role they play on the site. Their previously contributed content needs to be flagged that it was contributed by a past employee. It is really easy to miss this kind of detail when building out the first pass of a site, but it is worth planning for it, as it can become urgent if the person moves to a competitor.

Thinking through the language to describe the person's previously contributed content and allowing for a history of roles on their profile will help explain to other people on the site that Simon was the admin of this group, but he is no longer.

The profile is worth further discussion, if you are allowing people to create an affiliation, you need to allow for change and past affiliations. I've worked for the BBC and I'm now working at Nature, ideally I'd be able to list both of these appropriately on sites which cover the past 3-4 years. Maintaining a single bio or profile might not be enough. Conference websites are a good example of this, as your career progresses your bio will change, but the bio and the conference presentation come from a particular time period.

Technorati Tags:
time, social software, role, groups, moderation

I've been looking at the relationship between time, content and the flow of history over the past year or so. I've spoken or run sessions about it at eurofoo06, barcamp london 2 and xtech07.

I've been worried about our ability to understand our present after it becomes the past tense. Much of what we read makes sense for a limited timeframe. Time moves on and our context shifts to new current events. So heading back to try to understand the past becomes much harder. The analogy with well commented code springs to mind, maybe my desire is for comments on history. We understand much less of the humour in Shakespeare for this reason. Even moving five years into the past and it can be difficult.

The problem can widen out from here and I'll extend briefly, my context is different to your one, so I want different events and people related together. My family, my work and my friends and interests differ, so unlike code there is not one set of reference points. This is a substantially harder problem, so with that excursion, back to public news content.

I've been researching a proposed microformat hEpoch to deal with time based reference in news stories and I'll admit I've given up. The problem has becomes one of exactitude, stories are written to be read not used as code. Adrian Holovaty has written about this before in his essay on data as journalism. Stories reference past events using terms like "nearby" or "last month", an example story from the BBC. This means that it is currently impossible to make a microformat for linking these stories together. Yet if the data was there then all we need to solve is the time base reference, probably via a bit of POSH.

A good example of this is timelines, they are the pretty, but doomed product I've watched being made many times over. People work hard producing a nice interface and sometimes even separate content from presentation. Then the timeline is often left abandoned once the project or programme finishes. A small bit of care in terms of tags and time formats for news stories and they could auto update.

Time and geographic accuracy need to be flexible, with time we can be somewhat flexible, the hCalendar microformat allows just a year for dstart, which means fixing to a single year is possible, but a approximate date eg 1760s is impossible. I'm not knocking the hCalendar spec, just pointing out a space for new developments. the microformat principles are well defined and do deliver a workable product.

Geographic data is hard too, the geo microformat is superb for specific realtively modern locations, where there is enough context to determine the latitude and longitude for the place. However we frequently need to reference a region, like a city, but cities change over time. The centre of London is now Charing Cross, but it has shifted over time.

This post is entitled "a lament", but really there is a call to action in the tail. We can make this better, all the data exists, it just never makes it out of the editorial process. Similarly when blogging we omit lots of the specifics, as they make it easier to read. eg 06/07/07, how do you unpack that date? It is impossible to know for sure which year, month or day it is.

If you are a publisher then think of the value of being able to automate connecting story threads and managing content by time and location. If you are a tool provider, MarsEdit and MT or WordPress, then offer support for microformats. The same for the more corporate tool providers. If you are an author then think of being able to return to the item of content in months or years and understand the context. This is not easy, but it is possible. Next up, I'll be looking at search, then main user of this data.

Technorati Tags: history, microformat, provenance, time, semantic, journalism, story-telling

I've been designing Nature Network over the past year or so and through it we've created social software for scientists. This is a different type of social software to the more general platforms like Facebook or MySpace or Vox. We have designed it to fit the needs of scientists, it has captured some of the professional aspects of scientific life.
I'm keen to find more examples of this, I spoke to some people at foocamp and discovered ModelMayhem and SchoolLoop. Both of these are good examples of the kind of vertical market social software I'm after. Each of them presents reasonable (fair) barriers to entry and has a language of its own, mirroring the existing phraseology used within each subject area.
ModelMayem is a site for fashion models, makeup artists and photographers. You need to show work and be approved to properly join the site. SchoolLoop is aimed a teachers and parents of pupils in schools in the USA. You need to have a pupil at a school in the USA. Sermo and LinkedIn also spring to mind, LinkedIn less so, as it is quite generic. Sermo is for registered doctors in the USA

Can you tell me of other examples you found on the internet, thanks.

A while back I had one of those the internet as a service experiences. Lucy found a cd in our CD player and didn't recognise the wordless disk art, so to determine which CD it was, she popped it in her macBook and iTunes told her, via gracenote, that it was the excellent LFO album Sheath.

The internet has been a fundamental part of our house for a good while now, thanks to Be and WiFI, but it grows ever more so daily. I remember there was a guy researching this in early 94, a Norwegian Prof, who had his house on a 64k line and he described the experience as it becoming an appliance, only now we have the services to hook up to it, Google, amazon, gracenote, email etc.

Why did I just take an 18 month contract and a new N95 than wait six months with my Nokia 6630 and get an iPhone on O2?

The primary driver was availability, plus GPS, 3G, a decent camera and the rest (funny mocking ad comparing N95 and iPhone). I've waited through the N70, the N80 (briefly) and the N73 waiting for a phone which can act as a computer, camera and phone in one. I'm interested in how much of my persistence I can manage through my phone. I'm a twitter user and with the N95 I'll become a jaiku user. Then there is fire eagle and getting the flickr client working again on my phone.

The iPhone looks nice, but as willo and Duncan point out it is not without its flaws. Much of the UI strangeness comes from it feeling like an American product, not a European one. SMS is an important part of my life, as is decent 3G access. It seems unlikely that the iPhone will gain either 3G or proper GPS soon. The SMS issues, like only one recipient will be fixed, but the mapping aspect is important.
I lost my phone a while ago and I've not got round to moving the Lumisoft map application to my replacement 6630. I miss it a lot. I'd not realised how much a part of my life it had become for me in London. The N95 promised to offer this for the whole of the UK and potentially the rest of Europe. That is tempting.

Give me 18 months and an improved iPhone and I might be tempted, but for now the open environs of the N95 seem quite tempting, I'll be installing apache and Python soon. I really hope Omni allow S60 access to the rails app in OmniFocus though.

I tend to generate 12-20 tabs very easily when I'm researching a topic and then I can leave them open for a good while as I get around to reading them all. I can end up with 100 plus tabs open easily. In Safari this leads to heavy memory usage and on my 2GB PowerBook G4 this lead to 2 or more Gig of swap space and a basically unusable machine. Seconds of waiting between actions, at times. All cured by either quiting Safari and letting the VM recover or a restart. So, I decided to finally switch to Firefox.

I know it has taken me ages, but I'd got taken by Saft and saving my open tabs. Session Saver did this for Firefox 1.5 and it is now a slightly hidden feature on Firefox 2. A dropdown is not a good place to hide additional functionality. It is on the first tab in preferences, but looks like a drop down that controls the Home Page behaviour. So the other reason to move to Firefox, the lovely add-ons eg Operator and Tails which highlight microformats.

Firefox is not without its faults, it seems to be more CPU bound than Safari was, so my machine has a lower VM usage, 1 gig typically, but the CPU load sits about 2 most of the time. It makes my envious of Lucy's new MacBook.

I made a new gavinbell.com website last week. I've updated the site with details of the talks I've given over the past few years and refreshed my profile. I've also managed to pull all the writing I've done into one place too. It was a good experience building something afresh, skEdit and CSSEdit were a very helpful part of the process, with more to come. I feel enthused to make more stuff again.

I'm going to be at BarCamp in London on Saturday. I'm planning on giving an updated version of a session I ran at EuroFoo last September on time and the web.

Searching for content in the past is difficult, finding content in the same time period as older content is almost impossible. I mean finding 1960s stories about 1960s events, rather than finding 1990s and 2000s reporting on the history of the 1960s. Part of this is that we are very focused on now and the future (rss, I blame you), part of it is that the internet didn't exist then, so the content online is patchy.

However publishers are starting to put this kind of content on the web, the New York Times (sample article on the moon landing); the Washington Post; and my employer Nature spring to mind, though as paid for services. Google are offering a News Archive search with filtering by year and links to pay for archive stories from a range of USA papers, (Oscar Wilde from 1895).

We've got immense amounts of content already, how can we delve back into the past and find out about a period in history as more and more of our world is documented online and not in libraries.

I'll talk about what the issues are in exploring our past and how we can keep hold of what has happened, before all the primary witnesses die of old age. Persistence, geodata, time formats, content disambiguation and the grand sweep of history, how can you resist...?