Recently in writing Category

My book Building Social Web Applications is now generally available in print form, as an eBook and even as an iPhone application. Amazon US, Amazon UK, iTunes iPhone or iPod touch app, O'Reilly eBook and online access via Safari from O'Reilly. I'm really pleased to see it out there and I'm starting to get good reactions to it too.

My book, Building Social Web Applications is making its way through production at the minute, I've done the last substantial edits and am creating the website to support the book. It feels odd to be on this side of it, writing a book has been a large part of the last eighteen months or so. A good feeling though and I'm generally pleased with the results. It should be out in September and the book is available for pre-order from Amazon UK and Amazon US.

Another chapter outline for you to comment upon. I'm writing a book for O'Reilly and I'd love your help and suggestions. I don't write code for a living, so I need your input to strengthen this chapter. See earlier post for more information on the book. Let me know what is missing and what examples you'd like to suggest for the book. Comment below or send me an email me at gavinbell dot com or zzgavin on twitter. How social applications differ

• Identity is bound in, other people’s identity and interests become the focus of your site
• Linking, tagging and content reuse are important
• Asymmetric follow makes for tricky scaling issues
• Privacy complicates implementation
• Polymorphic nature of relationships between people, content and metadata, even functionality
• Small is good, let your community shape what you build, rather than launching a big app
• Rapidly evolving protocols and standards

Agile methodologies

• Preference for agile development approaches,
• flexible development a month at a time is better than rigid multi-month plans, ie release early and often.

Deployment and version control

• Retaining flexibility
• Importance of source control and the shift to distributed models like git or hg
• Dev, test, staging and production machines, continuous integration and TDD
• Conditional releases - one site different versions depending on who you are, tags and flags
• Alpha users (set list, or tagged groups)
• Tagged features available to certain tagged users (dopplr model and others)
• Testing with high loads and real data is hard, apache bench etc
• Rolling restarts for no downtime eg haproxy and mongrels
• Puppet and capistrano, managing collections of machines and code deploys.

Implementing the design of social applications

• Design is how it works
• Use cases for developers, understanding the whole picture vs implementing a feature
• The app has a point of view, understanding what fits and what fails as a feature
• How code review helps
• Linking the designed for activities to implementation
• REST approaches compared to RPC / CGI approaches,
• better urls, relying on the verbs in HTTP properly, not everything is a GET
• Feeds and APIs from your app

User interfaces and prototyping

• Failing gracefully is a strong capability for your site
• What makes a website tick (good error recovery)
• Accessibility is important, unobtrusive javascript is a great approach. (Not a tick list)
• Using the console for code level prototyping of ideas
• Templates and CSS vs application code and database schemas

L10N and friends

• UTF-8 and not ascii for data storage everywhere.
• Understanding internationalization and when to do it

Security

• SQL injections, cross site scripting, escape everything, only unescape content you know you really want to display.
• Let them use html and filter approach is doomed, whitelisting is best approach, CSS has similar vulnerabilities
• Cross site request forgery
• Google web accelerator means every link can get processed
• Ensure that form interactions really do come from your own site, use a token to prove this, unique per user and tied to session, this token is now extremely valuable
• Invisible iframes and click jacking
• Flash and browser plugins complicate the whole situation and web browser javascript and quirks mode means they are black boxes

Scaling and messaging architectures

• Forget WS-* unless you really need it for integration
• http://www.37signals.com/svn/posts/1509-mr-moore-gets-to-punt-on-sharding
• http://www.new.facebook.com/notes.php?id=9445547199
• http://highscalability.com/amazon-architecture
• Not about optimisation, architecture more important
• Load balancers, proxies and caching
• Realtime vs non-realtime
• XMPP etc and see http://www.loiclemeur.com/english/2008/12/xmpp-pubsub-aka.html
• NOT just XMPP, the importance of caching
• The qualitative argument for asynchronous development.
• From the SXSW Scaling Boot camp 
• "Design for parallel steps and use queuing as much as possible
• http://www.slideshare.net/rabble/beyond-rest-building-data-services-with-xmpp-pubsub

URLS and IA

• URLS should be as far as possible
• Short, meaningful, canonical, permanent and hackable, plus opaque if needed eg amazon asins
• Pagination and url structures, what was on &page=3 is a recurring problem for aggregation pages, providing date based aggregation alongside simple pagination can help. Imagine Twitter with yyyy/mm/dd aggregation pages too.

Search

• Search is critical to your site, but often left to the end.
• Understand what needs to be searchable and under what context
• Searchable items can grow, eg photos by place on Flickr
• Search can be provided by a third party eg summize and twitter
• Build your search based on the content you store, it is not enough to return simple text matches.
• Finding a photo means returning the picture and metadata, similarly for an event.

IDENTITY and management of user data

• REGISTRATION patterns
• Standard email address, password and screen name approach
• Deleted accounts and how to handle them
• Need for separate admin accounts alongside normal user accounts
• Staff will have admin account and user accounts with different capabilities

OPENID and OAUTH - the open web standards

• Why do they exist
• How to implement them and what impact they will have
• Relative importance of OAuth, balance between identity and access to data, selective permissions
• Portable contacts POCO and what it solves
• OpenSocial, plus the various connect products
• OAuth address book access

Federation

• Distributed systems linked via common protocols and data formats
• Not yet mainstream, but an important direction
• Ma.gnolia M2
• Identi.ca (openmicroblogging)
• http://www.jaiku.com/blog/2009/01/15/were-going-open-source/
• http://www.zengestrom.com/blog/2008/12/foreign-friends-from-a-servicecentric-to-an-objectcentric-social-web.html
• Activity Streams and DISO work

Reputation systems

• What to build and how.
• Data to base reputation upon
• How to aggregation and modify raw data
• Avoiding being gamed
• Bryce Glass and Yahoo pattern library.
• Managing and understanding your community

Need for admin tools

• When to build them
• What to put inside them
• How to maintain them loosely tied to TDD (no admin, no feature)
• Balance with just enough admin to get job done

Statistics

• Will your community want stats
• How to build a personal stats tool
• Granularity, fine is expensive
• Simple overviews eg favouited content etc
• Stats for your own use
• Collect and timestamp everything
• Avoid monthly reporting focus, allow for real time reporting
• Think too monthly and you’ll just get monthly stats
• Good metrics are vital

Collective intelligence

• Recommendation systems
• Machine learning
• Classifiers and filtering tools
• Starting simple and figuring out what data to start with
• Person, location and content are all valid places to build upon

Making your code green

• Average costs per person
• Optimisation of HTML and CSS etc
• Server side optomisation, caching etc
• Flickr ops data graphs and notes
• YSlow and energy efficiency
• Speed = less energy used in general.

The cover for my book, you can pre-order it from Amazon UK and Amazon US.
I'm fondly calling it the spiders book.

I've mentioned before that I'm writing a book for O'Reilly, entitled Building Social Web Applications. It should be published this August. I've written quite a lot of it based on a mixture of interviews and conversations with people at conferences and in London. I'm now writing the more technical chapters and I need your help. I'm an interaction designer and product manager with a good head for technical detail, but being honest I don't write code daily. So it would be great if I can get your input into these three chapters.

• Writing the application
• Extending the application, APIs and integration
• Launching and evolving

The earlier chapters cover the questions of why are you building a social application and look at how people behave and interact with one another online. These three chapters cover how to implement and extend the application. The intent is not to give a lot of technical detail and reams of code, but to explore how social applications differ. They are not the same as building a content publishing system or an online shop. I've created some outlines to give a framework for what I think should be in the chapters, but please let me know what is missing.

I'm not interested in recommending a particular language or framework, nor one particular approach to scaling. I'm also interested in your opinions on centralisation vs distribution and federation. Examples that you can share and suggestions of content to include are most welcome. Dopplr, Flickr, Twitter and sites like LinkedIn or Facebook are the kinds of things I've been featuring, but I've also been looking at adding social features to newspapers, magazine and product companies.

I'm going to put the API and integration chapter up first, the other two, tomorrow. I want to start writing the chapters on the 23rd of March, so about ten days or so from now, so please let me know what you think. I'm zzgavin on twitter and I'll be talking about the book there too.

Snow is a lovely book, it is a short tale of love, poetry and snow. Originally written in French by Maxence Fermine and translated wonderfully by Chris Mulhern . The writing is exquisite, simple short sparse prose with barely a word wasted. Each page invokes visual and emotional response, from the coldness of the mountains to loss and a vivid sense of colour. The story is of a young Japanese man finding his vocation as a haiku poet and the hard path he takes on the way. A perfect place to escape to on the way to work, though don't rush it, those 100 odd pages fast melt, like snow in a ditch.

The title of some email spam I was sent recently, I wonder at what scale of live it would make sense. Hydrogen is after all pretty damn tiny, making nanotech seem vast. Bosons and quarks fighting it out over who really is the god particle ?

Hi Macworld reader, I wrote the feature on blogging in the current issue of Macworld UK, I hope you enjoyed it. However things are a bit quiet here currently, lots of plans and less time. At the bottom of the page are some people I'd recommend reading and there are also links to my archives. Hope you have fun !

Recently I have found a new creative outlet to take my mind off the list of things going on in my life. From belt sanding the 300 sq foot floor of our bedroom, to the US election, product design pressures in work, the european constitution project I'm working on and the inevitable UK election and whether I can vote for Labour in the spring.

My new outlet is what is commonly called haiku, though often disparagingly. In fact mostly people write senryu. The key difference being haiku are about nature and include specific words to reflect season, the senryu are about human foibles.

Further references on haiku cover the life of Basho, one of the foremost masters of the art. There are a range of site covering the art haiku.com and Japanese site, the wikipedia definition. This is a lovely site, with some nice illustration, as is this one.

It is a thoughful pastime usually consisting of 3 lines and 17 syllables distributed in 5, 7 and 5. The intent is to capture or indicate a moment, a fleeting sensation usually of nature. My first efforts represent the lovely light from October.

dramatic light
veering sideways makes faces
glow as autumn leaves

More as I write them. The wonderful moleskine is the tool for writing haiku / senryu in, as they are perfect for writing on the train. Saves reading some tabloid over someone else's shoulder or feverishly typing on your powerbook.

I have lots of half finished notes sitting as partly written entries on this blog, it would seem that I'm not alone, a quick peruse of other peoples' blogs in NetNewsWire confirms this. MT has become the equivalent of the notebook, sort of, it doesn't capture everything, as it is not always with me.
Earlier in the year I wrote about how I had replaced my palm pilot with a sheet of paper, well it has been upgraded to a Moleskine, which I bought whilst in Grenoble recently. I've started to use it for capturing the inital "I might write about X thoughts". Todo lists live on as scraps of paper folded inside, and some travel writing fills the front, though I usually spend too much time staring out the window on trains and not writing.
Maybe the choice of notebook was inspired by Bruce Chatwin, who I'm reading at the minute, or maybe practicality. Whatever it is a useful medium to have, of all the notebooks cluttering up my flat I find that this is the right size for me to want to carry and not regret the size of it.
Chatwin's writing feels like weblog articles, you can see the constraint of the page size in his writing, each chapter is a couple of pages from his moleskin, the standard paperback page size dwarfs the writing at times. His writing also has the same immediate style, capturing the sights and the experiences straight onto paper. It is the immediacy that feels like weblogging drawing you into the periodic snapshots of his travels in Patagonia. It might just be that I've read few literary diaries, but the parallel is too strong for that I think.
So, back to the drafts, I still have them sitting on my server and I'm not sure what to do with them, some of them are interesting, others dated. I'm tempted to delete them and more on. Yet there is something of interest in them, maybe I'll roughly edit them and dump them on here.
Bruce Chatwin may have had a similar experience or half finished entries, yet working the linear medium of the Moleskine, he was much more constrained in terms of returning to unfinished work, this might be a good discipline to learn. Often the more quickly written articles are more popular than the longer missives I labour over, like this one is turning into.
On Monday night I spent a while talking over some of this with Tom, Euan, Paul and Dan, amongst others. Stuart Hughes talked about his experience of blogging in Iraq, then recovering from his injuries. He also touched on the differences between his day job as a journalist and his writing on his weblog. They are not the same thing, but the news media are interested in the freshness of weblogs, but the randomness of them in terms of fact checking makes them wary. The mainstream media prepare news for consumption, those who read blogs make their own narrative.
This lead into a debate on the changes that happen to a weblog if you write for an audience, or to get traffic. There is a continuum from niche blogs like gawker or pvrblog, to the more personal weblogs like this one. I write pretty much what I want, occasionally being sensitive to being a BBC employee, in that I don't blab about non-public projects, but that is not a BBC specific thing. Once you have an audience however small, it is difficult to ignore them, hence the occaisional apology for not posting that appears on various blogs. There is also the passing desire for fame via google and affiliate income from amazon or such, mind you the rates of pay are pretty low in terms of pounds per entry, even NUJ basic rates are better.
This desire to capture the passing flow of information and distill it is quite powerful, hence the linklogs that people have, this is a good example. There is a part of me that wants to make a linklog, in fact I've started one, which I've yet to link in to the main page. Yet links without context make me feel a bit sad, as they are just like bookmarks, which I seem to have stopped using altogether. I feel that I'll loose the context in which I found the link in the first place, again I've talked about this before.
Finding time to write enough articles to capture the world as it passes you by is important, maybe having the moleskine and the linklog will let me focus on the more interesting articles and "bookmark" the rest. There is also a balance between work and play, though arguably this is both.
Weblogs increase the amount of information you can read, RSS readers enhance this ability even more. Maybe I'm saying that reading less and maybe writing less might be a good idea, or at least letting go of some of it.