Some of my past projects with a little commentary about each. Most of this work is from the last 10 years. Where possible I’ve linked to the original site.
Qcon London 2013 §
I did a talk at Qcon in London I was a speaker on the Architectures of the Small & Beautiful track. The talk was about our re-architecture of Songkick’s rails application. I covered some of the reasons why we needed to clean it up and what that gave us after six months of work. It covers some of the same ground as my post on Songkick’s devblog but with more about why rather than actual discussion of the code although there was some code still in it. You can download a pdf of the slides if you like.
My work at the Guardian §
From May 2006 till May 2009, I worked at the Guardian on guardian.co.uk. It was without a doubt the largest and most ambitious project I had worked on. We redesigned and redeveloped (r2) the vast majority of the Guardian’s web site.
I started at the Guardian as the first client side developer and as such was in the privileged position of being able to decide on coding standards and having free reign in relation to template structure. Needless to say I made a lot of mistakes — I also think I got most of it right. By the end of my stay we had 280 templates over 300 components, 2MB of css and it all mostly worked every where. I’m going to try and describe what and how with some thoughts on what not to do thrown in along the way.
Templates & components
The templating language chosen for the r2 project was velocity, it has the virtue of being very simple. It did force some design decisions such as data sources and the very extensive nesting of components. In most cases velocity is easy and readable. I’m not going to discuss the language in any detail since the structure of our templates is generally applicable and velocity’s strengths and weaknesses are of limited interest.
At a template level our approach was very conventional — each page has a template and these templates have the bare bones of markup. We didn’t modularize this as much as we might have. A defining characteristic of our templates is template metadata, this is a map of properties and values that the cms uses to apply appropriate templates to content. It also sets the template’s name in the cms gui and sets a unique template id. The template id allowed us to move and rename the template files without affecting the operation of the cms. The use of a map meant the values were labeled and so in may ways this becomes self documenting. We also hooked our css merging tool into the template metadata so we knew what to call the merged css file for that template.
Our approach to components is quite a bit more interesting. A component is the basic way of getting code reuse and as such is the building block of the frontend. By the end of the r2 project some of our core components were being used in hundreds of places, this meant that adding some new functionality to every page on the site is often easier then adding it to a limited number of pages.
Possibly our best decision was that if a component has no data to display it will out-put nothing at all — no empty <div></div> tags, no empty lists, no comments, nothing! While this presents a problem when debugging (how we solved that I’ll come to later) it keeps page weight down and empty components cannot impact on the page layout. We also kept our comments in velocity so they never turned up in the html thus allowing us to duck the duplicate content bug in ie 6 and in truth comments in the html served to the web add nothing. That said we do have a few and we probably could get rid of them, but, I don’t think we have enough for it to matter deeply.
Since everything on the Guardian site is workflowed and we have our own development environments the application already runs in multiple modes. In preview mode or on a developer workstation our components are wrapped in comments, with the component name and a few other useful items. This is invaluable when debugging since finding the component causing the problem is otherwise hit or miss and some of our components are nested eight levels deep, just try tracking down the troublemaker without some kind of marker. I would recommend doing this, or something similar, in any templating system it will save hours of time and help keep your clientside developers sane.
The components themselves were developed to do one thing, this resulted in several layers to each piece of functionality. As mentioned we nest components deeply, often a component would be decision making it would choose which component is most appropriate to handle the shape of the data. It also means that a component will commonly be just 10 to 20 lines with some choices and a couple of tags and lots of calls to other components. The nesting of components is the primary way we get code re-use, while velocity does have macros they simply didn’t have the flexibility afforded by our components, particularly since macros could not have optional parameters.
This emphasis on lots of small components may seem to be storing up trouble for future as the number of places changes need to be made constantly increase. In fact because the individual components are generally very simple, reading and understanding them is easier. On occasion when there is a lot of complexity it is centralized. We also have a strong emphasis on deleting unused code and components.
Template vs layout
As I mentioned before we had approximately 280 templates, many of these did in-fact correspond to one layout, but not all. Some of our templates particularly ones for the front of the site or sections on the site or used to give special emphasis to a subject had multiple layouts.
The cms did not use wysiwyg editors (and this is a good thing) it did however allow the editors to place hints — when a new column should start, whether to use a special rendering and so on. These hints could then be interpreted by the template and produce an alternative layout. This is an incredibly powerful device and we used it extensively. It was also this hinting system that drove much of the component choosing behavior mentioned above. The other driver of component selection was image size, the component used would change depending on the image width (we almost never used height). This works so well because the Guardian’s grid meant image sizes were entirely predictable.
The distinction between a template and a layout has some interesting repercussions for training. While a one to one relationship between a template and a layout is easily understood, the way our templates reshape themselves depending on the content placed and the hints applied can seem quite mysterious.
When dealing with a site as complex and ambitious as the Guardian’s the way css is structured becomes critical. Writing maintainable css is not a topic that has received much discussion and certainly was not a hot topic in 2006.
The lesson we learnt over the course of r2 are relatively straight forward and easy to follow.
Create a style guide and stick to it
Consistency is important, keep the coding style guide short because otherwise no-one will read it. Make a point enforcing it.
Never use css shorthands
This is probably the most controversial comment but, I stand by it. Shorthands affect the cascade making it harder to interpret. Since properties can often be ordered arbitrarily it makes them impossible to search for. Some shorthands simply aren’t readable.
Don’t use hacks
For the real trouble makers (internet explorer) use conditional comments. For the other browsers the differences are now so small you should not need hacks.
Put css for a particular styling in its own css file
This makes the css modular and means when the styling needs to change it only needs to happen in one place.
Don’t comment out code — delete it
Since having versioning software means all changes are undoable, if you think some code is no longer needed delete it. Even if the removal is temporary make a note in you commit comment about the code needing to come back. Commented out code lingers around, causes confusion and if you are merging the css can lead to odd bugs.
Trust the browser
If the styling is different in Safari or Firefox (especially version 3) it is probably because you don’t understand what is going on
Put one selector per line
This plays nicely with merge tools and so make the css work better in source code management tools. It also makes the code more readable when there are many selectors.
Put one property and value per line and put a space after the colon
Again this works well with merge tools and makes searching the css easier.
Merge your css
This makes using lots of small files work — since you don’t serve the thousands of individual files to the client, but, you can still work this way.
The above list is not dogma it is what we learnt over the course of three years. It is also what makes wrangling 2MB of css possible. It is not to say that this will lead you to perfectly maintainable and readable code, but, it does help.
It is also worth noting that the recurrent bugs, the ones that took longest to solve, were hardest to diagnose and the fixes had the most unexpected consequences were invariably where we didn’t follow these principles.
Markup the good the bad and the unintelligible
Choosing the correct tag for the job is a fundamental part of client side development. It is very easy to invent a new markup language just using divs, spans and classes and ids but is counter productive. Html has slightly less than 100 tags, some are pointless (big anyone?), some are clearly the result of html being designed by computer programmers (var, code, samp, kbd and on …) and some are there to try and trip up the clever. The most notable of these is the definition list (dl, dd and dt). People just seem to want to use this whenever they can and a favorite use is captioning images, why I don’t know.
On the Guardian’s website we tried to follow a very simple system: label things what they are. This works quite well but occasionally you do find yourself wandering what something really is. This can lead to navel-gazing. We applied the naming principle to class names an ids as well. To decide what something was named we looked to the editors and the subs. We followed their naming conventions whenever possible. This was invaluable since it meant we all had the same vocabulary. When a bug was raised or a question asked both parties would be talking about the same thing using the same name. It also meant that if the function of some markup changed (and it does with remarkable regularity) then the tags, their class names and ids had to change also. This may sound like a large overhead but it isn’t, the time saved not having to translate badly chosen labels all the time more than compensates for the additional effort.
The use of the business language to define the labels used in the code is I believe one of the best decisions we made in r2. I would recommend anyone to do the same wherever possible. Often the business names have evolved through long usage and as such provide clarity and consistency not an easy thing to achieve.
The emphasis on the correct use of tags resulted in some interesting observations. For instance a while ago Google published an analysis of the use of tags in web pages. Looking at their numbers we end-up as outliers on the right of the graph. With on average 1500 tags per page and 33–35 different types of tags per page (the most common number being 18). It also gave us a very concrete thing to consider when interviewing candidates.
Invariably we would ask, ‘How would you mark this up?’. The replies would range from the terse, to the absurdly detailed. In interviews and in coding tests I started to notice some strategies around markup, choices made about which tags to use and which property values to use that become something of an alarm. The use of definition lists to put captions on images was one of the most clear signs of someone being too clever by half. The use of numeric values for the font-weight property suggested a lack of pragmatism and a failure to understand how browsers work. I know that Safari 4 and Firefox 3 have started supporting this, but, since most people have only two weights of their fonts installed and that font-weight: 500 doesn’t exactly scream bold, this kind of thing is best avoided. An over enthusiasm for absolute positioning was also a bad sign as absolutely positioned layouts tend to be brittle. Especially when the font size is increased. That is not to say it should never be used, just that, since absolutely positioned blocks are taken out of the flow if the flow changes the consequences are seldom pretty.
Never an easy task. What with trying to name things well, is this a list with only one item or is it actually just a paragraph? Do I sound like I’m about to disappear up my own arse? Well yes, but sometimes getting a name right is important.
So how do you avoid navel-gazing? Well start off with the principle that you can always change it later, then check to see do the users already have a name for this thing, if they do use it. If they don’t does it matter that much? Finally consider some other person coming to your code, will they understand what it is meant to do. This last is a good reason to avoid abbreviations what does vss mean? Who knows.
All this adds up to being careful about the names chosen and being willing to change them. And maybe a little navel-gazing is not such a bad thing in the long run, just, don’t get carried away.
Accessibility is a complex and subtle subject although you might not believe it if you listen to some of the accessibility zealots. I passionately believe web sites should be accessible by all, the part I’m having increasing difficulty with is how this is achieved.
Many of the accessibly guidelines, such as w3c and wcag present no evidence. We are expected to take it as a given that what is in the guidelines is actually true. I’m not convinced that they all are and neither are all the disability action groups. I remain unconvinced that setting the longdesc on an image when no user agent exploits it actually aids accessibility. The blind, sometimes frighteningly passionate, demands for these things I feel causes more harm than good.
Then there is the complexity, much of the accessibility debate revolves around the mythical screen reader user. This person is the poster child, and in some cases, the only subject considered. The emphasis on one disability is not helpful. Users with poor eyesight out number blind users by on order of magnitude (should we not cater for them?), what about motor difficulties? All these questions and precious little debate about what to do to really help and improve a sites accessibility. This becomes even more difficult with a site as large and complex as the Guardian’s.
As to our commenting system — we’re working on it ok?
So that I’m not accused of spouting unsubstantiated rubbish about accessibility advocates, a few links … Accessibility is a harsh mistress or this Why the Alt Attribute May Be Omitted (try reading some of the comments). I could go on, Joe Clark has some good rants about all this too. At the moment many web accessibility advocates are doing themselves and the people they claim to represent a lot of harm by being so extreme.
Solving the ‘but I want this thing there’ problem
This is not an easy one. Early on we realized that the customization of what appears on a page is a very desirable feature. The thing is, often these customizations only make sense when applied across many pages, per page editing is of limited value for certain items, like a weather widget. In some cases this problem is solved by creating a template with the customization required but this is of only limited use since you end up with 300 odd templates and maintaining that many becomes arduous.
For the first section we tackled in r2 (travel) we didn’t consider this problem. Having completed travel it was clear that some kind of mechanism would be required to customize which components appear on a template without requiring developer intervention. The need was particularly acute for our promotional content. Since the need was strongest there and it was generally confined to one area of our templates (the far right column) we devised a system which allowed the editors to make rules as to when a component should appear. This deceptively simple idea has far reaching consequences. It allows customization against many criteria and has been used for the last two years with some success.
As I left we were extending the rules idea to the rest of the template, it is not complete and some of the consequences are hard to predict but it will give an unprecedented level of editorial control over what appears on templates and in what order. It is in fact dangerously close to a fully customizable templating system. I wish them luck because I think this could be a revolutionary idea in how to build templates for content management systems and it might just solve: ‘but I want this thing there’, which would be good.
In the end I suppose like all engineering problems, web sites are an exercise in compromise. The complexity of the templates and the css has a cost in maintenance and a benefit in flexibility. While shiny new features attract developers and journalists alike, keeping things maintainable and durable argue caution and circumspection. If a site needs to last five years should we be making components that depend on flickr and twitter? Maybe the cost of being wrong most of the time is out weighed by being right just occasionally when it comes to new technology …
I have spent three years working on what is now a first class web site and cms, clearly I did not do it on my own. There is a team of incredibly talented people working on the site, just on the client side at the hight of r2 there were nine people, the whole team at the time ran to over 100. I’ve spoken about what the team I was leading did but none of it is possible without everyone involved.
I’ll be presenting a talk at @media about our experiences at the Guardian relating to css and making work reliably and keeping maintainable.
Working at the Guardian
Since January 2007 I’ve been working full time at Guardian unlimited. I also freelance occasionally at songkick.com where I do client side development and encourage good practice.
I have continued the work I started as a freelancer working on the travel site. We have since launched our new network front, Business, Money, Environment, UK news, World news, Politics, Science and Technology sections in the new design and cms.
While much of my work until march last year was focused on writing the templets and making the pages as accessible and standards compliant as possible. After that as the team grew from two to seven and soon to nine most of what I now do is strategic. Deciding which way to solve problems when to try new strategies and what we need to revisit.
One of the things I first noticed about the structure of the pages as they were being designed was if they were not articles (comment, reporting and commercial) they were lists of suggested articles. So one of the defining aspects of the markup is extensive use of lists. Using lists to chunk page content rather than using hundreds of
div tags is not something I’ve seen in many other pages, I may not be looking in the correct place, but, I think this might be a small innovation.
Naturally the separation presentation from content is not perfect. Bits of markup were added to give hooks for the css often this took the form of nested lists, so to achieve a two column layout I would use a list with two
li tags each containing a sub list of article trails. Those trails each having there own hierarchy of headings and paragraphs. Added to this we tried whenever possible to express meaning or intent when choosing
id names, although it turns out naming is hard.
Using lots of lists is not without problems. Even after you remove the browsers default styling there are still browser bugs to cope with. Notably Internet Explorer seems to have particular problems with lists, less so in version seven but still not great (combine that with ie’s quirky way of doing floats and things do get interesting). Firefox also has some problems with lists but they are minor by comparison. Another problem with using extensive nesting while trying not to go mad with class names is ie six’s lack of support for some of the more useful css selectors. This weakness means that when you look at code you will see a lots of
Nesting is a fundamental concept in html and as such it being a source of pain seems odd. If I create a list of lists I’d expect it to work — and at the most basic level it does. But when you choose to style these lists inside lists it all goes a bit to pot. You see ie (alright ie 6 and older siblings) only supports the descendant selector and not the child selector. As a result if if say —
ul.trail li — I’m stuck because now that styling affects all the nested
lis too. There are ways of dealing with this, the two most obvious, are use more class names and set the descendants descendant. Nether of these options is particularly appealing, all the same we did both.
So now what? So now we have a very solid (mostly semantic) html base with css for layout that scales well. Our css is starting to get to the point where it gives the new look and feel reliably and our vocabulary of class names allows us to create variations on the new look rapidly, and interestingly robustly, working cross browser with only a minimum of fuss.
The use of css means, contrary to what many commenters seem to have assumed, our site works quite well on mobile devices. Not that I’ve checked all of them, but, I have checked on Blackberry, Sony Erricson and Nokia phones, only a small subset, but in those and in Opera mini the page does work well.
Since I’ve not freelanced since 2006, none of the sites below, except the Guardian, still contain my work, having been redesigned and redeveloped. I’m going to leave the links though as they keep a record of my freelance work.
I was working at dlkw (now part of dlkw Low) on the halifax website.
I worked at Poke between August and November 2005 and I’ve worked on some interesting projects. I did a fair chunk on the orange entertainment site, although the bulk of the work on the css was already done. I also worked on a small site for SAB Miller which so I can’t really discuss, which is a shame as I am very pleased with the work I did on it.
On Rio Tinto my brief was to move their table based layout to a css layout and to improve the accessibility of the site. I was contracted to view. The final result is much simplified xhtml with semantic mark-up.
Just recently I finished wendysmith.co.uk. The site needed to present the artist work in a sympathetic manner and be easy to maintain. I designed the site and wrote the php, html, and css. The current version of the site is no longer my work.
Working as a freelance contractor for wheel:. The work was developing html and css for three clients: BT, Ernest Jones and H Samuel. The most important work I have done was on the new Unilever design.
The BT work was coding css for the new look on the BT small & medium business pages.
The Ernest Jones work was updating and adding products to their standard templates as was the H Samuel work. This mostly required changing or creating graphics and adapting their css to the new look. Unfortunately, I can’t point individual pages as they are a moving target. The last thing I worked on was the Bridget Jones promotion.
On Unilever I wrote most of the css and many of the html mock-ups. Obviously on a project this size many people are involved so I was just a small cog in the system. That said, I am very proud of the work I did on the Unilever css. The site has been redesigned and is no longer my work.
Training for Life
At Training for Life I worked on web design and development. The sites I worked on were usually for charities and small to medium businesses. My role was lead css and html developer as well as doing php development.
The largest piece of work I did was developing an intranet application for the NHS Confederation. It was implemented in php with mySQL as the database back end. If you are interested in looking at it please contact me so I can give you a url and a user name and password to view a demonstration installation.
Opportunities Child (no longer with us) I wrote the asp that connected to the access database for the site’s content. I also wrote the css. Again the layout is pure css.
Urban Inclusion we developed the design from their existing branding. I implemented the design, wrote most of the css and did the accessibility work. The site no longer users my work.
16 Hoxton Square (gone) & Hoxton Apprentice were developed for Training for Life as they are one of the partners in the scheme. On this I did quality control and cross browser testing.
At dct I did two things, I taught long term unemployed people computer applications, html and css and I maintained dct’s intranet.
From the teaching I learnt just how counterintuitive most computer applications are and saw the impact of poor design on people. It also gave me a ready pool of volunteers to use for testing designs (it can be quite humbling).