Usability is often overlooked during CMS evaluation, because it can be difficult to evaluate. Yet usability is a key factor contributing to project success, cost of ownership and user satisfaction.
Some years ago I joined the web development team at the Guardian newspaper just as they were about to launch their new network of websites under the "Guardian Unlimited" brand. The content management solution was built on a Vignette StoryServer platform: our team developed a suite of editorial tools that allowed journalists to input and edit content.
The core of the Guardian's content offering naturally originated with articles written for the newspaper. Every night articles were extracted from Quark Xpress and piped into the CMS, where a team of web staff set about proofing, cross-linking, repurposing and otherwise readying them for web publication.
The network launched not too far behind schedule, and to great success, soon picking up a slew of awards and one of the largest audiences of any UK news site. But behind the scenes there was a problem. The web production process was so convoluted and time-consuming that the web edition wasn't able to launch until 3am at the earliest. There was a cost to this in cash terms—overtime fees to staff—but there was also a cost in terms of staff morale: many of the editorial staff became frankly rebellious.
The problem: poor usability. The editorial tools had been constructed by a disparate group of developers in whatever idiosyncratic style they liked. There was no consistency, and there had been no significant user testing.
Without changing the underlying architecture, I redeveloped the user interface, giving it a consistent look-and-feel, consistent labelling and behaviour, and replacing cumbersome forms-based interfaces with WYSIWYG tools. With these usability improvements, it became possible to publish the online edition of the newspaper by midnight.
Benefits of usability
Usability brings significant and tangible benefits to a content management system.
There is a direct impact on cost of ownership. Strong usability:
• reduces learning curves and therefore staff costs,
• reduces help desk support costs, and
• increases staff productivity by minimising the time it takes them to perform their jobs.
Usability enables distributed authorship, one of the promises of content management. The more usable the CMS, the greater the chance you can give people throughout your organisation direct control over their own content, or even source content from outside your organisation.
Finally, good usability reduces user hostility to change. Rolling out a CMS usually involves a fair amount of organisational change, and inevitably some users will resent abandoning their traditional way of working. You can't solve this problem with usability, but you can mitigate it.
Poor usability is common in CMS
With all these clear benefits, you would expect usability to be one of the key drivers motivating adoption of a content management solution. Here's the bad news: the standard of usability of content management systems is generally very poor.
Why is that? There are two key reasons.
CMS usability is hard
First, CMS usability is hard. Content management systems are inherently complex applications—in fact, they are more like suites of applications, taking in a wide range of tasks covering everything from user administration and template design to content authoring and deployment. They are used by a wide range of users with different roles, some technical, some not. And they have to make large amounts of content of various kinds both manageable and navigable.
Also, many CMS products have web-based interfaces, both to fulfil the objective of "any time, any place" authoring, and to reduce desktop support costs. But web interfaces are immature and present particularly difficult challenges to application designers because of the reduced functionality available to applications running inside the browser. And web interfaces are evolving rapidly. Some CMS products will have evolved from "traditional"—and cumbersome—forms-based interfaces.
CMS usability is difficult to evaluate
Second, evaluating usability is difficult. Buyers often have little idea how to go about testing it.
• You might find you have limited opportunity to give the product a proper test drive
• You can easily be overwhelmed by the extensive feature set of most CMS products
• You have to consider usability from the perspective of a range of different roles
• The metrics to use in CMS usability evaluation are not clear cut.
The difficulty of evaluating CMS usability means vendors can get away with avoiding the difficulty of implementing good usability in the first place.
Even an entirely web-based CMS is not a website, it's an application. The basic principle of usability design remains the same however: it's all about respecting the user experience. This means paying attention to things like:
• Error tolerance
By familiarity I mean that the interface should follow standard user interface conventions where possible. It ought to look and behave like other applications (or operating systems) that users already understand.
Performing an evaluation
Make usability evaluation a separate exercise from evaluating the product feature set and its technical suitability for your requirements.
Don't settle for a demo. Before a product even makes your long list, make sure you've taken it for a test drive, with your hands on the wheel, not the vendor's.
It's unlikely that the content types defined in a demo installation will match your own content. Nevertheless, try and work through real use cases from your organisation, rather than testing product features in a general way.
Think about the basic principles of good interface usability I outlined above and consider how well the product conforms with them.
Ask yourself: can you perform simple tasks without training? If not, can you discover how to perform them without consulting the vendor?
Ask for references from other customers and follow them up.
On a project of any size, you should if at all possible run a pilot.