Evaluating CMS usability: a checklist
by Michael Kowalski, 23 Sept 2002
This checklist aims to give you some help evaluating CMS usability prior to purchase. It's not exhaustive, but if you find a product is failing on a significant number of these tests, alarm bells should start ringing.
The appropriate level of simplicity depends on the user role. Assuming the largest part of the CMS lifecycle will be spent authoring and deploying content (rather than, say, designing templates), you should try and ensure that these tasks in particular are made as simple as they can be.
Can you perform simple tasks without training? If not, can you discover how to perform them without consulting the vendor?
Bearing in mind differing user roles, is the interface as simple as it could be? For content authors, is it cluttered and overly technical?
Does the interface unnecessarily expose underlying architecture (eg. XML, DTDs)?
Does the interface open a confusing number of windows?
Are error messages easily understood? Do they suggest a course of action?
Are commonly used tasks separated out from infrequently used ones? For example, toolbars may become cluttered and difficult to navigate if they include all options: less frequently used commands might be better off confined to a right-click menu or an "Advanced" tab. Some CMS interfaces (particularly, I would say, open source ones) are designed by geeks for geeks. Which is fine if that describes your content authors...
The CMS will be more familiar to users if it looks and behaves like other applications (including the desktop) that users already understand. This has great benefits for reducing training costs and increasing productivity.
Does the interface look generally like other applications?
Does the interface behave like other applications? For example, do right-clicks, double-clicks, tooltips, and multiple selection work as you would expect?
Does the interface use language familiar to the user or introduce its own jargon?
Does the interface use standard UI components (menus, checkboxes, buttons) in non-standard ways? Does it unnecessarily introduce new UI components (common in Flash interfaces)? A surprising number of CMS interfaces I've seen attach a row of little tool icons to every entry in a list of documents, rather than having a single toolbar (which is both more familiar and simpler). Watch out for this one!
Again, the interface should aim not only to be internally consistent, but consistent with other applications the user might be familiar with.
Does the interface have a consistent look-and-feel?
Does it have a consistent labelling scheme? Does it always call the same thing by the same name?
Is interface behaviour consistent? eg. does double-clicking have a similar effect on every screen?
The interface should give the user unambiguous visual cues about what they can do, what they can't do and what they are doing.
Is the current status of content obvious at a glance?
Are disabled features greyed out or otherwise marked as disabled?
Do toolbar icons have text labels? It's better not to rely solely on tooltips to explain the function of icons.
Does the interface look frozen when it is carrying out a lengthy background process, or is there appropriate visual feedback (a "thermometer" progress bar or a spinning arrow)?
Is it clear when a task has been completed?
Are important navigation options hidden in dropdown menus? Top-level navigation should always be visible without mouse action.
User wait time should be minimised. This problem is generally more acute for web interfaces than for desktop applications because of the inherent latency of network traffic, but can be minimised with careful design.
Is the interface slow to respond to user input? For example, does the entire page reload whenever the user applies a change?
Does it take a high number of mouse clicks to complete common tasks?
Everybody makes mistakes. A good interface tries to prevent errors from happening, and lets you recover from them quickly and easily when they do.
Is there an undo facility?
Are you asked to confirm "destructive" actions like deleting content?
Are destructive commands placed right next to non-destructive ones? eg. the "Delete" button next to the "Save" button
How well does the interface cope with unexpected user action? For example, what happens if you double-click where you should be single-clicking?
Chances are a demo installation won't have much content in it. You need to think about what it will look like six months down the line, when the volume of content may have increased an order of magnitude or more.
How well will the interface scale as the volume of content goes up? Will you end up with very long scrolling lists that are slow to load and difficult to navigate?
Is there any way to quickly navigate to a specific item?
Can you view filtered lists of content items easily?
Can you select multiple items for processing together?
Are sensible default values supplied? Where appropriate, does the interface remember values you have entered previously or do you find yourself re-entering the same values over and over again?