In this blog series we’ll explore how heuristic evaluation can help you improve your website through concrete scores and data visualizations. Before we actually teach and walk you through the process of evaluation, we first must learn how to:
- Create heuristics
- Score your findings
- Visualize your scores
- Create tasks.
Starting your first website evaluation can be a doozy
I remember starting my first usability audit in the office and I can tell you it was not a walk in the park. Usability best practices were not in my UX toolkit yet and not knowing whether was I right or wrong is the worst part. This is where I realized that a structured review methodology can be a crucial arsenal in your toolkit, especially for first timers. Keeping things formal, organized and consistent will make our evaluation faster and consistent in the future. Luckily there are loads of online resources today that we can use.
What is a heuristic evaluation?
A heuristic evaluation is a review methodology or discount usability method for a quick, cheap and informative way of assessing your interface. A set of heuristics or general rules of thumb, are used to critique and score your website. Radar diagrams are also used to visualize weaknesses in the system. At the end of each heuristic evaluation you end up with an overall usability score of your website and a list of things you can improve on.
- Fast feedback for designers
- Feedback can be obtained early in the design process
- Inexpensive compared to other methods: this depends whether you get experts to assess your website
- Faster turnaround time than usability testing
Although Heuristic Evaluation provides quick and relatively cheap feedback to designers, it is best to be aware of some of the drawbacks that it has.
- Heuristic Evaluations will miss certain issues: This type of methodology is not meant to be used alone. It works best when paired with usability testing, by doing so you can validate your findings and determine other issues not seen during the evaluation.
- Heuristic Evaluations tend to uncover a lot more issues: The question is how do you prioritize which issues should be fixed first? Don’t worry we’ll discuss this on part 2 of the blog post.
- Heuristic Evaluations are more fruitful with experts: This is due to the fact that experts apply their years of experience to the evaluation.
- Not all heuristics will apply to you: It is important to know that some heuristics may not apply to your product. There are multiple heuristics sets available online which you can use depending on your project.
Who can perform a heuristic evaluation?
Anybody, as long as you have a set of reliable heuristics with you. Although three to five experts provides better results, assessing your website with a set of heuristics is better than doing nothing. The key is to have more than one evaluator (3 to 5) to assess your website, by doing so you can have a direct comparison of which are false alarms and which issues are reoccurring. Remember, the goal is to find issues you can fix, rather than how many problems you can uncover.
- 5 experts will uncover 75% of usability issues
- 5 non-experts will uncover 22 – 29% of usability issues
When is the right time to perform a heuristic evaluation?
It can be conducted at various stages of the development lifecycle, preferably at the start in order to determine issues early on in the project.
Heuristic evaluation can also be used to compare performance of competitors. This is particularly good in determining which aspects competitors fail in, and how might we avoid these failures in the future. But of course comparing multiple websites can be tedious and time consuming; this is why choosing the correct heuristics is crucial.
Before you start your evaluation
What you need:
- Three to Five evaluators: If you have the resources to hire a usability expert that would be awesome!
- A website to assess
- A set of heuristics: What we use is from User Focus’ Web Usability Guidelines, with a minor tweak. We included accessibility as a heuristic as well. All you need to do is include your rating and the excel sheet automatically calculates your score and generates a radar diagram. Thanks so much Dr. Travis for this!
- Tasks for your evaluators
- Pen and Paper: You never know what you’ll want to write down.
Creating or defining your heuristics
In order to properly review your website, all facets and layers should be evaluated. That includes your design, down to your functionality. For beginners you can download a set of heuristics that are available online. Here’s a few of my favourites:
- Jakob Nielsen’s 10 Heuristics: This of course is the most famous one.
- ISO 9241-110
- Dr. Peter Meyers’s 25-point checklist
- User Focus’ Web Usability Guidelines
Now getting back on track, as mentioned previously we use UserFocus’ set of heuristics which are:
- Home Page
- Task orientation and website functionality
- Navigation and information architecture
- Forms and data entry
- Trust and credibility
- Quality of writing and content
- Help, feedback and error tolerance
- Page layout and visual design
From these categories we can further slice them into specific questions. This allows us to create a formal checklist before assessing our website. One of the key advantages of a checklist is to make sure that nothing is forgotten.
Let’s use the heuristic, Task orientation and website functionality. Some questions you can form for this heuristic are:
- From only looking at the home page, does the first time user understand where to start?
- Is the value proposition clearly stated on the home page?
- Does the design of the home page encourage people to explore the site?
How to score your heuristics
Your heuristics can be scored in many ways, but the simplest we can do is to follow Leigh Howell’s scoring system from Smashing Magazine. Your scoring metric will go like this:
- 0 points: does not comply with the heuristic
- 1 point: kind of complies with the heuristic (ex. Some forms have error prompts but other pages don’t)
- 2 points: complies with the heuristic on all pages
All points will then be totalled per heuristic and an overall score will be derived at the end. Your formula would be like this.
(Actual Score / Possible Score) * 100 = Total
Home Page Heuristic: (38/40) * 100 = 95%
Overall score: (465/484) * 100 = 96.07%
Visualizing your scores
A radar chart is used to immediately pinpoint; visually, which parts of the website needs improvement without referring to to the Heuristic Table that has all numbered values. The more circular and larger the shape is, the better it performs. The more spikes seen in the shape, meaning more variation of scores. Take note that you read the points of the radar not the slices.
Radar charts are not for everybody. If you feel that using a different type of chart is better suited to present your data, feel free to do so. In our case, we find that radar charts is what fits best in our evaluations, especially for comparative reviews.
Please also be mindful that what’s important here is to identify the problems within your website and to have an actual usability standard on your digital products in all aspects (Navigation, Page Layout, Search, etc).The scoring is important but the goal here is finding the issues you can fix and applying the guidelines you have with you in a effective and much organized manner.
Since excel sheets are awesome, a radar chart or just about any type of chart is automatically generated for you! Just click the insert button and choose a chart to your liking. You can then go back to your data sheet or heuristic table to determine which issues you shall address first based on the chart.
Creating Tasks for your evaluators
The quality of the evaluation can be improved by providing specific tasks or scenarios for your evaluators to complete. The first step is to jot down a list of the most important tasks that people need to be able to do on your site. You can do this together with your evaluators; you can even gather some insights from the site owner as well.
Here are some tasks you might end up with:
- Sign up on my website.
- Place an order on the website by using Paypal.
The number of tasks will be up to you, but usually five major tasks is sufficient . Take note that more tasks will also mean more time evaluating.
A heuristic evaluation provides users with a concrete methodology to follow, both for beginners and expert UX Professionals. All facets and layers of the website shall be assessed from design to functionality. The scoring and data visualization immediately helps us create an informed decision in our design process when other methods are not appropriate.
So far so good! View part 2 of this article to find out step-by-step on how to perform an individual heuristic evaluation on websites.
If you guys want a tutorial on how to create your own excel sheet that automatically generate scores and radar diagrams, comment on this post below and I’ll create a tutorial on it.
Good luck guys! 🙂
Danino, N. (2001, September). Heuristic Evaluation – a Step By Step Guide Article. Retrieved from http://www.sitepoint.com/heuristic-evaluation-guide/
Ganapathy, D. (2013, July). Quick Research / Usability Methods: Heuristic Evaluation. Retrieved from http://productnation.in/quick-research-usability-methods-heuristic-evaluation/
Howell’s, L. (2011, December). A Guide to Heuristic Website Reviews. Retrieved from http://uxdesign.smashingmagazine.com/2011/12/16/guide-heuristic-website-reviews/#comments
Kaushik, A. (2010, August). Qualitative Web Analytics: Heuristic Evaluations Rock! Retrieved from http://www.kaushik.net/avinash/qualitative-web-analytics-expert-heuristic-evaluations/
Nielsen, J. (1995, January). How to Conduct a Heuristic Evaluation. Retrieved from http://www.nngroup.com/articles/how-to-conduct-a-heuristic-evaluation/
Sauro, J. (2010, August). 6 Things You Didn’t Know About Heuristic Evaluations. Retrieved from http://www.measuringusability.com/blog/he.php
Sauro, J. (2012, September). How Effective Are Heuristic Evaluations? Retrieved from http://www.measuringusability.com/blog/effective-he.php
Travis, D. (2009, July). 247 Web Usability Guidelines. Retrieved from www.userfocus.co.uk/resources/guidelines.html
Usability Government. (n.d.) Heuristic Evaluations and Expert Reviews. Retrieved from http://www.usability.gov/how-to-and-tools/methods/heuristic-evaluation.html