A Beginner’s Guide to Heuristic Evaluation (Part 1)

Introduction

In this blog series we’ll explore how heuristic evaluation can help you improve your website through concrete scores and data visualizations. Before we actually teach and walk you through the process of evaluation, we first must learn how to:

  1. Create heuristics
  2. Score your findings
  3. Visualize your scores
  4. Create tasks.
Starting your first website evaluation can be a doozy

I remember starting my first usability audit in the office and I can tell you it was not a walk in the park. Usability best practices were not in my UX toolkit yet and not knowing whether was I right or wrong is the worst part. This is where I realized that a structured review methodology can be a crucial arsenal in your toolkit, especially for first timers. Keeping things formal, organized and consistent will make our evaluation faster and consistent in the future. Luckily there are loads of online resources today that we can use.

 

What is a heuristic evaluation?

A heuristic evaluation is a review methodology or discount usability method for a quick, cheap and informative way of assessing your interface. A set of heuristics or general rules of thumb, are used to critique and score your website. Radar diagrams are also used to visualize weaknesses in the system. At the end of each heuristic evaluation you end up with an overall usability score of your website and a list of things you can improve on.

Advantages

  • Fast feedback for designers
  • Feedback can be obtained early in the design process
  • Inexpensive compared to other methods: this depends whether you get experts to assess your website
  • Faster turnaround time than usability testing
Disadvantages

Although Heuristic Evaluation provides quick and relatively cheap feedback to designers, it is best to be aware of some of the drawbacks that it has.

  • Heuristic Evaluations will miss certain issues: This type of methodology is not meant to be used alone. It works best when paired with usability testing, by doing so you can validate your findings and determine other issues not seen during the evaluation.
  • Heuristic Evaluations tend to uncover a lot more issues: The question is how do you prioritize which issues should be fixed first? Don’t worry we’ll discuss this on part 2 of the blog post.
  • Heuristic Evaluations are more fruitful with experts: This is due to the fact that experts apply their years of experience to the evaluation.
  • Not all heuristics will apply to you: It is important to know that some heuristics may not apply to your product. There are multiple heuristics sets available online which you can use depending on your project.
Who can perform a heuristic evaluation?

Anybody, as long as you have a set of reliable heuristics with you. Although three to five experts provides better results, assessing your website with a set of heuristics is better than doing nothing. The key is to have more than one evaluator (3 to 5) to assess your website, by doing so you can have a direct comparison of which are false alarms and which issues are reoccurring. Remember, the goal is to find issues you can fix, rather than how many problems you can uncover.

  • 5 experts will uncover 75% of usability issues
  • 5 non-experts will uncover 22 – 29% of usability issues
When is the right time to perform a heuristic evaluation?

It can be conducted at various stages of the development lifecycle, preferably at the start in order to determine issues early on in the project.

Heuristic evaluation can also be used to compare performance of competitors. This is particularly good in determining which aspects  competitors fail in, and how might we avoid these failures in the future. But of course comparing multiple websites can be tedious and time consuming; this is why choosing the correct heuristics is crucial.

 

Before you start your evaluation

What you need:
  1. Three to Five evaluators: If you have the resources to hire a usability expert that would be awesome!
  2. A website to assess
  3. A set of heuristics: What we use is from User Focus’ Web Usability Guidelines, with a minor tweak. We included accessibility as a heuristic as well. All you need to do is include your rating and the excel sheet automatically calculates your score and generates a radar diagram. Thanks so much Dr. Travis for this!
  4. Tasks for your evaluators
  5. Pen and Paper: You never know what you’ll want to write down.
Creating or defining your heuristics

In order to properly review your website, all facets and layers should be evaluated. That includes your design, down to your functionality. For beginners you can download a set of heuristics that are available online. Here’s a few of my favourites:

Now getting back on track, as mentioned previously we use UserFocus’ set of heuristics which are:

  • Home Page
  • Task orientation and website functionality
  • Navigation and information architecture
  • Forms and data entry
  • Trust and credibility
  • Quality of writing and content
  • Search
  • Help, feedback and error tolerance
  • Page layout and visual design
  • Accessibility

From these categories we can further slice them into specific questions. This allows us to create a formal checklist before assessing our website. One of the key advantages of a checklist is to make sure that nothing is forgotten.

Let’s use the heuristic, Task orientation and website functionality. Some questions you can form for this heuristic are:

  • From only looking at the home page, does the first time user understand where to start?
  • Is the value proposition clearly stated on the home page?
  • Does the design of the home page encourage people to explore the site?
How to score your heuristics

Your heuristics can be scored in many ways, but the simplest we can do is to follow Leigh Howell’s scoring system from Smashing Magazine. Your scoring metric will go like this:

  • 0 points: does not comply with the heuristic
  • 1 point: kind of complies with the heuristic (ex. Some forms have error prompts but other pages don’t)
  • 2 points: complies with the heuristic on all pages

All points will then be totalled per heuristic and an overall score will be derived at the end. Your formula would be like this.

(Actual Score / Possible Score) * 100 = Total

For Example:

Home Page Heuristic: (38/40) * 100 = 95%

Overall score: (465/484) * 100 = 96.07%

Heuristic Data Sheet

The above table shows total heuristic scores per category. It also includes the overall score of the website.

Home Page Heuristic Data Sheet

The above table shows the total score for the home page heuristic along with the possible score (total points that can be reached) and actual score (total points the evaluator has given).

Visualizing your scores

A radar chart is used to immediately pinpoint; visually, which parts of the website needs improvement without referring to to the Heuristic Table that has all numbered values. The more circular and larger the shape is, the better it performs. The more spikes seen in the shape, meaning more variation of scores. Take note that you read the points of the radar not the slices.

Radar charts are not for everybody. If you feel that using a different type of chart is better suited to present your data, feel free to do so.  In our case, we find that radar charts is what fits best in our evaluations, especially for  comparative reviews.

Please also be mindful that what’s important here is to identify the problems within your website and to have an actual usability standard on your digital products in all aspects (Navigation, Page Layout, Search, etc).The scoring is important but the goal here is finding the issues you can fix and applying the guidelines you have with you in a effective and much organized manner.

Since excel sheets are awesome, a radar chart or just about any type of chart is automatically generated for you! Just click the insert button and choose a chart to your liking. You can then go back to your data sheet or heuristic table to determine which issues you shall address first based on the chart.

Radar Diagram

A radar diagram of a website that performs well in all areas except “Accessibility.”

Creating Tasks for your evaluators

The quality of the evaluation can be improved by providing specific tasks or scenarios for your evaluators to complete. The first step is to jot down a list of the most important tasks that people need to be able to do on your site. You can do this together with your evaluators; you can even gather some insights from the site owner as well.

Here are some tasks you might end up with:

  • Sign up on my website.
  • Place an order on the website by using Paypal.

The number of tasks will be up to you, but usually five major tasks is sufficient . Take note that more tasks will also mean more time evaluating.

Conclusion

A heuristic evaluation provides users with a concrete methodology to follow, both for beginners and expert UX Professionals. All facets and layers of the website shall be assessed from design to functionality. The scoring and data visualization immediately helps us create an informed decision in our design process when other methods are not appropriate.

Stay Tuned!

So far so good! View part 2 of this article to find out step-by-step on how to perform an individual heuristic evaluation on websites.

A Beginner’s Guide to Heuristic Evaluation (Part 2)

If you guys want a tutorial on how to create your own excel sheet that automatically generate scores and radar diagrams, comment on this post below and I’ll create a tutorial on it.

Good luck guys! 🙂

References:

Danino, N. (2001, September). Heuristic Evaluation – a Step By Step Guide Article. Retrieved from http://www.sitepoint.com/heuristic-evaluation-guide/

Ganapathy, D. (2013, July). Quick Research / Usability Methods: Heuristic Evaluation. Retrieved from http://productnation.in/quick-research-usability-methods-heuristic-evaluation/

Howell’s, L. (2011, December). A Guide to Heuristic Website Reviews. Retrieved from http://uxdesign.smashingmagazine.com/2011/12/16/guide-heuristic-website-reviews/#comments

Kaushik, A. (2010, August). Qualitative Web Analytics: Heuristic Evaluations Rock! Retrieved from http://www.kaushik.net/avinash/qualitative-web-analytics-expert-heuristic-evaluations/

Nielsen, J. (1995, January). How to Conduct a Heuristic Evaluation. Retrieved from http://www.nngroup.com/articles/how-to-conduct-a-heuristic-evaluation/

Sauro, J. (2010, August). 6 Things You Didn’t Know About Heuristic Evaluations. Retrieved from http://www.measuringusability.com/blog/he.php

Sauro, J. (2012, September). How Effective Are Heuristic Evaluations? Retrieved from http://www.measuringusability.com/blog/effective-he.php

Travis, D. (2009, July). 247 Web Usability Guidelines. Retrieved from www.userfocus.co.uk/resources/guidelines.html‎

Usability Government. (n.d.) Heuristic Evaluations and Expert Reviews. Retrieved from http://www.usability.gov/how-to-and-tools/methods/heuristic-evaluation.html



About Christine Balatbat

Have you ever used a website or an application and got confused or frustrated? My job is to make sure that doesn't happen. People sometimes call us User Experience Designers, usability analysts or some other fancy word, but we pretty much do the same thing - make people's lives better. My goal is to share UX to the world, to debunk common myths about it and to inspire our future generations to innovate. You may sometimes see me talking about it. Don't be afraid to say hi! :)

5 thoughts on “A Beginner’s Guide to Heuristic Evaluation (Part 1)

  1. Fish says:

    Hi Christine! I’m so happy to have come across this article! I admit that I was kind of lost at first on how to go about evaluating a website but this somehow put structure on my evaluation process. Thanks so much for sharing this. It was a big help. 🙂

    • Hi Fish!

      I’m so glad that you’ve found my article helpful. Everything can be very overwhelming, especially if you are new to this field. If you have any questions or suggestions, don’t be afraid to say so. This is just one of many blog posts we will be creating. Good luck with your evaluation! I know you’ll do good. 🙂

  2. great..I learned alot from this

  3. eeklipzz says:

    Thanks so much for putting this together. It was very insightful. A good read for uxers of all ages. Recently I conducted a UX Assessment on a register system for a company. In reading your article, I find myself wishing I had read it a couple days ago on some parts. For the most part, I think I was inline with what you were saying above.

    I’ve documented my method for creating a ux assessment using Google Slides, which you may find an interesting read. I’d welcome your feedback…

    Creating a UX Assessment
    Thanks so much for sharing this information on. I think it’s also important to mention that this is a tool that allows us to find a bulk of the usability focal points. If the business is interested in the most accurate gauge of their design, then it’s probably going to need some usability testing next.

    I’ve created a how-to on creating a ux assessment that readers of this article may also find of value. I’m interested in getting your feedback as well. Thanks for the info.

    Creating a UX Assessment
    http://uxfindings.blogspot.com/2014/06/creating-ux-assessment.html

    • I’m glad you found my article helpful and I do agree that a more accurate gauge would be a usability test since Heuristic evaluations is a subjective form of assessment. Your blog post is pretty helpful as well! 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *


*