Here’s part 1 of this blog series in case you haven’t read it yet. I suggest reading it first to get a good look and feel of what a heuristic evaluation is.
I guess by this time you’ve already prepared the materials or even so, the people to do your evaluation. Before we start, I want to be perfectly clear that a heuristic evaluation can be a subjective form of an assessment. Remember, a number 2 score for another person might be a number 1 score for another. I could not emphasize enough to do a usability test to complement and debunk some of the issues you’ve encountered during your evaluation.
Conducting the Evaluation
There are two ways to conduct the evaluation, an individual evaluation or a group evaluation. I did this evaluation by myself so I have to go with an individual evaluation. Tune in on part 3 of our blog post to learn how to do a group evaluation with your team.
Individual Evaluation – each reviewer will go through your website by themselves. After they are done with the evaluation all the issues seen would be reported to you. You can choose to brainstorm with the other evaluators after they are done or just go through the list of issues by yourself. Conducting a brainstorm together is better though, you guys can dissect and prioritize issues together.
- Can be done remotely
- Allows reviewers to focus on their tasks
- Easier to facilitate, you basically brief them and get your results after
- A lot more issues will be reported
- May have a longer turnaround time
Group Evaluation – evaluators review the interface with the team while one person records all the problems. This person can be you or somebody else. The key here is that evaluators must assess the problem together and its severity. They don’t have to agree on the same problem, you guys can vote or leave it to the usability test to see which should be prioritized.
- One time facilitation
- Real time discussion
- Issues are prioritized on the spot
- Harder to prepare and schedule
Starting the individual evaluation
Start the heuristic evaluation by briefing your evaluators. Relay the goals and target market of the website and the main tasks that they have to go through. If your reviewer is not a usability expert, an explanation of the heuristics must be done. Since we have a heuristic dedicated to the home page, I usually start the evaluation there then proceed to the main tasks that I have done.
Here’s a short breakdown of my methodology:
- Find out who the website is for, what is its purpose or goal from the facilitator
- Receive tasks that I have to do from the facilitator (If you plan to do a usability test after the heuristic evaluation, the tasks for the usability test should be the same)
- Start the evaluation with the assessment of the home page
- Carry out my tasks while taking note of issues and problems I’ve encountered. At this point I’ll start giving a score on my heuristics.
Once I’m done carrying out my tasks and completed my evaluation. I’ll end up with an overall usability score of the website and a radar chart as well. I would then proceed to the next website I would assess.
I don’t know if this heuristic applies to the website
Hey, no worries. Not all heuristics are going to be used in the assessment. Simply skip that question and leave it blank.
I can’t do the task while giving scores on my heuristics
This is basically the problem I had when starting out. Just take it slow and do the tasks that are assigned to you. If you have to go over them again, do so. One trick I would do is to see where the problem I’ve found can be applied. For example Information Architecture, find a specific question tailored to that problem from your excel sheet. If there’s none, take note of it and discuss it with your facilitator and team.
I’ve done all the tasks but have a lot more heuristics to cover
Sometimes there are tasks that won’t have any forms involved; however you have it as one of your heuristics. What you can do is look for a form, if they have one, then assess it.
Can the facilitator be an evaluator as well?
Yes, sometimes the facilitator is just a member of your team or maybe a person who will lead the discussion later on.
Analysing your results
Once every one of your evaluator is done, you can start by reviewing each heuristic individually. Start at the home page and work your way down. Remove any similar or duplicate comments seen by the evaluators if you have more than one evaluator. You can do this by yourself or choose to do a brainstorming session with your evaluators after they are done. Next, tally the overall score of your website. Here’s the formula in case you need it.
Sum of evaluator scores / (100)(# of evaluators) = A
A x 100 = Average Score
Evaluator 1: 70% ; Evaluator 2: 55%; Evaluator 3: 60%
70% + 55% + 60% = 185%
(100)(3) = 300%
185%/300% = 0.61
0.61 x 100 = 61% Average Score
So I guess you’re thinking what does the score I’ve tallied mean? Is this a good score or a bad score? Ideally a website should at least get a 75% Usability Score. Each heuristic should at least have a 75% score as well to keep it balanced, something like the table below.
Here’s a sample analysis of a website as well.
This website has a fairly high score for the home page with a total of a 70% score. Navigation areas on the home page are not over-formatted and users will not mistake them for advertisements however text on the mega menu does need improvement. Excessive use of scripts, applets, movies, audio files, graphics and images has been avoided which means no banner blindness for the users. The site also avoids advertisements, especially pop-ups that may put off other users.
Referring to the radar chart, we could clearly see that Task Orientation should be improved first, garnering a percentile score of 36% which is the lowest. A number of common tasks are hard to accomplish and new windows are used to present other data. Forms are not validated before being submitted and some writing are too short to be informative. Overall the website needs improvement on all heuristics.
If you’ve evaluated a single website only, you can skip this part. But hey, a little more knowledge is better than none.
To emphasize each heuristic you have calculated you can overlay one radar plot on another.
See, now you have a clearer view of the results of each website.
As you can see all of them have strong and weak points to their website. All three websites have a relatively high score for their home page. Writing and content quality has the most similar figures out of all heuristics with search being the lowest for all three websites.
Seeing that you’ve pinpointed your competitor’s strengths and weaknesses, it is now up to you how to use this to your advantage.
Ok, now I have my score. What do I do next?
You can start by prioritizing which problems to tackle with your team. Since you have your excel sheet full of notes and scores, you simply have to segregate your data and fix the problems with the lowest value first. In my case the value (-1) is the lowest value on our scoring system.
Maybe now you’re thinking, “But there a lot of -1s in my scores, how do I know which to fix first?” Remember, it’s always easier to do quick fixes first. For example, things like copy changes and colours of call to action buttons. By doing quick fixes first you will immediately minimize the severity of the issues and it might even solve the problem completely. Consult with your team and see what you guys can do immediately.
Wait, there’s more!
Since we love you guys so much, Part 3 of our blog post would be how to conduct a group heuristic evaluation. We’ll walk you through the whole process and show you guys the difference of results with a group vs. an individual evaluation.
For questions just comment down below!
Good luck guys!
Danino, N. (2001, September). Heuristic Evaluation – a Step By Step Guide Article. Retrieved from http://www.sitepoint.com/heuristic-evaluation-guide/ – See more at: http://blog.foolprooflabs.com/2013/11/beginners-guide-heuristic-evaluation-part-1/#sthash.c50AyDyV.dpuf