Automatic Usability Evaluation Applications
I was fortunate enough to be a part of a discussion last week on LinkedIn about opinions around using usability evaluation tools as well as recommendations on what others have used successfully on client projects. If you want to simply access my bookmarks here. My immediate response was this:
—————
Automatic usability evaluation applications have their place in the testing and improvement of usability processes. However, to date there is no tool at hand that can effectively mirror human interaction. And for that matter, cognitive sciences still have not succeeded in determining exactly what the human abilities are. So when one approaches engaging these applications, they should consider the limitations and know that there is absolutely no magic bullet.
We have most certainly come a long, long way in anticipating the responses based on proven theories in design, form, function, typography, color theory, etc. But no tool can absolutely identify what the human mind will do in all conditions. Consider the process behind how humans learn from childhood. Thinking/learning through doing is the way we learned how to walk, talk and ride a bicycle. General understanding of the environment around us is a product of previous interactions.
In this respect, one might say that even another human may not be the best form of evaluating a human. We’re too unique.
To a certain degree, this is where having the assistance of tools like:
may be advantageous. While it would depend on the application at hand, most are built for a critical mass in mind. If your product is built within usability standards and an understanding of your audience in mind, you can use these tools to assist in your evaluations – alongside humans not in place of.
—————
Your best results are going to come from garnering a blend of qualitative and quantitative data. Ania Rodriguez, Customer Experience Expert, President Key Lime Interactive recommends to “clients that they start with a remote moderated study to uncover site critical opportunities and then run an automated study with a large N to get statistically significant data to support their decisions. The patterns that emerge from the remote moderated test are very insightful, but without the statistical validity of the data, the findings are just not as powerful.”
Ania also shared her insight for the real players in the automated usability evaluation space that offer much more then just observing what the users did. They offer more powerful evaluation through asking pre and post task questions. These tools include:
Bottom line. Understand the strengths of the tools. Understand they are to be used to compliment research uncovered from live methods such as focus groups and not as a replacement method.
