One challenge of survey design is whether to:<\/p>\n
One of the best-known usability questionnaires is SUS. Is it good enough?<\/p>\n
I’m going to start by mentioning the advantages and disadvantages of reusing questionnaires, and then talk about SUS in more detail.
\n<\/p>\n
An existing questionnaire (also known as an ‘instrument’ in the language of survey methodologists) has two clear advantages:<\/p>\n
It also has two clear disadvantages:<\/p>\n
The\u00a0hybrid approach lets you include missing questions, and correct\u00a0misunderstandings. But messing with the questionnaire may lose the\u00a0advantages of comparability.<\/p>\n
Jeff Sauro of Measuring Usability marked the 25th anniversary of SUS in his article Measuring Usability with the System Usability Scale (SUS). Jeff delves into the advantages of SUS. To summarize:<\/p>\n Before\u00a0you rush off to change over to using SUS as your main way of measuring\u00a0usability of your products, I’d like to mention some issues.<\/p>\n Jeff mentions one of the challenges of SUS:<\/p>\n “Even though a SUS score can range from 0 to 100, it isn’t a\u00a0percentage.\u00a0 While it is technically correct that a SUS score of 70 out\u00a0of 100 represents 70% of the possible maximum score, it suggests the\u00a0score is at the 70th percentile. A score at this level would mean the\u00a0application tested is above average. In fact, a score of 70 is closer Jeff quotes a graph that helps you to rebalance your scores into a real percentage and you can also buy his “SUS Guide & Calculator Package<\/span>” which contains spreadsheets to help you do it.<\/p>\n Even\u00a0so: is a 70% score good or bad for your users attempting their tasks\u00a0with your product, web site, or whatever? It sounds fairly good, but\u00a0what if your competitor is scoring 90%?<\/p>\n Much of the value of SUS rests on its longevity; during that time, it’s\u00a0been researched as well as used a lot. Not all of that research is\u00a0completely positive.<\/p>\n In a very helpful and thorough comparative study of questionnaires in 2004,\u00a0Tom Tullis and Jacqueline Stetson of Fidelity Investments and Bentley\u00a0College looked at SUS and a selection of other usability questionnaires. Overall, SUS came out on top: it was the best at showing which of the sites was more usable.<\/p>\n But\u00a0SUS wasn’t perfect. At the typical sample size that we might use in a\u00a0usability test, 6 users, none of the questionnaires was particularly\u00a0good – and SUS wasn’t any different, coming half-way up the results.\u00a0It’s just a statistical reality that small samples create odd\u00a0statisitical results, even though they’re usually plenty large enough As Jeff Sauro points\u00a0out, SUS was designed for assessing ‘green screen’ applications long\u00a0before the Internet was widely available. To make it comparable with the\u00a0other questionnaires in their study, Tom Tullis and Jacqueline Stetson\u00a0had to modify it slightly, replacing the word ‘system’ with ‘web site’\u00a0throughout.<\/p>\n But that’s not the only example of less-than-perfect wording. In a 2006 study, Kraig Finstad reported on The System Usability Scale and Non-Native English Speakers\u00a0(pdf). He found diffculties with item 8 in SUS: “I found the system In other\u00a0contexts, I have my doubts about item 5: “I found the various functions\u00a0in this system were well integrated”. What exactly would this mean if\u00a0you were trying to assess the usability of, say, a simple web\u00a0application with just one function? What about something like the\u00a0registration process for a site, where the issues might be around\u00a0whether the function should exist at all? What if your users simply\u00a0don’t understand the word ‘function’ in this context?<\/p>\n Overall,\u00a0though, SUS is a good old friend to the user experience researcher. If\u00a0you want to throw a few questions about usability at your users at the\u00a0end of a usability test, then SUS is a handy place to start. By all\u00a0means use it, and if you have time then do these things as well:<\/p>\n One challenge of survey design is whether to: use an existing questionnaire, or roll-your-own, or do some sort of hybrid. One of the best-known usability questionnaires is SUS. Is it good enough? I’m going to start by mentioning the advantages and disadvantages of reusing questionnaires, and then talk about SUS in more detail.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_seopress_robots_primary_cat":"","_relevanssi_hide_post":"","_relevanssi_hide_content":"","_relevanssi_pin_for_all":"","_relevanssi_pin_keywords":"","_relevanssi_unpin_keywords":"","_relevanssi_related_keywords":"","_relevanssi_related_include_ids":"","_relevanssi_related_exclude_ids":"","_relevanssi_related_no_append":"","_relevanssi_related_not_related":"","_relevanssi_related_posts":"","_relevanssi_noindex_reason":"","footnotes":""},"categories":[26],"tags":[],"acf":[],"_links":{"self":[{"href":"https:\/\/rosenfeldmedia.com\/wp-json\/wp\/v2\/posts\/184437"}],"collection":[{"href":"https:\/\/rosenfeldmedia.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rosenfeldmedia.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rosenfeldmedia.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/rosenfeldmedia.com\/wp-json\/wp\/v2\/comments?post=184437"}],"version-history":[{"count":3,"href":"https:\/\/rosenfeldmedia.com\/wp-json\/wp\/v2\/posts\/184437\/revisions"}],"predecessor-version":[{"id":186538,"href":"https:\/\/rosenfeldmedia.com\/wp-json\/wp\/v2\/posts\/184437\/revisions\/186538"}],"wp:attachment":[{"href":"https:\/\/rosenfeldmedia.com\/wp-json\/wp\/v2\/media?parent=184437"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rosenfeldmedia.com\/wp-json\/wp\/v2\/categories?post=184437"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rosenfeldmedia.com\/wp-json\/wp\/v2\/tags?post=184437"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\nSUS was developed by John Brooke to allow users to report their views\u00a0of\u00a0 the usability of a system immediately after a usability test,\u00a0something that many of us want to do. He explained the genesis of the questionnaire, and how to use it, in Brooke, J. (1996). “SUS: a “quick and dirty” usability scale” in P. W. Jordan, B. Thomas, B. A. Weerdmeester, & A. L. McClelland. Usability Evaluation in Industry<\/i>. London: Taylor and Francis.<\/span><\/span><\/p>\n\n
\nsimilar users, you’ll likely get approximately the same result.<\/li>\n
\nmeasure, that is whether or not the users perceive a system to be usable
\nor not.<\/li>\n
\nwith other people’s results from SUS to establish whether your system is
\nmore or less usable.<\/li>\n<\/ul>\nSUS isn’t the perfect usability questionnaire<\/h3>\n
SUS scores need interpretation<\/h4>\n
\nto the average SUS score of 68. It is actually more appropriate to\u00a0call it 50%”.<\/p>\nIs SUS really the best questionnaire?<\/h4>\n
\nThey tested the questionnaires’ ability to correctly identify which of\u00a0two web sites was more usable. Both chosen sites had similar aims and\u00a0audiences; study participants were randomly assigned to one site or\u00a0other, and each tried the same two tasks.<\/p>\n
\nfor us to make decisions about what changes to make next to our\u00a0products.<\/p>\nDoes SUS make sense to your users?<\/h4>\n
\nvery cumbersome to use.” If you do decide to use SUS, then it’s probably\u00a0best to replace ‘cumbsersome’ with ‘awkward’.<\/p>\nSUS is probably good enough<\/h3>\n
\n