This month, I’ve been working on my chapter on survey questions and revisiting my libary of books about questions. My favorite, easily, is:<\/p>\n
“The Psychology of Survey Response”<\/a> by Roger Tourangeau, Lance J. Rips and Kenneth Rasinski (2000) Cambridge University Press.<\/p>\n Let me take you through their main framework and some of my recent experiences with the book, and I hope that will explain why I like it so much. The book is based around their analysis of the components of the mental processes that respondents go through as they answer a question. In Forms that work: Designing web forms for usability, I explained these steps as:<\/p>\n Tourangeau, Rips and Rasinski describe the four processes, and their sub-processes, like this:<\/p>\n Comprehension:<\/p>\n Retrieval: <\/p>\n Judgment:<\/p>\n Response:<\/p>\n It’s a bit hard on them to present this rather complex list, full of technical terms, without their explanations – but I wanted you to realise that this is a really deep dive into questions. For example, when they are talking about the process “Represent logical form of question”. They investigate the grammar of the question, comparing easy pairs such as:<\/p>\n Iterrogative: “Do you have a home equity loan?” with the more complex ‘who, what, where, when, why, how’ questions, such as analysing this question: which has two possible interpretations of appropriate answers:<\/p>\n This illustrates both the one of the challenges and one of the strengths of this book. You’ve got to be willing to be challenged by topics like sentence parsing, but the authors will help you along by providing lots of examples.<\/p>\n The biggest chunk of the book is devoted to ‘retrieval’, with a chapter on memory followed by three chapters investigating how people use their memories to answer three types of questions: dates and durations, factual and numerical estimates, and attitudes. <\/p>\n One of the crucial nuggets from these chapters: we do better at recalling events if they are recent and significant to us. Example: I know exactly what I did all day on Saturday 4th December 2010, but had to check my diary to remind myself about Thursday 3rd December 2010. Why? Because my cousin’s wonderful wedding was on the Saturday, whereas Thursday was just an ordinary working day rather like any other.<\/p>\n Why does that matter for us in user experience? Because we often want to know about routine events that are relatively insignificant to our users, such as browsing a familiar web site or making an everyday purchase. Big deal to the site’s designer; not such a big deal (maybe) to the user.<\/p>\n The authors introduce the process of judgment in their chapter about attitudes, where they describe different models of the concept of attitude. Their preferred one includes a judgment phase, where “multiple considerations about the issue will come to mind, and the respondent will have to combine them to produce an overall judgment”.<\/p>\n They point out that these considerations may include:<\/p>\n They go on to explore these topics in greater detail in the chapter about attitudes and context effects: the way in which reported attitudes can apparently be changed by asking questions in slightly different ways or in different orders. <\/p>\n The final process of the four, response, is about presenting the answer in the format acceptable to the survey. In my terms: putting the answer on the form. They devote a chapter to this process, which includes a brief discussion of one of the most-researched topics in survey design: the number of points in a response scale<\/p>\n The final chapter about their four-element process looks at what they call ‘editing of responses’, by which they mean the way in which respondents make choices about what to reveal in their answers. You may be familiar with the concept of ‘social desirability bias’,: we like to portray ourselves as nice people and therefore tend to over-report behaviors and attitudes that we believe reflect well on us, and under-report ‘bad’ things. <\/p>\n A user experience example: telling someone that you don’t like their product might be seen as being an attack on them. So we find that users in usability testing struggle during their tasks, but then report satisfaction overall. Are they really satisfied, or are they editing their response to avoid appearing like nasty, attacking people?<\/p>\n The last two chapters move away from the main processes and look at ‘interview mode’ and ‘cognitive models’.<\/p>\n Interview mode is about how you deliver the survey: by telephone, mail, email, Internet or interviewer, and whether you use a computer or pen and paper. This is where the book shows its age a little: it was written in 2000, and the chapter needs to be read with that in mind.<\/p>\n Can I admit that I struggled a bit with the final chapter, on the application of cognitive models? It seems to be going into arguments for and against various threads in the academic discourse around surveys in the late 1990s, but as I’m not entirely familiar with those threads I found it a bit hard to follow.<\/p>\n So my suggestion: don’t worry too much about chapters 10 and 11. There’s more than enough in the previous nine chapters to justify reading this book.<\/p>\n I’ve tried to give you a flavor of the big themes in the book, which I find fascinating. I also enjoy reading the finer details within those themes. <\/p>\n For example: One topic that’s particularly relevant to user experience: reported ‘satisfaction’. Is that different to, or the same as, ‘happiness’? <\/p>\n They explain that “respondents may assume that two very similar items convey different questions when they appear in the same part of the questionnaire” and go on to quote an experiment that “asked respondents to rate both their happiness and their satisfaction with their lives. In one condition, the questions – How happy are you…?<\/i> and How satisfied are you…?<\/i> – appeared next to each other in the same questionnaire. In a second condition, on of these questions appeared as the last item on an intial questionnaire, the second as the first item on a second questionnaire. In line with the prediction, ratings for the questions tended to be more similar when they appeared in different questionnaires than in the same one”.<\/p>\n So: for satisfaction and happiness in life in general, the concepts are about the same if you ask the question at different times, but different if you ask the question at the same time. (Should you wish to follow up the experiment, it’s described here: Semantic and pragmatic aspects of context effects in social and psychological research.<\/a>Detail Only Available<\/span><\/span><\/span> Strack, Fritz; Schwarz, Norbert; W\u00e4nke, Michaela<\/span>; <\/span>Social<\/strong> <\/span>Cognition<\/strong>, Vol 9(1), Spr, 1991). <\/span><\/p>\n Hmm. What does that mean for us in ux? I’m thinking: it seems likely that the same effect happens when you ask about satisfaction and happiness for a product or experience. If you need to distinguish between them, you’ve got to ask about both at the same time.<\/p>\n Can you see why I enjoy this book so much? I find I keep reading a bit of it, start to think about the implications, want to follow up a reference – and next thing I know, an hour has gone by. <\/p>\n I won’t pretend it’s an easy read, but it does repay the effort. <\/p>\n","protected":false},"excerpt":{"rendered":" This month, I’ve been working on my chapter on survey questions and revisiting my libary of books about questions. My favorite, easily, is: “The Psychology of Survey Response” by Roger Tourangeau, Lance J. Rips and Kenneth Rasinski (2000) Cambridge University Press. Let me take you through their main framework and some of my recent experiences … Continued<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_seopress_robots_primary_cat":"","_relevanssi_hide_post":"","_relevanssi_hide_content":"","_relevanssi_pin_for_all":"","_relevanssi_pin_keywords":"","_relevanssi_unpin_keywords":"","_relevanssi_related_keywords":"","_relevanssi_related_include_ids":"","_relevanssi_related_exclude_ids":"","_relevanssi_related_no_append":"","_relevanssi_related_not_related":"","_relevanssi_related_posts":"","_relevanssi_noindex_reason":"","footnotes":""},"categories":[26],"tags":[],"acf":[],"_links":{"self":[{"href":"https:\/\/rosenfeldmedia.com\/wp-json\/wp\/v2\/posts\/184334"}],"collection":[{"href":"https:\/\/rosenfeldmedia.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rosenfeldmedia.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rosenfeldmedia.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/rosenfeldmedia.com\/wp-json\/wp\/v2\/comments?post=184334"}],"version-history":[{"count":3,"href":"https:\/\/rosenfeldmedia.com\/wp-json\/wp\/v2\/posts\/184334\/revisions"}],"predecessor-version":[{"id":186516,"href":"https:\/\/rosenfeldmedia.com\/wp-json\/wp\/v2\/posts\/184334\/revisions\/186516"}],"wp:attachment":[{"href":"https:\/\/rosenfeldmedia.com\/wp-json\/wp\/v2\/media?parent=184334"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rosenfeldmedia.com\/wp-json\/wp\/v2\/categories?post=184334"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rosenfeldmedia.com\/wp-json\/wp\/v2\/tags?post=184334"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\n<\/p>\nThe four components of the response process<\/h4>\n
\n
\n
\n
\n
\n
An example of the dicussion: logical form of question
<\/h4>\n
Declarative: “You have a home equity loan”<\/p>\n
“When did Lydia tell Emily the telephone would be fixed?”<\/p>\n\n
People are better at recalling recent, significant events<\/h4>\n
Judgment is about selecting an answer<\/h4>\n
\n
Response is about mapping judgments to survey answers<\/h4>\n
Two technical chapters: on interview mode and cognitive models<\/h4>\n
Is satisfaction the same as happiness?
<\/h4>\nOverall: a book that’s academic but intriguing<\/h4>\n