Survey book of the month, April 2011

All of us need to look out at our competitors, right? So I’ve been buying a selection of the various books on surveys that are aimed at the general market. Rather to my surprise, I’m making one of them my book of the month for April. It is:

Online Surveys for Dummies by Vivek Bhaskaran and Jennifer LeClaire (Wiley).

I’ve got a few of the ‘for Dummies’ series and my experience of them has been pretty good. For example, GUI Design for Dummies by Laura Arlov (1997) was a super book, very thoughtful and sensible. Unfortunately it rapidly went out of print. On the other hand, I’ll be honest and admit that all those years of reassuring participants who are blaming themselves for being stupid have rubbed off on me. I don’t like them claiming to be ‘dummies’, and so I always feel slightly shifty when hitting the ‘buy now’ button on a For Dummies book.

If you can get past that reaction, or maybe never had it in the first place, then there are some useful chapters in ‘Online Surveys for Dummies’.

The book only touches lightly on the first two stages of creating an effective survey:

• establishing your goals for the survey
• working with users to find out what they want to tell you about the survey’s topics.

Instead, the book focuses firmly on the later parts of creating a survey:

• building the questionnaire
• deploying the questionnaire to your sample
• analyzing the data.

Good advice on choosing a survey tool

I’m recommending this book mainly for chapters 3 and 4. Survey software and survey services come at every price point from free to thousands of dollars – a month. Although to some extent you get what you pay for, it’s also true that the top-end, high-cost services offer lots of complication along with their full feature set. These two chapters are a great introduction to the jargon used by survey tools vendors and the meaning of all those features.

In chapter 3, “Trying your hand at online surveys”, the authors introduce the basic steps in using an online tool to build a questionniare. In chapter 4, “Online survey tools you can use”, they help you to choose between the wide range of different types of survey tools on offer:

• free tools, such as the free trials or the opensource LimeSurvey
• paid-for online services, such as SurveyMonkey
• install-your-own software, such as Confirmit
• response-based services, such as InstantSurvey

Warning: this book could be subtitled “using QuestionPro”

Having recommended the book mainly for its chapter on choosing a survey tool, I ought to warn you that the authors plump for QuestionPro, one of the online services and a competitor to SurveyMonkey. Quite a few pages throughout the rest of the book are devoted to the specifics of how to do things in QuestionPro. I didn’t mind that and just skipped those parts.

Skip the ‘building a survey’ chapters

From a UX point of view, I’d also suggest skipping part II, chapters 5 through 8, or maybe just giving them a quick skim-read. For example, here’s the first (of two) paragraphs under “Keeping your brand name in mind”:

“Ideally, the colors you choose need to connect the product’s usage to its audience. If you’re conducitn gan online survey about power tools and adult males are your audience, you wouldn’t want to use lavenders and pinks. So when you choose colors for your online survey backgrounds, keep branding in mind”.

And the following paragraph is rather similar.
It’s not exactly that I think the advice in these chapters is downright incorrect, it’s more that it is rather basic and probably mostly familiar to those of us working in UX all the time.

Good introductory advice on deployment

I thought part III was more useful: “Distributing your survey to target audiences”. These four chapters go through:

• working out how to get the survey to the audience (e.g. email versus pop-up link)
• spam, and how to avoid being classed as a spammer
• a (very light, possibly too light) introduction to sampling and different types of sample
• thinking about how to protect your survey and avoid ‘ballot-stuffing’ (people repeatedly taking the survey)

The survey purists would probably blink a bit at the way that convenience sampling, and other non-probability samples, are described – but that’s everyday life for many organizations, where send-and-hope sampling may be the only type of sample you can get. (And when I say ‘blink a bit’, I’m using typical British understatement).

Data cleaning and getting an overview

One area that is often neglected, but gets a useful chapter in this book, is the first step in survey analysis: data cleaning. You’ve got to get to grips with the data you have, by reading it, thinking about the (inevitable) anomalies in it, and deciding how to deal with them. Those are the main topics in chapter 13, the first chapter of part IV.
I was less convinced by the other chapters in part IV – again, not really bad advice, but rather like part II, a bit too basic for the UX person who’s writing reports or doing presentations quite regularly.

Recommended, despite its weaknesses

One of the nice things about the ‘For Dummies’ series is that they (usually) have huge print runs. This makes them relatively inexpensive when brand new, and also means that you can usually pick them up for peanuts if you don’t mind a slightly used second-hand copy. For that, I’m willing to skip here and there and not worry if some chapters are irrelvant, provided there are some informative chapters in there. That’s why I’m recommending this book.