How to do a survey in six steps

Question: What’s the difference between a questionnaire and a survey?
Answer: A questionnaire is a series of questions and answers on a topic; a survey is the overall process of obtaining useful information using a questionnaire.

Question: OK then, what are the steps in the process?
Answer: Until recently, I was stumped on that one, but after a lot of help from others and some thinking, I have an answer for you…

Summary: the six steps

1. Questions: Start with the questions but don’t stop there
2. Goals: Work out the goals for your survey
3. Users: Talk to your users about the topics in your survey
4. Build: Create and test your questionnaire
5. Deploy: Send it out and watch the responses as they come back
6. Analyze: Investigate the data and create your report

1. Questions: Start with the questions but don’t stop there

You have to know what questions you want to ask. And the way to get to the right questions is to iterate: generate some questions, work on them, revise, try again.

2. Goals: Work out the goals for your survey

One definition of ‘the right questions’ is: ones that gather useful answers. Think about the answers you will get. What decision or change will you make based on the answers?

Then think about who you want to answer those questions. Who are the right users to reach, and how will you get to them?

If you don’t plan to make a decision or to change anything: stop now. Save your users’ goodwill for another time when you really, really need their answers.

Once you’ve got your goals sorted out, iterate back to step 1 – revise your questions to match the survey goals.

3. Users: Talk to your users about the topics in your survey

Now you know what answers will be useful, it’s time to find out if your users want to give you those answers or not. Meet some of them. (Face-to-face is best, but phone meetings will do in a pinch). This is also a chance to find out whether your plans for reaching your users are practical.

Do they understand your questions in the same way that you do? Do they want to talk to you about those topics? Is there another topic, more important to them, that you need to hear about?

If you talking to your users seems like a problem to you, then stop working on your survey and start working on that problem instead.

Once you’ve talked to your users about the topics in your survey, iterate back to step 1. You’ll have lots of ideas about how to improve your questions. Might be a good idea to have another little look at step 2 while you’re about it.

4. Build: Create and test your questionnaire

All those great questions … time to put them together. There are survey tools at every price from free to thousands, with features to match and learning curves to match the features. Or think about maybe an email questionnaire, or a Word document. (OK, maybe not the Word document unless that’s really the only option available). Or if you’re a programmer, or have some around the place, then try roll-your-own.

This is also the time to:

• construct your survey invitation and reminders and
• think about whether you need a pre-notice.

Then test, test, test. Try it out yourself, usability test it, run a pilot (same survey but on a very small sample). Don’t forget to test the pre-notice, invitation and reminders too.

If you don’t have time to test the questionnaire, you don’t have time to do the survey.

(And guess what: those tests will make you want to go back to step 1 and tweak your questions some more.)

5. Deploy: Send it out and watch the responses as they come back

This step is the most nerve-wracking but also the most fun. At last, you hit launch and out go your questionnaires, or the links to them, or whatever deployment you have chosen.

Watch those responses carefully, especially the first few. Anything odd happening? Do the answers seem sensible, individually and collectively? If not, pull the plug quickly. Better to grovel a bit than to waste that precious user goodwill on a survey that’s not working properly.

6. Analyze: Investigate the data and create your report

And at last, you can settle down to getting to grips with your data. Chances are you’ll have to to a bit of clean-up, making some decisions about part-completed questionnaires and other anomalies.

It’s also a good idea to do one last round of iteration, going back to your goals to make sure that your report refers to them.

Summary: the six steps

1. Questions: Start with the questions but don’t stop there
2. Goals: Work out the goals for your survey
3. Users: Talk to your users about the topics in your survey
4. Build: Create and test your questionnaire
5. Deploy: Send it out and watch the responses as they come back
6. Analyze: Investigate the data and create your report