Frequently Asked Questions
These common questions about designing for kids and their short answers are taken from Nate Bolt and Tony Tulathimutte’s book Remote Research: Real Users, Real Time, Real Research. You can find longer answers to each in your copy of the book, either printed or digital version.
- What is remote user research, anyway? Is it anything like focus groups or surveys?
Remote user research is simply a type of user experience (UX) research that’s conducted over the phone and Internet, instead of in person. In general, UX research seeks to understand how people interact with technology. Unlike focus groups and surveys, market research techniques that are used to learn people’s opinions and preferences, UX research focuses on studying people’s behavior. In that sense, remote user research isn’t really like market research; however, both remote user research and market research can be applied toward improving the design of existing technologies and inspiring new ones.
See Chapter 1, page 3. - What kinds of remote research are there?
That’s a huge question, and we spend a good chunk of this book introducing and describing the many varieties and specialties of remote research out there. In general, there are two branches of remote research: moderated and automated. In a moderated study, a researcher talks directly to the participants as they use the interface in question, and it’s good for obtaining rich, qualitative feedback. In an automated study, you use online tools and services to gather behavioral or written feedback and information automatically, without the researcher’s direct involvement.
For more about moderated testing, see Chapter 2, page 32, and all of Chapter 5. For more about automated testing, see Chapter 6. - I’m skeptical about remote research. If it’s so great, why haven’t I heard of it?
A lot of the misgivings that people have about remote research come from its novelty. The method is still cutting edge, and the technique requires a certain degree of know-how. Until now, there hasn’t been a book you could learn the method from—which is, of course, the reason we wrote it.Still, lots of companies have done it, with great success. We’ve done remote studies with Sony, Autodesk, Greenpeace, AAA, HP, Genentech, Wikipedia, UCSF Medical Center, the Washington Post, Esurance, Princess Cruises, Hallmark, Oracle, and Blue Shield of California, among many others. - I’m still skeptical. Can you really get valid behavioral feedback without seeing your participants in person?
Since remote research is conducted over the phone and Internet, many people worry about missing “rich details” like facial expressions and body language. First, we believe that for most user research studies, the way that users interact with the interface and their think-aloud comments are the only really necessary things to focus on. And on top of that, much of the tone does come through the user’s voice and language. We weigh the pros and cons of in-person research and remote research in Chapter 1, pages 5-15, and discuss moderating over the phone in Chapter 5, page 110.
Still skeptical? Then you should check out our exhaustively documented study for Wikipedia, complete with full-session videos and highlight clips at http://usability.wikimedia.org/wiki/Usability_and_Experience_Study. It includes both lab and remote sessions with identical goals, so it’s a good comparative case study.
The best way to see if a remote study is for you, however, is by getting your feet wet with a quick, painless pilot study, which we’ll walk you through in Chapter 2, page 30. - I want to cut costs for my user research study. Is using remote methods a good way to do that?
Not really. You might save on costs related to travel expenses, renting a lab facility, or hiring a recruiting agency, but then there are the expenses of the specialized remote research tools and services you’ll need, and where the researcher’s time, participant incentives, and project timeline are concerned, nothing is much different.
We cover the equipment requirements of a basic moderated study in Chapter 2, page 28, and the costs of many remote tools and services in Chapter 8. - Where can I get people to participate in my remote study?
You can technically use any methods to recruit for a remote study that you’d use for an in-person study: email contacts, recruiting agencies, and even craigslist ads (blegh!) are still an option. However, in this book, we introduce a method called live recruiting, with which you intercept visitors to your Web site by using a pop-up form to get them to participate in your study right away. We strongly believe this approach is ideal for remote research because it allows you to do what we call “Time-Aware Research,” which we introduce in Chapter 1, page 10 and discuss in depth in Chapter 3. - How on earth can you call this a book about research without way more academic references and doctoral degrees?
This book is aimed at people who want to hear practical information about how remote research is done in the real world.We don’t claim absolute peer-reviewed scientific rigor, but we are sharing what’s given us the best results after almost a decade of remote research experience: 2,615 moderated users, 2,676 automated users, 234 projects, 19,120 project hours, and 89 clients to date. Simply put, this is just what works for us, and we think it will work for you, too. (And for what it’s worth, Nate and Tony both have cognitive sciences/human-computer interaction degrees: yes, we roll that deep.)
