Whose Job Is User Research? An Interview with Celia Hodent

This is part 4 in the series Whose Job is User Research.

In this series I’ve been gathering perspectives from experts who develop different types of products. This time I spoke with Celia Hodent, Director of User Experience at Epic Games, a company known for cutting-edge technology and design in the video game industry. Celia also holds a PhD in Psychology so she has tremendous insight into how to use user research to improve the gaming experience.

Who Owns the Research?

It turns out that building games isn’t that different from building anything else. The user experience is a concern for everybody on the Epic Games development team. Research helps them understand more about their design. “The goal of UX research is to test whether design intentions are the ones actually experienced by the target audience,” Celia said. Research also identifies all usability issues that need to be resolved.

Like other great UX designers I’ve spoken with, she says that research should not be separated out from the design process. Research informs and supports the whole design process–which means that everybody needs to be involved.

“Owning the research is not just about conducting the research,” Celia says. “It guides the product team so that they ask the right questions, pose hypotheses, and make sense of the research results in terms of actionable items.”

Who Does the Research?

At Epic Games, Celia relies on expert researchers to conduct the research in the best possible way.

“When it comes to research methodology though,” Celia says, “the people handling research should be the ones who know how to design and carry it out.” Celia outlines four key steps: 

1. Defining the right experimental protocol to test the product team’s  questions
2. Running the research
3. Analyzing data
4. Communicating the results to the entire development team, as well as with other support teams such as Analytics or Customer Service to cross-check results against their data

Professional researchers, and sometimes outside researchers, can also help to reduce bias. “It is very hard to stay neutral when conducting research, even more so when you are part of the development team,” Celia says. Therefore, an expert researcher is more likely to get to the bottom of the research questions without preconceptions or confirmation bias.

When Do You Call in Researchers?

A common mistake Celia has seen companies make is reducing research to usability testing. Too often, product teams forget to involve researchers until after design is complete. By then, all they can do is find bugs.

Instead, Celia recommends bringing researchers into the entire game design process. “Whenever design iterations are conducted,” she says, “which happens as early as pre-production, the designers should pair up with a researcher to run quick-and-dirty tests.” You can use paper prototypes and interactive prototypes–before engineers start implementing a feature. It saves a great deal of time. You’ll find the high level UX issues early—they are cheaper and simpler to fix before you’ve implemented anything.

What To Do If You Don’t Have Researchers?

The truth is that many teams simply don’t have people familiar with good user research techniques. Celia has six tips to running research yourself:

1. Prepare your test well: be clear on what your design questions are and design the test to obtain relevant answers.

2. Tailor your protocol to your research questions and what you’re testing. For example, to test how gamers understand your heads up display (HUD), you can simply take them through a survey showing screenshots. But, if you’re testing how a new feature is introduced to a player, you’d use a “think aloud” protocol. Ask the participant to play through the game while explaining what their thought process. This shows you if they understand the interface as they go.

3. Test with people who don’t know you or your feature. People who know you will be more motivated to make an effort to figure things out just to please you and won’t be as open or honest in their feedback.

4. Encourage participants to ask questions out loud during research sessions. Avoid answering their questions to see how difficult it is for them to figure it out on their own. You won’t be there to help customers when they play them in the real world.

5. Ask fact-based, non-leading, exploratory questions. For example, “describe how you can use this feature” or “what was the objective of the mission you just played?” Avoid opinion-based questions like, “Was the feature easy to grasp?”
   

6. Look at everything as a whole when analyzing the results. Avoid highlighting only the part of the results that confirm your intuition. This is called “confirmation bias” and it can mislead you.

Learn More

Want to know more about conducting research well? Check out Interviewing Users by Steve Portigal for practical, easy to use advice on getting more out of your research sessions. Or join us this October for a one-day remote conference User Research for Everyone, featuring 8 of the most respected experts in the field.

Laura Klein is a Lean UX and Research expert in Silicon Valley who teaches companies how to get to know their users and build products people will love. She’s a Rosenfeld Media expert and author of UX for Lean Startups (O’Reilly). Her newest book, Build Better Products, is set for release later in 2016. Follow her on Twitter or subscribe to her blog and podcast at Users Know.