Whose Job is User Research? An Interview with Steve Portigal
Those of us who conduct user research as part of our jobs have made pretty big gains in recent years. I watched my first usability test in 1995 then spent a good portion of the 2000s trying to convince people that talking to users was an important part of designing and building great products. These days, when I talk to companies, the questions are less about why they should do user research and more about how they can do it better. Believe me, this feels like enormous progress.
Unfortunately, you still don’t see much agreement about who owns user research within companies. Whose job is it to make sure it happens? Who incorporates the findings into designs? Who makes sure that research isn’t just ignored? And what happens when you don’t have a qualified researcher available? These are tough questions, and many companies are still grappling with them.
So, I decided to talk to some people who have been dealing with these questions for a living. For this installment of the Whose Job is User Research blog series, I spoke with Steve Portigal, Principal at Portigal Consulting. He’s the author of Interviewing Users, which is a book you should read if you ever do any research on your own.
You still don’t see much agreement about who owns user research within companies.
Steve has spent many years working with clients at large and small companies to conduct user research of all types. He also spends a lot of his time helping product teams get better at conducting their own research. Because he’s a consultant, he sees how a large number of companies structure their research processes, so I asked him to give me some advice.
What Does It Mean to Own User Research?
“There are two aspects to ownership,” Steve says. “One is about owning the need. The other is about owning the actions where we build on what was learned in research. It doesn’t seem like there’s any perfect model for how research ownership works.”
As Steve points out, the concept of owning research is much more complicated than a single ownership model can describe. At a minimum, somebody needs to determine which business questions should be answered. Somebody needs to figure out how to get those questions answered. Somebody needs to figure out what to do with the results of the research. It’s not often the same somebody.
In fact, the different people involved in the research frequently are not even from the same department. Company org structures vary widely: researchers might be their own group or they might be part of marketing, product, or even engineering. The people requesting research and using the results might be product managers, ux designers, or marketers. That’s not even addressing the times when research is done by outside firms or by team members who aren’t trained in research.
There’s a reason this is so confusing. Despite the fact that various forms of user research have been used to develop products for decades, the widespread adoption of user research in the tech industry is still relatively new. Steve says, “People have more dogmatic theories about best practices, but I’m seeing so much variety and so much iteration as people try to figure it out.”
Hopefully with a few more iterations we’ll get a better idea of what works and what doesn’t. Although it’s likely there will never be one single answer. The best configuration may always depend on factors like company size, industry, and type of research that needs to be done.
Who Should Do the Research?
Despite the fact that Steve is a highly experienced research expert who conducts research for clients, he’s very upfront about the fact that internal teams should be heavily involved with the research. Often they should be doing it themselves. This is one of the reasons he teaches people how to be better at research and why he wrote a book that explains how to interview users more effectively.
Not every research study requires an expert at the helm. Quite a few products would benefit from having somebody on the main product team who could quickly get feedback or answers to simple questions. “Even a newbie researcher should be able to answer some factual questions about what people are doing or might want to do. They also have the opportunity to reflect on what assumptions they were holding onto that were incorrect,” Steve explains. “You’ll always get more questions to go with your answers, but hoo boy–it’s better than never talking to users and acting with confident ignorance.”
There are some questions you’re better off bringing in an expert, though. “The more help you need in connecting the business problem with the research approach and connecting the observations to the business implications, the more expert help you need,” Steve explains.
We should all know by now that things like usability testing make our products simpler and more intuitive for our users. There’s also a huge amount of information to help you run a basic usability test. But when you’re getting into some of the trickier questions around generating or validating business ideas–or turning early customer research into innovative solutions to problems, an expert can help guide the research process and make these complicated research studies run more smoothly.
How Do We All Work Together?
Of course, none of this answers the question of how we all work together. Steve feels like there’s not a single answer to this question, but it’s very important to decide this ahead of time so that everybody knows what to expect.
For example, consider where researchers live in relation to the people who need insights to inform product design. When your company has expert researchers, they may be part of an in-house silo, embedded in the product team, an outside consultant, or some hybrid of any of the above. Wherever they come from, you should determine five things as part of your research planning process:
- What do we need to learn?
- Who are we going to study?
- What will we do with the outcomes?
- How are we going to work together?
- How will we define success?
A standard predictor of success is how much the client is able to join the fieldwork.
“Negotiating these elements is part of what a good research person should be doing,” Steve says. “The team structures, the availability of stakeholders – these are all inputs.” In other words, there’s no one-size-fits-all when it comes to determining how research projects should happen.
According to Steve there is one constant: “A standard predictor of success is how much the client is able to join the fieldwork.” And he’s right. The best type of research is the research that people use to make better decisions. The more involved your team is with conducting the research, the more likely they will understand and pay attention to the results.
Learn More
Want to know more about conducting research well? Check out Steve’s book Interviewing Users for practical, easy to use advice on getting more out of your research sessions. Or join us this October for a one-day remote conference User Research for Everyone–featuring 8 of the most respected experts in the field.
Laura Klein is a Lean UX and Research expert in Silicon Valley who teaches companies how to get to know their users and build products people will love. She’s a Rosenfeld Media expert and author of UX for Lean Startups (O’Reilly). Her newest book, Build Better Products, is set for release later in 2016. Follow her on Twitter or subscribe to her blog and podcast at Users Know.
Product Teams – Who Does What?
When we started talking about putting on the PM + UX Conference, the first thing we asked was, “What sorts of things should we talk about?” Since the folks at Rosenfeld Media are, not surprisingly, extremely user-centered, the obvious answer was, “We’re not sure. How about we do some research and find out what questions our attendees might have?” So we did.
The most interesting thing to me was that a lot of the questions people asked boiled down to “Who does what on a product team?” This was curious. I mean, we’re all working on product teams or we’ve worked on them in the past, right? Shouldn’t we know what our jobs are? Shouldn’t we know what everybody else is doing? Well, yes! We should! And yet… when I started to dig around and have conversations with people, I got very, very different answers about how things really worked.
That was odd. It turned out that, although we all have job titles like Product Manager or UX Designer, many of us have very different ideas about what it is that we’re supposed to actually do all day. Do designers talk to customers? What about PMs? Who decides what features go into a product? Who makes wireframes? Does anybody do usability testing? If not, could they please start?
Like any good team faced with more questions than they started with, we did some more research. Ok, first we had a couple of stiff drinks. Then we did some more research. I was volunteered to lead the way.
The Research Methodology
I wanted to make sure that we hadn’t gotten weird data or misinterpreted the questions being asked- after all, it was a survey – so I conducted several qualitative interviews with PMs and UX designers in various sized organizations. I asked about what PMs and UX designers did at their companies. Incidentally, I also got a lot of “but the way we SHOULD do it is…” answers, but I was focused on reality for the moment. And reality seemed just as confused as the survey results.
After talking to a dozen or so people, I decided I wanted to collect exactly what tasks were performed at different companies so that we could see how much overlap existed. To get as many data points as possible, I sent out a very simple survey asking people what their role was, which tasks they thought were performed by UX Designers and which tasks they thought were performed by PMs.
I got dozens of responses from people on different product teams–mostly UX, PM, and researchers–and even some engineers, marketers, founders and managers. Of these responses, I got hundreds of different things that are being done by product teams.I am not exaggerating. It seems that PMs and designers are performing a dizzying array of tasks from generative research to making roadmaps to managing stakeholders to visual design.
I got curious. Who on these teams are doing all of these things? Seriously, most product teams just aren’t that big, so how do they get hundreds of things done? Who’s got that kind of time?? I narrowed down all the responses by deduplicating things that were similar. I picked the things that were mentioned the most, narrowed it down to 73 discrete tasks or processes–which is still a LOT of things for a team to be doing.
Next, I did a card sort: I asked people to review each task and tell me whether it was mostly done by designers, mostly by PMs, done about equally, or generally done by nobody or somebody besides these. I also said you could make your own categories for things that didn’t fit.
I ran the card sort remotely using OptimalSort, which was handy, since that meant that I could gather a lot more input than if I’d had to travel around with a set of index cards and watch each person do the sort. It also made it a lot easier to categorize, and you’ll see in just one second why that was important.
The Research Results
82 of you weighed in with how things worked at your company. And, surprise surprise, there was virtually no agreement. I mean, there were a few similarities. Everybody agreed that visual design was done by someone in the design department (except for those of you who said it was done by outside people or people in marketing). And almost everybody agreed that roadmaps were made by PMs. Or by execs. Or by managers. Or by someone else.
Oh, and nobody agreed about who was supposed to do user research, except that a lot of people made a category called something like “things we aren’t doing enough of” and stuck user research into it. So, that was fun.
Here are a few of the things people actually agreed on mostly.
What Are Designers Doing?
Interestingly, there was far more agreement about what designers do than about what product managers do across companies.
Visual Design
The most agreed upon tasks were things like visual design, design specs and style guides, image manipulation, illustration, and branding. Almost everybody agreed that these were mostly done by designers.
The unfortunate part about this particular agreement is that, while it’s true that these are more “design” tasks than they are “product management” tasks, it tends to confirm the idea that many organizations still see design as limited to visual design.
Interaction Design
It has “design” right there in the title, but 10% of the respondents said that interaction design is done equally by product managers and designers, and one sad person said that it’s not being done at all.
Other Mostly Design Tasks
Other tasks that were mostly done by designers were what you might expect:
- Mockups
- Wireframes
- Sketching
- Site Maps
- Information Architecture
Interactive prototypes were also very high on the list of tasks mostly done by designers, although at a few companies they’re still being made by engineers or developers. This may reflect the huge number of new prototyping tools available these days.
What Are PMs Doing?
There was a lot less agreement across companies about what PMs were doing, frankly.
Roadmaps
The most agreed upon task for product managers was creating and maintaining feature roadmaps, with 63 people saying that this was almost always done by product managers. In twelve cases however, roadmaps were maintained equally by designers and PMs, and in six cases, they’re being made by someone else or not being made at all. Project and program managers are sometimes the keepers of the roadmap, but very infrequently.
Feature Definition, Business Strategy, and Scheduling
Many of the most common product management tasks, unsurprisingly, are things that involve defining and prioritizing features, creating revenue models, or project management and scheduling tasks like sprint planning.
But there were really only a few of these tasks where more than half of the respondents agreed that they were done mostly by product managers and there were no tasks where more than three quarters of the respondents agreed were all or mostly done by product managers.
Business strategy, for example, was reported to be done primarily by PMs in only 50 of the 83 cases. In five cases, it was done equally by PMs and Designers and the rest of the time was done by anybody from executives to finance to nobody at all. In one case, the entire team worked together on setting business strategy.
What are People Doing Together?
The most inspiring part of the study for me was how many tasks people are doing together – either as a whole, cross-functional team, or at least across the PM and Design roles.
In 13 cases, the entire team reported brainstorming new features together, and in 51 cases, brainstorming was done about equally by PMs and UX designers. Hopefully that means that more people are getting involved in figuring out which new features to build for users.
Perhaps the most evenly split task was customer discovery, which worked out like this:
- Mostly designers: 17
- Mostly product managers: 17
- Equally by PMs and designers: 20
- Not done: 15
- Other: 16
I don’t like to see customer discovery not getting done almost 20% of the time, but it’s good that when it’s happening it’s a shared task. Needs finding had a similar split, as did most forms of generative research. At least, when user research is getting done, it’s getting done by the whole team rather than being entirely run by one group.
Why Does this Matter?
Having a shared understanding of what we do and what we’re supposed to do is hugely important to creating a well-functioning team. Imagine trying to build a baseball team where nobody agreed on what shortstops were supposed to do or what made somebody a great pitcher. It would be chaos.
And yet, every day, we hire people, add them to our teams, and give them titles like product manager or UX designer or researcher, without realizing that the last company where they worked might have had an entirely different definition of the role. Then we’re shocked when somebody who was a fantastic PM at their last company isn’t succeeding. Or we can’t believe that the new designer can’t do what we consider to be a trivial task. Worse, sometimes we have no idea what our coworkers are doing or what they’re supposed to be doing, which is why important tasks can fall through the cracks.
Learn More
There is actually one more bit of research that I’ve been doing in parallel for awhile now as I work on my new book, Build Better Products (Rosenfeld Media ‘16). I’ve been talking to teams in order to understand which ones are most effectively delivering value to users. There seems to be a very strong correlation between good outcomes and the teams that report doing certain of these tasks together. Teams that conduct research together are more likely to understand their users. Teams that use that research to ideate together are more likely to come up with feature ideas that users want. Teams that decide business strategy together are less likely to get into arguments about what should be prioritized.
If you want to learn more about how your team can work together more effectively, check out the upcoming one-day online conference devoted exclusively to Product Management + User Experience. I’ll be sharing activities you can run yourself to build team cohesion and get everybody working together. Your ticket gives you all-day access plus recordings to the full program. There are only a few spots left. Hope to see you there!
Laura Klein is a Lean UX and Research expert in Silicon Valley who teaches companies how to get to know their users and build products people will love. She’s a Rosenfeld Media expert and author of UX for Lean Startups (O’Reilly). Her newest book, Build Better Products, is set for release later in 2016. Follow her on Twitter or subscribe to her blog and podcast at Users Know.
Whose Job is User Research? An Interview with Jeff Gothelf
While doing research for the upcoming PM+UX Conference, several respondents requested guidance around how user research should managed. In fact, it was the most common write-in answer on our survey, and a question that comes up repeatedly whenever I give talks. Apparently, there seems to be very little consensus about who on a product team should own research. This makes it a lot harder to get user insights and make good product decisions.
In a way, this is good news. Five or ten years ago, there would have been more questions like, “How do I get my boss to consider doing user research?” and “What is user research good for?” Those still come up, but far more frequently, I’m hearing things like, “How do we make sure that everybody on the team understands the research?” and “Who is in charge of making sure research happens and deciding what to do about it?” Research, these days, is assumed. It’s just not very well managed.
To answer these questions, I interviewed several very smart people who know a thing or two about research and building products. I’ll share some of their suggestions in a series of blog posts.
First, I spoke with Jeff Gothelf, the author of Lean UX (O’Reilly, 2013) and Sense and Respond (Harvard, 2016).
Jeff spends most of his time training companies around the world on how to build usable products using Lean user experience design techniques. He’s seen his share of different organizations, and has an excellent sense of which ones are building the right things. So I asked him to help me figure out the best way to incorporate user research into the development process.
User Research is Owned by the Team
Companies that are functioning well and building great products are doing user research. And they’re doing it together. “Research is owned by the team,” Jeff says. “It is as important as design or writing code. That said, in most organizations, it’s the UX person who typically drives the research or, in their absence, the PM. It’s not necessarily the best approach, but it’s what I see most often.”
In this model, the person responsible for making sure that the research happens doesn’t own performing or communicating the research. What this means is that the product manager or UX designer will request that some specific research will be done.
Unfortunately, this often translates into the PM asking for research and a researcher who is completely unaffiliated with the team or even hired from outside going off and interviewing users and then coming back six weeks later with a report, by which time the team has almost certainly moved on. This generally means that the results of the study will never be used by the team, since the results will be out of date and irrelevant.
Create a Better Scenario
Rather than seeing research as something that is simply requested by someone on the team and then delivered, Jeff suggests a more collaborative approach. The best case involves the PM or UX designer sharing a specific business or user need that requires some research, and then having that turn into a shared activity where members of the team are involved in performing the research. “I’ve found it rare that a team can’t execute their own research,” Jeff says.
“I’ve found it rare that a team can’t execute their own research.” – Jeff Gothelf – Tweet This
There’s still a place for experts in Jeff’s model too. “There may be some situations where getting to the participants or communicating with them may prove challenging (language barriers, cultural differences, etc.) in which case an expert can help,” Jeff explains. “Also, if nobody on the team has research experience, bringing in an expert to train the team is helpful. If you already have some researchers in the company, they can train and deputize others to start collecting user insights.”
Never Outsource Research
Jeff is also not a fan of having outside researchers conduct studies, since it takes time and can lead to a one and done mentality. “I advise teams to never outsource their research. Letting someone else do your research means waiting for them to do it, synthesize it and write it up. Also, outsourcing the research typically means it’s a singular or at least a finite event. I believe research should be continuous.”
When teams bring outside people in to conduct studies and simply hand off results, they’re missing out on a huge opportunity to build skills within the team. Outsourced studies, even relatively simple ones, can cost thousands of dollars. If you’re making that budget decision every time you want to learn something, it’s going to keep you from doing all the research you should be doing. On the other hand, if members of your team can find ways to learn constantly from users, that price comes down considerably, as does the time it takes to run a study.
Finding ways to learn constantly from users brings the price of research down considerably. – Tweet This
There are a lot of options for getting your team up to speed: outside experts can come in and facilitate training sessions or work with the team to do the research rather than perform everything for them. Jeff also recommends Steve Portigal’s book, Interviewing Users, Giff Constable’s Book, Talking to Humans, and my own book (Thanks, Jeff!), UX for Lean Startups.
Get Out of Silos
Cross functional teams make many Lean UX practices possible. It’s easier to have “the team” be responsible for customer discovery if the team exists as a persistent entity throughout the life of the product. But the reality is that many companies still don’t work that way. Research, product, design, and engineering are often in their own silos, communicating only through deliverables, if at all.
The best approach is to get rid of silos entirely, but very few of us have the option to restructure how our companies work in order to make research more effective. If you’re in an organization where research has its own silo, Jeff has several suggestions for making those silos less restrictive.
“Brown bags, demos, ride-alongs, workshops – these are all good tools to show how to do research and what to make of your findings,” he explains. “At the same time, the PM/UX folks should seek this out from their researcher colleagues if it’s not offered up.” Even when your company is divided up by practice instead of by product team, there’s no reason why you can’t reach out to other people working on similar things and build bridges between the silos.
Learning what other people do and how to do it and teaching them what you do and how you do it can be a wonderful way of creating respect and unity within the team. It has the added bonus of making the product better because everybody ends up working together rather than stepping all over each other and ignoring real customer needs.
Learn More
Interested in helping your teams collaborate to create better research outcomes? Check out Jeff’s book Lean UX (O’Reily, 2013) or join us this October for a one-day remote conference User Research for Everyone, featuring 8 of the most respected experts in the field.
Laura Klein is a Lean UX and Research expert in Silicon Valley who teaches companies how to get to know their users and build products people will love. She’s a Rosenfeld Media expert and author of UX for Lean Startups (O’Reilly). Her newest book, Build Better Products, is set for release later in 2016. Follow her on Twitter or subscribe to her blog and podcast at Users Know.
