Those familiar with Steve Portigal‘s work know him as a widely-regarded expert in user research. Steve has spent over 15 years interviewing hundreds of people, from families eating breakfast, to rock musicians and radiologists. His latest book Doorbells, Danger, and Dead Batteries gathers 65 stories about research gone wrong. Because when you research real people, life is often unpredictable (and enlightening).
We felt it fitting to turn the interview tables around and ask Steve a series of 15 questions to learn more about what makes his brain tick. Enjoy.
1. Where were you born?
Winnipeg, Manitoba. Best bagels in Canada. So suck it, Montreal! Well, I probably prefer Montreal now.
2. Where did you grow up?
Burlington, Ontario, Canada. Although it was a small town back then (I remember when we got our first McDonald’s), now it’s basically a suburb of Toronto.
3. Three words that describe your childhood?
Kenobi. Simmons. Cheech.
4. Three things you never leave home without?
Wallet, keys, and an appetite (for destruction, of course).
5. What’s the best designed product you’ve ever used?
Timbits®—Bite-sized morsels of traditional donuts.
6. What’s the story behind how you got into user research?
I was working at a design agency that was tentatively experimenting with a new service offering—insights that were “left of the idea” (yes, that was actually how they tried to market generative research work). My putative boss literally stopped speaking to me, and wasn’t putting me on projects (the sort of thing that generally requires talking), so the team doing this research work took me in. In the beginning, they had me watch videos and make notes. Then they let me go into the field and hold the video camera. Eventually I got to ask one or two questions, and as time wore on, I began to lead interviews and then plan and manage research. During that time period Don Norman (or was it Don Knotts?) appeared before me in a dream, clad in diaphanous robes. He marked me with the Sigil of Lamneth and bid me sternly to pursue this holiest of professions. That sealed the deal for me.
7. What other profession would you like to try if you could?
I’m fascinated by the television writer’s room. I haven’t come across any depictions of it that make it sound pleasant, but the collaborative creativity is fairly seductive. Otherwise, something about tending to the emotional needs of bugs.
8. What’s the most embarrassing thing that’s happened to you in the field?
Once I was in the home of people who were relatives of Mayim Bialik, the girl who’d played “Blossom” on the TV show “Blossom.” I learned this because I saw her photo on the fridge. During the interview, I referred to her as “Blossom” and one of the family members pointedly corrected me, saying that her name is Mayim, and that Blossom was a character she played. The woman was right and I was being a bit insensitive. I think I was trying to be clever. Although this was after the show was off the air (Mayim was a college student at the time), that name and the essence of that character were strong cultural ideas. I mean, check out the show’s opening credits.
Okay, I’ve got one more. I was interviewing an African-American woman about music. She was really into artists and genres that are heavily African-American. As she told me about what she listens to, I kept looking over at this cool poster of Mick Jagger above her cabinet. When the interview was wrapping up, I tried too hard to find some common ground, musically, so I asked her, “Tell me about that poster of Mick Jagger?” She looked confused. It was Bob Marley. I DO know the difference between the two, but from where I was sitting, I swear he looked like Mick Jagger. I was embarrassed that my need to connect with her about “my” stuff looked like an inept and even-needier attempt to connect with her.
Takeway: Don’t mention pop culture figures by name?
9. What’s the most surprising thing that’s happened to you in the field?
Surprises are mostly internal moments, where I uncover a stub of my own judgment. As an example, I interviewed a man who was the head of an agency that shared his name. He was in his mid-60s with a head of white hair. I was steering the interview towards his past accomplishments, but he was so much more focused on his current goals. I realized I’d created my own narrative for this guy based on his age and that was completely inaccurate. So the surprise wasn’t about the fact that he was engaged and forward-looking. It was about the gap between my unspoken assumptions and the truth that unspooled before me. Honestly, the revealing of and subsequent dismantling of my assumptions is the most pleasurable part of doing fieldwork.
10. What’s the most heartwarming thing that’s happened to you in the field?
I tell this story in detail in my previous book, Interviewing Users. It involves a home interview where the participants were two young men still living at home, who hadn’t told their parents we were showing up for breakfast. But they wouldn’t speak in words and unwilling to talk with us. The parents were unsurprisingly hostile about our presence. Sitting in their kitchen, the mother (who we eventually pivoted to for the interview) told us that few people are welcomed into their house and that food is a carrier of meaning for their family and is not for strangers. We managed to have an incredible interview with her and her husband, after navigating extreme awkwardness and ambiguous permissions. When wrapping up, she told us, “No one comes here and doesn’t get food,” and made us some fried bread, fresh and hot. Given the horrible start, success was likely going to be not failing, at best. But instead, we ended up receiving her kindness and appreciation.
11. Tell us something people don’t know about the making of this book.
“Steve Portigal” is the pseudonym for an anonymous collective of heartists, Burning Man exonerees, and professional home stagers.
12. Which stories in the book did you personally learn the most from?
Oh, come on. I love all my children equally! The value of any story is most revealed when it’s considered in the aggregate. I learned from the process of analyzing and synthesizing the stories in order to create the book.
13. If someone is feeling burnt out on research, what story would you recommend they read from your book as a pick-me-up?
If you’re really burnt out on research, maybe go read about someone hiking the Pacific Crest Trail? If you aren’t quite at that stage, then maybe Susan Simon Daniels’ story “A Sigh Is Just a Sigh” which is touching as hell, or Jenn Downs’ hilarious (and slightly Bombeckian) “Burns, Bandages, and BBQ.”
14. Knowing what you know now, what advice would you give to your younger researcher self?
Don’t worry…someday there will be more researchers than you can imagine…and the demand for researchers will be more than that community can provide.
15. When you’re 90 and look back on your life, what would you like to be able to say to yourself?
“I still remember eating the last panda. Gosh, that was tasty!”
Steve Portigal is the founder of Portigal Consulting. He’s written two books on user research: Interviewing Users and Doorbells, Danger, and Dead Batteries. His work has informed the development of music gear, wine packaging, medical information systems, corporate intranets, videoconferencing systems, and iPod accessories. Follow Steve on Twitter or listen to his podcast Dollars to Donuts.
The other day I hopped the subway to the Soho Apple Store’s Genius Bar to get my dead iPhone fixed. Being suddenly phoneless is quite disorienting. Rather than folding myself over my little master as I normally would, I looked up and suddenly noticed… people! The sea of diversity you’d expect to see on a New York City subway. And as an old UXer, I was drawn to observe them, exercising dormant field research muscles.
That’s when I realized that I had a book with me: an advance copy of Steve Portigal’s new book, Doorbells, Danger, and Dead Batteries: User Research War Stories.
I couldn’t have had a better companion for the rest of that ride. I dipped into about a dozen of the 60+ field research war stories that make up the bulk of the book. The stories do what stories are supposed to do: engage. And the contributors have been through some experiences that will make you laugh, sweat with fear and discomfort, and—let’s face it—enjoy a bit of schadenfreude.
But it’s wrong to see Steve’s new book simply as a compilation of user research war stories. Let me explain why with a bit of my own publishing war story.
When Steve came to me with the idea for his new book a year or so ago, he was concerned that I wouldn’t want to publish it. After he explained the idea, I wasn’t sure either. I generally hate compilations, as they tend to drown out the main author or editor’s voice. And how useful could a book of user research war stories really be?
Then I thought some more. And I realized that some people have a knack for combing through ideas to arrive at a greater truth. Steve is one of those master synthesizers. I began to believe that if he dedicated the time to really digging into these stories, his sum would be greater than the parts.
In Doorbells, Danger, and Dead Batteries, Steve comes through: he delivers a broader framework that’s useful for making sense of user research—or, actually, situations with people. Eleven chapters deliver eleven principles that you must know if you’re doing any kind of research:
- Chapter 1: The Best Laid Plans
Expect your plan to never to go according to plan.
- Chapter 2: Those Exasperating Participants
Be prepared for people to surprise (and sometimes frustrate) you.
- Chapter 3: Control is an Illusion
Be prepared for research contexts to surprise (and sometimes frustrate) you.
- Chapter 4: Cracking The Code
Be prepared to be challenged by differences in language and culture.
- Chapter 5: Gross, Yet Strangely Compelling
If you feel disgust when observing people, counter it with empathy.
- Chapter 6: Not Safe For Work
Be prepared for research contexts that are unpleasant and occasionally morally challenging.
- Chapter 7: To Live Outside the Law You Must Be Honest
Know your ethics and your obligations before you begin.
- Chapter 8: The Perils of Fieldwork
Be prepared for the discomfort and even danger you may face in the field.
- Chapter 9: People Taking Care of People
Be prepared for people’s lives and situations to pull you from observation to participation.
- Chapter 10: Can’t Stop The Feeling
Like it or not, your emotions will impact your research.
- Chapter 11: The Myth of Objectivity
And, like it or not, observing and learning from people will inevitably change you.
So I’m glad to have my assumptions questioned about what books merit publication. Thanks for that, Steve—and, more importantly, for writing Doorbells, Danger, and Dead Batteries and opening up a greater truth about field research.
It’s been quite a year here at Rosenfeld Media HQ. Four successful events, two new book imprints, and today, we bring your our seventh title of 2016: Blind Spot: Illuminating the Hidden Value of Business (by Steve Diller, Nathan Shedroff, and Sean Sauber).
It’s fair to say that if you’re in any type of business, you and I continually struggle to gain and strengthen customer loyalty. It’s one of our biggest challenges. But despite decades of books and discussion, we haven’t seen a clear way forward. As with other intractable problems, the best approach is often to pause, step back, and reframe.
And this is where Blind Spot nails it: the authors move us from a mindset of mining our customers for loyalty (and money) to creating sustainable relationships that benefit us both.
Relationships with our customers go beyond traditional values of function and finance. In Blind Spot, you’ll learn to understand and develop premium types of value—emotional, identify, and meaningful—which offer far greater opportunities for creating better relationships.
You’ll also learn a new method for modeling these relationships called the waveline, that Bruce Nussbaum calls “the 21st century replacement for the consumer journey”. Finally, Blind Spot offers a 7-step approach to innovation built upon a foundation of strong and healthy relationships.
As always, I hope you’ll pick up a copy—either from the Two Waves site or Amazon. Blind Spot is, like all of our books, available in a lovely paperback edition, as well as four DRM-free digital formats (PDF, ePUB, MOBI, and DAISY). To your success!
Happy book launch day!
It took months of cajoling but I talked Laura Klein into writing Build Better Products: A Modern Approach to Building Successful User-Centered Products. And guess what–she delivered.
Build Better Products is a comprehensive handbook for anyone designing or managing products. It provides easy-to-follow exercises and methods to improve products in six critical areas:
- Goal: Defining the key business need
- Empathy: Understanding user behaviors and needs
- Creation: Designing a new user behavior to meet both business and user needs
- Validation: Identifying and testing assumptions
- Measurement: Measuring changes in user behavior
- Iteration: Doing it again and again–improving each time
This framework—and the advice it delivers—is hugely practical and applicable for just about any product. The more complex your product, the more you need this book. Laura addresses another key need: bringing together the best of product management and user experience design in one place.
And there are two other reasons to pick up a copy of Build Better Products:
- Tricks of the trade from a stellar lineup: Cindy Alvarez, Janice Fraser, Learie Hercules, Avinash Kaushik, Amy Jo Kim, Steve Krug, Dan Olsen, Steve Portigal, Chris Risdon, Kate Rutter, Teresa Torres, and Christina Wodtke.
- It’s funny. Really funny. That’s why I spent all those months cajoling her. You’re very welcome.
I recently got to interview Steve Krug, author of Don’t Make Me Think and Rocket Surgery Made Easy, as the lunchtime entertainment during the User Research for Everyone conference. I always enjoy talking with Steve, which meant that we spent most of our time chatting and then ran out of time to answer all the questions submitted by the audience.
Sorry! To make up for that, Steve and I have answered some of those questions here. A few have been edited slightly for clarity and brevity.
Question: Our company ran an unmoderated usability test where we tasked users to try to find a bit of information via search. 95% of people didn’t type anything in the search box before clicking done. They just wanted the incentive. I looked through all the records of their sessions to see who seemed to put in a good faith effort and who didn’t. But that took a lot of time and effort. How can we get around this?
Laura: The first thing I thought when I read this question was, “You should be watching all the sessions.” That’s the value of usability testing–seeing people use your product. If you’re just making people use the product and then giving them a survey at the end, you won’t get very good feedback.
Most companies that help you do this sort of testing let you rate your participants. If you feel like the participants really aren’t fulfilling their end of the bargain and are just clicking through at random for the incentives, give them a 1-star rating and ask for your money back. Companies aren’t interested in recruiting bad testers for their panels.
In your case, I want to know why they didn’t search. Was it because they couldn’t find the search box? Or because they could find what they were looking for without searching? Did they understand the task? You’ll only understand their motivations for not completing the task if you actually watch the sessions you’re paying to have people run.
Steve: Well said, all of it! You’ll only understand their motivations if you watch what they do (and listen to them thinking aloud, of course). This is fundamental. It’s the reason why qualitative methods give you insight into why things happen, as opposed to quantitative methods that only tell you what happened.
Question: Roughly what % of companies do you believe actually conduct usability testing?
Laura: I have no idea, but it’s far too low. In my experience, it varies wildly based on industry. Even what people count as usability testing varies. For example, a lot of consumer packaged goods companies do lots of customer testing, but they’re not run in the same way that a software company might run it. A lot of companies say they usability test, but what they really mean is that they’ve run a few tests in the past once in awhile. Or they talked to that user only one time.
I’m curious why it matters to you? The important thing is that the percentage of those who test keeps going up!
Steve: 21.5% (in keeping with the notion that 64.4% of all statistics are made up on the spot). Laura’s right: whatever the actual number is, it’s far too low. It’s why I spend a lot of my time showing people how they can reduce the amount of time and effort required, and make sure they’re really doing a usability test (i.e., watching people try to use the “product” while listening to them think aloud). The good news is that even though the percentage is still way too low, it’s much higher than it was a few years ago. And it continues to grow.
Question: What’s the simplest user research we can do?
Laura: As Steve said during the session, and in greater detail in the interview I did with him for my new book Build Better Products (shameless plug–you can buy it now), the simplest form of user research to do is usability testing on competitive products. It’s a fantastic and useful method for getting feedback on products.
Cindy Alvarez also mentioned using this method in her talk. She pointed out that it’s a great way to get people excited about usability testing and start to learn the techniques without threatening to make anybody inside the organization look bad.
Steve: And there are other research methods that are reasonably simple: like doing some user interviews. Interviews can be particularly simple if you do them remotely since you don’t even need to physically “get out of the building.” (Sidenote: the need to at least metaphorically get out of the building is one of the many terrific things the Lean UX movement—and Laura’s books, etc.—have introduced a lot of people to.) Interviews produce whole different kinds of insights than usability tests, but they both can be very rich.
Question: I am currently the UX strategist for a global law firm where I am tasked with designing tools for attorneys. My users are within reach, which is a great thing. But lawyers are pressured to fill all working hours during a working day with billable hours. I can’t figure out how to reach these users given their tight time constraints. Any ideas?
Laura: The easiest thing you can do is to compensate them for their time. Whenever you’re recruiting professionals, it’s important to recognize that their time is valuable and offer them something. It might not be money. It might be a donation to charity or a bottle of wine. But it has to be something that they think is valuable.
Also, try getting smaller chunks of their time. Asking for an hour is harder than asking them for a 15 minute call or screenshare. Offer times at lunch and before or after work. Try to do it remotely or go to them so that you’re not asking as much. Look for other types of lawyers: some in-house counsel jobs have regular salaries, and those lawyers aren’t as likely to have to account for every second. If you’re just doing usability testing, you don’t necessarily need the exact lawyers that you work with to help. You might be able to use other lawyers to see if they understand the interface. That will give you a lot larger pool of possible participants.
Steve: All great. And what about working with management to get the time they spend testing categorized in some way that recognizes its value?
Apart from this corner case (billable hours), testing internal tools (or intranets) is often easy, because you know exactly where to find a large pool of actual users who have a pretty good incentive to help. Often the best subset is new hires, because while they’re actual users with domain knowledge, they’re not yet familiar with the internal tools.
Question: How important is it to do usability testing on a user’s own computer/device/environment? Compared to bringing them into an office?
Laura: It always depends on what you want to learn. When possible, I prefer remote to forcing people to come into your office, since it means you’re not being geographically biased in your participant sample. On the other hand, seeing their environment in person can give you a lot of insight into their context of use.
Steve: And of course remote has so many other advantages, like no travel time and expanding the recruiting pool from “people who live or work near you” to “almost everybody.”
Laura: I’d say that contextual learning (home visits, etc.) is a lot more important for user research as opposed to usability testing. If you’re just interested in seeing if people can perform certain tasks given a particular interface, I’d go with remote testing. If you’re really digging into the way people work or perform tasks in their natural environment, consider whether you need to be more immersed in that environment.
Steve: Only one caveat: if you’re bringing them in, you need to make some effort to ensure that your computer setup isn’t going to get in their way. The most common case is sitting people down at a laptop with a touchpad when they’re only comfortable with a mouse. The solution, of course, is to have both available. You may also run into issues with Mac vs. PC users, but only if the test tasks are going to involve interacting directly with the operating system. If you’re testing something in a Web browser for instance, it probably won’t matter.
Question: What user research would you recommend for non-homogeneous target groups? From users with basic knowledge to super-professionals?
Laura: I’d recommend focusing on a specific subset of people and making things better for that identifiable group. Often, when you try to make improvements “for everybody” you end up making things better for nobody. Understand who you think your changes/new features/improvements are for and what behavior you expect to change, and then usability test on that subset of your users.
Steve: And I’d recommend beginning by focusing on the people who live on the “basic knowledge” end of the scale. If they find it usable, the people with more knowledge/experience will probably be able to figure it out. Remember, you’re not dumbing things down: you’re making things clear. Everyone appreciates clarity—even power users.
Laura: If you are making a global product change, make sure you understand the different segments of your users and recruit participants from each segment. You’ll most likely want to run a few more sessions than you would otherwise, to make sure you don’t have just one of any group.
Question: What are some ways you’ve successfully distributed findings from usability testing—particularly when some of the most interesting findings aren’t relevant to the topic you were researching?
Laura: It depends on who the user is of the findings. I wrote a post on deliverables that might be helpful. Whenever you create a deliverable of any sort, treat the recipient of the deliverable as a user of a product. Present the findings in an appropriate way for that person.
In general, the best way to make people familiar with the findings is to invite them to help with analysis, but I understand that that’s not always realistic.
Question: We have a really distributed team. What are some good strategies for getting everyone to watch people use our software when it means asking everyone to watch two hours of video?
Laura: Does everybody need to watch all two hours of video, or could you cut a highlights reel? I’ve had better luck going through the videos and grouping responses or tasks together to make it easier for people to consume.
Another option is to schedule an online analysis session with stakeholders where you use a digital whiteboard like Murally to share insights. You can require stakeholders to watch the research ahead of time. Just givethem a deadline and a reason to encourage them to watch the videos.
Steve: If you don’t have the time to create a highlights reel, you can simply crop out the time-wasting parts. And then give the viewers a tool that lets them watch what’s left at faster-than-normal speed. I never watch a usability test video at less than 1.4x normal.
Question: If you only have three slots and test three different pieces of functionality, is there enough enough feedback to recommend changes?
Laura: It depends, unfortunately. There have been usability tests where I knew there were enormous problems after half a session. I still ran a few more sessions after that to confirm, but it was very clear. In general, you run the sessions you need to run to start to see patterns.
Question: For an overall junior team, is it fine to start out doing internal usability testing–despite its limitations due to things like bias–to get up to speed and gain confidence?
Laura: Sure. In fact, I really like the suggestion from one of the speakers that you run a pilot on a member of your own team (like the designer or PM) at the beginning of all usability testing, regardless of how senior your moderators are.
Just keep in mind that what you’re learning from internal people will likely be extremely biased. Use it as a training tool, but I wouldn’t take the results nearly as seriously as I would results from real or potential users.
Also, ask yourself whom you’re protecting by keeping your junior folks away from users. If you feel like they’re going to damage customer relationships, try getting them some one-on-one coaching or training. Then have them practice on friendly customers. If you’re worried that they’ll get bad results, they’re no more likely to get bad results from external participants than they are from internal ones.
Steve: Yes, pilot tests. It’s too bad we didn’t get to talk about pilot tests since they’re absolutely essential. If you don’t do a quick pilot test before you bring your participants in, you’re not doing it right.
Question: How strongly do you feel about having one person per session? I’m part of a team developing an internal tool for a large company. I’ve got access to users in-person (our employees) twice a month and my boss thinks it’s better to test or interview two or three people at a time.
Laura: I have very strong feelings about this. As I mentioned during the Q&A session, I’d want to know why my boss wanted to interview two or three people at a time. Is it for convenience? Or because they think you’ll get better results? Hopefully you can talk them out of it, because neither scenario is true. You’ll end up getting a lot of echoing and learn more about power dynamics in your company by interviewing multiple people at once.
If it’s a tool that people will use jointly, it can make a lot of sense to structure sessions in a way that helps you understand how that will happen.
Question: The big benefit of the model that you’re describing for usability testing comes from synchronous participation (and discussion afterwards.) How do you apply this model in globally distributed teams?
Laura: Well, you can get huge benefits from usability testing when it’s just the researcher/designer/PM doing the research, finding problems, and fixing them. You don’t technically need everybody involved in that process.
The more people you have involved, the easier it will be to get the changes made. Especially if you’re in an environment where people tend to want to ship it, forget it, and move on to the next thing. Get creative—create things like video highlight rolls that you can share widely or snippets of information that are easy for people to digest.
Steve: It’s true: you don’t technically need everyone involved in the debrief process. But I think it’s worth bending over backwards to get as many people involved as you can (hence my “make it a spectator sport” maxim). Having more people involved does increase your chances of having durable buy-in and getting the changes made. But you’re also filtering the problem through more minds with different perspectives and valuable bits of information about where the bodies are buried. I also think that the process of observing and debriefing makes people better at their jobs, something I think of as “informing their design intelligence.”
For globally distributed teams, maybe some people watch the recorded sessions at a more convenient hour, then come together the next day for a full-group debriefing session.
Question: Are your users who you think they are?
Question: Laura, what are the three most common questions you get about research?
- How do I recruit people?
- How do I answer this particular question?
- How do I convince people that research is valuable?
Not necessarily in that order and not necessarily in those exact words. I was happy to see that those were big themes in our research before the conference, since now I can point people to a bunch of experts answering those questions.
Question: In a publishing environment, the editor is considered the authority on editorial content. What’s the best way to disavow editors of the notion that their editorial power also carries into UX of that editorial content?
Laura: Ha! You could literally replace “editors” with any other professional who is an expert in their thing – lawyers, doctors, scientists, etc.
Be open to what they’re recommending. Remember, they do have more domain expertise than you do. Understand why they want what they want. It may be that there’s a third solution that takes their concerns into account better than what you’re proposing.
And never underestimate the power of testing—either usability or a/b. If you’re advocating for one solution and your stakeholder is advocating for another, you’re going to have to devise a test that will definitively show which one of you is correct. This is literally what a/b testing is great for if you have enough users. If you don’t, then having a neutral party run usability tests on two different variations of a prototype can also be very eye opening.
Question: What do you think about pinging other UX professionals for usability and testing of prototypes other UX pro’s are working on?
Laura: I’m not a fan of asking other UX folks to do usability testing of prototypes, unless, it’s a product for other designers. You’re typically much better off testing on a few of your real users or potential users rather than asking someone who does this for a living.
Remember, UX designers aren’t magic. We don’t automatically know what problems real people will have and instinctively know how to fix these problems. If we did, we wouldn’t need to usability test! UX designers have the curse of knowledge. We don’t look at a product the same way other people look at it. If it’s a product for a specific group of people, designers can lack the domain expertise to give you decent feedback about what might be confusing.
That said, after you’ve run usability tests with your target market, other UX designers can often be very helpful in suggesting other ways to approach the problems you’ve identified. They may have seen other users have similar problems and have ideas about what worked to fix those problems.
But you still have to test with users.
Question: What’s the approach for testing content heavy, inspirational websites? Think TED.com.
Laura: Same approach for everything else–figure out what you’re actually testing and then come up with something that will help you learn what you need to know. Are you trying to test whether the website is usable? Inspirational? Understandable? Useful? Those are all very different types of experiments that you want to run.
The type of product is not what determines how you test. You test based on what you want to learn.
Question: I’m wondering if we’re too biased to effectively help others.
Laura: Depends on what you mean by “we” and “help.” Humans are extremely biased. It’s kinda our thing. There are ways to acknowledge and counter that bias to some degree, although I don’t think that most of us will ever even come close to getting rid of it entirely.
In general, the more time you spend listening to others and the less time you spend thinking about you and how you would do things, the less biased you’ll be.
Question: I often hear “I would have designed it differently.”
Laura: Yeah. We all do!
The best follow up is, “I see. Why would you do it that way? What problem would that solve for you?”
Frankly, I don’t care if they want a big red button to push that says “push me” on it. I want to know why they think that button will help them. Think of solution requests as a jumping off point to understand what their problem is.
Steve: We in the usability racket like to say “users aren’t designers.” And the truth is, they aren’t. Every once in awhile (once in a great while, actually), a test participant will make a terrific suggestion. You know when this has happened, because everyone in the observation room slaps their forehead simultaneously and says “Why didn’t we think of that?” But 98% of the time, design suggestions from participants aren’t great. I always recommend listening politely while they make the suggestion so they know you’re paying attention to them. Then at the end, after they’re finished with the tasks and you’ve moved on to open-ended probing, you can remind them of their idea and ask them to explain how it would work. In my experience, they’ll almost always end up by saying some variant of “But actually, I probably wouldn’t use [the new version] because [reason]. It’s better the way it is.”
Question: Is there a point of diminishing returns with too little user research? That is, is there a fear that a little learning can be a dangerous thing, leading to a false sense of UX security?
Laura: Sure.Talking to one person and then immediately diving in and solving all of their problems can certainly cause problems. If that person is an outlier. You don’t necessarily want to design your product around exactly one user. On the other hand, you can also spend an awful lot of time trying to “prove” yourself right.
I wrote a blog post on Predictive Personas that is probably applicable here. You can use this for usability testing just as easily as for predicting who will use your product. The idea is to run some sessions, try to spot patterns, predict what you’re going to see next, and then see if your predictions hold. Once you’re making predictions accurately, you’ll have a good idea of the recurring problems in your product and can more confidently start fixing them.
Laura Klein is a Lean UX and Research expert in Silicon Valley who teaches companies how to get to know their users and build products people will love. She’s a Rosenfeld Media expert and author of Build Better Products, and UX for Lean Startups (O’Reilly). Follow her on Twitter or subscribe to her blog and podcast at Users Know.
Steve Krug is best known as the author of Don’t Make Me Think: A Common Sense Approach to Web Usability and Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability Problems. Steve spends most of his time teaching usability workshops and consulting. Follow him on Twitter.
Buy on-demand access to the User Research For Everyone program.
I’ve got two kids, 8 and 12 years old. With issues like race relations, income inequality, wars without end, and presidential politicking here in the US, it’s harder and harder to make sense of the world for them.
It’s hard to even explain it to myself. While I’m doubtful that the good old days were all that good (thank you, Steven Pinker), I do know that there are many, many factors—from where we choose to live, to (ahem) Facebook—that make me feel like we’re all busily locking, loading, and reinforcing our concrete bunkers. And doing everything we can to build walls between ourselves and the Other. Whoever Other is for us. Opportunities to interact with and understand people with different perspectives are disappearing as fast as the ice caps. It’s not a happy, much less tenable, situation.
That’s why launching Dave Gray’s new book, Liminal Thinking, is so meaningful for me.
Sure, it’s always exciting to publish something new. I like to think our UX books have, in some small way, made the world a better place for users.
But Liminal Thinking—the first in our Two Waves series—has the opportunity to make the world a better place for people. Not just for the ones who make and use products and services. But for anyone who understands the need to break down their walls of belief and connect with others.
Liminal Thinking is, if anything, a self-help book, and it’s quite practical. Six principles, nine practices, 184 pages, and lots of Dave’s deceptively simple illustrations. That’s it. You’ll read it in a couple hours, and it might just change your life. Seriously. See what others have shared in Amazon reviews so far.
A decade ago, there was no publishing house focused solely on user experience design, so I founded Rosenfeld Media. If the world needed better experiences, then UX practitioners needed more and better books to help them design those experiences.
We started out publishing books on highly practical UX methods, like card sorting, prototyping and mental models. Many twists and turns later, our recent and upcoming titles cover as much strategic topics—like product management—as UX. UX has evolved into its own field—and it’s not close to being “done”.
To make sure that the books we publish stay relevant to UX people, we need to make them relevant for other people too. Let me explain.
Many of the people who’ve been in UX the longest now lead teams, business units—even companies. As a design leader, you’re likely to be more focused on infrastructure, like design systems, than on design itself. You’ve had to learn how to lead organizational change, recruit and hire talent, influence and negotiate and master other “soft” skills. More and more, design leaders and changemakers have their fingers on the levers of strategy.
The senior UX people I’ve spoken with find themselves collaborating with people who are outside the UX tribe: founders, product managers, C-level and middle managers, IT and marketing people. These “others” share a common realization: UX delivers business value. It gives products and organizations a clear competitive advantage. “Traditional” business people are looking to UX to solve larger scale problems and tap business opportunities.
I love that these two waves are converging. But what do we call this bigger umbrella? Design thinking? Design strategy? Meh. I just want Rosenfeld Media to be there and, as a publisher, help guide this movement in a productive, positive way. That’s why we’re launching Two Waves Books.
Two Waves Books will cover a spectrum of creative leadership. We’re working with authors who’ve got experience surfing and navigating both waves. We’re leading off with Dave Gray’s Liminal Thinking (which is now available for pre-order), and Blind Spot, by Steve Diller, Nathan Shedroff, and Sean Sauber.
I’ll report more signings over the coming months; can’t wait to share these with you.And we’ve just signed Catherine Courage and Richard Dalton to write Design to Drive.
With Two Waves, our other new line Digital Reality Checks, and our continuing line of UX books for practitioners, I’m excited that Rosenfeld Media can help define another emerging field, and build bigger and better umbrellas.
The other day, I was doing a final check for a voter guide project. An elections office had created a sample PDF, asked me to check it and identify any problems, then figure out how to solve them. We’d gone back and forth for a few days.
When I finally sent off the email saying all the ducks were in a row, and that the office could start printing the PDF booklet, I breathed a sigh of relief.
And then, like you do, I took to Twitter to share some of my frustration. Much of my time had been devoted to handling the tedious minutiae of setting, checking, and double-checking technical settings I’d chosen in Word–wondering if I’d checked off the right option. So I tweeted this:
It’s not that you can’t create an accessible pdf from an accessible Word file.
It’s that it’s so easy to get it wrong.
I didn’t expect the tweet to launch a Twitter conversation that went on for two days. The response prompted me to write a longer form post.
PDFs are not always the best way to share information. Most of the time, you’d find me standing at the head of the line advocating for HTML. Sometimes though, PDFs are the right solution. Particularly for long reports or other documents.
For the voter guide project, PDF is the better option. And assuming you begin with an accessible file, turning it into a PDF ought to be the easiest, fastest way to get from digital source to an accessible printed report. Sadly this isn’t the case.
In the voter guide project I just worked on, the source file was Word. I’d heard that InDesign does do a better job of creating PDF files. And while that’s good advice, it doesn’t address the question of why it’s so hard to make a relatively simple, accessible Word file into an accessible PDF.
Worse, I’ve taken a look at the documentation on creating an accessible PDF from InDesign and it doesn’t look that easy. Powerful, yes. But not easy.
Why does it matter if it’s easy or not? You shouldn’t need an expert to create accessible documents! There simply aren’t enough experts to go around. Templates can help. Cheatsheets and checklists are good reminders. If accessible documents require experts to make them, it’s no wonder there are so few accessible documents out there. Without accessible documents, millions of people lose the ability to get the information to participate in their country’s elections.
But this is about more than elections. In every business and university, people create documents every day. People whose expertise or main focus probably isn’t technology or accessibility. This isn’t a criticism. It’s the real world–and we’re not yet serving everyone’s needs.
Designing for accessibility ought to be easy. It ought to be the default for software. The features to create documents shouldn’t be hidden behind 3 or 4 clicks–every single time you need to use those settings.
Here’s what a typical conversation sounds like helping someone through the process:
Them: I know I’m supposed to put alt text on this image, but I can’t figure out where it goes.
Me: It’s in the formatting properties.
Them: Where? I see Glow and Artistic Effects and things like that…
Me: See the little blue square with the arrows.
Them: Oooooh. That’s where it’s hiding.
And that’s a good conversation. Sadly, too many sound like this:
Me: See the luggage tag on the left. Open that. Now click on the thing that says “Tags” at the top of the panel and…
Them: Wait. What….what is all this? This looks…complicated. How am I going to learn this?
Me: Just send me the file and I’ll fix it.
In both cases, creating accessible information has been made harder than it should be, or downright scary. When accessibility is too much work, it’s a hidden tax.
The good news is we can fix this. I suggest two principles for programs that could make the world more accessible:
Make software defaults that support accessibility.
And make sure the defaults don’t actively harm accessibility. For example, the default Word option to “Bitmap text when fonts may not be embedded” seems like a good thing, but it turns text into little snippets of images. You might not notice this with a visual inspection, but it causes real problems for screen readers. Don’t take the option away…just make it clear what will happen. And don’t make it the default.
Make accessibility features easy to find.
Don’t hide or require several clicks to get to these features. Or make users guess at what they mean. For example, the table property to “Repeat as header row at the top of each page” also sets the row as a header for screen readers, but how would anyone know that this is important (or why it’s important). Maybe the default should be that the first row is set as a heading row. It can be changed, but if you do nothing, you have a more accessible table.
So, what happened to the elections project?
If you’re curious, the booklet was turned into 85 versions. They will be voter guides with information about the candidates and measures on the ballot (along with some basics about ways to vote).
There are 85 of them because there’s not just one ballot for the whole county or state. In New Jersey, where we have a lot of local government, each borough, town, and township has a different ballot. In many places, there are overlapping water districts, fire districts and school districts on top of the boundaries for local offices. I’ve seen the GIS mapping layers for suburban Cook County outside of Chicago, and just thinking about them makes my head hurt.
A large county can have hundreds of different ballots. Even a small county might have dozens. Each is a unique combination of contests for one geographical area. Because the schedule is crazy fast, there’s only a few days between when the list of candidates and measures are certified and when the booklets (and ballots) have to be ready to go to the printer and be posted on the web.
Getting it right is important. The right ballot content. The right languages. The right legal notices. And getting accessibility right, too.
But most of all, easy counts.
It’s the only way we’re going to make real progress.
What you can do
P.S. Since the most obvious recommendation is to “take it up with the vendors” when you have a problem–I did. Because if they never hear about the problems, they can’t fix them. I’ve been talking to both Adobe and Microsoft. They both have online forums. If this matters to you, let them know.
Whitney Quesenbery is the co-founder of the Center for Civic Design. She’s passionate about making interactions with government effective and enjoyable, and bringing more design literacy to elections and government workers. She’s co-authored three books— A Web for Everyone: Designing Accessible User Experiences (with Sarah Horton), Storytelling for User Experience: Crafting Stories for Better Design (with Kevin Brooks) and Global UX: Design and Research in a Connected World (with Daniel Szuc). Follow Whitney on Twitter.
Let’s be honest: many organizations don’t know how to successfully do user research. Too often research gets treated like an item to knock off a checklist just before products get shipped out. Or it’s skipped altogether because “there’s just no time.” Even with good intentions, the lack of in-house expertise can stymie us. We end up conducting focus groups. Or approaching strangers in Starbucks–asking them how much they’d pay for our new product.
We researchers, designers, and product managers want to understand what makes great products that customers will buy, use and recommend to friends. But without research, this can feel impossible.
This is such a pervasive pain point that Rosenfeld Media has made user research the topic of our next virtual conference: User Research for Everyone. The goal: to help you figure out how to build a team that is excited about research and empower it with the tools to make research useful.
Before designing the program, I helped Lou conduct some research (obviously). Research on user research–yep, pretty meta. Almost 200 of you weighed in and told us the areas you struggle with most, and who you most want to learn from. The responses clustered nicely into six prevailing topics:
“How do I convince people research is important?”
Convincing leaders and teams to see the value of user research is the top question you submitted. We agree. Research works when everybody is on board, but making people care is challenging. Especially if you work in a large organization and need to persuade multiple stakeholders with competing agendas.
“How do I turn this research into better products?”
We’re often told to get out of the building and talk to users, but that can create a five-story pile of research. Distilling and choosing the right insights to inform product decisions is time consuming and daunting.
“How do I make sure everybody understands the research?”
Research can only inform design decisions if everybody on the team understands (and agrees on) the results. How many of you have created a 30-page deck that nobody ever looks at? Many of you said you’re eager to discover more compelling ways to share and evangelize research that people want to absorb and use.
“How can we do this faster and cheaper?”
Most of you face real budget or time constraints–making research seem like a pipe dream. But it doesn’t have to stop us from learning from users. A lot of respondents want techniques for doing remote research to cut costs, or to shorten the timeframe to work better in an agile environment. Luckily, there are a lot of tools and tactics available these days to help make good research achievable and affordable for any budget.
“How do I pick which research to do?”
Many of you have difficulty picking the right methodology to answer your questions. Even experienced researchers can struggle to understand how to incorporate quantitative data into the research process. Many of you request better frameworks for deciding which research to do when.
“How do I get more participants?”
Even when you’ve got a good understanding of who your users are, there’s still the challenge of getting them to talk to you. This can be especially tough when you’re in the early days of a product development and don’t yet have real users. In established companies, you need to coax sales or marketing people to let you talk to customers. There are better ways to reach the right users without resorting to begging or bribery.
Getting to Answers
Join us on October 11, where we’ll tackle these important questions with the experts you requested most: Abby Covert, Steve Krug, Erika Hall, Nate Bolt, Leah Buley, Cindy Alvarez, Julie Stanford and me (as moderator). The program includes Q&A opportunities with every expert–so you can ask your most top-of-mind questions.
Buy a ticket for yourself–or your team and build the user research toolkit you’ve wanted. Your ticket also gives you unlimited access to downloads and replays of the full program in case you can’t make it.
Thanks to all of you who participated in the survey. Hope to see you there! And hit us up with any questions and thoughts below.
Laura Klein is a Lean UX and Research expert in Silicon Valley who teaches companies how to get to know their users and build products people will love. She’s a Rosenfeld Media expert and author of UX for Lean Startups (O’Reilly). Her newest book, Build Better Products, comes out in late 2016. Follow her on Twitter or subscribe to her blog and podcast at Users Know.
I’m thrilled to let you know that our next virtual conference—User Research for Everyone—takes place October 11. If you have a team or colleagues (e.g., developers and product managers) that need to get familiar with the basics of user research, this is spot on. Or if you’re a new user researcher or just need to brush up your skills, you’ll definitely benefit from attending and learning from a fantastic lineup of speakers.
User Research for Everyone is a one-day affair that works well for a team ensconced in a conference room (don’t forget to order in). Or join in from home in the comfort of your slippers and PJs. If you have to get up and walk the dog, no worries—the entire day’s recordings are included with your registration.
Laura Klein and I are curating the event, and we did extensive user research to develop the program and speaker lineup (and if you helped: THANKS!). So we’re pretty confident that you’ll enjoy what we’ve come up with:
- Just Enough Research with Erika Hall
- The Right Research Method For Any Problem (And Budget) with Leah Buley
- How to Find and Recruit Amazing Participants for User Research with Nate Bolt
- Do-It-Yourself Usability Testing Discussion/Q&A Session with Steve Krug and Laura Klein
- Creating a Virtuous Cycle: The Research and Design Feedback Loop with Julie Stanford
- Making Sense of Research Findings with Abby Covert
- Infectious Research with Cindy Alvarez
One last note: the early bird registration deadline is September 13. Hope you’ll join us in October!
- Chapter 1: The Best Laid Plans