Lou Rosenfeld tackles the differences between producing books and conferences, and how to make conferences a little deeper—like books! Read on in Medium.
Those who know me well will laugh, but I actually started out as a project manager. It was the late 80’s. I was in grad school when group projects first became the rage. But professors didn’t bother to teach us how to manage the projects they assigned us. So my teammates and I would scramble around like ants without a trail to follow. We’d duplicate each other’s work, fall behind schedule, point fingers at each other. Eventually I’d volunteer for the dreaded responsibility of tracking our projects. Except I wasn’t equipped for the role so things turned from bad to worse. I could’ve used some guidance back then. Like Brett Harned’s project management book, Project Management for Humans–that just came out today!
Maybe like me, you fell by accident into project management. Or you work with project managers and yet, things feel close to coming unglued. Project Management for Humans teaches you how to recoup your time, resources and sanity. It’s a short, practical and enjoyable playbook you’ll want to read and keep handy to help you resolve problems before they mushroom into crises.
Even if you’re a professional project manager, Brett’s project management book can help you too. It goes beyond teaching traditional systems. You’ll learn how to tackle the interpersonal challenges that can often derail a project in unexpected ways.
The timing for our newest book Designing Agentive Technology couldn’t be better. AI has moved from being the “next big thing” to being the thing for designers to grapple with. I’ve even done some research that demonstrates how important AI and machine learning are to UX people.Technology has been getting smarter for years, and many of us have already been integrating AI into designed experiences. Think Siri, Alexa, wearables, automatic pet feeders, self-driving cars.
If you’re venturing into the world of AI this year (or just thinking about it), there are two reasons you should consider picking up a copy:
You can read a free excerpt from Designing Agentive Technology from Designing Agentive Technology in today’s A List Apart. The book—like all of our titles—is available in just about every format you might need: a lovely color paperback, and PDF, MOBI, EPUB, and DAISY digital formats. You can purchase your copy from us directly, or make Jeff Bezos even richer. Let us know what you think of Designing Agentive Technology.
- It’s practical. When it comes to AI, there simply aren’t a lot of books—yet—that provide such practical guidance to designers. Kudos to Chris for making designing agents clear and concrete.
- It’s necessary. Technology always races ahead, forcing us slow-moving humans to catch up with its impacts. Those impacts can be troubling and even destructive. We need to work harder to humanize the technologies we create, and no single group will be more important to those efforts than designers. See this book as preparation for rehumanizing AI.
We’ve cued up our next virtual conference for Tuesday, April 25 and it’s called To Be Designed. If you’ve been wondering how to keep pace with new trends in design, don’t miss this one.
To Be Designed will give you a taste of the “near future” of design—the stuff that’s far out enough to be fascinating, but not so far out that you can’t imagine ever working on it in your career.
What kinds of topics will we cover? Well, like always, we asked YOU. 331 of you responded, and here’s what you suggested:
By popular demand, the presenters will cover aspects of how AI intersects with design, wearables, conversational UI, smart objects, and—perhaps most importantly—the ethics required to navigate the near future.
And you’ll hear from six speakers we know you’re going to love—because you suggested them! We’ll announce them soon! In the meantime, you can register today at the early bird rate and take it on faith that like our previous conferences, we’ll deliver a well curated program for you.
Or wait until March 24, when the speaker lineup goes public––and when the regular ticket rate will apply).
Looking forward to having you join us on April 25!
Those familiar with Steve Portigal‘s work know him as a widely-regarded expert in user research. Steve has spent over 15 years interviewing hundreds of people, from families eating breakfast, to rock musicians and radiologists. His latest book Doorbells, Danger, and Dead Batteries gathers 65 stories about research gone wrong. Because when you research real people, life is often unpredictable (and enlightening).
We felt it fitting to turn the interview tables around and ask Steve a series of 15 questions to learn more about what makes his brain tick. Enjoy.
1. Where were you born?
Winnipeg, Manitoba. Best bagels in Canada. So suck it, Montreal! Well, I probably prefer Montreal now.
2. Where did you grow up?
Burlington, Ontario, Canada. Although it was a small town back then (I remember when we got our first McDonald’s), now it’s basically a suburb of Toronto.
3. Three words that describe your childhood?
Kenobi. Simmons. Cheech.
4. Three things you never leave home without?
Wallet, keys, and an appetite (for destruction, of course).
5. What’s the best designed product you’ve ever used?
Timbits®—Bite-sized morsels of traditional donuts.
6. What’s the story behind how you got into user research?
I was working at a design agency that was tentatively experimenting with a new service offering—insights that were “left of the idea” (yes, that was actually how they tried to market generative research work). My putative boss literally stopped speaking to me, and wasn’t putting me on projects (the sort of thing that generally requires talking), so the team doing this research work took me in. In the beginning, they had me watch videos and make notes. Then they let me go into the field and hold the video camera. Eventually I got to ask one or two questions, and as time wore on, I began to lead interviews and then plan and manage research. During that time period Don Norman (or was it Don Knotts?) appeared before me in a dream, clad in diaphanous robes. He marked me with the Sigil of Lamneth and bid me sternly to pursue this holiest of professions. That sealed the deal for me.
7. What other profession would you like to try if you could?
I’m fascinated by the television writer’s room. I haven’t come across any depictions of it that make it sound pleasant, but the collaborative creativity is fairly seductive. Otherwise, something about tending to the emotional needs of bugs.
8. What’s the most embarrassing thing that’s happened to you in the field?
Once I was in the home of people who were relatives of Mayim Bialik, the girl who’d played “Blossom” on the TV show “Blossom.” I learned this because I saw her photo on the fridge. During the interview, I referred to her as “Blossom” and one of the family members pointedly corrected me, saying that her name is Mayim, and that Blossom was a character she played. The woman was right and I was being a bit insensitive. I think I was trying to be clever. Although this was after the show was off the air (Mayim was a college student at the time), that name and the essence of that character were strong cultural ideas. I mean, check out the show’s opening credits.
Okay, I’ve got one more. I was interviewing an African-American woman about music. She was really into artists and genres that are heavily African-American. As she told me about what she listens to, I kept looking over at this cool poster of Mick Jagger above her cabinet. When the interview was wrapping up, I tried too hard to find some common ground, musically, so I asked her, “Tell me about that poster of Mick Jagger?” She looked confused. It was Bob Marley. I DO know the difference between the two, but from where I was sitting, I swear he looked like Mick Jagger. I was embarrassed that my need to connect with her about “my” stuff looked like an inept and even-needier attempt to connect with her.
Takeway: Don’t mention pop culture figures by name?
9. What’s the most surprising thing that’s happened to you in the field?
Surprises are mostly internal moments, where I uncover a stub of my own judgment. As an example, I interviewed a man who was the head of an agency that shared his name. He was in his mid-60s with a head of white hair. I was steering the interview towards his past accomplishments, but he was so much more focused on his current goals. I realized I’d created my own narrative for this guy based on his age and that was completely inaccurate. So the surprise wasn’t about the fact that he was engaged and forward-looking. It was about the gap between my unspoken assumptions and the truth that unspooled before me. Honestly, the revealing of and subsequent dismantling of my assumptions is the most pleasurable part of doing fieldwork.
10. What’s the most heartwarming thing that’s happened to you in the field?
I tell this story in detail in my previous book, Interviewing Users. It involves a home interview where the participants were two young men still living at home, who hadn’t told their parents we were showing up for breakfast. But they wouldn’t speak in words and unwilling to talk with us. The parents were unsurprisingly hostile about our presence. Sitting in their kitchen, the mother (who we eventually pivoted to for the interview) told us that few people are welcomed into their house and that food is a carrier of meaning for their family and is not for strangers. We managed to have an incredible interview with her and her husband, after navigating extreme awkwardness and ambiguous permissions. When wrapping up, she told us, “No one comes here and doesn’t get food,” and made us some fried bread, fresh and hot. Given the horrible start, success was likely going to be not failing, at best. But instead, we ended up receiving her kindness and appreciation.
11. Tell us something people don’t know about the making of this book.
“Steve Portigal” is the pseudonym for an anonymous collective of heartists, Burning Man exonerees, and professional home stagers.
12. Which stories in the book did you personally learn the most from?
Oh, come on. I love all my children equally! The value of any story is most revealed when it’s considered in the aggregate. I learned from the process of analyzing and synthesizing the stories in order to create the book.
13. If someone is feeling burnt out on research, what story would you recommend they read from your book as a pick-me-up?
If you’re really burnt out on research, maybe go read about someone hiking the Pacific Crest Trail? If you aren’t quite at that stage, then maybe Susan Simon Daniels’ story “A Sigh Is Just a Sigh” which is touching as hell, or Jenn Downs’ hilarious (and slightly Bombeckian) “Burns, Bandages, and BBQ.”
14. Knowing what you know now, what advice would you give to your younger researcher self?
Don’t worry…someday there will be more researchers than you can imagine…and the demand for researchers will be more than that community can provide.
15. When you’re 90 and look back on your life, what would you like to be able to say to yourself?
“I still remember eating the last panda. Gosh, that was tasty!”
Steve Portigal is the founder of Portigal Consulting. He’s written two books on user research: Interviewing Users and Doorbells, Danger, and Dead Batteries. His work has informed the development of music gear, wine packaging, medical information systems, corporate intranets, videoconferencing systems, and iPod accessories. Follow Steve on Twitter or listen to his podcast Dollars to Donuts.
The other day I hopped the subway to the Soho Apple Store’s Genius Bar to get my dead iPhone fixed. Being suddenly phoneless is quite disorienting. Rather than folding myself over my little master as I normally would, I looked up and suddenly noticed… people! The sea of diversity you’d expect to see on a New York City subway. And as an old UXer, I was drawn to observe them, exercising dormant field research muscles.
That’s when I realized that I had a book with me: an advance copy of Steve Portigal’s new book, Doorbells, Danger, and Dead Batteries: User Research War Stories.
I couldn’t have had a better companion for the rest of that ride. I dipped into about a dozen of the 60+ field research war stories that make up the bulk of the book. The stories do what stories are supposed to do: engage. And the contributors have been through some experiences that will make you laugh, sweat with fear and discomfort, and—let’s face it—enjoy a bit of schadenfreude.
But it’s wrong to see Steve’s new book simply as a compilation of user research war stories. Let me explain why with a bit of my own publishing war story.
When Steve came to me with the idea for his new book a year or so ago, he was concerned that I wouldn’t want to publish it. After he explained the idea, I wasn’t sure either. I generally hate compilations, as they tend to drown out the main author or editor’s voice. And how useful could a book of user research war stories really be?
Then I thought some more. And I realized that some people have a knack for combing through ideas to arrive at a greater truth. Steve is one of those master synthesizers. I began to believe that if he dedicated the time to really digging into these stories, his sum would be greater than the parts.
In Doorbells, Danger, and Dead Batteries, Steve comes through: he delivers a broader framework that’s useful for making sense of user research—or, actually, situations with people. Eleven chapters deliver eleven principles that you must know if you’re doing any kind of research:
- Chapter 1: The Best Laid Plans
Expect your plan to never to go according to plan.
- Chapter 2: Those Exasperating Participants
Be prepared for people to surprise (and sometimes frustrate) you.
- Chapter 3: Control is an Illusion
Be prepared for research contexts to surprise (and sometimes frustrate) you.
- Chapter 4: Cracking The Code
Be prepared to be challenged by differences in language and culture.
- Chapter 5: Gross, Yet Strangely Compelling
If you feel disgust when observing people, counter it with empathy.
- Chapter 6: Not Safe For Work
Be prepared for research contexts that are unpleasant and occasionally morally challenging.
- Chapter 7: To Live Outside the Law You Must Be Honest
Know your ethics and your obligations before you begin.
- Chapter 8: The Perils of Fieldwork
Be prepared for the discomfort and even danger you may face in the field.
- Chapter 9: People Taking Care of People
Be prepared for people’s lives and situations to pull you from observation to participation.
- Chapter 10: Can’t Stop The Feeling
Like it or not, your emotions will impact your research.
- Chapter 11: The Myth of Objectivity
And, like it or not, observing and learning from people will inevitably change you.
So I’m glad to have my assumptions questioned about what books merit publication. Thanks for that, Steve—and, more importantly, for writing Doorbells, Danger, and Dead Batteries and opening up a greater truth about field research.
It’s been quite a year here at Rosenfeld Media HQ. Four successful events, two new book imprints, and today, we bring your our seventh title of 2016: Blind Spot: Illuminating the Hidden Value of Business (by Steve Diller, Nathan Shedroff, and Sean Sauber).
It’s fair to say that if you’re in any type of business, you and I continually struggle to gain and strengthen customer loyalty. It’s one of our biggest challenges. But despite decades of books and discussion, we haven’t seen a clear way forward. As with other intractable problems, the best approach is often to pause, step back, and reframe.
And this is where Blind Spot nails it: the authors move us from a mindset of mining our customers for loyalty (and money) to creating sustainable relationships that benefit us both.
Relationships with our customers go beyond traditional values of function and finance. In Blind Spot, you’ll learn to understand and develop premium types of value—emotional, identify, and meaningful—which offer far greater opportunities for creating better relationships.
You’ll also learn a new method for modeling these relationships called the waveline, that Bruce Nussbaum calls “the 21st century replacement for the consumer journey”. Finally, Blind Spot offers a 7-step approach to innovation built upon a foundation of strong and healthy relationships.
As always, I hope you’ll pick up a copy—either from the Two Waves site or Amazon. Blind Spot is, like all of our books, available in a lovely paperback edition, as well as four DRM-free digital formats (PDF, ePUB, MOBI, and DAISY). To your success!
Happy book launch day!
It took months of cajoling but I talked Laura Klein into writing Build Better Products: A Modern Approach to Building Successful User-Centered Products. And guess what–she delivered.
Build Better Products is a comprehensive handbook for anyone designing or managing products. It provides easy-to-follow exercises and methods to improve products in six critical areas:
- Goal: Defining the key business need
- Empathy: Understanding user behaviors and needs
- Creation: Designing a new user behavior to meet both business and user needs
- Validation: Identifying and testing assumptions
- Measurement: Measuring changes in user behavior
- Iteration: Doing it again and again–improving each time
This framework—and the advice it delivers—is hugely practical and applicable for just about any product. The more complex your product, the more you need this book. Laura addresses another key need: bringing together the best of product management and user experience design in one place.
And there are two other reasons to pick up a copy of Build Better Products:
- Tricks of the trade from a stellar lineup: Cindy Alvarez, Janice Fraser, Learie Hercules, Avinash Kaushik, Amy Jo Kim, Steve Krug, Dan Olsen, Steve Portigal, Chris Risdon, Kate Rutter, Teresa Torres, and Christina Wodtke.
- It’s funny. Really funny. That’s why I spent all those months cajoling her. You’re very welcome.
I recently got to interview Steve Krug, author of Don’t Make Me Think and Rocket Surgery Made Easy, as the lunchtime entertainment during the User Research for Everyone conference. I always enjoy talking with Steve, which meant that we spent most of our time chatting and then ran out of time to answer all the questions submitted by the audience.
Sorry! To make up for that, Steve and I have answered some of those questions here. A few have been edited slightly for clarity and brevity.
Question: Our company ran an unmoderated usability test where we tasked users to try to find a bit of information via search. 95% of people didn’t type anything in the search box before clicking done. They just wanted the incentive. I looked through all the records of their sessions to see who seemed to put in a good faith effort and who didn’t. But that took a lot of time and effort. How can we get around this?
Laura: The first thing I thought when I read this question was, “You should be watching all the sessions.” That’s the value of usability testing–seeing people use your product. If you’re just making people use the product and then giving them a survey at the end, you won’t get very good feedback.
Most companies that help you do this sort of testing let you rate your participants. If you feel like the participants really aren’t fulfilling their end of the bargain and are just clicking through at random for the incentives, give them a 1-star rating and ask for your money back. Companies aren’t interested in recruiting bad testers for their panels.
In your case, I want to know why they didn’t search. Was it because they couldn’t find the search box? Or because they could find what they were looking for without searching? Did they understand the task? You’ll only understand their motivations for not completing the task if you actually watch the sessions you’re paying to have people run.
Steve: Well said, all of it! You’ll only understand their motivations if you watch what they do (and listen to them thinking aloud, of course). This is fundamental. It’s the reason why qualitative methods give you insight into why things happen, as opposed to quantitative methods that only tell you what happened.
Question: Roughly what % of companies do you believe actually conduct usability testing?
Laura: I have no idea, but it’s far too low. In my experience, it varies wildly based on industry. Even what people count as usability testing varies. For example, a lot of consumer packaged goods companies do lots of customer testing, but they’re not run in the same way that a software company might run it. A lot of companies say they usability test, but what they really mean is that they’ve run a few tests in the past once in awhile. Or they talked to that user only one time.
I’m curious why it matters to you? The important thing is that the percentage of those who test keeps going up!
Steve: 21.5% (in keeping with the notion that 64.4% of all statistics are made up on the spot). Laura’s right: whatever the actual number is, it’s far too low. It’s why I spend a lot of my time showing people how they can reduce the amount of time and effort required, and make sure they’re really doing a usability test (i.e., watching people try to use the “product” while listening to them think aloud). The good news is that even though the percentage is still way too low, it’s much higher than it was a few years ago. And it continues to grow.
Question: What’s the simplest user research we can do?
Laura: As Steve said during the session, and in greater detail in the interview I did with him for my new book Build Better Products (shameless plug–you can buy it now), the simplest form of user research to do is usability testing on competitive products. It’s a fantastic and useful method for getting feedback on products.
Cindy Alvarez also mentioned using this method in her talk. She pointed out that it’s a great way to get people excited about usability testing and start to learn the techniques without threatening to make anybody inside the organization look bad.
Steve: And there are other research methods that are reasonably simple: like doing some user interviews. Interviews can be particularly simple if you do them remotely since you don’t even need to physically “get out of the building.” (Sidenote: the need to at least metaphorically get out of the building is one of the many terrific things the Lean UX movement—and Laura’s books, etc.—have introduced a lot of people to.) Interviews produce whole different kinds of insights than usability tests, but they both can be very rich.
Question: I am currently the UX strategist for a global law firm where I am tasked with designing tools for attorneys. My users are within reach, which is a great thing. But lawyers are pressured to fill all working hours during a working day with billable hours. I can’t figure out how to reach these users given their tight time constraints. Any ideas?
Laura: The easiest thing you can do is to compensate them for their time. Whenever you’re recruiting professionals, it’s important to recognize that their time is valuable and offer them something. It might not be money. It might be a donation to charity or a bottle of wine. But it has to be something that they think is valuable.
Also, try getting smaller chunks of their time. Asking for an hour is harder than asking them for a 15 minute call or screenshare. Offer times at lunch and before or after work. Try to do it remotely or go to them so that you’re not asking as much. Look for other types of lawyers: some in-house counsel jobs have regular salaries, and those lawyers aren’t as likely to have to account for every second. If you’re just doing usability testing, you don’t necessarily need the exact lawyers that you work with to help. You might be able to use other lawyers to see if they understand the interface. That will give you a lot larger pool of possible participants.
Steve: All great. And what about working with management to get the time they spend testing categorized in some way that recognizes its value?
Apart from this corner case (billable hours), testing internal tools (or intranets) is often easy, because you know exactly where to find a large pool of actual users who have a pretty good incentive to help. Often the best subset is new hires, because while they’re actual users with domain knowledge, they’re not yet familiar with the internal tools.
Question: How important is it to do usability testing on a user’s own computer/device/environment? Compared to bringing them into an office?
Laura: It always depends on what you want to learn. When possible, I prefer remote to forcing people to come into your office, since it means you’re not being geographically biased in your participant sample. On the other hand, seeing their environment in person can give you a lot of insight into their context of use.
Steve: And of course remote has so many other advantages, like no travel time and expanding the recruiting pool from “people who live or work near you” to “almost everybody.”
Laura: I’d say that contextual learning (home visits, etc.) is a lot more important for user research as opposed to usability testing. If you’re just interested in seeing if people can perform certain tasks given a particular interface, I’d go with remote testing. If you’re really digging into the way people work or perform tasks in their natural environment, consider whether you need to be more immersed in that environment.
Steve: Only one caveat: if you’re bringing them in, you need to make some effort to ensure that your computer setup isn’t going to get in their way. The most common case is sitting people down at a laptop with a touchpad when they’re only comfortable with a mouse. The solution, of course, is to have both available. You may also run into issues with Mac vs. PC users, but only if the test tasks are going to involve interacting directly with the operating system. If you’re testing something in a Web browser for instance, it probably won’t matter.
Question: What user research would you recommend for non-homogeneous target groups? From users with basic knowledge to super-professionals?
Laura: I’d recommend focusing on a specific subset of people and making things better for that identifiable group. Often, when you try to make improvements “for everybody” you end up making things better for nobody. Understand who you think your changes/new features/improvements are for and what behavior you expect to change, and then usability test on that subset of your users.
Steve: And I’d recommend beginning by focusing on the people who live on the “basic knowledge” end of the scale. If they find it usable, the people with more knowledge/experience will probably be able to figure it out. Remember, you’re not dumbing things down: you’re making things clear. Everyone appreciates clarity—even power users.
Laura: If you are making a global product change, make sure you understand the different segments of your users and recruit participants from each segment. You’ll most likely want to run a few more sessions than you would otherwise, to make sure you don’t have just one of any group.
Question: What are some ways you’ve successfully distributed findings from usability testing—particularly when some of the most interesting findings aren’t relevant to the topic you were researching?
Laura: It depends on who the user is of the findings. I wrote a post on deliverables that might be helpful. Whenever you create a deliverable of any sort, treat the recipient of the deliverable as a user of a product. Present the findings in an appropriate way for that person.
In general, the best way to make people familiar with the findings is to invite them to help with analysis, but I understand that that’s not always realistic.
Question: We have a really distributed team. What are some good strategies for getting everyone to watch people use our software when it means asking everyone to watch two hours of video?
Laura: Does everybody need to watch all two hours of video, or could you cut a highlights reel? I’ve had better luck going through the videos and grouping responses or tasks together to make it easier for people to consume.
Another option is to schedule an online analysis session with stakeholders where you use a digital whiteboard like Murally to share insights. You can require stakeholders to watch the research ahead of time. Just givethem a deadline and a reason to encourage them to watch the videos.
Steve: If you don’t have the time to create a highlights reel, you can simply crop out the time-wasting parts. And then give the viewers a tool that lets them watch what’s left at faster-than-normal speed. I never watch a usability test video at less than 1.4x normal.
Question: If you only have three slots and test three different pieces of functionality, is there enough enough feedback to recommend changes?
Laura: It depends, unfortunately. There have been usability tests where I knew there were enormous problems after half a session. I still ran a few more sessions after that to confirm, but it was very clear. In general, you run the sessions you need to run to start to see patterns.
Question: For an overall junior team, is it fine to start out doing internal usability testing–despite its limitations due to things like bias–to get up to speed and gain confidence?
Laura: Sure. In fact, I really like the suggestion from one of the speakers that you run a pilot on a member of your own team (like the designer or PM) at the beginning of all usability testing, regardless of how senior your moderators are.
Just keep in mind that what you’re learning from internal people will likely be extremely biased. Use it as a training tool, but I wouldn’t take the results nearly as seriously as I would results from real or potential users.
Also, ask yourself whom you’re protecting by keeping your junior folks away from users. If you feel like they’re going to damage customer relationships, try getting them some one-on-one coaching or training. Then have them practice on friendly customers. If you’re worried that they’ll get bad results, they’re no more likely to get bad results from external participants than they are from internal ones.
Steve: Yes, pilot tests. It’s too bad we didn’t get to talk about pilot tests since they’re absolutely essential. If you don’t do a quick pilot test before you bring your participants in, you’re not doing it right.
Question: How strongly do you feel about having one person per session? I’m part of a team developing an internal tool for a large company. I’ve got access to users in-person (our employees) twice a month and my boss thinks it’s better to test or interview two or three people at a time.
Laura: I have very strong feelings about this. As I mentioned during the Q&A session, I’d want to know why my boss wanted to interview two or three people at a time. Is it for convenience? Or because they think you’ll get better results? Hopefully you can talk them out of it, because neither scenario is true. You’ll end up getting a lot of echoing and learn more about power dynamics in your company by interviewing multiple people at once.
If it’s a tool that people will use jointly, it can make a lot of sense to structure sessions in a way that helps you understand how that will happen.
Question: The big benefit of the model that you’re describing for usability testing comes from synchronous participation (and discussion afterwards.) How do you apply this model in globally distributed teams?
Laura: Well, you can get huge benefits from usability testing when it’s just the researcher/designer/PM doing the research, finding problems, and fixing them. You don’t technically need everybody involved in that process.
The more people you have involved, the easier it will be to get the changes made. Especially if you’re in an environment where people tend to want to ship it, forget it, and move on to the next thing. Get creative—create things like video highlight rolls that you can share widely or snippets of information that are easy for people to digest.
Steve: It’s true: you don’t technically need everyone involved in the debrief process. But I think it’s worth bending over backwards to get as many people involved as you can (hence my “make it a spectator sport” maxim). Having more people involved does increase your chances of having durable buy-in and getting the changes made. But you’re also filtering the problem through more minds with different perspectives and valuable bits of information about where the bodies are buried. I also think that the process of observing and debriefing makes people better at their jobs, something I think of as “informing their design intelligence.”
For globally distributed teams, maybe some people watch the recorded sessions at a more convenient hour, then come together the next day for a full-group debriefing session.
Question: Are your users who you think they are?
Question: Laura, what are the three most common questions you get about research?
- How do I recruit people?
- How do I answer this particular question?
- How do I convince people that research is valuable?
Not necessarily in that order and not necessarily in those exact words. I was happy to see that those were big themes in our research before the conference, since now I can point people to a bunch of experts answering those questions.
Question: In a publishing environment, the editor is considered the authority on editorial content. What’s the best way to disavow editors of the notion that their editorial power also carries into UX of that editorial content?
Laura: Ha! You could literally replace “editors” with any other professional who is an expert in their thing – lawyers, doctors, scientists, etc.
Be open to what they’re recommending. Remember, they do have more domain expertise than you do. Understand why they want what they want. It may be that there’s a third solution that takes their concerns into account better than what you’re proposing.
And never underestimate the power of testing—either usability or a/b. If you’re advocating for one solution and your stakeholder is advocating for another, you’re going to have to devise a test that will definitively show which one of you is correct. This is literally what a/b testing is great for if you have enough users. If you don’t, then having a neutral party run usability tests on two different variations of a prototype can also be very eye opening.
Question: What do you think about pinging other UX professionals for usability and testing of prototypes other UX pro’s are working on?
Laura: I’m not a fan of asking other UX folks to do usability testing of prototypes, unless, it’s a product for other designers. You’re typically much better off testing on a few of your real users or potential users rather than asking someone who does this for a living.
Remember, UX designers aren’t magic. We don’t automatically know what problems real people will have and instinctively know how to fix these problems. If we did, we wouldn’t need to usability test! UX designers have the curse of knowledge. We don’t look at a product the same way other people look at it. If it’s a product for a specific group of people, designers can lack the domain expertise to give you decent feedback about what might be confusing.
That said, after you’ve run usability tests with your target market, other UX designers can often be very helpful in suggesting other ways to approach the problems you’ve identified. They may have seen other users have similar problems and have ideas about what worked to fix those problems.
But you still have to test with users.
Question: What’s the approach for testing content heavy, inspirational websites? Think TED.com.
Laura: Same approach for everything else–figure out what you’re actually testing and then come up with something that will help you learn what you need to know. Are you trying to test whether the website is usable? Inspirational? Understandable? Useful? Those are all very different types of experiments that you want to run.
The type of product is not what determines how you test. You test based on what you want to learn.
Question: I’m wondering if we’re too biased to effectively help others.
Laura: Depends on what you mean by “we” and “help.” Humans are extremely biased. It’s kinda our thing. There are ways to acknowledge and counter that bias to some degree, although I don’t think that most of us will ever even come close to getting rid of it entirely.
In general, the more time you spend listening to others and the less time you spend thinking about you and how you would do things, the less biased you’ll be.
Question: I often hear “I would have designed it differently.”
Laura: Yeah. We all do!
The best follow up is, “I see. Why would you do it that way? What problem would that solve for you?”
Frankly, I don’t care if they want a big red button to push that says “push me” on it. I want to know why they think that button will help them. Think of solution requests as a jumping off point to understand what their problem is.
Steve: We in the usability racket like to say “users aren’t designers.” And the truth is, they aren’t. Every once in awhile (once in a great while, actually), a test participant will make a terrific suggestion. You know when this has happened, because everyone in the observation room slaps their forehead simultaneously and says “Why didn’t we think of that?” But 98% of the time, design suggestions from participants aren’t great. I always recommend listening politely while they make the suggestion so they know you’re paying attention to them. Then at the end, after they’re finished with the tasks and you’ve moved on to open-ended probing, you can remind them of their idea and ask them to explain how it would work. In my experience, they’ll almost always end up by saying some variant of “But actually, I probably wouldn’t use [the new version] because [reason]. It’s better the way it is.”
Question: Is there a point of diminishing returns with too little user research? That is, is there a fear that a little learning can be a dangerous thing, leading to a false sense of UX security?
Laura: Sure.Talking to one person and then immediately diving in and solving all of their problems can certainly cause problems. If that person is an outlier. You don’t necessarily want to design your product around exactly one user. On the other hand, you can also spend an awful lot of time trying to “prove” yourself right.
I wrote a blog post on Predictive Personas that is probably applicable here. You can use this for usability testing just as easily as for predicting who will use your product. The idea is to run some sessions, try to spot patterns, predict what you’re going to see next, and then see if your predictions hold. Once you’re making predictions accurately, you’ll have a good idea of the recurring problems in your product and can more confidently start fixing them.
Laura Klein is a Lean UX and Research expert in Silicon Valley who teaches companies how to get to know their users and build products people will love. She’s a Rosenfeld Media expert and author of Build Better Products, and UX for Lean Startups (O’Reilly). Follow her on Twitter or subscribe to her blog and podcast at Users Know.
Steve Krug is best known as the author of Don’t Make Me Think: A Common Sense Approach to Web Usability and Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability Problems. Steve spends most of his time teaching usability workshops and consulting. Follow him on Twitter.
Buy on-demand access to the User Research For Everyone program.
I’ve got two kids, 8 and 12 years old. With issues like race relations, income inequality, wars without end, and presidential politicking here in the US, it’s harder and harder to make sense of the world for them.
It’s hard to even explain it to myself. While I’m doubtful that the good old days were all that good (thank you, Steven Pinker), I do know that there are many, many factors—from where we choose to live, to (ahem) Facebook—that make me feel like we’re all busily locking, loading, and reinforcing our concrete bunkers. And doing everything we can to build walls between ourselves and the Other. Whoever Other is for us. Opportunities to interact with and understand people with different perspectives are disappearing as fast as the ice caps. It’s not a happy, much less tenable, situation.
That’s why launching Dave Gray’s new book, Liminal Thinking, is so meaningful for me.
Sure, it’s always exciting to publish something new. I like to think our UX books have, in some small way, made the world a better place for users.
But Liminal Thinking—the first in our Two Waves series—has the opportunity to make the world a better place for people. Not just for the ones who make and use products and services. But for anyone who understands the need to break down their walls of belief and connect with others.
Liminal Thinking is, if anything, a self-help book, and it’s quite practical. Six principles, nine practices, 184 pages, and lots of Dave’s deceptively simple illustrations. That’s it. You’ll read it in a couple hours, and it might just change your life. Seriously. See what others have shared in Amazon reviews so far.