Although we couldn’t make it down to Buenos Aires for Interaction South America, thanks to the magic of Skype I was able to present Designing the Problem at over the weekend.
Too often we assume that doing research with users means checking in with them to get feedback on the solution we’ve already outlined. But the biggest value from research is in uncovering the crucial details of the problem that people have; the problem that we should be solving.
As the design practices mature within companies, they need to play an active role in driving the creation of new and innovative solutions to the real unmet needs that people have. In part, driving towards this maturity means looking at one’s own culture and realizing the value of being open-minded and curious, not simply confident. This is a challenge to each of us personally and as leaders within our teams and communities.
Below you’ll find slides, audio and sketchnotes. I’ll repost when the video go up.
The talk is just over 40 minutes and there are two questions (which you can’t hear but which should be obvious enough from my response).
[audio:Steve Portigal – ISA14 – Designing the Problem.mp3]
To download the audio Right-Click and Save As… (Windows) or Ctrl-Click (Mac)
Here is my huge head during the Q&A segment (image via Juan Marcos Ortiz)
Sketchnote by Kat Davis (click for full size)
Sketchnote by Thiago Esser (click for full size)
Embedded above is a fantastic and disturbing episode of Love + Radio. Nick van der Kolk and Noah Morrison visit Jay Thunderbolt, who upon beginning the interview, aggressively reiterates his demand for payment (not possible for public radio, which Jay knew). Noah ends up going on a liquor run instead, as Jay offers him a pistol or Kevlar vest. Jay never stops insulting the interviewers, and stories of violence abound; indeed at one point he points a gun at Nick’s head. Meanwhile, they are interviewing Jay about the strip club he runs out of his house.
Yikes. This sounds like some of the War Stories, doesn’t it?
I don’t know what is going through Nick’s head as he’s doing this interview, but as I listen I find myself strongly repelled by Jay. And while the interview here is edited, so we don’t know all that happened, but Nick never reveals discomfort or lets Jay’s obvious provocations get to him. His patience and tolerance create room for Jay’s story to come out, and while Jay is not an appealing individual, you begin to understand and accept him as he is. Well, I did. Your experience may vary.
Nick finally responds to Jay’s taunting at the end, when he asks Jay “Do you think you understand the way I feel about you?” and Jay admits that he doesn’t. It’s a powerful moment in an intense interview.
The first episode of Working (a new podcast) features Stephen Colbert explaining in great detail the process of creating The Colbert Report. The entire episode (embedded above) is really good process stuff (creativity, collaboration, finding the story, media firehose, working under pressure) but I want to call out the section about how he prepares and uses the questions for interviewing his guest, as it’s is quite consistent with what I wrote in Interviewing Users.
And then I read the two sheets of questions that the writers have come up, what their ideas are. I usually pick 10 or 15 of those. But I don’t look at them. I don’t look at them until right before I go over [to the set], and then I read them over once again in front of my producers to get a sense of, oh, this is how my character feels about this person.
Come show time…I take them out and I go, oh, yes, these are the questions I chose. And then I try to forget them and I try to never look at the cards. I just have a sense in my head of how I feel. And the cards are in front of me, but I try not to look at them at all. I’m pretty good. Maybe I look once a week at the cards. I put my hand on them, so I know I have them if something terrible happens, but as long as I know what my first question is for the guest I kind of know what every other question is, because I really want to react to what their reaction to my first question is.
And I usually end up using four of the 15, and the rest of it is, what is the person just saying to me? Which makes that the most enjoyable part of the show for me. Because I started off as an improviser. I’m not a standup. I didn’t start off as a writer, I learned to write through improvisation, and so that’s the part of the show that can most surprise me. The written part of the show, I know I can get wrong. You can’t really get the interview “wrong.”
In 1984, I was 23, and working for a market & social research firm in San Antonio, Texas. They sent me down to McAllen to collect voter opinions on the upcoming national elections. McAllen is a sleepy little town near the bottom tip of the state, just a few miles from the Mexican border, mainly populated with low-to-moderate income Hispanic families.
I was on my second day of door-to-door polling, asking voters their opinions on policy matters, and their thoughts on the state and presidential candidates. The work was progressing well. As usual, I was getting a high rate of interview completions, with lots of useful data. After four years of working in market and social research, I was quite confident in my neutral, non-threatening “aw shucks, I’m just one of you” act, and its ability to deliver great results.
But my confidence was shaken when I met Maria, a shy housewife in her early 30’s.
It was about 4 pm on a warm, dry Thursday afternoon when I knocked on the door of a modest, well-kept ranch house in a suburban section of McAllen. Maria opened the door part way. She was half-hiding behind it, sizing me up like a rabbit peering through tall grass at a coyote in the distance…curious, but poised to flee.
Me: “Hello, my name is Patricia, and I’ve been sent here by (XYZ Research) to gather public opinions on the upcoming elections.”
Maria: “Oh, hi.”
Me (turning on the charm): “May I ask you some questions? Don’t worry, I’m not selling anything!”
Maria: “Uhh, sure, I guess?”&
Me: “Great, thanks! This won’t take long.”
Wide-eyed, Maria flashes a shy smile before her jaw slacks again. This one’s cagey, I thought to myself, but I’ll get her talking.
Me: “Now, thinking about (Candidate X), what comes to mind?”
Maria: “Uhh, I don’t know? Is he a good guy?”
Me (shrinking): “Well, I really don’t have any thoughts on (Candidate X). Besides, my bosses didn’t send me all this way to talk about my opinions. He wants to know your opinion.”
Maria: “I don’t know. He seems okay?”
Now, I don’t think Mary is incapable of forming opinions. I suspect she’s simply never been asked to share her thoughts about such important things, so far from home. And she may never be asked again. But on this day, I was determined to make her opinion count.
Me: “Well, you’ve heard of him, maybe seen him on TV?”
Me: “So, what did you think of him? Is he someone you would vote for?”
Her eyes darted across my face, scanning every crease and twitch, searching for clues. Those big rabbit eyes begged mutely for help. I stared back, apologetically. I took a few slow breaths, trying to ground us both, so she might relax into talking more naturally. Each time she hesitates, I carefully repeat the question, altering the wording and inflection to make them sound as simple and benign as possible.
Me: “Really, we’re just interested in what you think. Whatever you think is fine. Do you think you’ll vote for him, or not?”
Maria: “Uh…yes?” (seeing no reaction from me) “No?”
Me: “Okay, that’s fine. Alright. Now, thinking about (Issue A), is that important to you? Do you think it’s good or bad?”
Maria: “Uhh…I think it’s good?”
The back and forth went on for several minutes. I’m trying to go completely neutral and void of any emotional expression, but my contortions only intensified the awkwardness. The interview was in free-fall. I was failing miserably to collect any genuine responses from Maria. A hot wave of panic washed over me. How can I get this back on track?
In that moment, I just had to let go.
I quit fighting it, and fell back on connecting with Maria as a person. As Maria answered my questions, I began riffing on her responses, affirming and adding detail to them. While trying not to reveal my personal opinions, I offered supportive words and gestures to elevate everything she said, so that she might open up and elaborate. Eventually, she did relax, and her answers flowed a bit more freely.
Me: “So, what about the presidential candidates?”
Maria: “I guess I’ll vote for (presidential candidate B).”
Me: “Great! Is it because he is for (issue B)?”
Maria: “Oh, that’s good. Yeah, (B) is good for us.”
Although Maria was warming up to me, I felt I was way off book. It seemed impossible not to sway her answers. Whatever I wrote down, I feared it might be swept away by the slightest shift in body position, or an eyebrow lift. Well – at least she was talking, I told myself.
Finally, we got to the end. Walking back to my car, I breathed a huge sigh of relief. The hardest interview I’d ever done was over. I went out for a well-earned drink and a tragicomic debrief with my co-workers.
Sometimes you just get a dud subject, and it is what it is. But something about that 15-minute exchange with Maria struck a deeper chord in me. As I drove out of town, troubling questions lingered. What is the value of a skewed interview? Was this the only time I’d failed to be impartial? Or, had this been happening all along, in more subtle ways? How can I ever know that the data I’m collecting is pure?
Maria taught me two important things that day.
1. People make stuff up as they go along. And, we can’t always see the flaws in self-reporting.
2. The observer effect is unavoidable. Interviewers shade their work in unpredictable ways.
I’m as diligent as ever about delivering valuable insights through my research. But ever since that incident in McAllen, I draw my conclusions with a fuzzy border, in humble deference to flawed inputs and shadow projections, on both sides of the clipboard.
Steve will be presenting Designing for Unmet Needs at the Warm Gun conference in San Francisco on December 4.
Alex Blumberg has a podcast about his journey to start a podcast-related business. A recent episode of This American Life included an excerpt from this podcast (called StartUp), in which Blumberg is half-heartedly pitching his idea to investor Chris Sacca.
They talk for a while, and Alex is having difficulty in explaining his idea and what he’s asking for.
Alex Blumberg: So it’ll take a million and a half dollars, I think. And–
Chris Sacca: Take out the “I think.”
AB: Yeah. It’ll take a million and a half– I’m looking for a million and a half to $2 million in seed-stage funding.
CS: No, no, no, no, no.
CS: You were looking for a very specific amount of money.
AB: I’m looking for– [LAUGHS NERVOUSLY]
Finally, Chris decides he’s just going to show Alex how to pitch his idea and he very masterfully riffs a confident and coherent bit of persuasion. It’s certainly worth listening to, but here’s the excerpt from the transcript.
Hey, look, can I get two minutes from you? So here’s the thing. You probably know me, producer of This American Life, been doing it for 15 years. You know it’s the most successful radio show, top of the podcasts in iTunes, et cetera.
So here’s the thing. I realize there’s a hunger for this kind of content out there and there’s none of this [BLEEP]. It’s just a bunch of jerk [BLEEP] podcasts. Nothing’s out there.
Advertisers are dying for it. Users are dying for it. And if you look at the macro environment, we’re seeing more and more podcast integrations into cars. People want this content. It’s a whole new button in the latest version of iOS.
So here’s the thing. Nobody else can make this [BLEEP]. I know how to make it better than anybody else in the world. And so I’ve already identified a few key areas where I know there’s hunger for the podcast. We’ve got the subject matter. We’re going to launch this [BLEEP]. I know there’s advertisers who want to get involved with it.
But here’s the unfair advantage I have. Because of what I’ve done in my past careers with This American Life and with Planet Money, people are actually willing to just straight-up pay for this stuff. And I’m not just talking about traditional subscriptions. I’m talking– we did this T-shirt experiment at Planet Money where we got $600,000 coming in, where people actually gave us money to buy a t-shirt with our logo on it as part of the content. It was integrated directly. And I know we can replicate that across these other platforms.
So here’s what we’re doing. We’re putting together a million and a half dollars. That’s going to buy us three, four guys who are going to launch these three podcasts in the next 12 months. We think very easily we could get to 300,000, 400,000 net subscribers across the whole thing.
With CPMs where they are in this market right now, I know on advertising alone, we could get to break even. But as we do more of this integration, we get people texting in to donate to this stuff, buying some of this product, doing some of these integrated episodes, I know that we’re going to have on our hands here something that will ultimately scale to be a network of 12, 15 podcasts. The audience is there. They want it. Nobody else can do it like we can. Are you in?
It’s so painful to hear Alex stumble and when Chris takes over, I felt a sense of relief and a certain excitement, to hear an idea presented in a way that was designed to engage and persuade. This is a valuable skill in many aspects of professional life, especially when we’re in the business of sharing ideas. The superlative example in this podcast is quite inspiring.
The relevant section starts at 19:21 in the embedded widget below.
It’s been well over a year since Interviewing Users came out. Below is a roundup of links to various bits connected with the book. I’ll republish this occasionally with accumulated updates. If you haven’t already, get your copy here! And if you have, it would be great if you wrote a brief review on Amazon here.
- Buy from Rosenfeld Media
- Buy from Amazon
- Foreword by Grant McCracken
- Excerpt from chapter 2
- The cover
- Images and diagrams
- Society for Technical Communications (members only; contact me if you’d like to see this)
- Web Designers’ Review of Books
- How To Interview Your Users And Get Useful Feedback by Garrett Moon, Founder of TodayMade
- Reviews on Goodreads
- QRCA (Qualitative Research Consultants Association) Views Magazine (PDF)
- New: Reviews on Amazon
- Why You Should Read ‘Interviewing Users’ by Steve Portigal
- New: Tuesday #TechHour
- UX Discovery Session
- Wise Talk
- The eLearning Coach
- UX Magazine
- UX Matters
- UIE Book Corner (audio)
- 33 voices (audio) [summary slidedeck]
- UX Booth part 1, part 2
- Business 901 (audio)
- Huffington Post
- Ethnography Matters (and in Spanish)
- Interview with Denise Lee Yohn (audio)
- The User Experience podcast with Gerry Gaffney
- Steve speaks with Jared Spool about field research
- SpoolCast: Steve Portigal’s Interviewing Tips
- IXDA Los Angeles/LA UX Meetup – slides, video, alternate video, tweetstream
- recording of O’Reilly webinar #1
- recording of O’Reilly webinar #2
- A 5-minute talk about The Power of Silence