It was my first field assignment out of school. Okay, technically it wasn’t my assignment–a contractor would be conducting the interviews, and I would be along to observe and record. But I’d spent the previous two years and six months in a lab writing code, so I would take what I could get. To say that I was excited would be an understatement. I was stoked.
Plus, I’d get to fly to California! I’d be on an honest-to-goodness business trip! It was going to be great.
It certainly started out great. In the shuttle from the airport to the hotel, I counted citrus trees. Citrus trees! Growing in the ground! In people’s yards! And no one seemed shocked by this! Of course, I had plenty of time to count those trees, as we crawled through traffic for hours. But the weather was glorious and I, like it, was ebullient.
Things began to look dicey, however, when I met the researcher I would be working with. She was a smart, gregarious woman, who also happened to be sick. Very sick. Down-a-bottle-of-Nyquil-and-sleep-it-off-for-a-week sick.
Rest and recuperation, unfortunately, were luxuries we could not afford. The project was on a tight timeline and already behind. At least one of the interviews we had planned had been rescheduled once. Stakeholders across three organizations were chomping at the bit. It was, in the melodramatic way of business schedules, do or die.
And so we did. We pre-gamed with Thai food, guzzling tom yum soup for its sinus-clearing properties before returning to the hotel for an early-to-bed. The following morning we set off, my compatriot fueled by a powerful cocktail of cold medicine and espresso, myself running mostly on nerves and the delicious feeling of being free of my cubicle confines.
Still, we felt uncomfortable bringing sickness into the homes of our participants. “Give us your insight, and we’ll give you the plague!” is not the most enticing slogan a researcher could come up with. We tried to minimize risks. I shook hands with the participants; she abstained. She positioned herself as far away from them as their living rooms and rapport-building would allow, with me, a human note-taking buffer, in between. We strove not to be vectors of disease.
Given the circumstances, the first two interviews went well. But after hours of driving hither and yon across the north Bay Area, in traffic that I would have avoided navigating even with a clear head, my partner’s energy was flagging and the cold medicine wearing off. She tossed back an emergency booster of DayQuil in a Starbucks parking lot and we steeled ourselves for the final interview. It was perhaps more disorganized than the first two interviews, but we muddled through together.
And then, as the sun sank below the side of the endless freeway, it was over and we were once again untroubled by the inflexibility of a corporate system that put us in the ethical quandary of whether to conduct field work–or work at all–while ill. We parted ways at a BART station. She headed home to collapse into a restorative, cold-medicine induced coma; I went in to the city to spend a few days basking in the glow of more-or-less-successful fieldwork.
My basking didn’t last long, of course. In no time at all, I had a cold.
Jon McNeill is the Principal of Hunter.
Relatively early in my career, as I began stepping out and leading studies on my own, I was in Miami Beach doing ethnographic interviews with participatory “drive-alongs” for a luxury car brand. It was the last day in town, and I, with client in tow, had three 3-hour interviews scheduled that had to get done before we could fly out in the morning, the last one being scheduled for 9pm. This last interview was with Kenny, a guy who was actually supposed to be interviewed earlier in the week, but had to cancel because his yacht broke down and he was stranded for the day on a small island off the coast. We hear a lot of different excuses for non-participation, but that was a new one.
My client and I get through our first two interviews that day at around 8, hop back in the rental car, and start the trip to interview 3, feeling hungry and tired, having missed dinner. I called Kenny to confirm that we were coming, in case he was on another island. He answered in an energetic but distracted tone: “Yeah, laying out the drinks right now. We’ll get in the car, go get some speed, and come back and I’ll give you whatever you need.” Click.
“Speed? Oh no. Who is this guy? He must mean going fast, in his car,” I thought to myself.
I warned my client that we might have a live wire on our hands, but that we’d just go get the interview that we needed and then grab a bite.
We arrive to the address to see Kenny out front, waiting for us. “My wife is putting the kids to bed right now,” he told us, “so I’d rather not go in just yet and disturb them. Why don’t we get in my car, do the drive, go get a beer, and then come back and do the interview thing?”
We usually did the drive-along as the last part of the interview, but as intrepid researchers! Going with the flow is what we do best! Plus, at this point in the day, a drink sounded pretty good. My client and I nodded our agreement and squeezed into Kenny’s convertible: me riding shotgun, and my client folded into the tiny backseat area, holding the camcorder.
As soon as I buckled my seatbelt, Kenny hit the gas and I saw the speedometer jump up to 110 mph. I looked back at my client, white knuckled and – like a champ – rolling video on the whole thing.
We rocketed through a number of dark, mostly empty Miami streets. I was disoriented but loving the way the car gripped the pavement as we took turns in high gear. Just as I was wondering why he was choosing to take us to a bar that was so far from his home, I noticed a police cruiser waiting at a stop light ahead of us. Either Kenny didn’t notice, or he wasn’t worried; we flew through the intersection, still doing over 100.
I flashed on how the rest of the evening might unfold: sirens, mug shots, bailing my informant out of jail… but the cruiser didn’t even give chase. I think the officer knew he wouldn’t catch us.
Finally we pulled into a large parking lot, full of expensive cars, in front of a small oblong building. Two huge bouncers stood out front.
Kenny turned to us and said, “Welcome to the best all-black strip club in Miami Beach!” and headed for the entrance before I could fully process what that meant. My client’s mouth was agape.
Neither my client nor I are what you might call “strip club people”. He had been telling me about how he and his partner were remodeling their house into a real mid-century modern masterpiece. As I looked down at myself, I saw with dismay that the polo shirt I was wearing kind of made me look like the guy on Blue’s Clues.
Since this experience, I’ve heard stories of researchers obliging their clients by taking them to strip clubs, all in the name of client services. And Miami’s relationship to strip clubs did seem to be more casual than other parts of the country, because a few of our other participants had mentioned in passing eating lunch or getting a drink at a strip club. But I was mortified – this was not something I was anticipating. Yet at the same time, I felt cuffed: I knew we had to get this interview checked off, and I didn’t feel like I could demand that we return to his home without ruining our chances at building strong rapport.
I turned to my client and said, “I am so sorry. If I had any idea that he was taking us here, I wouldn’t have agreed. But at this point, I’m worried about insulting him; so let’s just go in, have a quick drink, and head out.”
My client, a saint, shrugged and said, “This is just what happens when you do ethnography, right?” Right.
The bouncers patted us down and we walked inside. Not having a depth of experience in this area, I had to take Kenny’s word for it being the best of its kind in Miami. Kenny was already at the bar, waiting with our drinks.
“So, what do you want to know?” he asked me, as he handed me a beer.
I struggled to remember my protocol questions, and we talked for about five minutes before Kenny excused himself to go to the bathroom. I looked over at my client and we both made a silent acknowledgement that we were done with our beers and ready to go.
Just then, Kenny came back with a stripper on his arm. He turned to my client: “Hey, I bought you a lap dance.”
My client’s face went white. The room began to spin. My client tried to politely decline.
Kenny, confused, said, “No, she’s great, I’ve had her before!”
My client politely declined again, and suggested Kenny go for it.
Kenny asked him, “What is it? Are you married?”
“You have a girlfriend that would disapprove?”
“Well, then, what is it?”
My client started stumbling over his words, trying to come up with a firmer excuse. Then Kenny laid down his trump card.
“Look, man, I’m doing this because everyone thinks you’re cops. You’re white, clearly not having a good time, and if you don’t do this, they’re probably going to take us outside and beat us up.” He waited for my client to answer.
My client looked at me the way survivors of a shipwreck must look at the person holding a life preserver. To my shame, I looked away.
My client, resigned, was led back to a private room. I turned back around in my seat and started processing all that had happened: my conversations with my client, some of the things he said that I hadn’t caught at the time, his answers to Kenny just then… and it all suddenly clicked for me, with a sickening certainty.
Kenny handed me another beer and said, “You know, I think your colleague might be gay.”
“Yeah,” I told him, “I just figured that out myself. But what you don’t know is he’s actually not my colleague, he’s actually my client. You just gave a private lap dance to my gay client.”
I felt ill. Kenny started laughing.
“That’s really funny, man. That’s really funny.”
I think Kenny really felt badly about the whole thing. After my client returned, we left and Kenny took us out to dinner at a kitschy piano bar owned by an old gay friend of his. We all laughed and told stories about crazy things that had happened to us in our lives, and at the end, without us knowing, Kenny paid for everything.
The night ended back at Kenny’s house, in front of a literal parking lot full of his Audis, Porsches, and huge SUVs. He was a fantastic informant, and helped me craft the recommendations for the brand based on his interview.
The car ride back to the hotel was pretty quiet. “Strange night, huh.” I said. My client nodded his head.
We shook hands at the hotel elevator and said goodnight. That was the last time I saw him – he wasn’t at the final presentation, and I heard that he had left the company not too long afterward.
At the end of the study, we sent him a client satisfaction survey, which was standard practice for us at that time. To my shock, it came back straight 10s. My client was a saint.
Unlike many of the other War Stories, this doesn’t paint me in the best light – mistakes were made, character flaws became apparent. But in some ways, the ability to realize that you’ve made mistakes and are flawed is one of the things I treasure most about anthropology — ever since my Intro to Anthro college courses where I began to learn about the long, illustrious line of mistaken and flawed anthropologists who came before me. In fact, often those mistakes and faux pas were the keys to unlocking some heretofore hidden cultural truths. And I think that night was no different, although I don’t think the cultural truths that were unlocked for me were necessarily about luxury automobiles.
I can’t see myself getting into the same situation now – there were at least two inflection points that night where today I would have directed things differently – but it could be that going through that experience together, the three of us, led to a deeper connection and (eventually) a successful interview. It certainly led to a War Story.
Doug Cooke is founder of Tinder, a research consultancy focused on people-centered innovation.
In a recent research and strategy project focused on defining a new global platform for a medical device, our research plan required us to shadow clinicians and others as they used existing devices in the “context of care.” With minor issues like HIPAA protecting patient privacy and other security issues at big urban hospitals in the US, our team decided that conducting research in Europe provided a better opportunity to understand these devices and their users.
Planning started with all the usual steps: multi-day client sessions to assess the domain, issues and problems; auditing reams of client data and documents; becoming familiar with competitive products, etc. We developed a research protocol that went through many rounds of revision with a large, multi-location client team, arriving at a clear understanding of relevant and important user issues. We developed screening criteria for participating medial institutions. Pilot studies were run at US hospitals. Months of preparation were spent in making sure our research team was fully prepared to bring back insights and perspectives that would help define the next generation global respirator platform. Ready, set, on to Europe!
Our first stop was a hospital in Wales. They had lined up the appropriate people for us to shadow and interview, including department heads, physicians,and medical techs. We spent two days shadowing, probing and gathering, and everything worked according to plan. Wahoo!
At our second stop in London (hauling two large model cases that would not fit into London’s very spacious cabs), we arrived at the check-in desk and ask to see Dr. Smith (or so we’ll call him). Upon arrival at his department wing, we learned that Dr. Smith was not in. Even more concerning was that Dr. Smith was out of the country at a conference and had not let anyone else know we were coming. After speaking with a few more people, the answer was “Please come back at another time when the doctor is in.” Ouch! In spite of all the planning, effort, and resources to get here, a few uncooperative people were about to jeopardize our research program.
How could this happen? Well, I ignored one of my primary rules: never let the client take on a critical path item that could endanger the project’s success and my firm’s reputation. Specifically, because of the difficulty of gaining access to the right people and institutions, and extremely high cost if we were to use a traditional recruiting process, our client took on the responsibility for arranging our visits to hospitals through Europe. Few clients understand the level of effort needed to screen, schedule and triple-confirm each participant. When the “research gig” is complex and requires the participation of a number of people carefully choreographed in a short time, it is essential to have a dedicated, experienced resource to make that happen.
We made it all work in the end. With no Dr. Smith and an apparent dead end, we literally started on-the-spot networking, walking up and introducing ourselves to doctor after doctor until we had made some friends that would grant us two days of access in the ICU and ER. It worked out in the end, but presented unforeseen delays and stress to an already pressure-filled project. Painful but constructive outcomes, nonetheless.
The rest of the trip in Germany and Italy presented various levels of preparedness on the part of hospitals we visited. Some hospitals were planning on hosting us for our full two day itinerary and some were expecting only a few hours meeting (which we were able to extend by turning on our best charm).
I have always been a very careful planner and can fastidiously orchestrate research logistics. I know what it takes to gather user insights. But the lessons learned from this European research foray is a clear reminders that whenever I can, I must control the recruiting and scheduling process. I hope to never again knock on any unsuspecting doors.
UX architect Chauncey Wilson shares a rather scary story about permissions gone missing.
In the 1980s, I worked for about 7 years at Digital Equipment Corporation (DEC) as a usability engineer. My group was led by John Whiteside, who pushed to make usability a serious discipline informed by metrics, fieldwork, and lab studies. The method of contextual inquiry was developed in our group by John, Karen Holtzblatt, Sandy Jones and Dennis Wixon. We did a lot of fieldwork to refine our methods and inform product teams about how to improve their products.
During my tenure at DEC, I set up a set of interviews with a major client who must still go unnamed. The client did military research and used some of our products. I got clearance to interview people at the site with the caveat that all videos, tapes, and notes would be surrendered when I left. I would analyze the data at their site and do a presentation about my findings, leave all data, and not discuss any details of my interviews. I got to the site early in the morning and signed in at the front desk. In those days, we had 8mm video cameras as our primary tool for field interviews. I had permission from the senior security chief to videotape the screens and record sound for 5 different users of our DEC products. I started setting up my equipment for the first interview and about the time I got to mounting the video camera on a tripod, three really large security guards with weapons blocked the exit to the office and asked me what I was doing (“I’m here doing some research for DEC”), then they grabbed my equipment and took me to a holding area and proceeded to interrogate me. I said that I had sought permission and had an agreement with the chief security officer – but that agreement was not to be found.
My name had been on the visitor list and the people I was interviewing vouched that I had set things up with them, but there was no clear approval for videotaping. I asked if they could contact their security chief, but he was on a vacation in the Virgin Islands. While they called and left messages for him, I spent a few hours in the holding area (you might call it a “cell”) concerned that I might go to prison. Though it took a while, they did catch up with the security chief and took me back to the cube where I had started my set-up and let me continue.
I spent a week at this site and noticed that the guards walked by and checked in on me a lot. Every night when I left during the week, they had me empty my pockets and remove every item from my briefcase. On Friday, I put together a report and presented to an audience of very serious people who asked no questions. I left all the data, submitted to my final contraband search and left the most bizarre field visit of my entire career.
I was researching, with my colleague Patrizia Bordignon, how people thought about and dealt with home renovations.
One of the methods was a diary study (“cultural probe”), and we had carefully recruited – or so we believed – a small set of participants with whom we would work for several weeks.
Warning bells sounded fairly early with one of the participants, who showed up very late for the initial briefing. These things can happen, so we ran a separate briefing session for him, gave him his kit of reporting materials (camera, diary and so on) and sent him on his way. Let’s call him Mr. W.
Three days after the briefing we telephoned each of the participants. It’s a good idea to do this to remind people about their commitment, to redirect as necessary, and to address any issues that arise. All our participants were on-track, with the notable exception of Mr. W, who seemed somewhat evasive in his answers.
At the end of the first week, we visited the participants. Again, this is good practice; it’s an opportunity to see how the data is being gathered, and what changes might be needed to the process. We also use that opportunity to make a part-payment to the participants, which can serve as a nice motivation.
We were delighted with what we saw. Participants had kept bills and receipts, photographs and magazine clippings, they showed us their renovations or their plans, and we were confident that we were getting plenty of highly relevant data.
When we visited Mr. W’s house, however, it was evident from the first moment that his home was different. The front gate didn’t work properly and the hinges squeaked, the garden was unkempt and the house gave an overall sense of dilapidation. Inside it was a similar story. Every room was in dire need of immediate restorative work, but none was evident. I felt a tad depressed as we drank tea from cracked mugs and listened to Mr. W list the things that needed to be fixed.
Mr. W was not an enthusiastic renovator. His house represented a series of urgent and necessary tasks, none of which had been tackled.
It looked like we would collect no useful data from Mr. W, and as we traveled back to the office we talked about our disappointment and reexamined our recruiting strategy.
However, as we moved into data analysis, we found ourselves referring quite often to Mr. W, and gradually came to realize (no doubt this should have been obvious earlier) that Mr. W’s world was in fact directly relevant to our project. While the enthusiastic renovator was undoubtedly a key consideration, the unenthused or reluctant could also present great opportunities. Their needs and goals were different, their attitudes were different, and the way that we would design for those characteristics was different.
In many ways, in fact, Mr. W was an ideal participant specifically because he didn’t fit our expectations. He challenged the underpinnings of the project, and made us examine our design decisions in a much more rigorous fashion.
I often reflect back on this experience when I’m doing user research, and I specifically watch out for negative reactions and experiences, because they can often teach us things that we might not otherwise learn.
I still believe it’s important to recruit carefully, but perhaps we should be more open to the idea that the “wrong” participant is sometimes precisely the right one.
Steve Sato is the Principal at Sato+Partners, a customer-centered strategy and stakeholder-centered organization design consultancy.
We were three days into our 18-day research trip. The clock was ticking and our progress had been frustratingly slow. We had nary an insight to show for our time spent here so far. It was 9 o’clock in the morning and we were already hot and sweaty after having walked a quarter of a mile on the footpath, the only way to a remote village in Uganda. Our team was doing field research on making microfinance more efficient and reliable, so banks and other financial institutions would find it profitable for them to extend their services to include microfinancing. The current system of paper and pencil, traveling back and forth to an office two hours away, and then transcribing notes onto a PC (“sneaker net”) was inefficient and fraught with errors and omissions. Furthermore, what was required was not only an IT system that could span “the last mile” but we had 15 days left to prototype an interaction model that would augment the device. It needed to be a process that the field agents and their clients would trust and adopt without much help. On top of that we had to identify what other not-for-profit and for-profit organizations (e.g., medical, agriculture, manufacturing and so on) would find the field device useful (so we could size the potential market for the device).
I was responsible for the research and the results. I really was feeling the stress and the jet lag and I had heartburn non-stop from the first day here.
We arrived at the village and our team was introduced by the microfinance agent to a group of a dozen women who were her clients. After a few minutes of conversation the women gathered and sat down, with the field agent, on the ground in a large circle. Two researchers stationed themselves behind the agent while the rest of us positioned ourselves around the perimeter of the circle. I turned on the video camera and thought “Whew! We’ve been prepping this for nearly a month and now we’ll finally get to make some interesting discoveries!” But then I spent the next half hour struggling to stay focused, to listen to the conversation and watch the exchange between a woman and the field agent. Then some amount of self-awareness seeped into my head: “The breeze feels so good, gosh! I’m so exhausted, I could go to sleep right now…let me see, it’s 11ish at night in Portland…Ohh! I promised I’d call my wife today!”
Without thinking, I pulled out my cell phone and looked to see if I had a signal. To my surprise I had one bar! By walking away from the group towards a little rise I could get 2-3 bars which was good enough!
It was good to hear my wife’s voice. I closed my eyes while talking with her for about five minutes, like I was only a block away. I felt calm relief return.
But then my eyes popped open, because with the relief came a realization, triggered by my ability to connect to my wife halfway around the world while I’m in the African back country, gazing at a group of women sitting in the grass under the shade of a huge tree, with puffy white clouds against a bright blue sky. It was surreal and so powerful. I experientially understood our mission: to connect the people here to the world in a way that would make their everyday lives better, as was happening to me in the moment. Suddenly I was re-energized and fully present. Throughout the rest of the trip I kept coming back to relive this experience. It kept me energized, engaged and focused, no matter how exhausted I felt. I honestly believe it made a positive difference in what we discovered, what we surmised and in our final designs.
Alice Goffman’s ‘On the Run’ Studies Policing in a Poor Urban Neighborhood [NYT] describes a sociologist who commits deeply to truly immersive fieldwork. At one level this simply reminds us of the differences between academic and industry work, but beyond that it surfaces just how personally demanding it is to deeply engage in a culture, requiring us to forgo much of ourselves (In Interviewing Users that’s Check Your Worldview At the Door) in order to understand the people we are interested in (Embrace How Other People See the World).
Ms. Goffman comes from a home where intensive fieldwork was something of a family business. Her father, the eminent sociologist Erving Goffman (who died when she was a baby), posed as an employee of a mental hospital for a year to research his 1961 study, “Asylums.” Her mother, Gillian Sankoff, is a sociolinguist at the University of Pennsylvania who has done studies in Papua New Guinea and French Canada; her adoptive father, the sociolinguist William Labov, also at Penn, has done pioneering field research on African-American urban vernacular, among other subjects.
Ms. Goffman, who grew up in the Center City neighborhood of Philadelphia, said she took her first field notes as a teenager, recording observations about the Italian-American side of her family in South Philadelphia. By her sophomore year at Penn, she had moved full time to a mixed-income African-American neighborhood and was hanging out on a tough strip, fully immersing herself in local culture.
She abandoned her vegetarian diet, listened only to mainstream hip-hop and R&B, and adopted local “male attitudes, dress, habits, and even language,” as she puts it in a long appendix, describing her research methods. While drugs, and drug selling, pervaded the neighborhood, she did not use them, she writes, partly because “it hampered writing the field notes.”
By her own account, she lost most of her college friends, and struggled to complete her non-sociology requirements.
It may sound “absurd” now, Ms. Goffman said of her extreme immersion. “But I was trying to take the participant-observer approach as seriously as possible.”
It can be hard to square the very ordinary-seeming academic who recalls her teenage affection for “My So-Called Life” with the young woman of her startlingly confessional appendix, which ends with a moving account of a close friend’s death in a shootout.
It’s been one year (wow!) since Interviewing Users came out! Hooray! Below is a roundup of links to various bits connected with the book. I’ll republish this occasionally with accumulated updates. If you haven’t already, get your copy here! And if you have, you should write a brief review on Amazon here.
- Buy from Rosenfeld Media
- Buy from Amazon
- Foreword by Grant McCracken
- Excerpt from chapter 2
- The cover
- Images and diagrams
- Web Designers’ Review of Books
- How To Interview Your Users And Get Useful Feedback by Garrett Moon, Founder of TodayMade
- Reviews on Goodreads
- QRCA (Qualitative Research Consultants Association) Views Magazine (PDF)
- New: Reviews on Amazon
- Why You Should Read ‘Interviewing Users’ by Steve Portigal
- New: UX Discovery Session
- New: Wise Talk
- The eLearning Coach
- UX Magazine
- UX Matters
- UIE Book Corner (audio)
- 33 voices (audio) [summary slidedeck]
- UX Booth part 1, part 2
- Business 901 (audio)
- Huffington Post
- Ethnography Matters (and in Spanish)
- Interview with Denise Lee Yohn (audio)
- The User Experience podcast with Gerry Gaffney
- Steve speaks with Jared Spool about field research
- SpoolCast: Steve Portigal’s Interviewing Tips
- IXDA Los Angeles/LA UX Meetup – slides, video, alternate video, tweetstream
- recording of O’Reilly webinar #1
- recording of O’Reilly webinar #2
- A 5-minute talk about The Power of Silence