Doorbells, Danger, and Dead Batteries Blog

User Research War Stories

Posts written by Steve Portigal

  • David’s War Story: Suit yourselves

    Posted on

    David Serrault is an Information Architect and Experience Designer for SNCF, in France.

    It took a long time and several discussions with the stakeholders from SNCF (the French railways operator) to receive authorisation for a field investigation using old-age simulation suits. We were to focus on Gare Du Nord, the largest railway station in Europe. The station hosts more than 190 million passengers a year, including commuters, long distance travelers from all across Europe, businessmen and tourists from anywhere in the world. Gare Du Nord is filled with people in movement, following their daily routine or finding their way in this complex ecosystem of platforms, corridors and stairs, all managed by several operators.

    I have always been fascinated by railway stations. Even more than the airport, I think that they are like temples of modernity, ruled by time and noisy, heavy, industrial technology.

    The goal of this onsite experiment was to evaluate the gaps in the information provided to travelers, especially older ones. The old-age simulation suit makes the user feel the constraints of an elderly person. The suit, made in Japan, looks like a kind of kimono such as one might wear for practicing judo. But this one is green and red, covered with belts and weights in order to simulate the physical constraints of an old person. Also, you have to wear glasses to restrict your visual field and reduce your ability to perceive contrasts as well as ear plugs to reduce your hearing.

    We were four: my friend and colleague Christophe Tallec (a French service designer) and two students. We set out in the morning for our day of experimentation. Indeed we were a bit worried by working in such a crowded public space. Even with an official authorisation from the station officer it was difficult to imagine what the reaction of the police authority and the users of the station would be. The place had been a scene of a spontaneous riot a few months before, when a young boy from a low wage suburb commonly called “Banlieusard” had been arrested and assaulted by the police.

    The main difficulty we faced was more a logistical one. Each of us was to attempt a pre-defined journey through the railway station but we had to find a place to put on the suit. Eventually we did this behind some ticket vending machines that were close to the bus station. Not a very intimate place, but good enough!

    Surprisingly the police simply came to ask us what we where doing and then let us do our job. We didn’t encounter special reactions from the passengers who seemed to deliberately ignore us.

    This experiment was a revelation, especially for the youngest members of the team. We realised how many constraints someone with physical limitations can face during journey through a public space like this one. These spaces are dedicated to people flow, but disabled or challenged individuals are like aliens. There are many initiatives to make these areas accessible and readable but how can this be a priority for an organisation that has to manage daily security and transit efficiency for millions of people. It is a huge challenge for the designers and the architects who work for these places.

    Eventually we realised that indeed a few passengers were staring at us, especially ones who were dressed as Sailor Moon, Naruto or other manga and video game heroes. The day of our research was also one of the largest conventions for Japanese culture aficionados in Paris. Japan Expo is known for its concentration of “cosplay” practitioners; many these young adults were simply in their costume to the event by public transit.

    From their side, I suppose they were wondering what kind of manga persona we were playing. From my side, I felt like I was at the meeting point between modern and post-modern times.

    People Like Us

    Posted on

    People Like Us was a mockumentary that ran on BBC radio in the late 90s before becoming a TV show. The radio shows are hilarious and a great illustration of what can and does go wrong in fieldwork. Each episode is essentially a total War Story.

    The shows follow a hapless reporter, Roy Mallard, investigating the lives and work of ordinary people: a bank manager, an artist, a stay-at-home mom, an actor and so on. Things go awry: despite being married (Really? You?) he finds himself awkwardly attracted to an interviewee, only to realize that another interviewee is her bitter ex-boyfriend. He’s a passenger in a recklessly driven car. He’s witness to firings, incompetency, violence, relationship hassles. He trips, drops things, is sneezed upon, breaks a washing machine, and more.

    At the same time, his attempts to interview people and get to the heart of what their lives are about are thwarted. If not by circumstances, then by the inarticulateness of his interviewees, or by their sheer misinterpretation of his questions (e.g., Q: You’ve been here for a long time. What kind of things have changed? A: My hair.)

    People Like Us manages to be completely absurd yet with an eye-rolling kind of truth that any user researcher (and journalist, I imagine) will identify with.

    The radio series has been posted to YouTube and I’m embedding all the episodes as a playlist below. Check ’em out and let me know what you think.

    David’s War Story: Footloose

    Posted on

    Interaction designer David Hoard shares a story where even his best intentions are not sufficient to prevent a perplexing gaffe.

    Researcher Chinami Inaishi and I were on a 10-day trip to Tokyo to interview kids and young adults about their video game use. It was 1995 and the console wars were in full effect. Chinami is Japanese, but had lived in the US for many years. So she was the perfect local guide to help me understand the cultural nuances we were witnessing. She also helped us navigate the nearly impossible house numbering system in Tokyo, where house 31 was next to number 6, which was next to 109. This echoed one theme of the trip: squeeze things in wherever you can find space for them. Every square inch will be utilized.

    The visits were fascinating and enriching at each stop. We saw small beautiful homes with Western-style furniture next to Japanese Tatami rooms. We interviewed a young man with the smallest apartment ever, a tiny 8′ x 8′ space packed to the gills with Western-oriented magazines, blue jeans, skateboards, and a (unused) full-size surfboard. The kids were impressive, with their beautiful calligraphy work and exacting toy collections. In all cases, no square inch of space was unused, and that made me rethink the design were considering. A low, wide game console was perhaps out, replaced by a slim vertical unit that could fit in one of their dense bookcases.

    Before the trip, I had done my best to read up on Japanese culture and manners. There’s no way to learn a culture from a book or two – my goal was simply to avoid making a big mistake. I practiced and practiced the few phrases I would need (Chinami was doing simultaneous translation for 98% of it). I knew my two-handed business card presentation technique, and I nearly understood the rules for bowing.

    We’d been through most of the visits, and so far so good. All of the sessions had gone fairly well, and we were learning a lot. But then I did something bad. Something wrong.

    We had been visiting a house near the end of a train line, slightly out of the city-center. The session was over; it was time to pack up the camera and notes and head out. We were doing our now-normal goodbye ritual, trying to check off the right etiquette boxes. And then it happened: I misstepped. Near the front door, I stepped my sock foot just off the wood floor and onto the carpet. With one shoe on already. Unknown to me, I just violated an important manner about exactly where your shoes go back on when you leave.

    Instantly the whole family erupted in hysterical laughter, with everyone pointing at me. Suddenly I was in a mayday situation, with my manners in a dangerous nosedive. Confused, I did my best to get my shoes on as Chinami pulled me out the door and onto the street. She was like a commando extricating someone from an international hotspot.

    “What was that??” I asked, once we were out on the street.

    Chinami informed me that laughter (apparently hysterical laughter) was how the Japanese cope with a faux-pas or embarrassing situation. Embarrassment was indeed what I created, and I felt it too. Intense embarrassment comes with a whole set of physical sensations. You’re flushed, addled, and dazed. You’ve got great regret, but it’s too late to fix it.

    When we go out to do field research, we often feel we are going out to observe a strange species in its native habitat. We are the scientists, they are the creatures to be documented. We go to great lengths to help them feel comfortable with our scientist-like presence. We feel like we are the smart ones.

    But guess what? The research participants are in their native habitat, and are experts on their own lives. We the researchers are the weird aliens. We’re the ones not getting their nuance. We’re the ones who are sometimes worthy of mockery.

    But it’s all in a days work when you’re out doing research; you’ve got to be light on your feet. Every research session I’ve ever been on has been a dance to cover the material and sniff out insights right below the surface. All while you try to make everyone comfortable and keep the conversation flowing. It’s that dance that makes it exciting; just try to keep your toes in the right place.

    Sharon’s War Story: Broken Windows Theory

    Posted on

    Sharon Cartwright, a consultant with Optimal Experience, shares her story about uncovering context.

    We were surprised that Anna wasn’t at home when we arrived at her house. We could see through the lounge windows that the house was empty of furniture and personal belongings, adding further to the intrigue. We had followed our usual protocol of sending an email confirmation clearly stating the time of the research session and Anna had been called the day before to confirm.

    We decided to try calling Anna. She picked up our call as she was coming up her driveway. She was running late and seemed a bit agitated, telling us that she had locked herself out an hour earlier. She was just moving into this house and didn’t have a spare key yet. She’d decided, in the interest of proceeding, that she was going to smash a window to get in and had just called around to her brother’s house to pick up the appropriate tools.

    Anna seemed like a practical woman, and smashing her front door window didn’t seem to daunt her. I indicated that there was no need to take such drastic action for our sakes. But she was adamant and was soon taking a hammer to the glass panel in the kitchen door.

    Once inside, Anna, myself and the client (who’d come along for the ride) began the clean-up. I manned the vacuum cleaner and mused over the start to the session. There is something bonding about a shared clean-up!

    We were there to observe Anna set up wireless broadband. On a good day the process wasn’t straightforward; we had already seen several participants struggle through it. Over the next two hours we observed Anna encountering several technical issues with hardware and software. She managed to resolve some issues on her own. Many times she resorted to calling the contact centre, although their advice was mixed – sometimes helping Anna, sometimes complicating matters.

    We had scheduled the session for 1.5 hours expecting this to be sufficient, but many sessions – including Anna’s – extended over this. After two hours Anna had not succeeded in setting up her wireless broadband connection. Unable to stay any longer, we were disappointed to leave without seeing Anna ultimately succeed. We wished her luck, as she was clearly going to need it.

    During the two hours Anna revealed a few things to us. Her long-term relationship had recently ended, explaining her move to Auckland. She was also looking for work. Her ex-partner’s teenage son had generally taken the lead on the technology front in their home, and it was dawning on her that this was now her role.

    With the significant life stresses she was facing it was hard to watch her struggle through a technology set-up that should have been easier, dare I say simple. While we learned that setting up wireless internet often happens during a time of stress, as it’s one task of many when you move house, I felt for Anna and the difficult time she was going through.

    Elysa’s War Story: Keep The Swiffer On Your Right

    Posted on

    Senior strategist Elysa Soffer heads into the field where an adventure in building rapport awaits.

    She’s Caucasian, 75 years old, retired, married with 2 grown kids, lives in Berkeley, and cleans her floors 2 – 4 times a week. This woman fits our qualifications as a participant for our research study. But you never know who you might be talking to and how best to get them to open up to you-a stranger entering their home.

    Two of us spent a few hours interviewing this woman in her home for a floor cleaning study. We asked warm-up questions about her and her household. She mentioned that she was a writer and lived there with her husband. She showed us around her house, and pointed out the rooms where she spends the most time cleaning floors.

    During the next part of the interview, she demonstrated her cleaning process and we asked her to test a few prototypes we brought along. She showed us everything from how she stores her tools, cleans, and puts everything away. It was as simple as that.

    This was one of the last in-home interviews out of about a dozen conducted for this project. It went just as smoothly as the previous ones. We felt like we gathered insights. So, we asked our wrap-up questions and packed up. Once the video camera and recorder were off, we made small talk while heading to the door.

    My research partner stopped next to the door to look at a handmade shrine-like structure sitting on a tchotchke shelf. It was made out of bones! Not fake Halloween-decoration bones, but real human-looking bones. We couldn’t resist, we had to ask.

    The woman’s face lit up, and she was excited to tell us the story about her adventures visiting a tribe of cannibals in Africa. She explained that she published articles about this tribe and took many trips over the years to study their culture. She also pointed out how the shrine was made of animal, not human bones. To top it all off, she confessed that she “may” have been fed human during some of her expeditions there. Whoa!

    We asked a few more questions about her adventures, but unfortunately we had to go. The vibe in the room had completely changed. She was enthusiastic and seemed more comfortable than she did for the previous few hours. Had we noticed the statue on the way in, would the interview had been different? If the camera was still on would she have reacted the same way?

    Back in the car, I looked at the recruiter’s sheet again. Cannibal wasn’t listed anywhere. I wondered what secrets the previous floor cleaning participants hadn’t shared.

    Jaimes and Aico’s War Story: Sumimasen!

    Posted on

    Jaimes Nel of Connected Futures collaborated with Aico Shimuzu to tell this story about their rapid research in post-quake Japan.

    Research often feels like a process of managing confusion and uncertainty until we find conceptual tools to understand a situation. Failure (and subsequent redemption) are, after all, a well-worn trope in ethnography. The journey from outsider to insider is an effective literary technique that boosts the credibility of the storyteller.

    My experience is an ordinary day’s work on a more asymmetric battlefield than a typical client project. On the 11th of March 2011, a magnitude 9 earthquake struck off the east coast of Japan, causing a tidal wave that devastated towns up and down the coast of Tohoku province. Travelling to Japan later that year, I was interested in how the earthquake had affected people around the country. They had been experiencing rolling power blackouts for several months in the aftermath of the Fukushima nuclear emergency and I wanted to get a sense of whether this might prompt change in a country that has experienced sluggish growth for nearly two decades. I was only in Tokyo for a week, with other commitments, but managed to set aside a day for some research time with Aico Shimuzu, a friend who works as a design researcher and innovation consultant in Tokyo.

    Our very straightforward approach was to meet at a busy train station, a little way away from the usual Tokyo hotspots of Shibuya or Shinjuku, and simply approach people in the street, asking for a few moments of their time. We spent around 3 to 4 hours wandering through the streets around the station, striking up conversations when we could. As you might expect, more people said no to us than yes. It was probably quite helpful that we were a man and a woman, and Japanese and a foreigner. In a culture that can be very reserved, it was much easier for me to approach strangers than it was for Aico. This small cultural sin was more easily forgiven of a foreigner. My faltering Japanese also made for a great opening line, as Aico would have to step in and rescue our victims! In all we managed to have brief conversations with 7 people (some in pairs and some on their own). They kindly agreed to let us photograph them and told us about their experiences since the earthquake and what they thought had changed in Japan.

    Their stories were moving, and worrying. They all agreed that things had to change, but the events of 11th March had shaken their confidence in the authorities. Some emphasised the need to appreciate daily life, as the things that are ordinarily taken for granted had become impossible for people in Tohoku. One woman felt Japan had become selfish, and needed to re-emphasise connections between people, telling us “In the end, we can never live alone. We have to help each other.”

    Aico and I were struck by the earnestness with which the people we spoke to desired change. This was truly an event that made people notice that the infrastructure around them could not always be relied on. In a way, an event such as this is like a giant “breach” experiment, in which the unspoken assumptions we rely on to get along with each other have broken down. As we walked, Aico and I talked about all of these issues and the people who took the time to chat with us helped us learn a little bit more about our own perspectives and gave us fuel to think about things. In many ways, this type of guerrilla work is as much a tool for your thinking as it is a way to understand a topic. This is what I mean when I say it’s almost inevitably a failure. I don’t mean that it’s not successful, but rather that it can never hope to represent a coherent view. It’s simply too random. The people we spoke to had little connecting them, apart from the fact they were all going through the same experience in some fashion Doing this work outside Tohoku, we were a little distant from the events there and so were our participants. We also noticed that many of those who were willing to break stride and talk to us were from outside Tokyo, which says a little about both big cities and our process!

    Our guerrilla tactics may limit the claims we can confidently make from this data, but we can still explore them as individual cases, and use them to frame our own thought processes. Eventually, our day of guerrilla research left us with more questions than ever, just like our participants, but often that’s entirely the right place to be in a situation which is fluid and unresolved. Discussing this piece now, a year and a half later, Aico makes the point that repeating the exercise would help understand whether the social ripples from the quake were still being felt, or if people were forgetting and moving on. At the time, Japan was still working on making sense of what happened to it all those months ago and this day helped sensitise us to that atmosphere of uncertainty and doubt.

    See Aico’s pictures from Tokyo just after the quake and from Tohoku a month later.

    Apala’s War Story: Whose side is the researcher on?

    Posted on

    Apala Lahiri Chavan, Chief Oracle and Innovator at HFI relates a tale about her own distress when faced with a difficult household situation.

    In one of the very early research projects I worked on in India, we learned many best practices that became part of our research guidelines. However, there were parts of the experience that I have not quite gotten over even, after 12 years.

    We were researching in-home media usage in India for a multinational tech company from the USA. The research targeted lower middle class households from across the country, both urban and rural. My colleague Amita and I were covering the northern Indian state of Punjab. This particular evening we were visiting a semi-urban household near the city of Chandigarh.

    It was 6pm when we reached the location. We had to leave the car behind and walk the last few yards since the lane was too narrow. As we entered the gate of the house, a group of ladies came out of the house and welcomed us. They had marigold garlands for us to wear and a lit earthen lamp with which evil spirits were warded off.

    We were welcomed inside the house and ushered into a tiny but neat living room. One elderly lady who had garlanded us was the mother-in-law. She invited us to sit on a rug on the floor. She sat down too and so did a number of other people – the entire neighbourhood was there.

    The mother-in-law and some of the other elderly women from the neighbourhood did their own in-depth interview where we the participants! We were asked about our age, income, husbands, caste, education, number of children, size of house, TV serials we watched, and so on. On hearing that neither of us had children though we were married, a hush descended upon the group. After a few tense moments, the mother-in-law told us to have faith in God and not give up hope. We looked down into our plates, wondering what would happen if we told them we did not have children out of choice!

    Some young girls had also joined the gathering inside the living room (which had spilled out to the gate and into the kitchen). One girl rushed home and get us some dessert that she had made that day!

    By the time the gup-shup (friendly chatter) was over it was 7:30pm. Even though we were used to the fact that time stretched quite a bit in India and hence a session slated for an hour could stretch to an hour and a half (after all, the concept of time in India was circular unlike the linear concept of time in the West) this was beyond what either of us had experienced. But there was no possibility of hurrying anybody. How could we tell a group of extremely hospitable and warm people that our time was more important than their company?

    At last, we started the session with the mother-in-law and daughter-in-law. As we realized that the neighbours were not about to leave, Amita and I looked at each other and decided that it was fine to have the neighbours since they were clearly like a large, extended family.

    As the father-in-law, two sons and the other daughter-in-law were not back from work yet, this was women’s time. However, it became clear within the first ten minutes that this daughter-in-law was not going to speak her mind in front of her mother-in-law and the other neighbourhood matrons. We did some quick improvisation: I asked the daughter-in-law to show me the house (they knew this would be part of the interview) while my colleague continued to speak with the mother-in-law. This worked very well and while she showed me the house I got to hear the daughter-in-law’s story (e.g., how she never got to watch any television she wanted or listen to any music she chose, and so on).

    It was 8:30pm or so when the men of the house returned from work. Instantly, the atmosphere changed. Until then the mother-in-law and the other matrons had been the ‘bosses’ of the session but now everything changed. The father-in-law and the two sons were clearly in control. Mother-in-law now scurried about between kitchen and living room and many of the neighbourhood matrons and girls left. This was a very high power distance culture after all!

    We were fortunate that we had the time to speak with the ladies while the men were not around. Had they been around from the beginning, we would have not heard any of the ladies speak. Even as we carried on the conversation with the men, we suddenly noticed that there was a young lady standing at the living room door. There was a hint of aggression in her posture that made it difficult to ignore her. However, everyone else in the room seemed to not even see her.

    Then as the elder son was telling us about his media usage preferences, a most unexpected thing happened. The young lady at the door suddenly stormed into the room and started addressing us.

    She screamed and howled about the physical abuse and dowry harassment she was being subjected to by her husband (the son who was speaking) and the mother-in-law. She shouted “He made my poor parents buy him that expensive TV that he has been talking about. Now he wants them to buy him a VCR. Where are they going to get the money for that?”

    We were absolutely stunned and just as we began to get our wits together, her husband got up, caught her by the hair and gave her two resounding slaps across the face. He told her – in the most abusive language – to go up to her room.

    Enough was enough. Both of us sprang up to our feet and stood between the man and his wife. He was unperturbed and tried to pick up the thread of the conversation with us, as if nothing had happened. His parents said “This daughter-in-law is a bad woman and needs to be kept in control since she shames us.”

    We were still standing in front of the daughter-in-law who was now alternating between weeping bitterly and screaming in rage. We had to make a critical decision. What should we do? What was our role in this situation? We were researchers who were supposed to maintain our neutrality and objectivity but was that the right thing to do? Should we not call the police’s Crimes Against Women section and complain about the husband and his parents? Should we not protect this woman who was being abused?

    But before we could do anything, the daughter-in-law ran out of the house screaming and made a dash for the main gate. Not one of her family responded and told to ignore the nautanki (drama) and continue the conversation.

    We said to the family that we would not be able to continue, given what had happened. We handed over the cash incentive and walked out of the house. We looked up and down the lane but we could not see the daughter-in-law anywhere. It was little after 10pm when we got into our car and drove off.

    I have not been able to come to terms with that one incident even today. I felt that in order to be a “good” researcher I had betrayed that girl who was being abused and a much larger cause. Should one not be an activist ethnographer, when the situation demands?

    Carla’s War Story: A dirty diaper sitting in the mud

    Posted on

    Consumer insights professional Carla Borsoi encounters the outlier that illustrates a greater truth.

    There is nothing like home research to challenge your notions of whether or not everyone lives like you. Earlier this year, we were doing research on how people use multiple devices (phones, tablets ad computers) – what they are doing with each, what they feel about each device and how these are shared (or not). We were particularly focused on three audiences: Moms, Entertainment Junkies, and Earlier Adopters. Yes, in my world, we use Title Case to label our different audiences. At any rate, we picked three areas with high density for devices and plenty of each of these audiences in spades: NY, Seattle and Austin.

    I headed to Seattle in late March to meet with people and to talk to them about what they do. The first interviews went swimmingly: one Dad told us how he used his tablet to collect coupons, his computer to develop his Saturday shopping plan with coupons, and his phone to go through with his plan. He also told us about watching movies during lunch at work on his tablet. An Earlier Adopter told us how he obsessively followed tech news as he rode the bus. Good, I thought, these interviews are going really well. The Seattle weather was appropriately grey and rainy, but these folks lived in warm and welcoming homes. Normal, to me, at least, with the typical toys in the home with kids, the nice entertainment system, clean kitchens, and so on.

    It was our last day of interviewing. The rain had been pouring down the night before and I hoped it would hold off until I got to the airport at the end of the day. We were interviewing a young Mom who lived past Sea-Tac. I drove down pseudo-country roads and pulled up to the property for the interview. The driveway was full of mud. Thankfully, I was wearing wet weather boots. As I walked up with my colleagues to the front door I passed a dirty diaper sitting in the mud. Huh, I thought. Their garbage probably got torn apart in the storm last night. The house was old, but that’s how these things go. We were greeted by the young Mom and entered the house. Immediately the stale smell of cigarettes and mildew hit my nose.

    Uh-oh.

    The mom proved to be a bright young woman, who tended bar a couple nights a week, while going to school and parenting the rest of the time. I looked down at the dirty table in front of me while we continued talking. She had some great insights about how she used her tablet (often on loan to her parents who would watch the kids), how critical her phone was to keeping in touch, and how her computer was there as she worked on projects for school. However, the smell assaulted my senses. I could feel my two colleagues shifting in their seats, covering coughs. Our interview was scheduled for two and half to three hours, but after about 45 minutes, I knew I wouldn’t be able to handle it much longer. Someone asked if anyone had more questions. I quickly spoke up “Nope, think we’re good.” No one disagreed.

    We walked out the door and I noticed more garbage outside – but breathed in the sweet clean air. I realized that as researchers we occupy a place of privilege. People allow us into their homes, without embarrassment or shame. This is their life. They allow us to see a window into it. People often participate in research for the chance to earn a little cash. This woman had spoken of how much they had saved to be able to rent this small, mildewed space. It reminded me that I have a lot of advantages that other people don’t. It’s a reminder that when we’re creating products, we’re doing it not just for some sexy early adopter, but real for people who are just trying to make ends meet and get started with their life. It also reminded me to go home and wash down the walls of our stairwell, covered in grime.