If you’re starting up a new company or product, it wouldn’t hurt to scan these 204 Startup Failure Post-Mortems.
In a twisted sort of way, I particularly enjoyed this postmortem of SunEdison’s backruptcy:
SunEdison at its core is a boring construction company, that earns the trust of its institutional investors by being boring and managing risks … [but the company’s senior executives] didn’t want to be boring, they wanted to be a technology company.
Tom Wood is one of the partners at Foolproof, an experience design firm based in the UK.
About 10 years ago I was trying to understand online poker playing behaviours on behalf of a gaming company. We’d recruited for a study across their various target segments, but the hardest to find were the high-value, semi-professional players. They prized their anonymity and guarded their playing secrets.
One of the respondents I did find was a part-time property developer, part-time drummer, but his passion was poker. He was close with players from the city’s professional soccer team who were happy to lose large amounts of money in order to pick up skills in poker: an important accomplishment for the professional sportsman in the UK.
The interview did not go well at first. The respondent was a regular online player but his behaviour when using the subject site was stilted and he seemed so disengaged that I began to worry that he was out of his depth online. Eventually I decided to reframe and go back the beginning of the discussion, where we had talked about his usage habits on his regular site. This time, because he was getting more relaxed in my company, I suggested doing this by watching him play. The key behaviour this revealed was how he found a table he wanted to join. This involved simultaneously watching a large number of games in progress – an almost incredible skill. What he was studying was the weaknesses of the players at the various tables: their inexperience, bravado, impatience, petulance. His whole demeanour changed, and I had a feeling like being a naturalist watching a lion selecting the impala that it is going to turn into lunch. Compelling but horrifying at the same time. It was clear that the subject site I’d asked him to use had poor affordance for this important process, but because it was a basically unchivalrous activity he had been guarded about discussing it.
This change in tack got me this and other insights which informed our design advice. And resolved me never to take up poker.
Most experience design folk enter the field because they understand that they themselves don’t have all the answers. I’m fond of this story because it was when I properly realised that I didn’t even have all the questions. I suspect that this job made me a better researcher, and certainly made me approach certain types of work in a completely different way. At Foolproof we always preface our discussion guides with words to the effect that the discussion guide is just that, a guide – and that we reserve the right to take any approach we need to in order to meet the research objectives.
I admit I don’t have a lot of experience with children but the opportunity to shadow a patient through an entire day’s hospital visit was one to not pass up. The patient being 13 years old added another layer of consent and assent, a mythical ethnographic research unicorn of sorts.
The goal of shadowing was to understand the experience of the entire visit from start to finish, through multiple provider visits, labs, tests, and the waiting times in between. I met the patient and her mother as they were pulling into the parking garage and started the day with a scan. During the next two hours she patiently laid in a claustrophobic tunnel, and did everything as asked, from changing positions ever so slightly, holding her breath for 30 seconds at a time, and breathing at a specific pace.
Having fasted since the previous evening, she was ready for lunch but wanting to get everything done before their provider visit, she and her mom decided to get a blood test done before lunch.
We arrived in the pediatrics department and her mother stood in line to check in while I joined the patient in the waiting area. After a few minutes, a volunteer came over for what I felt was a break in our somewhat awkward small talk.
The volunteer was a kind elderly man with a book cart offering free books for patients to take home. The patient, tired from the scan and possibly feeling out of place in the bright and cheerful pediatrics environment shrugged and said there wasn’t anything she liked. Determined, the volunteer took out a “magical coloring book” which colored itself with a flip of a page. She was still not impressed.
Then came the pièce de résistance. From the cart the volunteer pulled out a heavy woven rope and introduced the patient to his friend, Mr. Stick. Mr. Stick had a magic ability you see, with a grand gesture he could become taut. In order to turn back into a rope, the patient was instructed to ask, “Mr. Stick, will you go down?”
The shade of red across the teen’s face had long passed lobster and she and I stared at each other in disbelief. Her mother was still in line across the way, and as the adult I felt responsible but conflicted on what to do. Surely the man had no idea what he was implying? Being a very good sport, she complied and sure enough Mr. Stick fell limp.
But the volunteer didn’t stop there. He turned to me, holding the middle of Mr. Stick, now back in its rigid state. He asked me to tell Mr. Stick to go down, which I did. Nothing happened. The volunteer said I must say “please”, which I did. And again nothing happened. He then said, “I guess Mr. Stick doesn’t go down if you’re not a child.”
“Hey, I think they’re calling your name,” I quickly said to the patient. And with that we escaped the somewhat creepy, but good intentioned volunteer.
“That was awkward,” she said.
It wasn’t until after the blood test and during lunch that we were able to debrief and talk about the encounter with the volunteer. I was afraid the mother would be upset that I hadn’t intervened sooner. She was shocked but laughed, wondering if someone could really be that clueless. As I started to explain what had happened, the patient (who been sitting right next to the volunteer) intervened:
“No, its name was Mr. Stiff, not Stick.”
Me: “Oooh, that’s even weirder.”
Mother: “I’m really curious how you’re going to write this up.”
Ilona Posner is a User Experience and Usability consultant with more than 25 years of experience. In this story, she is challenged in different ways to leave her participants in relatively good shape.
Around the year 2000, homes with internet service were rare. AOL was plastering the planet with CDs that promised free internet. Modems were uncommon and expensive. Online access usually required a modem card installed inside a computer case by a service technician, at a significant cost. My client, the largest Internet Service Provider in Canada, was redesigning their Self-Installation Package for its DSL service; today this would be called a DIY kit.
The goal of our research project was to evaluate the customer experience. It entailed contacting customers who had just ordered the package, interviewing them about their order experience, and asking to visit their homes to observe the installation of the hardware and setting up the service. We visited many homes and observed people with diverse technical experience trying to install this package. The success rate of the customers completing this self-installation within our allotted 2 hours was very low. We had to suffer silently watching their ordeals: searching among numerous papers and user manuals that accompanied the package for the correct documents and locating the required identification codes; mixing up phone and internet cables; moving their furniture so that the provided cables would reach their destinations; and trying to explain their problems in repeated phone calls with technical support. In some cases, after observing them struggle for 2 hours and realizing they were incapable of completing this task unaided, we felt so sorry for them that before departing we completed the installation process on their behalf. We felt bad that they would have to spend additional days waiting, making additional phone calls to arrange for a technician’s visit, and dealing with the additional costs of assisted installation. That way, we also were able to witness their excitement and gratification of getting online; for some it was their first time.
I clearly remember one participant who actually was able to successfully complete the installation, and it “only” took him 1.5 hours to do it. He was a male in his early 30s, technical writer by profession. His PC had 32 MB of RAM, and was running Windows 95. He already had a modem but was switching to this new High Speed Service. He had to remove the internal ISA modem card from his PC tower in order to install the provided Ethernet card. He was more confident and comfortable at this task than most of our other participants. While our camera rolled, he confidently skimmed documents and manuals, even when they were different manuals from the devices he was dealing with at the time. He opened his PC without difficulty. He proceeded to remove the internal modem card from deep inside his PC case. In the process, he cut his hand on one of the sharp internal edges of the metal case. His hand started to bleed! Blood got on his hardware. We had to interrupt our observations to assist him in stopping the bleeding.
After completing our research, we redesigned the package. We reduced the number of documents and numbered each one for easy reference (unfortunately, this simple and usable solution only lasted until the next rebranding exercise conducted by the marketing department, who did not inherit our design rationale). We rewrote the instructions, using beautiful visuals. We also included a special highlighted warning, “Please be careful when opening your computer case, there are many sharp edges inside.”
I wonder if anyone ever noticed that warning message.
From New York Times Corrections for April 7, 2013 comes another demonstration of a) the importance of context in understanding what interview respondents are saying and b) the necessity of an actual recording of what was said.
An accompanying feature transcribed incorrectly a comment from Callie Khouri, creator of the television drama “Nashville,” about what she would put on her Easter playlist. Khouri said she would include music by Pops Staples, the late patriarch of the singing family the Staple Singers. She did not say she would include “pop staples.”
Kavita Appachu shares her story about uncovering emotion where she hadn’t expected to find it.
Finance has never been my thing, and where possible I leave the chore of managing my finances to others. That changed somewhat a few years back when I started working for a company that makes financial software, specifically tax software. This threw me right in the middle of people’s financial lives.
What I had not realized was that while the task of managing finances may be very functional, everything else related to money and taxes is at its core very emotional. I have lost track of the innumerable times participants have poured their hearts out as they describe how they manage their finances, from the twenty-something who referred to her mom as ghetto, or the hulk of a guy who rattled off the choicest of expletives for his ex-wife. The one story that has stood out in all this is about a mom, wife and editor in Seattle.
On a rare sunny day, we pulled up to a community of condos with well-manicured yards. We rang the doorbell and my fellow researcher and I were greeted by our participant, who welcomed us into her very tastefully done home. There were pictures of the kids, family vacations, sporting events. It seemed like a happy home. The kids were at school and our participant had the morning off so she had decided to catch up on her finances, specifically her investments. We talked about the members of her household, her husband’s job, her job and their approach to financial planning. She was concerned their savings were not going to be enough for retirement and the kids’ education.
She had all her papers spread out on the dining table beside her laptop. We observed her going through the process of logging into both her and her husband’s 401(k) accounts, monitor her mutual funds and stocks and even place a sell order. Nothing out of the ordinary…and then she broke down in tears.
We were a little taken aback. She had a helpless look on her face and kept sobbing and muttering that woman, that woman. We calmed her down and then asked her if she wanted to share what was bothering her. She told us that as part of her husband’s divorce settlement from his earlier marriage he was required to pay for her stepchildren’s college. That was making a deep hole in their pockets and she was unable to save for her own children’s college education, take vacations or save for retirement. She hated the ex-wife and held her husband somewhat responsible for giving in to the ex-wife’s demands. She avoided tracking finances if she could because it was a painful reminder of her dire situation.
That was my aha moment. I had known all along that personal finances are very closely entwined to one’s life, but this really brought it home: personal finances are a mirror of your inner joys, sorrows and insecurities.
Whitney Hess is the author of Pleasure & Pain, and the founder and principal of Vicarious Partners, an independent consultancy specializing in strategic user experience. She believes empathy builds empires.
I interviewed Holocaust survivors. Four words that still send shivers down my spine. Their stories were meant to shape my research; they ended up shaping me.
It was the project of a lifetime. I was asked to conduct user research for the United States Holocaust Memorial Museum with web design agency Happy Cog. Together we identified several constituents of the Museum to explore: visitors, students, teachers, scholars, activists, volunteers, donors…and survivors. Survivors of the Holocaust. I would be performing the interviews, crafting personas, and reporting on findings to the Museum’s executive board.
As a rule, when I engage with a research participant, I, Whitney Hess, cease to exist. It is a skill I have honed over many years of conducting research. I don’t get hungry, I don’t get tired, I don’t have to pee. I shed my beliefs and my assumptions and my identity. My only need is to listen. My only purpose is to absorb – with total objectivity.
Would it be possible then for me to objectively study Holocaust survivors? I am a Jew.
At first I told myself that being Jewish somehow qualified me to understand their stories and empathize with their pain. Then I feared that I would get so emotional that I wouldn’t be able to make it through an interview.
I was wrong on both counts.
I had the honor and the privilege of interviewing seven survivors – from Germany, Lithuania, Czechoslovakia, Austria, Poland, Slovakia, and Great Britain – all volunteers at the Holocaust Museum in varying capacities. Some interviews were in person at the Museum, others were over the phone. They shared their stories of survival, and they shared their feedback on the website. Both extremes were just as relevant. I listened with reverence and I asked probing questions. I was so busy taking it all in, I didn’t have time to feel anything about it. I was working.
When it came to crafting personas, I started with the teachers and students, moved on to activists and scholars, and eventually I could postpone it no longer – it was time to review my findings from the survivors.
Reading back through my notes and the interview transcripts, I maintained my composure. I kept reminding myself, You have work to do. But in a moment of weakness, I allowed myself to listen to a recording. And then another. Day became night and I was still listening. They recounted the abuse they’d endured, the brutality they’d witnessed, the family they’d lost…it was so raw, so real. I let myself go. I cried, bawled. For what they had overcome, for themselves, for their families, and for me.
In the end, I decided not to create a persona of a survivor, and my teammates and clients understood my reasoning. Their stories were unique; they could not be merged.
Instead I gleaned a few key quotes, to convey the essence of the individuals. What they had to say changed my whole perspective on what we were doing and why we were doing it. Their message had to be heard. I had to share it. I got to share it.
And it changed everything.
I was interviewed by Tomer Sharon (as part of his incredible series of interviews for It’s Our Research). Our conversation ranged from about why it’s hard for people to do user research, the collaboration between agency and clients, and how to think about organizational and stakeholder challenges as design problems.
The 22-minute video is embedded below and can also be seen at the first link.
Marta Spurgeon plays the roles of design researcher, innovation capabilities consultant, and sometime photographer at Doblin in Chicago.
I was fresh out of the Peace Corps. Somehow, through an educational background in photography and videography and a unique set of personal contacts, I landed myself in a contracting gig at a design strategy firm. It turns out this was the perfect place for me. It makes sense that all roads led here, though I wouldn’t have been able to characterize it at the time. My experiences on that first project would solidify both my approach to ethnographic-style research and my interest in innovation in the business sector. The techniques and tasks associated with international development share some remarkable similarities with those we utilize in business innovation and design strategy.
In our Peace Corps training, we were encouraged to “do nothing” for the first 6 months of our service, to just sit with the host-country nationals in their day-to-day activities, observe, ask questions. And indeed, most of my Peace Corps experience was composed of these moments of quiet observation, learning about a culture so foreign to me as to ultimately challenge my beliefs about my own. Many of us are familiar with this participant-observer stance as one of the foundations of ethnographic study. It requires us to put aside pre-conceptions and biases, to embrace the other as a credible expert despite the legions of difference between us. In both international development occupations as well as strategic design efforts, this anthropological approach to learning about the people we’re designing for is the foundation of an often ambiguous process to create and launch new concepts that will be adopted by those-or people like those-whom we’ve studied, and ideally help them improve their lives.
Though I certainly had more casual experience doing this as a Peace Corps Volunteer, I was on my first professional foray into ethnographic-style user research. Our team was learning about people’s experiences using medical devices in the home. At this point, we had spoken with a couple dozen medical practitioners in their professional settings and patients in their homes. We were on our final interview of the study. My role had been to photograph and video the interviews, take notes, and generally to follow the lead of my teammates who were directing the session. But for this final interview, my colleagues asked if I’d like to conduct the conversation, and I took them up on the opportunity to lead my first formal in-context interview.
We drove to a relatively remote location in Connecticut to see a middle class family of two parents and three boys. Two of the boys had an immune condition that required them to pump medication for one to two hours every two weeks. The parents had decided that rather than stigmatize or de-vitalize the process of the boys’ drug infusion, they would celebrate it by joining together as a family for a pizza party and movies on Friday night. This celebration was in full-swing as we entered their home.
It was a lively atmosphere. It turned out this wasn’t just a family of five; they lived with a menagerie of animals in their small home-cats, dogs, birds, rabbits, reptiles, and guinea pigs, bringing their total number to between 20 and 30 inhabitants. We were introduced to the guinea pigs and shown the rabbits. Everyone was supremely generous and inviting. They gave us a little tour, encouraged us to get comfortable offering us food and drink several times. Cats snuggled up beside us, intermittently disrupting our video equipment or the conversation, while birds squawked in the background. Comfortable and confident amongst one another, this family moved freely and raucously around me and my two colleagues, us all a bit squished onto too few pieces of furniture for all eight of us humans.
The parents graciously answered our questions about their children’s health and their medical needs, as the boys played video games and watched cartoons energetically, occasionally peppering the conversation with commentary or a boisterous request for attention-“Watch this! Watch!” They showed us how they hooked up the medication pumps, from prepping their sons’ skin to inserting the needles, demonstrating how it all worked. Father proudly brought out two large toolboxes full of medical supplies that they took along whenever they got in the car. He had come up with the idea of creating toolkits for the supplies they needed to be mobile. The interview continued successfully, if a bit disordered, given all the different activities happening. Not at any moment were they embarrassed or ashamed of the boys’ condition or the things they had to do to treat it. To them, this was just their life.
And when we finally all said goodbye, and the door shut behind us, I think all three of us researchers breathed a small sigh of relief. Truthfully, it had all been quite chaotic, though we had done our best to take it in stride. But our last interview was complete, and we got into the car, heading toward New York to fly home the next day.
Driving along, one of my teammates offhandedly said, “Well, I don’t think we learned anything useful from that. That scene was a complete mess! What a waste of time.” This somehow infuriated me. Sure it was intense, chaotic, indeed a less tidy environment than might be desired. They had more animal friends than a small farmer might. The lifestyle this family lived was obviously busy and disorganized. Certainly they had some health problems, probably some difficulty making ends meet, and a shortage of square footage for all of the living things in their home. But they also clearly loved one another and were just doing the best they could to live full, healthy, enjoyable lives. I may have been totally green and unfamiliar with utilizing this research practice for new business innovation, but I knew it wasn’t our place to judge, whether we approved of their lifestyle or not.
I was so angry. Never one to hold back, I told this teammate exactly what I thought. That these people had generously and openly invited us into their home so that we might learn about how they live, how they experience their medical conditions, how they interact with these essential medical devices. Whether we found their lifestyle appealing or disgusting, it was valid. Their experiences were real, and we were there to learn about them. It was unfair and totally inappropriate to judge them, and it missed the entire point of what we were there to do. I said all this, I’m sure, not nearly as eloquently as I say it now, and likely with less respect than my colleague deserved as he had more experience and knowledge on the subject than I did. He actually took it relatively well all things considered, and we remain friends today despite the words exchanged on that trip.
But I’ve found this to be one of the formative moments of my career-a moment when I expressed with passion and understanding just exactly what our purpose was there. And I’ve found similar sentiments coming to my lips again and again (with increasing grace and respect, of course), as I’ve had to remind most often clients but sometimes colleagues why we do this work. For an hour or two, we go into the home of a stranger, with a respect and appreciation for the validity of each individual’s experience. We must practice empathy, reserving judgment, allowing ourselves to stand in the other’s shoes, understand how he lives, why she does what she does, what they want to achieve, what makes that hard for them. So that in the end we might create better solutions that help them do it and make theirs and other people’s lives better and healthier. Sometimes we just have to remind ourselves.
IDEO. NYC. Early 2010.
I had been summoned from Europe to lead a project about the future of education in the US. At IDEO, there is a well-established a code of ethics for site visits. This code takes extra measures to protect the privacy of informants – especially their identities and contact data. IDEO also has sensible, street-smart guidelines for fieldwork in sketchy environments. In previous jobs, I had seen a situation in which two of my female design researchers had to go to remote, sparsely populated parts of the Midwest and visit big, burly, smiling men who stored every conceivable power tool in their dungeon-like tornado cellars.
There is never a shortage of people in NYC though, and recruiting there offers many delights. For instance, NYC is one of only several places where it is possible to recruit for impossibly specific profiles like: “Seeking 3 single dads who have volunteered with their children at a local charity organization within the past 2 weeks, and who also must struggle with their own gender identity and make at least $150K/year.” In the Tristate, if you are one in a million, by definition there are at least 22 of you.
Our recruiter used Craigslist for most projects and straightaway found us one of our targets: a working mother who had successfully completed a BA online while still raising a family. I had a new team and my associate design researcher was an eager, empathic and articulate ethnographer doing her first project at IDEO. We headed out to Inwood in Brooklyn for our first site visit, hoping to get insights from this working, baccalaureate mom.
During the ride, I played the senior mentor guy, offering advice about doing ethnography “in a design context.” We arrived at the address in Inwood, an obscure part of Brooklyn that looks like a sad, dilapidated part of Queens that in turn, tries to look like a nondescript suburb in Long Island. We were buzzed into the building, walked up to a door and were greeted by a large woman with a curly red mane of hair. Her name was “Roberta-but-call-me-Bert.”
She let us in. The apartment was dim. It smelled of litter box mixed with burnt Dinty Moore beef stew that Ramon, Roberta-call-me-Bert’s husband had overheated on the stove. The dingy plaster walls were covered in old shopping lists, written in a mangled scrawl that suggested vaguely menacing pathologies and personality disorders suffered by their author.
The sofa we sat on smelled of cat piss, and the living room offered up no pretense of ever having been cleaned. We sat up straight, made eye contact in that standard, pious, non-judgmental manner that earnest ethnographers often adopt. We began the paperwork. We were offered water and politely declined.
I asked her about work, family, free time; all of the perfunctory questions before we got into her BA experience. Since I was the seasoned professional, I led the discussion, “Tell me a story about your favorite class…”, “Did you make friends with your classmates?”, “Do you still keep in touch?” Since my associate was taking notes, I focused on keeping the discussion moving and letting Roberta-call-me-Bert lead us to all sorts of exciting insights.
The trouble was, she didn’t.
“Oh, I don’t remember much about that class,” she said about her favorite statistics course she took just before graduating 18 months ago. “Yeah, I pretty much kept to myself, because I had to work and raise a family, you know?” I nodded my head earnestly.
I began asking her questions about change: “Do you view your daughter’s education differently now since you got the degree?” “Not really,” she said, as her daughter ate ice cream from a container while watching a YouTube video about dog fighting.
We eventually went on our way. Once out the door, I was about to launch into the debrief. Since I was the experienced one, I was going to teach my associate a simple, time-honored 20 minute structure I often use for debriefs: Interesting Behaviors/Motivations and Drivers/Problems and Frustrations/Opportunities.
I noticed that she was grimacing.
“What’s wrong?” I asked.
“That was a waste.” she replied.
“What do you mean?” I asked.
“She lied, she never went to college.”
I was gobsmacked.
And she was absolutely right.
There were no interesting behaviors. There were no drivers or motivations. There were no problems or frustrations. There were no opportunities.
There was no diploma. It was “packed away somewhere.”
We returned to the office. Another colleague was leading a project in men’s fashion and desperately trying to recruit shop-along dyads of couples in their 40’s and 50’s where wives selected the husband’s clothes. She said they had already recruited one couple on Craigslist and that her name was Roberta from Inwood, Brooklyn.