{"id":2789,"date":"2021-03-13T21:05:38","date_gmt":"2021-03-13T21:05:38","guid":{"rendered":"https:\/\/rosenfeldmedia.com\/advancing-research-2021\/?page_id=2789"},"modified":"2021-03-14T14:00:34","modified_gmt":"2021-03-14T14:00:34","slug":"ar2021-what-did-i-miss-the-hidden-costs-of-de-prioritizing-diversity-in-user-research-megan-campos-madpow","status":"publish","type":"page","link":"https:\/\/rosenfeldmedia.com\/advancing-research-2021\/ar2021-what-did-i-miss-the-hidden-costs-of-de-prioritizing-diversity-in-user-research-megan-campos-madpow\/","title":{"rendered":"AR2021-What Did I Miss? The Hidden Costs of De-prioritizing Diversity in User Research (Megan Campos, Mad*Pow)"},"content":{"rendered":"

\"\"<\/p>\n

\u2014> Hi everyone, and I hope everyone is staying safe and healthy<\/div>\n
<\/div>\n
\u2014> \u00a0Today\u2019s talk will be about the importance of diversity in UX research<\/div>\n

 <\/p>\n

\"\"<\/p>\n

\u2014> This topic has its roots in my own experience in the nuances of identity, especially with her own background as being half-Puerto Rican.<\/div>\n
    \n
  • \n
    This has impacted how I approach UX research recruitment<\/div>\n<\/li>\n<\/ul>\n

    \"\"<\/p>\n

    \u2014> At MadPow, we are always examining practices and how they can be improved for clients and users<\/div>\n

     <\/p>\n

    \u2014> Now, to frame this talk:<\/div>\n
    <\/div>\n
      \n
    1. \n
      This conversation is ongoing. Our goal is to expand the scope of conversation and learn together<\/div>\n<\/li>\n<\/ol>\n
        \n
      1. \n
        I don\u2019t have the solution, and we need to work together to iterate and evolve on it<\/div>\n<\/li>\n
      2. \n
        Diversity for UX research refers to race, gender, economic, social status, disability and other aspects of identity<\/div>\n<\/li>\n
      3. \n
        We need to avoid the term \u2018under-represented\u2019, as we don\u2019t want to center dominant characteristics as the standard for our research. Mis-represented is a better word choice<\/div>\n<\/li>\n<\/ol>\n

         <\/p>\n

        \"\"<\/p>\n

        \u2014> How\u2019d I get here?<\/div>\n
          \n
        • \n
          I noticed that all screeners called for a mix of criteria like gender, race, income<\/div>\n<\/li>\n
        • \n
          But actual user studies had a homogeneous talent pool that was all cis-gendered, and mostly white and middle-class<\/div>\n<\/li>\n<\/ul>\n
          <\/div>\n
          \u2014> I wanted to know the reasons why a desire diverse factors didn\u2019t have impact on how research was conducted<\/div>\n

           <\/p>\n

          \"\"<\/p>\n

          \u2014> Demographics matter, and shape how we experience the world<\/div>\n
          <\/div>\n
          \u2014> There are many essays, books, journal, posts that capture how the world looks different to you depending on skin color, sexuality, class status, etc.<\/div>\n
            \n
          • \n
            From how you walk into a store, to everyday conversations, you are shaped in visible and invisible ways by your demographic make-up<\/div>\n<\/li>\n<\/ul>\n

             <\/p>\n

            \"\"<\/p>\n

            \u2014> For \u00a0a recent example, look at how Black populations were impacted by the coronavirus pandemic, due to pre-existing \u00a0health-care system discrimination<\/div>\n
              \n
            • \n
              Importantly, this discrimination was visible only when stress occurred to the healthcare system in the form of the pandemic<\/div>\n<\/li>\n<\/ul>\n

               <\/p>\n

              \"\"<\/p>\n

              \u2014> So how do we get demographic mix for UXR efforts?<\/div>\n
              <\/div>\n
              \u2014> First, I wanted to validate to see if I was only person who saw need for changing our approach. So I ran a survey<\/div>\n

               <\/p>\n

              \"\"<\/p>\n

              \u2014> I wanted to get a sense of how people think about demographic criteria,<\/div>\n
                \n
              • \n
                I reached out to people who run UX research studies, and recruitment screeners for these studies<\/div>\n<\/li>\n
              • \n
                I also reached out to personal\/professional networks<\/div>\n<\/li>\n<\/ul>\n

                 <\/p>\n

                \"\"<\/p>\n

                \u2014> At a high-level my survey indicated that demographic criteria is less important than project criteria, or contextual criteria<\/p>\n

                 <\/p>\n

                \"\"<\/p>\n

                \u2014> Then there was a hierarchy of importance In terms of which demographic data was important for UX research<\/div>\n
                  \n
                • \n
                  Race, ethnicity, gender, sex, age, location, income, education, disability, and profession<\/div>\n<\/li>\n<\/ul>\n
                  <\/div>\n
                  \u2014> Race, ethnicity, gender, sex were seen as less important relative to other criteria like age, which was seen as the most important rating<\/div>\n

                   <\/p>\n

                  \u2014> Location, profession, household income, fell into the middle<\/div>\n

                   <\/p>\n

                  \"\"<\/p>\n

                  \u2014> Demographic data with more invisible structures was given less priority in terms of research relevance<\/div>\n
                    \n
                  • \n
                    The highest priorities were profession and age which clearly impact a person\u2019s experience<\/div>\n<\/li>\n
                  • \n
                    The lowest priorities were race and gender which do not have as visible and impact<\/div>\n<\/li>\n<\/ul>\n
                    <\/div>\n
                    \u2014> I began creating a ranked order to assign characteristics with higher\/lower area<\/div>\n
                    <\/div>\n
                    \u2014> My hypothesis: Since we want to believe gender, ethnicity, and race don\u2019t have an impact on our lived experience, and the experiences we observe in others, we have a tendency to downplay these differences<\/div>\n
                    <\/div>\n
                    \u2014> As a result, we are comfortable saying differences matter in things that are more obvious to us like occupation and age, than in things that are less visible like race and gender<\/div>\n
                      \n
                    • \n
                      Seeing race and gender impacting people\u2019s experiences makes us see the inequality in other’s experiences, \u00a0and that makes us feel uncomfortable<\/div>\n<\/li>\n<\/ul>\n

                       <\/p>\n

                      \"\"<\/p>\n

                      \u2014> So why not emphasize diversity?<\/div>\n
                        \n
                      • \n
                        I heard two kinds of responses for my request to recruit for diversity<\/div>\n
                          \n
                        • \n
                          1) Demographics don\u2019t shape participant behavior<\/div>\n<\/li>\n
                        • \n
                          2) We know that demographics matter, but we can only do our best<\/div>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n

                           <\/p>\n

                          \"\"<\/p>\n

                          \u2014> Here are some erbatim quotes that say demographics don\u2019t matter<\/div>\n
                            \n
                          • \n
                            In\u00a0their view, a person\u2019s identity and the experiences they have, doesn’t matter in a study<\/div>\n<\/li>\n<\/ul>\n

                             <\/p>\n

                            \"\"<\/p>\n

                            \u2014> However, by neglecting diversity, we are giving in to existing structural and social inequality<\/div>\n
                            <\/div>\n
                            \u2014> We can\u2019t expect people to have behaviors and experiences completely separate from their identities<\/div>\n
                              \n
                            • \n
                              And we are also missing an opportunity to serve our users<\/div>\n<\/li>\n<\/ul>\n

                               <\/p>\n

                              \"\"<\/p>\n

                              \u2014> On the second belief, people stressed they did what they could to make sure participants were diverse<\/div>\n
                              <\/div>\n
                              \u2014> I agree with this sentiment, but shouldn\u2019t \u00a0our \u201cbest” be open to improvement and evolution?<\/div>\n
                                \n
                              • \n
                                We should be able to shift our perspectives to adjust our practices accordingly<\/div>\n<\/li>\n<\/ul>\n
                                <\/div>\n
                                \u2014> We should apply a willingness to improve our practice<\/div>\n
                                  \n
                                • \n
                                  Stagnation isn\u2019t good for everyone at best, and can be detrimental to some at worst<\/div>\n<\/li>\n<\/ul>\n

                                   <\/p>\n

                                  \"\"<\/p>\n

                                  \u2014> Now, for some examples<\/div>\n
                                  <\/div>\n
                                  \u2014> We wanted to figure out what the health-care experience looked liked for independent, employer-insured adults in Massachusetts.<\/div>\n
                                    \n
                                  • \n
                                    Nothing in this case study was explicitly race specific, but health-care \u00a0in the United States is stratified along racial lines<\/div>\n<\/li>\n<\/ul>\n

                                     <\/p>\n

                                    \"\"<\/p>\n

                                    \u2014> We set up a panel quota of non-white participants, which was pulled from census data estimates<\/div>\n
                                      \n
                                    • \n
                                      Our recruiter struggled to get participants for quota, but we insisted on it, and we finally got six non-white participants<\/div>\n<\/li>\n<\/ul>\n
                                      <\/div>\n
                                      \u2014> These non-white participants didn\u2019t have a totally different experience from the white participants, but their personal stories had distinctions that reflected other trends, which included:<\/div>\n
                                        \n
                                      • \n
                                        A wariness of being under-prescribed and over diagnosed<\/div>\n<\/li>\n
                                      • \n
                                        A fear of being unable to find Black doctors, as subjects wanted someone who could relate to them<\/div>\n<\/li>\n<\/ul>\n

                                         <\/p>\n

                                        \"\"<\/p>\n

                                        \u2014> One participant expressed a concern about being under-prescribed and over-diagnosed.<\/div>\n
                                        <\/div>\n
                                        <\/div>\n
                                        \u2014> Another felt his doctor dismissed and ignored him due to his race.<\/div>\n
                                          \n
                                        • \n
                                          Being Black influenced his health-care experience,<\/div>\n<\/li>\n<\/ul>\n
                                          <\/div>\n
                                          \u2014> These differences weren\u2019t rooted in profession or age, but rooted in experiences in the subject’s identities as Black men and women<\/div>\n
                                            \n
                                          • \n
                                            These differences are important<\/div>\n<\/li>\n<\/ul>\n
                                            <\/div>\n
                                            \u2014> Interviews gave us personal stories and let us craft solutions to address pain points<\/div>\n
                                            <\/div>\n

                                             <\/p>\n

                                            \"\"<\/p>\n

                                            \u2014> So what happens when don\u2019t insist on diversity?<\/div>\n
                                            <\/div>\n
                                            \u2014> Well, we don\u2019t know what we miss when diversity is avoided in recruitment<\/div>\n
                                              \n
                                            • \n
                                              We can\u2019t ask questions to people that were excluded, since they are out of the conversation by default<\/div>\n<\/li>\n
                                            • \n
                                              All we have are hypotheses<\/div>\n<\/li>\n<\/ul>\n

                                               <\/p>\n

                                              \"\"<\/p>\n

                                              \u2014> For example, in another project, our goal was assessing the comprehension and actionability of a \u00a0health insurance \u201cwelcome letter\u201d for new members<\/div>\n
                                                \n
                                              • \n
                                                The audience was 80% non-white and had a large segment of non-English speakers<\/div>\n<\/li>\n<\/ul>\n
                                                <\/div>\n
                                                \u2014> Despite the clear need for diversity, the participant list was not reflective of the population served by the provider<\/div>\n
                                                  \n
                                                • \n
                                                  All participants spoke English as their first language<\/div>\n<\/li>\n
                                                • \n
                                                  The client felt it was too expensive to hire a Spanish-speaking moderator<\/div>\n<\/li>\n<\/ul>\n

                                                   <\/p>\n

                                                  \"\"<\/p>\n

                                                  \u2014> As a result, all the people in the study understood what the insurance welcome packet was asking them do<\/div>\n
                                                    \n
                                                  • \n
                                                    The impact of homogeneous recruitment was that the client couldn\u2019t say anything with certainty about their customer\u2019s experience<\/div>\n<\/li>\n
                                                  • \n
                                                    They could only provide assumptions about what wasn\u2019t heard, like cultural barriers, language comprehension issues, etc.<\/div>\n<\/li>\n<\/ul>\n
                                                    <\/div>\n
                                                    \u2014> We don\u2019t know if these cultural barriers are real though, so we can\u2019t solve for these barriers<\/div>\n
                                                      \n
                                                    • \n
                                                      All we have are assumptions that otherize groups, and treat the group we talked to as the norm for our recommendations<\/div>\n<\/li>\n<\/ul>\n

                                                       <\/p>\n

                                                      \"\"<\/p>\n

                                                      \u2014> So how do we ensure diversity?<\/div>\n
                                                      <\/div>\n
                                                      \u2014> I will discuss an initial approach and suggestions<\/div>\n

                                                       <\/p>\n

                                                      \"\"<\/p>\n

                                                      \u2014> First use a third party to get research recruits<\/div>\n
                                                        \n
                                                      • \n
                                                        Get very specific in asks for third-party recruits<\/div>\n
                                                          \n
                                                        • \n
                                                          Need to be demanding with numbers, and making sure the recruiter can deliver<\/div>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n
                                                          <\/div>\n
                                                          \u2014> If we accept that demographics matter, and shape experiences, we need to be specific and demanding for quotas<\/div>\n
                                                            \n
                                                          • \n
                                                            If you make it clear it\u2019s not a suggestion, you get what you ask for<\/div>\n<\/li>\n
                                                          • \n
                                                            Recruiters should not treat this as optional<\/div>\n<\/li>\n<\/ul>\n

                                                             <\/p>\n

                                                            \"\"<\/p>\n

                                                            \u2014> Next, try screening recruiters, knowing you want to recruit diverse participant pool. \u00a0Be upfront with recruiter and ask if they can meet your needs or not<\/div>\n
                                                            <\/div>\n
                                                            \u2014> If it can\u2019t work, find someone who can execute on an address<\/div>\n
                                                              \n
                                                            • \n
                                                              Hire a recruiter who can get job done accurately<\/div>\n<\/li>\n
                                                            • \n
                                                              Recruiters will make change only when you incentivize them to do so<\/div>\n<\/li>\n<\/ul>\n
                                                              <\/div>\n
                                                              \u2014> Your responsibility to make diverse number of recruits a priority<\/div>\n

                                                               <\/p>\n

                                                              \"\"<\/p>\n

                                                              \u2014> Regarding guerrilla research, \u00a0people tend reach out to people who look a lot like them.<\/div>\n
                                                                \n
                                                              • \n
                                                                You need to go outside your network, and tailor your recruitment to appeal to a broader audience<\/div>\n<\/li>\n<\/ul>\n

                                                                 <\/p>\n

                                                                \"\"<\/p>\n

                                                                \u2014> Many researchers have already done this, and I encourage you to do so<\/p>\n

                                                                 <\/p>\n

                                                                \"\"<\/p>\n

                                                                \u2014> The following suggestion comes from my experience with teams who are homogeneous<\/div>\n
                                                                  \n
                                                                • \n
                                                                  Many of us are operating from a position of privilege as a result, and are bringing a similar perspective to the research process<\/div>\n<\/li>\n<\/ul>\n
                                                                  <\/div>\n
                                                                  \u2014> Hiring diverse teams, can help you catch people who don\u2019t get recruited for user research<\/div>\n

                                                                   <\/p>\n

                                                                  \"\"<\/p>\n

                                                                  \u2014> Diversity is worth the effort<\/div>\n
                                                                    \n
                                                                  • \n
                                                                    If demography matters, we should recruit participants who reflect the make-up of our user demographics<\/div>\n<\/li>\n
                                                                  • \n
                                                                      \n
                                                                    • \n
                                                                      We need to be insistent on finding the pool of recruits<\/div>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n
                                                                      <\/div>\n
                                                                      \u2014> Otherwise we will get flat dimensions, and making assumptions about wants, needs, and experiences of our users<\/div>\n

                                                                       <\/p>\n

                                                                      \"\"<\/p>\n

                                                                      \u2014> This presentation is not the end of the conversation<\/div>\n
                                                                        \n
                                                                      • \n
                                                                        There\u2019s no guaranteee that diverse participants will reflect our user audience<\/div>\n<\/li>\n
                                                                      • \n
                                                                        But if we continue to work with homogeneous recruitment pools there is a 100% chance of ignoring the needs of a diverse population<\/div>\n<\/li>\n<\/ul>\n

                                                                         <\/p>\n

                                                                        \"\"<\/p>\n

                                                                        Q&A<\/span><\/div>\n
                                                                        <\/div>\n
                                                                          \n
                                                                        1. \n
                                                                          \u00a0When talking with specific audiences where diversity is a known systemic issue (such as college students), how do you tackle recruitment in a way that does not result in any one or two participant feeling like they are \u2018tokenized\u2019?<\/div>\n<\/li>\n<\/ol>\n
                                                                          <\/div>\n
                                                                          A: You can\u2019t represent everyone, but see how you can do the most good with the work that you had<\/div>\n
                                                                          <\/div>\n
                                                                          \u2014> Use words that are inclusive, and be conscious of how audience wants to be spoken to<\/div>\n
                                                                          <\/div>\n
                                                                          \u2014> Questions asked should cover holistic experience, not just focusing on one characteristic<\/div>\n
                                                                          <\/div>\n
                                                                            \n
                                                                          1. \n
                                                                            Would you recommend using purposive sampling to draw more on experiences of those marginalized by various factors (i.e. over-representing them in the sample) rather than using representative sampling?<\/div>\n<\/li>\n<\/ol>\n
                                                                            <\/div>\n
                                                                            <\/div>\n
                                                                            A: We are currently over-representing white\/cis people, so when needs are there, it makes sense<\/div>\n
                                                                            <\/div>\n
                                                                            <\/div>\n
                                                                              \n
                                                                            1. \n
                                                                              What is your process of deciding what the specific mix should look like? Does it start from understanding the product user base?<\/div>\n<\/li>\n<\/ol>\n
                                                                              <\/div>\n
                                                                              A: Start with asking if client has audience breakdown of existing user population, and understanding where the product wants to go<\/div>\n
                                                                              <\/div>\n
                                                                              \u2014> If clients don\u2019t have the data, reference US census data as a backstop<\/div>\n
                                                                              <\/div>\n
                                                                                \n
                                                                              1. \n
                                                                                For those researchers undertaking their own recruitment, do you have any recommendations for how to reach out to misrepresented groups?<\/div>\n<\/li>\n<\/ol>\n
                                                                                <\/div>\n
                                                                                A: Caveat, in that I use third-party \u00a0recruitment and not going into space<\/div>\n
                                                                                <\/div>\n
                                                                                \u2014> Emphasize getting outside of the comfort zone, but defer to experts<\/div>\n
                                                                                <\/div>\n
                                                                                <\/div>\n
                                                                                  \n
                                                                                1. \n
                                                                                  Any advice for B2B companies that primarily have members of a specific profession as users? Working in healthcare, I have buy-in that diversity is key for patients. But there’s this (incorrect) idea that being a physician overrides every demographic category.?<\/div>\n<\/li>\n<\/ol>\n
                                                                                  <\/div>\n
                                                                                  A: \u00a0Depends on scope of study. It depends whether B2B use clients to do work beforehand to pull evidence showing experience along racial\/gender lines \u00a0might influence how users interact with product<\/div>\n
                                                                                  <\/div>\n
                                                                                  \u2014> Incorporate diversity study pool piece by piece and that will get you there<\/div>\n
                                                                                  <\/div>\n
                                                                                  <\/div>\n
                                                                                    \n
                                                                                  1. \n
                                                                                    I work for a company that makes apps for outdoor activities like hunting and offroading. Our customer base is mostly white men, and my research participants mostly match this. I know that adding diversity is the right thing. But how do you champion bringing in diversity if the diversity doesn\u2019t match your customer base?<\/div>\n<\/li>\n<\/ol>\n
                                                                                    <\/div>\n
                                                                                    A: The wider the audience, the more you can sell, and you can get a larger purchasing audience<\/div>\n
                                                                                      \n
                                                                                    • \n
                                                                                      You will have to make case to support types of marketing efforts for broader user base<\/div>\n<\/li>\n<\/ul>\n
                                                                                      <\/div>\n
                                                                                      \u2014> It\u2019s been found that when you have findings that suggest solutions of smaller population, you will serve larger population as well<\/div>\n
                                                                                      <\/div>\n
                                                                                      \u2014> Moreover, there are places are there for a diverse recruitment pool, but people assume they don\u2019t exist.<\/div>\n
                                                                                        \n
                                                                                      • \n
                                                                                        NPR podcast Code Switch helps tell story of these kinds of populations<\/div>\n<\/li>\n<\/ul>\n
                                                                                        <\/div>\n
                                                                                        <\/div>\n
                                                                                          \n
                                                                                        1. \n
                                                                                          What are the known barriers that you often hear about when recruiting these populations?<\/div>\n<\/li>\n<\/ol>\n
                                                                                          <\/div>\n
                                                                                          \u2014> Main barrier is recruiters with mostly white panels, as they struggle to find non-white people in New England for user research studies<\/div>\n
                                                                                          <\/div>\n
                                                                                          \u2014> Another barrier is strict adherence to getting representative sample<\/div>\n
                                                                                            \n
                                                                                          • \n
                                                                                            Putting screener in way that\u2019s very specific and asking questions in a sensitive way<\/div>\n<\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"

                                                                                            \u2014> Hi everyone, and I hope everyone is staying safe and healthy \u2014> \u00a0Today\u2019s talk will be about the importance of diversity in UX research   \u2014> This topic has its roots in my own experience in the nuances of identity, especially with her own background as being half-Puerto Rican. This has impacted how I […]<\/p>\n","protected":false},"author":334,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_seopress_robots_primary_cat":"","footnotes":""},"acf":[],"_links":{"self":[{"href":"https:\/\/rosenfeldmedia.com\/advancing-research-2021\/wp-json\/wp\/v2\/pages\/2789"}],"collection":[{"href":"https:\/\/rosenfeldmedia.com\/advancing-research-2021\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/rosenfeldmedia.com\/advancing-research-2021\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/rosenfeldmedia.com\/advancing-research-2021\/wp-json\/wp\/v2\/users\/334"}],"replies":[{"embeddable":true,"href":"https:\/\/rosenfeldmedia.com\/advancing-research-2021\/wp-json\/wp\/v2\/comments?post=2789"}],"version-history":[{"count":9,"href":"https:\/\/rosenfeldmedia.com\/advancing-research-2021\/wp-json\/wp\/v2\/pages\/2789\/revisions"}],"predecessor-version":[{"id":3121,"href":"https:\/\/rosenfeldmedia.com\/advancing-research-2021\/wp-json\/wp\/v2\/pages\/2789\/revisions\/3121"}],"wp:attachment":[{"href":"https:\/\/rosenfeldmedia.com\/advancing-research-2021\/wp-json\/wp\/v2\/media?parent=2789"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}