Day 2– Theme Panel

  1. We are being asked to do much more with research now. Given all that context seems more important to lean into mixed methods. Building practice of new methods of that work and time for when it happens?
    1. NSR: Goes back to idea that it comes from wanting to learn why it is happening and mixed methods are way to have intuitive way for survey, focus group and participatory design — doing what it takes to learn what is happening even when talking to students and asking why they did this, and what worked, and what didn’t
      1. After using each tool, provide critique of tool itself and other tools itself— knowing that everything is temporal as tool, and can be built upon, gives me open mind for research process
  2. AB: That resonates for me, and job involves saying yes, when people curious about running a new tool and do evaluation, and evolve research toolkit.
    1. Client requests are wild as well, and need to figure out new way to meet moment and be up to date with research methods and tools
  3. RM: For me moment when decide to move to a method is when research question I have can’t be answered using existing methods
    1. Alternatively when I have attitude or behavior and don’t know what you are searching for — using method that doesn’t have known outcomes
  4. KH: Methodological inspiration from conferences, and keep it in back of mind until time arrives
  5. Any new methods to try later on today?
    1. KH: Doing something participatory, but interesting
    2. AB: Formal literature review as some questions with aspects of product not answered through direct testing and taking inspiration from talk and playbook approach
    3. RM: Giving a second chance to focus groups, based on talks and discussion and planning to try again
    4. NSR: Looking at Katie’s deck for experimental techniques and applying to students as a method — and experimental
  6. Tons of methods talked about today. How about those using quant and secondary research keep lived experiences of those participants in your minds and that of stakeholders and colleagues?
    1. KH: With trust experiment worked with qual researcher at Thumbtack and combination in one unified deliverable of numbers with participants and rich quotes and having both on joint perspective made it hit home and brought user to life
    2. AB: Finding places to sprinkle primary research and secondary research — few people in client organizations and helping assess if relevant and finding balance for pressure testing for new environment and bringing to life for stakeholders
    3. NSR: Secondary research can be useful when you don’t want to get immersed in community content, and people feel far more vulnerable and for that matter is where you should rely on data of census and American surveys
      1. People spending so much time
    4. RM: Idea of lean research, right respect, relevance, and right size and how to approach communities in way community feels respected and interviewing others and frameworks to treat data with lots of respect and right-sizing research and thinking through most quick questions and right-sizing research for participants as we are asking so much of them
  7. Opportunity we have in new emerging world and influence strategy, how should we up game to meet the moment? Skills to brush up on?
    1. AB: Now in executive role and thinking through big picture and how to support goals as broadly as possible and best thing it can be and constantly reorienting and organizational objectives and linking back to those things
      1. Doing work in some cases and specific test and collateral and applying to other clients to and collateral for other clients
      2. Advocating for budget and timeline at executive level and showing value
    2. KH: Getting comfortable with company goals and objectives, and grasping goals of people at table and how research will get them to where they need to go
    3. NSR: In teaching class called “Design Leadership” and feeling confident where it will be design leaders and situational leadership model of being coach, director, supporter, or delegating tasks
      1. Situational leadership and who is above you and below you — exploring methods and making them think leadership and looking at challenges for situation and helps up think through that to address challenges and how we talk about it
    4. RM: For strategy, is that every org is focused on delivering value, an emergent property of a system, and no single thing in org that doesn’t effect strategy or is not affected by strategy
      1. You can connect dots with system thinking and seeing effects as part of systems that effects strategy
  8. How can you do research when you are faced with complex internal systems where users are frustrated and stakeholders have baked design decisions in, without consulting users?
    1. NSR: Summarizing Slack, it’s important to get to root cause as to why someone went ahead with solution and why problem is happening, and in relationship to each other and included in conversation and so traditional in how we address institutional problems
      1. Relying on external expertise versus internal expertise. Challenge of stakeholders and decisions of having the power and external expertise — internal system to address challenges
    2. RM: Stakeholders didn’t want us to tell others in org that product iteration should be killed, and that goals are aligned for company to be successful
    3. AB: Find they pick battles and do what client wants to do, if initiative won’t harm anyone
      1. If you don’t fight every single battle, you can win on the ones that matter the most to you
      2. Specific example and good of business and mutual goal here
    4. NSR: Read question from perspective of stakeholders making consulting decision without considering others lower in status and doing process improvement — but don’t go to people delivering process and policy and local-state-federal level
    5. KH: Find that secret-sauce is talking to frontline employees and how they do the work and get it done
  9. Commercial resistance to investing in long-term content, and participatory stuff. How have you handled it and value and compromises made and how you feel about this?
    1. RM: Did long-term study on ending racism in design for Meta, Amazon, Google. Found that biggest problems was time of research, and too short to invite communities impacted
      1. Include “equitable pauses” to slow down research
      2. Demands to speed up artificial intelligence and make it worst — no solution in problem in short-term
    2. AB: See in my research it takes month or years, along with UX research as well and having a mixed program and overall research approach and patience for longer-term things
      1. Company rooted in science and way to do the work with marketing and commercial environment — not a complaint but hard
    3. KH: Idea of balanced portfolio of short and big studies, and success of big study proves out value of further big studies
    4. NSR: Right-sizing problem given the time available, and if problems are beachballs there one person can hold, other problems are beach balls that need to be held by five people
      1. The size of the “racism in design” beachball is very big, but specific challenges allow right-size beachball given people to commit to research and weeks to commit to it
      2. Small interventions to get trust from partner and more space for participatory stuff
  10. Colleagues taking on research and customer exposure and methods available. How do you help colleagues grow their own skills?
    1. AB: Recommend events to those looking to grow skills and recommend engage with local communities like UXPA and Rosenfeld Media
      1. Training in psych and learned design skills on the job
      2. Volunteering to partner with others when they show an interest and finding right balance to help and advise you on this and up-skilling people with approach
    2. Mentioned briefly that shared tools developed with people and HMW questions from people and have so many of tools that if someone asked me if they want to explore something have conversation with them on what thinking about and how they are thinking about problem and plant idea in head they can do it
      1. Can be useful to give tool for hour long session to see if they want to do something or not
      2. Giving chance to try out a. tool and learn a particular skill they want to
    3. KH: Understand context and see what learning process works for the tool and kind of vendor we need and methods
      1. Knowing what’s safe for a novice to do, and clearly communicating it and asking questions if the tool made sense to use