Day 2–Redefining the Research Toolkit: Expanding methodologies for a changing world

Panel #1:

Jenna Ahmed: Lead Curator for Rosenfeld Media

Eduardo Ortiz: Founding Owner, Conforma

Megan Blocker: Justworks

Discussion

Jemma

1. Gonna get spicy today and field undergoing significant dynamics as relationships shift to be from linear and hierarchical to plural and relational. Required to expand and adapt methods we use— lot of excellent methodologies which reflect times built
   1. Times have changed and value we need to demonstrate as changed
2. Need to acknowledge knowledge built pluralistically in a network— and changes how we demonstrate credibility, and what we are being asked to explore by orgs in which we sit
   1. We need integrative techniques and be fluent with other insight teams are doing, to act as connectors and curators
3. Accept building knowledge relationally and with participants and committees
   1. Need to embrace participatory methods, and non-extractive lens to work, and current frameworks that don’t address needs and integrate into practice quickly
      1. No need for purpose and values
   2. Prime tool using and foster shared knowledge at scale.
4. Believe outdated methodologies must be shaped by collective wisdom of many, and doesn’t exist in single person’s head
   1. And share how we can build methodology by community and for community and times we are in
5. Spend time with favorite people and talk through necessity of change, and not a zero-sum game when it comes to demonstrating value to our org
   1. When we get this right, we demonstrate solid business value, and serve time and people to hold much better
6. I am researcher’s researcher, and will nerd out

Meg

1. Role of research is changing and we need to change too, and not being asked to figure out what users want, and more strategic conversations like ‘what to build next’, ‘driving systemic change in marketplace’
   1. You can’t answer strategic questions with tactical methods—you will hit walls
2. Not just about methods, and organizational nerd with team structure, and thinking how research fits into organization
   1. At height of design thinking had comfortable home, and need to shed attachment to design-land and spend out into broader org
      1. Moat that protected us as source of insight, has disappeared, and everyone in insight game
   2. Moat is now enabling org to make use of insights we find, and building insight fluency and how to integrate across teams
      1. Partnering with business strategy and operations, and expanding methods to handle dimensionality of problems we tackle
      2. Do this well, more decisions driven by high-quality data and every team if we do this well
   3. Insight will become life-blood of org, and we will influence functions with critical decisions
3. How might we actively shape role in org, as opposed to having someone define it us
   1. Evolve and lead or founder and be left to behind

Jemma

1. To build on this, we need to have educational practices to execute on this
2. Speaking on educational experience in ongoing learning and early career seems to me that training practices have shifted
   1. More observational on current training modality and available to us with people under resourced
      1. Even with external barriers there are things within our control
   2. Believer in education to manage dynamics
3. Thinking of training in practice
   1. Considered trained as junior-to-low weight researcher for first five years, and not as senior and weekly observation session of moderation
   2. Grew up in public service and leveraged formal courses like AQR, meet-ups, conferences, journals, and time to observe culture and life to get broader info to triangulate within study learning
      1. Cost time, but not necessarily money
   3. Why say this?
      1. As mid-weight researcher, year 5-7, had rigorous and expanding toolkit, and knew how to dial rigor up and down through my in-depth knowledge and pick and mix methods— based on context in front of me
      2. Don’t see this continuous learning throughout the industry and say this as challenge to rise up and tackle to help raise the tide for us all — as needs have evolved and role in org has shifted
         1. Sometimes traditional methods are enough, but sometimes are not to build credibility for stakeholders
         2. As stakeholders become more fluent in how research and methodologies, will lose credibility with stakeholders — in-depth continuous and MBA and research and keep demonstrating and credibility
      3. Touched on growing feedback on researchers on how work can feel extractive at time, and align with that shift
         1. Applying non-extractive ways of working as it’s good research and less so with broad toolkit to meet person in context where they are
   4. Prioritizing continuing ed helps us adapt to shift, and proving out our value and all want credibility and respect for work
      1. Aside ethical and adaptive aspects, increasingly to get arms around this
         1. As engagement and participant with people not taking part in poor research experience
      2. Genuine crisis with CX and market research taking it seriously, and risk losing data quality and participant access— which is foundational work

Eduardo 

1. Can relate to this, as someone on both sides of organization while also having unique opportunity of doing something about
   1. Company has professional dev stipend similar to what larger org has, and helping people improve knowledge base
2. Not exciting, but necessary

Megan

1. Agree with you never stop needing professional development, and level of expertise for knowing when things will break and reminds me of cooking and baking experience
   1. Baking you need to deeply understand principles to understand how the discipline works— otherwise you get inedible cookies
   2. Be ready to admit where you are beginner and need to seek recipe or advice
2. When it comes to adding to toolkits need to seek out advice and get good, before playing fast and loose with it

Jenna

1. In era of overwhelming data, with AI, data-science, and democratization. As researchers we don’t win by adding more data
   1. Change of building knowledge pluralistically and getting data cheaper and faster
2. Analogy to Amazon coming along, and analogy of us as Etsy versus Amazon
   1. Define where we add value and our role in relation to data— as we can’t win on data any more
3. When you think of pure data, that type of insight is about introducing inductive reasoning into an org,
   1. Other sources can do deductive reasoning
   2. By accepting add more value when we think our role as introducing critical thought and reflection, and understanding
      1. What we do, that other sources can’t
4. Using wide variety of artifacts like play and WhatsApp conversations, and less onerous on participants
5. Rethinking our value, and methodology applications, might unlock sources we weren’t thinking about

Megan

1. Spent lot of time talking about crisis and crucible, but also exciting
   1. Proactively pulled into strategic conversation as collaborator and exciting moment as opposed to scary one
   2. Applying it to our work as we previously working in rigid siloes
2. Much more fluidity and systems thinking with disciplines talking to each other and informing product work
   1. Upshot is that. people can now challenge and get up in your face about the insights they get
      1. Need to be more aware of insights, and data people are using
   2. Keeping tight-eye on duplicative work, as so many teams getting in insights game, and risk of doing duplicative work, being less acceptable and reduce duplicative work
3. Asking how much something is landing and expected impact of customer experience and success of something created, and broader skillset of launches and landings and leaning into special skillset and challenges

Eduardo

1. In doctorate program to level up and study up to follow conversation and will do best to keep conversation at level they have already set
2. Era where data is ever available and present, and value from insight — yet participant always seems to be absent or missing with same data funnel
   1. Our funnel view and lens ia an interpretive one and way to understand info not part of same funnel and needs of different stakeholders and insights
   2. Need new frameworks, tools, and change how we think about ourselves as researchers
3. Too often we are looking for relationship between X and Y, and observing things in sterile setting, and as researchers have ton of power to center participants for stories and experiences for way that’s effective and useful
   1. On us to articulate importance of this, versus seat at table and budget
      1. Fighting to compensate people, real representation in our research, empowering toolkits to show people as they are
         1. Real data over synthetic data.
            1. Why should we care? As data has no ethics or moral code— with direct observation as having big impact to heal or harm
      2. Taking extractive lens versus relational ones and answer is classic ones of it depends
   2. Shifting from owners to being coaches and guides, and creating organizational knowledge is creating path
      1. Accounting for well-being and rep and let people to provide unique insights
4. Advent of founder-mode and tons of risk in this
   1. None of us have all insights or information, and risk causing harm, intended or not, to ourselves and others
5. Imagine cost of “I know it all” attitude, with where airplane got impacts
   1. Initial solutions of reinforcing areas where plane was being hit, but solution lay in what was not seen or omitted
      1. Planes that were returning had bullet holes, but what about the ones that didn’t come
      2. Selection bias is ever present, and getting things right is crucial in preventing this

Jenna

1. Final why is culturally where participants and researchers are now pushing back to paradigm
   1. We don’t have choice as this is direction where people insight field will take us in
   2. Embrace this by non-extractive technique and about instituting communicating learning and value learns — often about increasing this
2. Culturally living with trauma epidemic and lower connection and participants and researchers are holding more psychologically than ever
   1. Often giving only space to be heard and provide that
   2. Potential to heal and harm contexts, and coupled with ohter things going on, means we must upskill
3. Consequence of meeting people where at, and onus to protect us and them, and consequence in this
4. Being mindful of time will share how to get involved

Q&A

1. When you see adjacent fields which ones do you have in mind?
   1. Varying on organizational context as people have adjacent fields in them and managing the data and customer support and those signals
   2. Professional researchers and those with research signals and CX and multilple market researcher teams, and people involved with gathering data and insight
   3. Should be able to what org system looks like
2. In Monday evening talked about loss of appetite for learning, and making learning cool and accessible for peers and future practicioners?
   1. Shouldn’t tell people how to be cool and methods to try things out and working with case study and doing service storming and practical method for service blue-print and activity to engage with them and be silly and learn thing in way for it to apply thing quickly — and designing programs by adults by learning by doing
      1. Learning by doing would be very fun
   2. Part of it to be one each of us, and some want nothing to be done, and keep up with conversations — so it comes down what drives you
      1. Your agency is something no one can take from you