Day 1- Curating Insight: Strategies for integrating knowledge across research functions

Panelists

— Shadi Janansefat (SJ): Senior Data Scientist—Meta

— Alan Wright (AW): UX Leader—Google

— Jen Cardello (JC): Head of Research and Design Operations— Capitol One

Moderator

— Jemma Ahmed (JA):

Moderator Panel

JA: One of consequence of shifting from researcher-org-participant schema, is that we are building knowledge with adjacent insight teams and collaborating plurally and relationally, and what this means for UXR in future, and research teams, and our research as a craft

1. In world for where knowledge-building teams are converging with each other, what do you think implications of it are for how we see ourselves in our role, in mindset and practicalities?

— JC: Worked on teams that have been integrated insights that combined market, advertising, user research and CX. Helped us figure out where we play and responsible for

• Our overall goal is to accelerate organizational learning velocity to provide value for customers
  • Specific altitudes of research questions, to answer on behalf of company of larger group
    • Less about lines of methodology, but research question and what level it sits at, and how to answer it with primary/secondary research

— SJ: Also bringing out collaboration, and looking at how to answer questions holistically, regardless of tool and collaboration between different functions and helps the org make better and more informed decisions

— AW: Comes at this from perspective as recovering researcher, and more consumer of insights and interesting to make transition

• Seeing interesting shift going on with boundraries getting blurred and tasked with understanding things
• Less about structure of org, and division of labor for team, but more about how to enable knowledge transfer to happen so that teams in org can work from shared set of facts and proceed form there— it’s a question of internalization of insights
  • Skills are more critical, and participatory exercises to help people pressure test insights, and make it their own
  • That is the need and not sitting off to the side

—JC: Doesn’t matter how much data you have, unless you act on the right question

• Whether org design, and partnering with researchers across org, figure out what questions are being asked, and what should be asked
  • Don’t need to know answer to 91 questions to deliver value, and seeding that in an agreed approach and not competing against each other
• In order to deliver value through digital experiences, we need to answer certain questions throughout the product dev lifecycle
  • Focus on aligning if these are the key research questions to answer

SJ: Guiding team to come to you to ask the right questions, and product needs, and see value in being missed as not embedded function,

• Hard to ask right questions without embedded context of product, and important to figure out how to bridge that gap

AW: Reasonably well to have it work, with understanding roadmaps and strategic planning roadmaps, and align self at beginning of strategic planning lifecycle and triage them together

• Best methods to answer questions and roadmap for research
  • Effort to pool questions at beginning of who would tackle what

JC: Helped well with new initiatives and created roadmap to visualize what questions to answer first, and what would come later in process

• Then visual artifact works to simplify what process is and answer questions in a specific way and no arguing through out the initiative

SJ: Creating roadmapping and what to build for users and questions to answer, and what to understand [both day-to-day and three-month view] and things to map out and next three months

• Grasping how to get ahead of execution for making decisions and having insights in place

JA: To recap, consequence of moving to more plural way of knowledge for other teams, and mindful of questions asking and not getting flooded, and shared learning agendas

1. Know you worked in centralized insight function, what are the benefits of it?

JC: Centralized means different things for people, and can include researchers dedicated to product areas, and report up all to central research and insight

• Context in domain in particular, and research professionals

— Courses on behavioral science and persuasive design, and like working with people who have different experimentation methods

• Embedding with market research and their practices and techniques and partnering on projects and building job map and different techniques had benefits

— People being in domains and in with product teams to surface questions through a universal intake form, and putting research questions at right altitude and answer questions so that more people benefitted from research together

1. What does working with other insights mean for skills to understand each other? Does it require people to learn more about analytics?

— JC: Yes, I’m a lifelong learner, and grasping other domains and what the job that others are trying to do, and awareness of what they are trying to accomplish

• Have had people move teams and lovely to have cross-pollination and mobility for researchers

— SJ: Helping them to ask the right questions, and working effectively and need to speak to each other in collaborative way

• Minimum bar to have cross-disciplines communication and knowing about others domains

— AW: When you have insights function more together, raises bar for the rigor of the work, and improving precision of reliability of insights and tradeoff

• Rigor versus impact and seen as overly centralized function and day-to-day and ebb and flow of team
• Embedded model makes sense— researchers can be blended into design and reporting to UX lead,
  • In the mix, and can response in agile way, but risk of losing their own identity as neutral actor

— Ideal to have semi-embedded option of being in a team, and also in a broader research community of practice

• Always a balancing act

— JC: Experienced this several places, where researchers supported individual product team

• They could assemble concept of portfolio of insights, but wasn’t cumulative and additive, but instead had lots of overlapping insights, and redundancy
  • Our progress involved bringing insights up to better-level, and not learning same things over and over again
• Risk of federated teams serving same information repeatedly and then digging into assessment and feature functionality

— Now track insights, and actionability of insights, and timeline something will be done

• Having insights to action ratio, and catalog to show what was learned and how we responded to it, to tune the research approach

— AW: Hard to track impact, and research can be too squishy and very literal

— JC: This was asked and what was learned and what was actioned on

• Delivering info to non-research leaders lets them value the practice
• Information we encounter reveals insights about org design, as well as product insights
  • i.e. Are dependencies and tech backlog too much
  • Incredibly valuable insight for organizational health

1. JA: What skills should UXRs getting arms around to adapt to building knoweedge with others?

— SJ: Extent to which we enable people to consume insights and data, and creating dashboards for PMs and engineers— experimentation and initial analysis already somewhat automated

• Additional ability for researchers to do more and take on complex projects and holistic question, and product areas
  • Being able to embrace more complex problem space
• Simpler tasks may be automated

— JC: Root-cause analysis as skillset tapping worth tapping into, and understanding why phenomenon is happening. Doing assessments i.e. no one is using feature created, and seeing where completion rates are low or task duration is too high when it shouldn’t be

• Skillset and triangulation

— Harvesting insights also big too, like secondary research and looking at data and existing insights and findings for good decisions

— AW: If you getting good use of AI tools to synthesize big chunks of documents and distilling common threads between them

• Not difficult, but capturing in practice

— For secondary research is leaning more into trends analysis and forecasting and strategic foresight, which are typically not part of toolkit

• Lot of cases especially for UXR, tendency to anchor findings and outputs delivered of identifying problems of what’s broken
  • Researchers embracing story-telling and narrative opportunities to lead with opportunity rather than problem, as the key for product strategy

— Bigger opportunities are more aspirational

1. JA: Playing threads we talked about and more strategic work, and adjacent insights. Do you think it might mean over coming time, might start seeing rise of more generalist insights roles, and less emphasis on evaluative work?

— AW: Interesting question. To extent, certain insights might be more easier to automate, and possible to scale the work being done, ability to tell story and frame a narrative becomes the real important skill — frame presentation to bring people along the ride

• Skillset for facilitating discussion will become even more important

1. Audience Q&A: Do insights and recommendations need to be structured in specific format to track insight over time— so how do you go about doing that? How do you start the knowledge flow across organizational silos?

— JC: Have specific insight syntax of what-so-what-now what

• Observation, why it matters, what to do about it
• Findings live in study debrief and tracker
  • Entered into database and Tableau, and tracking insight-to-action
• Two research repositories exist
  • Research study repo, and company wide and insights tracking that live in reports and can query that environment
    • Just looking at insights here is one specific repository
• In each roadmap cycle, look at roadmap and see what aspect of roadmap influenced by insights delivered

— AW: Seconding what-so-what-now-what framework as great mental model