AR2021-User Science: Product Analytics & User Research (Marieke McCloskey, Humu)

—> Thank you so much for having me
—> I’m excited to talk with you about user science

 

—> I work at Humu, a HR Tech start-up to make work better for everyone
—> I was assigned a customer with new kind of shift worker, and I needed to figure out how to adjust the product for these new types of workers
  • So I started with interviews at new customers as well as with people in similar roles

 

—> Meanwhile, a data scientist at the company conducted an analysis of the new shift worker and found that there were two kinds of employees
  • 1) This who choose a variety of work locations
  • 2) Others who choose only two locations
—> This lead to follow-up questions for me
  • However, it was easy for me as researcher  to get stuck in narrow focus of questions I  could answer, based off my domain knowledge

 

—> I want us to collaborate on research questions between data team and user research to share findings, and insights as well
—> Heard user science from Brent Tworetzky, who is trying to convince people to understand behavior and user needs
—> In my work at User Testing, I liked idea of combining data and user research
  • Data roles are just as messy for analytics as they are for us researchers
—> I chose product analytics, as it describes people who interact with product for opportunities for improvement

 

—> I was stunned to read that respondents feel they rarely work at doing proactive customer-driven decision making
  • The goal is to increase this

 

 

—> I believe you do have the data skills to do this type of analytical work
  • If you study people, you understand data
—> We are int this together, and data teams at company are focusing on making products to be successful and for the company to be successful
—> As UX researchers, we are uniquely positioned to form bonds, as product leadership is  often swamped.
—> We are good at cross-functional work, good at storytelling, and involved at development process at every stage
  • This gives us a strategic edge to build collaboration

 

 

—> Its important to find the right opportunity and right data partner
  • Opportunity: A chance for mutual benefit
  • Partners: Curious about each others role
—> I got lucky at User Testing through my friendship with a data scientist, leadership buy-in for idea
  • Humu didn’t initially have this, but I soon found a way

 

—> Before we go further though, what does Humu do?
  • It is is HR tech focused on what everyone uses at works, including all employees
—> It’s hard for employees to learn new habits and skills
  • So Humu leverages nudge theory to send nudges to employees, breaking complicated habits into actions that can be taken to teach a skill to the employees
—> People have told me they love the product
—> But my every attempt to identify the  “power users” of Humu had failed
  • This included a trip to a site where she thought she had found power users, but it turned out not to be the case

 

 

—> Last summer, an opportunity came up
—> I realized people were asking the same question and stuck with how people were engaging with Humu’s product
—> Leadership:
  • Getting alignment with company on what mattered
—> UXR
  • Figuring out what made Humu users, super-users? And who does Humu work for best?
—> Product
  • How to show engagement with Humu product with administrators?

 

—> I then needed a partner, and looked for these three things
  • Someone who cared about people behind data and made sure what was there for people needed to be accounted for
  • Someone who understood what happens to the product after people use it
  • Someone intrigued by idea of collaborating user research

 

—> I found a partner in a recent arrival to the data science department
—> Invited him to a user interview, and he was impressed by the work
—> Our first project was to dive in, and see how we track nudge engagement.
  • Many meetings/doc edits later, we presented the following document

 

—> The result of our collaboration is shown above
  • X is how long our users have used the product
  • Y is  the number of user utilizations of the product
  • Colors were bucketed to show engaged users from disengaged users
—> We used UXR to figure out what was a good signal of engagement, expertise of non-analytics people scientists, and spoke with customer success managers
  • It’s really good to help figure out what matters and what doesn’t

 

 

—> It turns out our most engaged users were “activators” who want to move Humu’s product throughout the company
—> This helps narrow down who to talk to, and find people with the highest score
  • The study established a shared language in the company as to who a “power user” was

 

 

—> There are many more ways to build solution and be aligned with leadership
—> Using an opportunity/solution tree is an example:
  • Green circles are user problems and what we can build solutions for
  • Yellow circles are different solutions
  • Blue circle represents desired outcome, based on company strategy  or vision.
  • Orange circle represents experiments
—> Data science can figure out what drives that desired outcome, figure out what to see in behavior to be a renewing customer, and adjust actions accordingly.
—> Different people own different parts of the solution tree and are not shy on what to pursue

 

—> In my case, my work was presented to the board, and the company CEO said finding out more about the most active users was a key priority for the year ahead
  • This shows how user science can work to drive serious company impact

 

 

—> So use your time to invite data partners and see what they bring up and ideas that they have. It will be a very effective partnership
—> Thank you!

 

Q&A
  1. What are obstacles to preventing data science and user research from collaborating? What do you think are the obstacles to this – why isn’t everyone doing it?
A: There are different kinds of insights, and potential distance within the org.
  1. Using these measures, do you worry about striving for more engagement in light of the “habitual use” ethical concerns? Is more use always better?

A:  It depends on what is useful for the business, but we try not measure everything

 

3. So much of my work has involved employee experience research. so glad to hear this perspective. How often do you think employee experience and customer experience are integral to one another? And what would you say to product teams that they need to think beyond the external “customer?”

 

A:  Thinking so much about what can be learned from employee experience and what we can share
—> There is a lot to borrow from employee experience world

 

4. Analytics data are convincing to stakeholders because they are “real”. Whenever analytics data and qualitative user research conflict each other, analytics data are more likely to be the “winner”. How do you address the issue?

A:  Recently have not been in place where the two have been in conflict.
—> But, collaborate with data team as early as possible, and present info as soon as possible

 

5. As primarily qualitative user researchers, how much knowledge do you believe is needed for us to have re: quant and analytics as compared to partnering with an expert in those domains?

A:  You can have no knowledge, but you need to be honest about lack of knowledge you have.

 

6. Do you have any tips on how to show our strengths and contributions to data teams who may not naturally be as collaborative?
A: Don’t assume data scientists know everything.
  • Recognize the power you have in your insights with primary stakeholders