Day 2- Building a Product Insights Team

—  Thanks everyone for introduction.

 

— As Bria mentioned, I was head of business intelligence at HotJar, where I appreciated combining qual + quant insights to drive business growth

 

— I’m going to talk through the process of building our product and stats team

 

— I have a toddler, and here he is skateboarding
  • But why is he in the picture?

 

— We are familiar with Spotify, which provides custom playlists to its listeners
  • After I had my kid, the playlist changed to from what’s above

 

— To the playlist here

 

—  My original playlist was replaced with songs by a smiley faced watermelon, which helped soothe my son, but this wasn’t what I wanted from my experience from Spotify
  • What I liked from Spotify was that it was my go-to for music.
  • If you looked at my data alone, it’s easy to think my listening increases, but the types of music I listened have suddenly changed, hence the new playlist
— A qualitative approach can capture these changes in human behavior to produce better solutions
  • People have experienced services like this, and its up to companies to understand these things much better
— Even a message acknowledging from Spotify my life change would have been huge
  • Now Spotify is one of the best businesses today, but there is opportunity for room for growth and improvement

 

— The combination of what and why is where true insights live, and where we will get maximum impact for the company

 

—  Why this happens today, is that on one-end you have data analytics that live on one side , with lots of resources

 

— And on the other end, UX research is smaller, and more neglected with less budget than desired
  • Rarely do these teams work together in organizations
— This is a big missed opportunity

 

— Both teams have the same goals though, to enable insights with maximum impact
  • We build these teams and give them same goal, but don’t have them work together
  • We know as professionals we will see the most meaningful results

 

— What do both of these disciplines do in practice?
  • Both start with user surveys and collecting data
  • Then they cleanse the data
  • Then they analyze the data
  • They they ultimately try to deliver insights to a business
— The analogy is that of sifting through sand for nuggets of gold
  • Goal is to create refinery to create nuggets and gold bars of insight for an organization

 

— So how did we get started?
  • The change wouldn’t happen overnight
— It was a huge transition at HotJar, as the company relied on qualitative research, with relatively little rigor in the analytics stack

 

— But as the BI team was built out, amount of requests and demand rapidly increased
  • We moved to solely quant, before moving to happy medium
— In our case, we developed a shared hypothesis of the ICP or ideal customer profile, which was a great place to get started
  • It’s a good use case where research of qual and quant could be combined

 

 

— We got stated by bringing in an analyst, manager, and user researcher
  • We did the natural thing of looking at the data, taking the longest paying customers and reviewing demographic profile data alone
  • But there was great insight within qualitative areas like marketing and broader analytics data
— As we grew the ideal customer profile, looking at the data alone was like only looking at product and marketing done today
  • But this is not where biggest opportunity lies long-term
—We combined research we had done to inform our decisions

 

— We began with data collection to understand what data we had about customers, and associated market trends

 

— Then while doing data analysis, we bounced our ideas off each other, which we saw during the analysis
  • Analysts and UXRs interacted
— When insights were delivered, we had shared understanding, alignment in data collection, and cross-pollination of info during analysis
  • This led to more buy-in from the company, as the insights were integrated into company decision making
  • We set foundations and best practices to figure out how to roll-out the information to the org

 

— So how to roll out the information across the org?
  • Even small size and scale for HotJar, challenges are similar with minimal buy-in
— We found effectiveness by showing decreases in redundant research efforts, and increasing insight actions based on combined research done
  • Even a few cases, where we showed time-saved in one instance, or applied actions in another, had a big impact

 

— We then thought of the model that will best for work for business.

 

— No right or wrong model that can be chosen over another
  • Models include an embedded model
    • Self-sustaining team or squad in each, broken down by area of expertise
      • Pro: Can build domain expertise
      • Con: No built in redundancy and gains in knowledge. Poor knowledge sharing

 

— Another model is a Center of Excellence, and centralized requests coming into the team
  • This model began at HotJar, as we could have broad overview of business drivers, and making sure our work was aligned with company OKRs,
    • Made it easy for us to say no to requests
    • Good for setting best practices and sharing knowledge in org, but not good in building specialized knowledge

 

—  We slowly moved to another hybrid model, with COE and embedded teams, focused on individual teams within the organization
  • If one goes on holiday, we will be able to deploy
  • Analysts and researchers could build domain expertise, while experiencing general knowledge sharing
— Don’t think of right model to choose, but look at your org, and see what works best
  • In any case, take control of the backlog and position yourself as delivering value for the company, as opposed to being just a service center for the business
  • Take time to communicate effectively in the org, and help understand where you are driving impact for the company
    • If you are clear in your communications you will win at the end of the day

 

— Many organizations like Spotify, and Shopify have leveraged these analytics practices

 

— We want to break down data silos to broader process, where we work together and grasp what data is collected, cleansing, and building out a refinery for golden insights for our organization

 

— This will let us create more experiences that inspire, and help us build products that we love
  • Always more room for improvement, and I’d love to hear from you
  • Reach out to me via Twitter, LinkedIn
— Thank you!

 

Q&A
  1. I love the idea of a shared hypothesis and shared insights concept? Have you found it useful to bring it to the marketing team?
—> Absolutely. Marketing will be biggest advocates for company and are the best way to communicate value effectively to rest of company
  1. For a hybrid model, how is workload distributed across squads?
—> We have an analyst with domain expertise like acquisitions be deployed to projects dealing with specific  OKRs

 

—> Business structured was reflected in analysis structure for projects, which flowed down to the rest of the team
  1. How to manage resistance to collaboration between UX and marketing?
—> Little resistance, due to scale and size of company. More resistance within a big organization
  • Since we were deliberate, we were able to streamline effectively
  1. How does content and UX writing fit into the insights team?
—> Something I have not thought about before, but something that needs to be included
  • Doing a lot of research for writing, and it can be good collaboration to consider
  1. How to keep track of all data sources coming in and synthesize them?
—> Not easy, and I started from scratch at one point
  • I had segment analytics with same data-set across different tools, and have consistency across the board
—> Originally just moving information into Confluence, but Confluence is not ideal for this service.
  •  I left HotJar to build out a more ideal type of repository tool at Avrio