Day 2- Building a Product Insights Team
— Thanks everyone for introduction.
— As Bria mentioned, I was head of business intelligence at HotJar, where I appreciated combining qual + quant insights to drive business growth
— I’m going to talk through the process of building our product and stats team
— I have a toddler, and here he is skateboarding
-
But why is he in the picture?
— We are familiar with Spotify, which provides custom playlists to its listeners
-
After I had my kid, the playlist changed to from what’s above
— To the playlist here
— My original playlist was replaced with songs by a smiley faced watermelon, which helped soothe my son, but this wasn’t what I wanted from my experience from Spotify
-
What I liked from Spotify was that it was my go-to for music.
-
If you looked at my data alone, it’s easy to think my listening increases, but the types of music I listened have suddenly changed, hence the new playlist
— A qualitative approach can capture these changes in human behavior to produce better solutions
-
People have experienced services like this, and its up to companies to understand these things much better
— Even a message acknowledging from Spotify my life change would have been huge
-
Now Spotify is one of the best businesses today, but there is opportunity for room for growth and improvement
— The combination of what and why is where true insights live, and where we will get maximum impact for the company
— Why this happens today, is that on one-end you have data analytics that live on one side , with lots of resources
— And on the other end, UX research is smaller, and more neglected with less budget than desired
-
Rarely do these teams work together in organizations
— This is a big missed opportunity
— Both teams have the same goals though, to enable insights with maximum impact
-
We build these teams and give them same goal, but don’t have them work together
-
We know as professionals we will see the most meaningful results
— What do both of these disciplines do in practice?
-
Both start with user surveys and collecting data
-
Then they cleanse the data
-
Then they analyze the data
-
They they ultimately try to deliver insights to a business
— The analogy is that of sifting through sand for nuggets of gold
-
Goal is to create refinery to create nuggets and gold bars of insight for an organization
— So how did we get started?
-
The change wouldn’t happen overnight
— It was a huge transition at HotJar, as the company relied on qualitative research, with relatively little rigor in the analytics stack
— But as the BI team was built out, amount of requests and demand rapidly increased
-
We moved to solely quant, before moving to happy medium
— In our case, we developed a shared hypothesis of the ICP or ideal customer profile, which was a great place to get started
-
It’s a good use case where research of qual and quant could be combined
— We got stated by bringing in an analyst, manager, and user researcher
-
We did the natural thing of looking at the data, taking the longest paying customers and reviewing demographic profile data alone
-
But there was great insight within qualitative areas like marketing and broader analytics data
— As we grew the ideal customer profile, looking at the data alone was like only looking at product and marketing done today
-
But this is not where biggest opportunity lies long-term
—We combined research we had done to inform our decisions
— We began with data collection to understand what data we had about customers, and associated market trends
— Then while doing data analysis, we bounced our ideas off each other, which we saw during the analysis
-
Analysts and UXRs interacted
— When insights were delivered, we had shared understanding, alignment in data collection, and cross-pollination of info during analysis
-
This led to more buy-in from the company, as the insights were integrated into company decision making
-
We set foundations and best practices to figure out how to roll-out the information to the org
— So how to roll out the information across the org?
-
Even small size and scale for HotJar, challenges are similar with minimal buy-in
— We found effectiveness by showing decreases in redundant research efforts, and increasing insight actions based on combined research done
-
Even a few cases, where we showed time-saved in one instance, or applied actions in another, had a big impact
— We then thought of the model that will best for work for business.
— No right or wrong model that can be chosen over another
-
Models include an embedded model
-
Self-sustaining team or squad in each, broken down by area of expertise
-
Pro: Can build domain expertise
-
Con: No built in redundancy and gains in knowledge. Poor knowledge sharing
-
-
— Another model is a Center of Excellence, and centralized requests coming into the team
-
This model began at HotJar, as we could have broad overview of business drivers, and making sure our work was aligned with company OKRs,
-
Made it easy for us to say no to requests
-
Good for setting best practices and sharing knowledge in org, but not good in building specialized knowledge
-
— We slowly moved to another hybrid model, with COE and embedded teams, focused on individual teams within the organization
-
If one goes on holiday, we will be able to deploy
-
Analysts and researchers could build domain expertise, while experiencing general knowledge sharing
— Don’t think of right model to choose, but look at your org, and see what works best
-
In any case, take control of the backlog and position yourself as delivering value for the company, as opposed to being just a service center for the business
-
Take time to communicate effectively in the org, and help understand where you are driving impact for the company
-
If you are clear in your communications you will win at the end of the day
-
— Many organizations like Spotify, and Shopify have leveraged these analytics practices
— We want to break down data silos to broader process, where we work together and grasp what data is collected, cleansing, and building out a refinery for golden insights for our organization
— This will let us create more experiences that inspire, and help us build products that we love
-
Always more room for improvement, and I’d love to hear from you
-
Reach out to me via Twitter, LinkedIn
— Thank you!
Q&A
-
I love the idea of a shared hypothesis and shared insights concept? Have you found it useful to bring it to the marketing team?
—> Absolutely. Marketing will be biggest advocates for company and are the best way to communicate value effectively to rest of company
-
For a hybrid model, how is workload distributed across squads?
—> We have an analyst with domain expertise like acquisitions be deployed to projects dealing with specific OKRs
—> Business structured was reflected in analysis structure for projects, which flowed down to the rest of the team
-
How to manage resistance to collaboration between UX and marketing?
—> Little resistance, due to scale and size of company. More resistance within a big organization
-
Since we were deliberate, we were able to streamline effectively
-
How does content and UX writing fit into the insights team?
—> Something I have not thought about before, but something that needs to be included
-
Doing a lot of research for writing, and it can be good collaboration to consider
-
How to keep track of all data sources coming in and synthesize them?
—> Not easy, and I started from scratch at one point
-
I had segment analytics with same data-set across different tools, and have consistency across the board
—> Originally just moving information into Confluence, but Confluence is not ideal for this service.
-
I left HotJar to build out a more ideal type of repository tool at Avrio