Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

UX Lessons from running more than 1,200 A/B Tests
Wednesday, July 10, 2024 • Rosenfeld Community
Share the love for this talk
UX Lessons from running more than 1,200 A/B Tests
Speakers: Erin Weigel

Summary

Knowing how to solve the right problem is only one small part of the product-success equation. And nailing the execution is harder than most people think. As Erin says in this talk, “There are far more ways to fail than there are to succeed” when it comes to experimentation. To help you learn, Erin shares some of the silly mistakes she made while A/B testing her designs. That way you can avoid those pitfalls as you work to make your digital product better—not just different.

Key Insights

  • Over 90% of A/B tests at Booking.com fail, emphasizing the need for rigorous testing.

  • The execution often matters more than the idea itself in determining success.

  • A culture of experimentation leads to better user experiences and business growth.

  • Understanding user behavior is crucial for designing effective tests and features.

  • Small changes in design can significantly impact user experience and conversion rates.

  • Continuous re-evaluation of experiments is essential as technology and consumer behavior evolve.

  • Qualitative feedback should complement quantitative metrics to understand user interactions better.

  • Design choices, such as font selection, can have unexpected effects on clarity and conversion.

  • Edge cases in user experience can lead to significant issues in usability and conversion.

  • Investing in tracking and analytics tools enhances the ability to gather useful data for informed decision-making.

Notable Quotes

"I'm actually a really big failure."

"About 90% of all the work that I did for booking.com ended up in the garbage can."

"If a problem presents itself over and over again, it's not necessarily the concept that fails."

"The invisible decisions we make can impact a customer's experience as much as what they see on the screen."

"Typography matters. It matters a lot."

"Good design often lies between what's already there and what we think is best."

"We democratized user research. The researchers were there to enable us to do our own research."

"You can't fail 90 times out of 10 without finding humor in it."

"If you don’t need to, just don’t use fancy typefaces."

"A little line of CSS that may have made a browser crash last year might not make a browser crash anymore."

More Videos

Alnie Figueroa

"Communicate your value continuously; it’s how you’ll break down perceptions and build support."

Alnie Figueroa

Teamwork: Strategies for Effective Collaboration with Other Program Management Teams

September 8, 2022

Melissa Eggleston

"Building trust happens by laying down pieces brick by brick."

Melissa Eggleston Maya Israni Florence Kasule Owen Seely Andrea Schneider

Practical People Skills for Building Trust on Teams and with Partners

December 9, 2021

Jilanna Wilson

"Support software used to be really shitty and expensive. We sought out to change this experience."

Jilanna Wilson

Distributed Design Operations Management

October 23, 2019

Alan Williams

"The providers app is the most popular tool for low-income Americans to manage their finances."

Alan Williams Rose Deeb

Designing essential financial services for those in need

February 10, 2022

Edgar Anzaldua Moreno

"We wanted to share immediately because that would become a hub of shared understanding."

Edgar Anzaldua Moreno

Using Research to Determine Unique Value Proposition

March 11, 2021

Prayag Narula

"Cold calling became essential in our engagement strategy due to the nature of our users."

Prayag Narula Abhinav Krishna

Dialing for Research: How to Reach the Unreachable

March 10, 2022

Ovetta Sampson

"One way or another, it doesn't let up."

Ovetta Sampson

Turning UX Passion into Real Product Influence

June 7, 2023

Catherine Blizzard

"The proof of concept was in the delivery, proving our model works through tangible outcomes."

Catherine Blizzard

Using Integrated Insight to Drive Growth

March 10, 2022

Sarah Coyle

"You can create an information experience; it’s not just about the data."

Sarah Coyle

Design and Analytics with Sarah Coyle

July 30, 2020