Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Measuring What Matters
Gold
Wednesday, October 23, 2019 • DesignOps Summit 2019
Share the love for this talk
Measuring What Matters
Speakers: Kristin Wisnewski

Summary

Research and analytics are the eyes and ears of DesignOps. Validating whether strategies are effective or ineffective allows course correction and adaptation. Identifying problems that require action is essential for facilitating collaboration and ensuring that team members are engaged. Prioritizing user needs is the heart of user experience and Agile development. None of this can be done well without an effective research program. We’ll share how we scale research efforts to collect feedback from thousands of users per month and track the impact of dozens of projects involving 120+ designers.

Key Insights

  • IBM's CIO design team represents only 1% of the IT department but aims to support 385,000 IBMers.

  • The team emphasizes empathy in creating effective user experiences to meet the challenges posed by a rapidly changing workforce.

  • Over half of IBM's employees have been with the company for less than five years, making onboarding and user experience critical.

  • The design team's responsibilities extend beyond tools to influence culture and the physical work environment.

  • Continuous improvement and data-driven decision making are essential to enhance user experience at IBM.

  • Regular feedback from employees through systems like the Voice of Employee program helps in real-time adjustments to services.

  • Metrics such as Net Promoter Score (NPS) and goal completion rates guide the team's strategy and accountability.

  • A strong collaborative relationship among design, research, and operations teams ensures cohesive user experience improvements.

  • Keeping the user at the center is fundamental; measurement informs better service rather than just retroactively validating decisions.

  • Flexibility in processes and embracing chaos is necessary for growth in such a large organization.

Notable Quotes

"Our mission is to create a productive environment for IBMers, continuously improving the work experience."

"The state of IT is a daily reflection of what the company thinks and feels about its employees."

"We believe in measurement: what you measure is what improves."

"Every minute spent struggling with an IT system is a minute of time not spent delivering value for IBM and our clients."

"You cannot just focus on a single task; you must consider the entire ecosystem."

"We’ve learned to always be testing versus always be guessing."

"We're trying to influence the tone and culture of the company through design."

"What you measure is what improves, and we are ready to put those metrics out there for everyone to see."

"Data helps you make the right decision, not just support the decision that you've already made."

"As imperfect as NPS is, it's uniformly imperfect, and if you manipulate results, you're harming the entire system."

More Videos

Theresa Neil

"Specialization could be an opportunity for a lot of people."

Theresa Neil

Designing for Wellness: Specializing in Healthcare

May 22, 2024

Ricardo Martins

"Quantitative methods provide essential insights that qualitative research alone cannot deliver."

Ricardo Martins

Unlocking the power of advanced quantitative methods

March 12, 2025

Bria Alexander

"Remember that community voices must be included in the process, not just consulted."

Bria Alexander Ariel Kennan Charlotte Lee Sarah Brooks Emily Lessard Gordon Ross Joanne Dong

Reflect and Chart Forward

December 10, 2021

"We just decided that we would become consultants internally and we started doing work."

Discussion

June 9, 2017

Benjamin Real

"I was able to explain to my team which big themes we were focusing on."

Benjamin Real

Maturity Models: A Core Tool for Creating a DesignOps Strategy

October 1, 2021

Clara Kliman-Silver

"Trust is paramount when integrating AI into UX processes."

Clara Kliman-Silver

UX Futures: The Role of Artificial Intelligence in Design

June 7, 2023

Noz Urbina

"Having a starting point gets people's juices flowing and collaboration going in ways I couldn't have imagined."

Noz Urbina

Rapid AI-powered UX (RAUX): A framework for empowering human designers

May 1, 2025

Melinda Belcher

"Flexibility needs to go both ways."

Melinda Belcher

Insider preview of Enterprise Experience 2020

May 28, 2020

Edgar Anzaldua Moreno

"If you frame the research in terms they can understand, you’re going to make them research hungry."

Edgar Anzaldua Moreno

Using Research to Determine Unique Value Proposition

March 11, 2021