Evaluating Designers with Ignacio Martinez

Giving feedback to subordinates can be just as stressful as receiving it. Yet evaluations are a critical component of retention, employer/employee expectations, and production in general. Having an evaluation framework and system in place creates efficiencies, fills voids, and benefits everyone on the team. Enter Ignacio Martinez, associate director at Grand Studio in Chicago. He’ll be delivering a talk at the DesignOps Summit in September, “Fair and Effective Designer Evaluation”.

In this podcast episode, Ignacio and Lou explore the importance of a well-structured evaluation framework that highlights “glows and grows” in the areas of craft, quality, client interaction, and teamwork. Ignacio’s system, built on the very accessible Google Sheets, combines quantitative metrics with qualitative feedback to reduce bias and offer a comprehensive assessment of designers’ performance. His framework allows for continuous feedback from peers, project directors, and supervisors.

What You’ll Learn from this Episode:

• The principles and methodologies behind creating a fair and effective designer evaluation system
• How contributions from peers, career managers, and directors can create a robust evaluation system
• The importance of a structured framework with clear categories and traits such as craft, quality, client interaction, and teamwork
• The benefits of incorporating both quantitative metrics and qualitative feedback to minimize bias and provide comprehensive evaluations
• How to create a framework for continuous ongoing feedback from peers, project directors, and supervisors
• How evaluation criteria may evolve based on internal priorities

Quick Reference Guide:
0:13 – Intro of Ignacio
3:54 – Evaluating designers then and now
6:32 – Gut feelings versus using a system
8:27 – Defining desired traits and levels
11:49 – The framework of the documentation
14:54 – The Rosenverse
17:34 – Who are the evaluators? Are they biased? Are they anonymous?
21:33 – The frequency of evaluations
22:36 – Consider what makes the business run
26:05 – The importance of transparency
26:51 – Ignacio’s gift for listeners

Ethan Marcotte on the Tech Industry, Unions, and AI

In a time of massive layoffs across the tech industry, and with the inevitable advancement of AI, is it time for tech workers to organize — as in, unionize? I know, I know. You thought unions were for 1950’s factory workers. Not so. Ethan Marcotte, author of You Deserve a Tech Union (and coiner of the term “responsive web design”) thinks it’s high time for tech workers to protect themselves by coming together and deciding what’s most important to them as a collective.

Certainly tech workers don’t face the same kind of potential life-threatening working conditions of industrial America, but they still deserve a seat at the table when important decisions about their work are being discussed. With issues related to equality, transparency, workplace harassment, and how AI is shifting roles and affecting how work gets done, there’s a lot to talk about.

Ethan will bring his perspective on tech workers and how they’re being impacted by AI to the upcoming Designing with AI virtual conference in June.

What You’ll Learn from this Episode:

• What’s attractive about unionizing for tech workers of the 2020s
• What tech workers would change if they could
• About tech walkouts and unions that have already happened
• Helpful resources for starting conversations with coworkers
• The potential relationship among AI, reskilling, and worker unions

Quick Reference Guide:
0:20 – Introduction of Ethan
3:35 – How Ethan became interested in the idea of tech unions
6:04 – “Weren’t unions for the manufacturing industry in the 1950s?”
9:32 – The things tech workers would change if they could
11:14 – Conversations among employees – are they safe? Are they protected?
13:28 – On organizing for the greater good of humanity
17:11 – Plug for Managing Priorities: How to Create Better Plans and Make Smarter Decisions by Harry Max
19:06 – How we should feel about AI
22:36 – AI, reskilling, and when workers don’t want to leave mundane tasks behind
31:08 – Employees “voting with their feet” is costly for organizations
33:24 – How future workers may organize as it relates to AI
36:30 – Ethan’s gift for listeners

We Need to Talk with Joshua Graves

Tough conversations can feel like real-life horror stories—but they don’t have to. In We Need to Talk: A Survival Guide for Tough Conversations, Joshua Graves offers a practical, psychologically grounded toolkit for navigating conflict with clarity and courage. Drawing on insights from neuroscience, psychology, and his own lived experience, Graves explains why our brains react so strongly to tension and conflict—and what we can do about it.

Lou and Joshua discuss workplace power dynamics, emotional triggers, and avoidance patterns, showing how even a moment of pause can shift the outcome. Joshua’s advice? Slow down. Breathe. Ask questions that begin with what or how instead of why. And remember, you’re allowed to step away and come back—conflict doesn’t need to be resolved in the heat of the moment.

Whether you’re facing pay disputes, boundary violations, or breakdowns in trust, Joshua’s goal isn’t to script your response but to equip you with flexible, self-aware tools you can adapt to your own voice.

What You’ll Learn from this Episode:

• Why Joshua Graves, an artist with a love for spooky aesthetics, is an unexpected but deeply thoughtful guide to conflict.
• How his personal discomfort with confrontation led to years of research in psychology, neuroscience, and communication.
• Why our brains interpret digital conflict—like emails or Slack messages—as real threats, and what that means for how we respond.
• What it means to treat tough conversations like design problems, working within human constraints rather than against them.
• How slowing down and asking the right questions can transform emotionally charged moments into opportunities for clarity and connection.
• Why one-size-fits-all advice doesn’t work—and how Graves’ approach helps you develop your own voice in conflict.

Quick Reference Guide:
0:12 – Meet Joshua Graves
2:45 – The background of Joshua’s book, We Need to Talk
7:30 – Helpful rabbit holes when researching and writing We Need to Talk
10:00 – Advice for tough conversations
16:23 – Why you should use the Rosenverse
18:38 – What to do when someone is out of control
22:07 – “Conversations” to have with yourself
25:20 – Joshua’s gift for the audience

Unleashing Swarm Creativity to Solve Enterprise Challenges

Enterprises, even those with mature design practices, find it difficult to tap into the creativity of all of its workforce. Yet unleashing that broad creativity is now needed more than ever as success of teams depends on having the nimbleness of an ant farm to adapt and find their way around obstacles. Enterprise design processes, systems and ops are often tied to old top-down command/control organizational models. Design Swarms is an approach that has been used and adopted by teams within companies like Amazon, Amgen, Autodesk, Callison, Deutsche Bank, Lilly, T-Mobile, Microsoft, and REI to unleash swarm creativity at scale.

But Do Your Insights Scale? with Katy Mogal

When stakeholders have access to real-time data about millions of user interactions, how can qualitative researchers articulate the value of small-sample studies for product and business strategy?

Katy Mogal, UX Research Lead at Google Assistant, joins Lou to offer a preview of the case study she’ll share at Advancing Research 2021, including learnings about how human-centered researchers can effectively collaborate with functions like data science and business strategy, and how to persuade analytically-minded stakeholders to embrace rich qualitative data about people’s needs and motivations as an input to business strategy.

Future Orientations to Everyday Life: Futures Anthropology as a Methodology

UX research is inherently future-oriented. As researchers, we are often on the lookout to understand what users expect, what they anticipate to happen, and what they hope will come next. How we orient ourselves towards the future is part of how we experience the present. By drawing attention to the nuances of different future orientations in everyday life, we can unlock new depths of understanding to improve product and service design. Join this session to learn how to center the future as an object of study and apply frameworks from futures anthropology to advance tactical and strategic research.

How Your Organization’s Generative Workshops Are Probably Going Wrong and How to Get Them Right

Generative workshops are a critical generative component of any product development process. But in my 20+ years conducting product user research, I have seen more product harm come from so-called “workshops” or “design sprints” than good. In this tutorial, I will share more about my experience and what I’ve found are critical components of generative workshops — whether they last five hours or five days.

Contrary to popular belief, a design sprint is a highly structured and carefully designed series of exercises, not a brainstorm, design jam or free-for-all. The whole point is to drive a cross-functional team to the right outcome, and this requires a set of structured exercises which weave the thread of user needs, behaviors and attitudes throughout. This involves more than reviewing the research at the start and then moving on to create without that research in context.

A true design sprint takes us from user insights — even broad user insights — to user-evaluated concepts or designs. The generative phase of a product is deeply impactful, and design sprints are a fantastic tool for driving this needed impact. However, many are practicing brainstorms or design jams rather than true design sprints. One can make a mismatched concept extremely usable throughout the product development process, but that will not remedy the fact that it is not the right concept.
Researchers are ideal design sprint organizers and facilitators, but researchers are sometimes not even considered a critical component of the sprint. It’s important for knowledgeable researchers to drive design sprint impact.

A New Vantage Point: Building a Pipeline for Multifaceted Research(ers)

The world is evolving. By 2044 the U.S. will be a majority-minority population. Africa and India are projected to fuel the world’s population growth over the next 30 years. With demographics shifting under our feet, how will we generate accurate and relevant insight if the interpreters are not cultural natives to the new demographic mix. This presentation will explore strategies to create pipelines for the next, most diverse generation to have open pathways and onramps to the next chapter of research, insights, and innovation.

Enterprise UX Playbook (Videoconference)

Many of the most successful software development practices — like agile and UX — emerged in the consumer facing private sector. Think Airbnb, Spotify, Uber. But what happens when we try to map those practices onto large enterprises that typically serve internal employees rather than the public? For example, are UX’ers prepared to think about how large systems connect and interact? How about the challenges of HR and roles and responsibilities? Challenges such as these are highly relevant in enterprise spaces, and perhaps even more so in the public sector where systems are often quite old and ways of working have calcified. This talk focuses on “gaps” in Enterprise UX, and how we might seek to close them.

QuantQual Book Club: Small Data (Videoconference)

In a context dominated by conversations about AI and big data, it’s a great moment to revisit the concept of “small data,” and discuss the significance of small observations and intimate details in understanding consumer behavior and its impact on making successful business decisions. Martin Lindstrom’s classic book Small Data sheds a bright light on how sometimes big data and big decisions stand firm on small data nuggets. Join us for a discussion of Martin’s work—even if you’ve not yet read Small Data (but extra credit if you have!). Bring your questions and experiences to share.