Day 3 Session Notes–Sturgeon’s Biases

— Great to be here and hello everyone

• A question for you: What was last bad experience you had with another discipline?
  • Everyone I know in design or ops, has story or ten involving product managers, developers, etc.

— When you think about experience, you are more aware of how things went wrong, and how you had to change your work to avoid these things

— People have similar stories about designers, and have story about having such a bad experience that they changed how they worked with design

— With that, I’ll kick off with introducing Theodore Sturgeon, who wrote many sci-fi stories and TV scripts

— Sturgeon is known for his response to critical views of sci-fi genre as being filled with cruddy content

• Sturgeon had a revelation…

— 90% of everything is crud, and this revelation is called Sturgeon’s Law

• Helps me understand problems we encounter when disciplines start butting heads
  • We have PMs vs designers vs engineers all ranting about how the other is doing a bad job
• And leaders struggling in spaces where all disciplines overlap

— We are butting heads frequently, especially in times of organizational transformation and disciplines redefining relationships with each other

 

— Two biases in how disciplines perceive each other

• Because it turns out that Sturgeon’s law applies to group expertise too
  • Problem for sociologist to point out

 

— Why does this happen?

• Imagine community with 100 people [lawyers, devs, doesn’t matter], but let’s say designers
• Some folks will be great, others will be less so, with everything in between

 

— You have some kind of intuitive sense of people who are better or worse in some domain, or raw ability

• There is a mental model of how communities are broken down, and that people have good sense of spread of ability in discipline

 

— But distribution of people in practice isn’t random, and good people tend to cluster together with each other

 

— They also raise up people around them through influence and education or push them away

 

 

— This means best practitioners of community of practice have overly biased view of their ability, as spread of ability between best and worst is narrow and biased toward the good end of the distribution

— People outside the bubble of excellence get radically different experience

• The best in the community experience that community as 90% awesome for everyone, while in reality it’s only 10% of entire population

— This is the first Sturgeon bias

• So is DesignOps crud?

— The most skilled part of any community of practice will be a minority that many people won’t experience

— My first 10 years of working, I disliked quality assurance testers, as people worked with were all terrible at testing, with no automated testing or independent thought

• Example of this poor performance involved releasing a curse word in application copy, and this word got seen but not called out, because it wasn’t on the written quality checklist
• These bad experiences got in my own way, as I seriously underestimated value of what awesome testing could bring in

 

— Voices of people in community are not equal as well

 

— You hear voices of top performers who have more visibility than anyone else

• Few conference keynotes are about things going wrong and failing

 

— Presentation of community is 90% awesome, but this is a minority view

• This dissonance causes problems, so we dismiss awesomeness and relevance of craft as a lie
  • For example,  you experience Agile as process-heavy action, instead of a radically better way to develop software
  • DesignOps can be wonderful, but seen as top-down prescriptive design system, and hard to take seriously even when it could help

 

— I’d like to emphasize these biases are massive oversimplification at best

• Ability isn’t fixed and people can improve, and their performance can get worse if they don’t adapt to changing circumstances
• Ability can be constrained by external factors, and you won’t get better without opportunities with training

— Domains are messy and overlap with others, and no clean borders at all

 

— Yet we do experience enough problems in interacting with other disciplines, which make you think you understand problems better than the actual practitioners

• Downstream results include: UXR being ignored, design-thinking being presented as snake oil from executives

 

— Really easy for awesome bubble to not see the bad, and someone outside to never experience the good of a domain

• To give an example, a new leader brought in to operationalize design, and called embedded designers into central internal agency and standardized workflow and products for projects
  • Bunch of reasons for approach and worked with certain parts of org,
  • But for other groups it was a terrible decision as teams were doing long-term product dev work, and had existing system and processes in place
    • They were already happy and working effectively
  • Pulled away to central team, and created additional artifacts that provide no value
• Not what good DesignOps leader would do
  • But teams experienced operations work as power-grab and make work, and having their autonomy removed
  • Never experienced good designs, and took experience of bad design practice to every job they went to afterwards

— How to turn this around?

 

— First, believe people’s lived experiences

• Don’t have argument as to whether things are good or not
• Be curious and understand what people’s experiences have been

— Especially important at executive level, as they have more experience to draw in, and if those experiences were bad, they are more impactful

• Great testing lead listened to my concerns and believed and empathized first

 

— Next, ask for stories about experience, and last experience of working with a discipline, to understand my baselines

• We should be good at this, but need to double check what people’s experiences of DesignOps has been

 

— We should use different labels for work, or specific word or more general

• Prior experience with label might have been poisonous

— Need to double check if we are actually communicating things, that we think we are communicating

— Hit this a bunch with UXR, as people viewed it as empty report [slow expensive, not useful]

• Use other labels like ‘discovery sprints’ ‘diary study’ to have conversation of intent and outcomes without pre-conceptions

— Talk about outcomes first, as opposed to details of what is done

• Focus on being explicit about outcomes, as people will make assumptions based on previous experiences
• Work of financial organization— and exec teams were resistant to design and research staff, which was driven by bad experiences of ineffective teams
  • Speaking about outcomes that were familiar to financial org, and changes in working practice to help achieve that
  • Outcomes first and helped them question previous experiences

 

— Next, question what bubble you are living in:

• Whether awesome bubble or crud bubble, it’s still a bubble

— How can we find out? Question assumptions and immediate reactions to community or discipline

• New and innovative practices will be in minority of any field and previous experiences can block us from learning something new
• Treat this as a research question— we should be good at this

— Massively skeptical of AI hype cycle, personally, as I have been working in AI space, and previous failures have colored experiences

• Because of bad experiences paying attention to AI-adjacent sessions, want to figure out good things you are missing

— Keeping biases can help you see your own bubbles and help other get out of their

 

— Two corollaries to revelation:

• Don’t feel too bad about the bad in your domain, as everyone else has that same problem
• Don’t forget awesome bits of your domain are pretty awesome

— Design is going through transformation from AI but…

 

— We are not special and all domains are going through changes of AI

— We are not special, but we are not alone in our struggles

• DesignOps can help but only if people can hear about it

— Can’t make orgs more resilient until we understand our weakest points

• Can’t make impact if fighting misconception of work with ourselves and everyone else

— Understand good and bad

 

— So to summarize: Believer the lived experiences of others, and use different words to get past pre-conceptions and question your own experiences

• Links to worksheets in conference notes and QR codes

 

— So when you encounter problems with different disciplines, keep biases in mind, and find awesome a little bit more often

Q&A

1. If you had to pick name of Slack channel for questions of good experience what would it be?
   1. Failure Swap-Shop, activity and learnings from failure
2. Practices to help with this mindset?
   1. Ask people to repeat back what you told them, and looping for understanding
      1. Get people to cross-validate meanings
   2. Shared artifacts for single source of truth rather than multiple interpretations of it
      1. Rather than disagreement, but discussion about common artifact
3. How to talk with executive burned by bad experience with UXR? How to turn perspective around?
   1. Ask for stories about experience —
      1. For financial services team results were: stream of doing work but not much value for it, another part was stuff we had wasn’t clickable by time it had to be used
   2. Listened to stories and validated feelings, and then spoke about what we wanted to do and outcomes to achieve and expected that this was an experiment for the organization and what was true or not for thing
      1. Activities to figure X out to make this kind of decision, as we linked practice to activities that would lead to particular outcomes
         1. UXR wasn’t seen as random stage-gate process— but about feedback loops and how research could give direction to organization and be more common
4. How to balance believing your own experiences and questioning your own experience?
   1. Heuristic: Watch out if you believe a thing, listen to stories that contradict that belief, and you automatically dismiss those stories
      1. In example of design, I was working in software development as Agile was developing
      2. I thought Agile was nonsense, contrary to previous experiences of doing design up-front
         1. Started doing Agile and found it super effective and then started learning more
   2. If other people saying something is good, and I automatically think it is nonsense, need to poke at it more and double-check
      1. If you see diametrically different opinions, spend time in the place, and validate if assumption is right or wrong