Day 1- Actions and Reflections: Bridging the Skills Gap Among Researchers

— Speaking from Jakarta, it’s 2:00 AM EST, and I’m awake and topic is about actions and reflections

 

— This talk builds on bridging skills gap for researchers

 

— In 2018, I returned to Indonesia from the United States, where I led tech organizations at two unicorns
  • I had access to researchers without the same pedigree
— I will talk about the learning framework for researchers,
  • No matter where you work, you face the same issues where you have UXRs with limited working experience in a team
  • If we want to be a truly inclusive practice, that captures many life experiences, we need to do a better job of creating opportunities for researchers for non-traditional backgrounds

 

— I will talk through three things
  •     The Challenge
  •     The 3 Philosophies
  •     The Learning Frameworks

 

— The challenge: I had the pleasure of leading two teams with 50+ researchers, with the following educational background:
  • 3-4 years of UXR
  • Background in STEM
  • 10% had a Master’s degreee
  • 63% had no prior design experience
  • 13% were new hires

 

— Compare this to the former research team at Uber
  • People were groomed as design researchers at famous studios programs, and had graduate background in social sciences
—  I had a team filled with potential, but which didn’t have the necessary experience
  • Nothing wrong with this, but I needed to bridge gap in skills and practices
  • So how to improve practice and skills?

 

— I brought continuous learning to the team as a general principle
  • Attended conferences
  • Brought in experts
  • Built own research conference
  • Set up research buddy programs
  • Tons of research feedback sessions
— While impressed by these activities, the skill growth of UXR lagged behind design teams
  • Why though? Both teams had similar backgrounds,
  • UXR would always get evaluations of  more room for improvement from stakeholders and recognized, and there was a sense of impostor syndrome among the researchers
— This approach didn’t make sense to me, given all the necessary education development that had been committed so far

 

— What caused the gap in the design team having more positive outcomes?
  • The Initial hypothesis was that stakeholders didn’t grasp same level understanding of research relative to design
  • Since product managers and c-levels may know design more, they may misunderstand research performance
— So I shadowed design team and their activities to get sense of what worked
  • Three rituals stood out

 

— These three activities were the main difference between the design team and research team
  • Design critiques
  • 1 on 1 sessions
  • Paired Designs

 

— Designers and researchers used vigor differently:
  • Researchers were  incredibly focused on rigor for their weekly sessions
  • Designers used rigor in more relaxed way referencing feelings, and balance

 

— The second was that designers focused on live designing, copying existing design problems, and focused on redefining design problems
  • We used direct demonstrations of skills and discuss centered on how things were done differently
  • It was rare to witness real-time research evaluation in research teams
    • UXR teams rarely conducted analysis together
    • Their discussion was primarily verbal and anchored in general terms

 

— I did a secondary study to see if others encountered similar discrepancies and came across three philosophers that touched upon issue

 

— The philosophers were Erik Stolterman, Donald Schon, and John Dewey

 

— After looking through all three had eureka moment

 

— First Stolterman explained the difference between science complexity versus design complexity
  • His argument was that when design research doesn’t support design practice, it is too reliant on scientific rigor
  • Design research practice should be built upon design methods
    • Design research faces design complexity, rather than scientific complexity

 

—> Researchers per Stolterman, should focus on design rigor vs scientific rigor

 

—> Four types of rigor were idetnified
  • SImple tools
  • Non-perscriptive frameworks
  • Concepts open for implementaion
  • High-level theoretical ideas

 

— Why does this matter?
  • Designers did adhere to standards, but not scientific one I was more familiar with
— Schon said researchers are designers, rather than scientific actors

 

— Design rigor talked about here is applicable to practice and Schon challenged the type of rigor which researchers tend to adhere to

 

— Challenged idea that scientific rigor was adequate to meet design problems, which are messy and full of surprises
  • We need a different kind of rigor called knowing-in-action that was intuition that designers grasp through practice, a kind of intuition that is involved with physical activities like riding a bike
  • Next, practitioner developed this thinking, by the  concept of reflection in action, where practitioner improved rigor through
    • 1) Capturing intuitions of how things work
    • 2) Encountering surprises that don’t fit the mental model
    • 3) Reflecting on the surprises and new strategies
    • 4) This would lead to different actions and on-the-spot experiments
    • 5) Finally,  the designer would reflect on the move itself
— Practitioner can refine rigor by following this step, and researchers should follow-up

 

— UXRs wanted to push back on the lack of rigor, but cycle described previously fits with activities we do as researches
  • 1) We need to be non-judgmental in our interviews and research, as well as use neutral language
  • 2) We review contradictory information
  • 3) We modify our interview techniques
  • 4) We are opinionated about our goals
  • 5) We are neutral in capturing results of tools
— I learned from all this:
  1. Researchers are designers, and should focus on design rigor
  2. Intuition, is a design rigor that we researchers can use
  3. Reflection in Action is one framework for us to build on our intuition

 

— Finally Dewey, proposed two factors for learning activities
  1. Learning by Doing
  2. Demonstrations by Senior Practitioners, as opposed to verbal instructions
— Students could be coached to absorb lesson, as opposed to taught

 

— How did these ideas explain the gap between the design and UXR?
  • I found that design activities aligned with  an effective learning framework, while UXR was focused on developing the wrong skills in the wrong way
— I then took the next step of building a more effective framework for team, based on all these insights

 

— Created two frameworks that supports reflection in action 3Rs and DI

 

— Rs
  • Recount: Describe intuition used in UXR activities, and explaining  procedures.
  • Relate: Taking intuition to a particular problem or concept
  • Reframe: Questioning intuition used, and looking at the problem
— Goal to dissect learner’s intuition so that others could learn

 

— UXRs would synthesize this framework

 

 

— Second framework dealt with educating practitioners through demonstrating and imitating, where people reflect on the work they did

 

— Teams did through this through paired research, and 1 on 1 research sessions
  • Paired Research, where researchers did UXR together
  • 1 on 1 had mentors and learners imitate these challenges
— These changes were awkward to begin with, but we got used to it

 

 

— My initial hypothesis was off, that stakeholder value on design wasn’t main cause

 

— Instead the design team, whether they knew it or not, was applying successful learning philosophies

 

— By introducing 3Rs and DI framework, UXR team quickly caught up, research practice became more effective, and received more positive stakeholder feedback

 

— I invite you to use your own intuition to absorb this information, glean your own insight, and create frameworks that work for you

 

— This will help us beat our impostor syndrome, and gatekeeping in design research
  • Much more effective than instruction being thrown at learners
— Confident this reflective learning approach, will help demystify design research for those outside it
  • Our programs need to be broader to those who don’t check all the UXR boxes, and apply to everyone
Q&A

 

  1. Are you putting together an article regarding these thoughts?  If not, what do you suggest to read?

 

—> Reflective Practioner by Donald Schon is good literature to start with