Day 1- Actions and Reflections: Bridging the Skills Gap Among Researchers

— Speaking from Jakarta, it’s 2:00 AM EST, and I’m awake and topic is about actions and reflections


— This talk builds on bridging skills gap for researchers


— In 2018, I returned to Indonesia from the United States, where I led tech organizations at two unicorns
  • I had access to researchers without the same pedigree
— I will talk about the learning framework for researchers,
  • No matter where you work, you face the same issues where you have UXRs with limited working experience in a team
  • If we want to be a truly inclusive practice, that captures many life experiences, we need to do a better job of creating opportunities for researchers for non-traditional backgrounds


— I will talk through three things
  •     The Challenge
  •     The 3 Philosophies
  •     The Learning Frameworks


— The challenge: I had the pleasure of leading two teams with 50+ researchers, with the following educational background:
  • 3-4 years of UXR
  • Background in STEM
  • 10% had a Master’s degreee
  • 63% had no prior design experience
  • 13% were new hires


— Compare this to the former research team at Uber
  • People were groomed as design researchers at famous studios programs, and had graduate background in social sciences
—  I had a team filled with potential, but which didn’t have the necessary experience
  • Nothing wrong with this, but I needed to bridge gap in skills and practices
  • So how to improve practice and skills?


— I brought continuous learning to the team as a general principle
  • Attended conferences
  • Brought in experts
  • Built own research conference
  • Set up research buddy programs
  • Tons of research feedback sessions
— While impressed by these activities, the skill growth of UXR lagged behind design teams
  • Why though? Both teams had similar backgrounds,
  • UXR would always get evaluations of  more room for improvement from stakeholders and recognized, and there was a sense of impostor syndrome among the researchers
— This approach didn’t make sense to me, given all the necessary education development that had been committed so far


— What caused the gap in the design team having more positive outcomes?
  • The Initial hypothesis was that stakeholders didn’t grasp same level understanding of research relative to design
  • Since product managers and c-levels may know design more, they may misunderstand research performance
— So I shadowed design team and their activities to get sense of what worked
  • Three rituals stood out


— These three activities were the main difference between the design team and research team
  • Design critiques
  • 1 on 1 sessions
  • Paired Designs


— Designers and researchers used vigor differently:
  • Researchers were  incredibly focused on rigor for their weekly sessions
  • Designers used rigor in more relaxed way referencing feelings, and balance


— The second was that designers focused on live designing, copying existing design problems, and focused on redefining design problems
  • We used direct demonstrations of skills and discuss centered on how things were done differently
  • It was rare to witness real-time research evaluation in research teams
    • UXR teams rarely conducted analysis together
    • Their discussion was primarily verbal and anchored in general terms


— I did a secondary study to see if others encountered similar discrepancies and came across three philosophers that touched upon issue


— The philosophers were Erik Stolterman, Donald Schon, and John Dewey


— After looking through all three had eureka moment


— First Stolterman explained the difference between science complexity versus design complexity
  • His argument was that when design research doesn’t support design practice, it is too reliant on scientific rigor
  • Design research practice should be built upon design methods
    • Design research faces design complexity, rather than scientific complexity


—> Researchers per Stolterman, should focus on design rigor vs scientific rigor


—> Four types of rigor were idetnified
  • SImple tools
  • Non-perscriptive frameworks
  • Concepts open for implementaion
  • High-level theoretical ideas


— Why does this matter?
  • Designers did adhere to standards, but not scientific one I was more familiar with
— Schon said researchers are designers, rather than scientific actors


— Design rigor talked about here is applicable to practice and Schon challenged the type of rigor which researchers tend to adhere to


— Challenged idea that scientific rigor was adequate to meet design problems, which are messy and full of surprises
  • We need a different kind of rigor called knowing-in-action that was intuition that designers grasp through practice, a kind of intuition that is involved with physical activities like riding a bike
  • Next, practitioner developed this thinking, by the  concept of reflection in action, where practitioner improved rigor through
    • 1) Capturing intuitions of how things work
    • 2) Encountering surprises that don’t fit the mental model
    • 3) Reflecting on the surprises and new strategies
    • 4) This would lead to different actions and on-the-spot experiments
    • 5) Finally,  the designer would reflect on the move itself
— Practitioner can refine rigor by following this step, and researchers should follow-up


— UXRs wanted to push back on the lack of rigor, but cycle described previously fits with activities we do as researches
  • 1) We need to be non-judgmental in our interviews and research, as well as use neutral language
  • 2) We review contradictory information
  • 3) We modify our interview techniques
  • 4) We are opinionated about our goals
  • 5) We are neutral in capturing results of tools
— I learned from all this:
  1. Researchers are designers, and should focus on design rigor
  2. Intuition, is a design rigor that we researchers can use
  3. Reflection in Action is one framework for us to build on our intuition


— Finally Dewey, proposed two factors for learning activities
  1. Learning by Doing
  2. Demonstrations by Senior Practitioners, as opposed to verbal instructions
— Students could be coached to absorb lesson, as opposed to taught


— How did these ideas explain the gap between the design and UXR?
  • I found that design activities aligned with  an effective learning framework, while UXR was focused on developing the wrong skills in the wrong way
— I then took the next step of building a more effective framework for team, based on all these insights


— Created two frameworks that supports reflection in action 3Rs and DI


— Rs
  • Recount: Describe intuition used in UXR activities, and explaining  procedures.
  • Relate: Taking intuition to a particular problem or concept
  • Reframe: Questioning intuition used, and looking at the problem
— Goal to dissect learner’s intuition so that others could learn


— UXRs would synthesize this framework



— Second framework dealt with educating practitioners through demonstrating and imitating, where people reflect on the work they did


— Teams did through this through paired research, and 1 on 1 research sessions
  • Paired Research, where researchers did UXR together
  • 1 on 1 had mentors and learners imitate these challenges
— These changes were awkward to begin with, but we got used to it



— My initial hypothesis was off, that stakeholder value on design wasn’t main cause


— Instead the design team, whether they knew it or not, was applying successful learning philosophies


— By introducing 3Rs and DI framework, UXR team quickly caught up, research practice became more effective, and received more positive stakeholder feedback


— I invite you to use your own intuition to absorb this information, glean your own insight, and create frameworks that work for you


— This will help us beat our impostor syndrome, and gatekeeping in design research
  • Much more effective than instruction being thrown at learners
— Confident this reflective learning approach, will help demystify design research for those outside it
  • Our programs need to be broader to those who don’t check all the UXR boxes, and apply to everyone


  1. Are you putting together an article regarding these thoughts?  If not, what do you suggest to read?


—> Reflective Practioner by Donald Schon is good literature to start with