Day 1- Actions and Reflections: Bridging the Skills Gap Among Researchers
— Speaking from Jakarta, it’s 2:00 AM EST, and I’m awake and topic is about actions and reflections
— This talk builds on bridging skills gap for researchers
— In 2018, I returned to Indonesia from the United States, where I led tech organizations at two unicorns
-
I had access to researchers without the same pedigree
— I will talk about the learning framework for researchers,
-
No matter where you work, you face the same issues where you have UXRs with limited working experience in a team
-
If we want to be a truly inclusive practice, that captures many life experiences, we need to do a better job of creating opportunities for researchers for non-traditional backgrounds
— I will talk through three things
-
The Challenge
-
The 3 Philosophies
-
The Learning Frameworks
— The challenge: I had the pleasure of leading two teams with 50+ researchers, with the following educational background:
-
3-4 years of UXR
-
Background in STEM
-
10% had a Master’s degreee
-
63% had no prior design experience
-
13% were new hires
— Compare this to the former research team at Uber
-
People were groomed as design researchers at famous studios programs, and had graduate background in social sciences
— I had a team filled with potential, but which didn’t have the necessary experience
-
Nothing wrong with this, but I needed to bridge gap in skills and practices
-
So how to improve practice and skills?
— I brought continuous learning to the team as a general principle
-
Attended conferences
-
Brought in experts
-
Built own research conference
-
Set up research buddy programs
-
Tons of research feedback sessions
— While impressed by these activities, the skill growth of UXR lagged behind design teams
-
Why though? Both teams had similar backgrounds,
-
UXR would always get evaluations of more room for improvement from stakeholders and recognized, and there was a sense of impostor syndrome among the researchers
— This approach didn’t make sense to me, given all the necessary education development that had been committed so far
— What caused the gap in the design team having more positive outcomes?
-
The Initial hypothesis was that stakeholders didn’t grasp same level understanding of research relative to design
-
Since product managers and c-levels may know design more, they may misunderstand research performance
— So I shadowed design team and their activities to get sense of what worked
-
Three rituals stood out
— These three activities were the main difference between the design team and research team
-
Design critiques
-
1 on 1 sessions
-
Paired Designs
— Designers and researchers used vigor differently:
-
Researchers were incredibly focused on rigor for their weekly sessions
-
Designers used rigor in more relaxed way referencing feelings, and balance
— The second was that designers focused on live designing, copying existing design problems, and focused on redefining design problems
-
We used direct demonstrations of skills and discuss centered on how things were done differently
-
It was rare to witness real-time research evaluation in research teams
-
UXR teams rarely conducted analysis together
-
Their discussion was primarily verbal and anchored in general terms
-
— I did a secondary study to see if others encountered similar discrepancies and came across three philosophers that touched upon issue
— The philosophers were Erik Stolterman, Donald Schon, and John Dewey
— After looking through all three had eureka moment
— First Stolterman explained the difference between science complexity versus design complexity
-
His argument was that when design research doesn’t support design practice, it is too reliant on scientific rigor
-
Design research practice should be built upon design methods
-
Design research faces design complexity, rather than scientific complexity
-
—> Researchers per Stolterman, should focus on design rigor vs scientific rigor
—> Four types of rigor were idetnified
-
SImple tools
-
Non-perscriptive frameworks
-
Concepts open for implementaion
-
High-level theoretical ideas
— Why does this matter?
-
Designers did adhere to standards, but not scientific one I was more familiar with
— Schon said researchers are designers, rather than scientific actors
— Design rigor talked about here is applicable to practice and Schon challenged the type of rigor which researchers tend to adhere to
— Challenged idea that scientific rigor was adequate to meet design problems, which are messy and full of surprises
-
We need a different kind of rigor called knowing-in-action that was intuition that designers grasp through practice, a kind of intuition that is involved with physical activities like riding a bike
-
Next, practitioner developed this thinking, by the concept of reflection in action, where practitioner improved rigor through
-
1) Capturing intuitions of how things work
-
2) Encountering surprises that don’t fit the mental model
-
3) Reflecting on the surprises and new strategies
-
4) This would lead to different actions and on-the-spot experiments
-
5) Finally, the designer would reflect on the move itself
-
— Practitioner can refine rigor by following this step, and researchers should follow-up
— UXRs wanted to push back on the lack of rigor, but cycle described previously fits with activities we do as researches
-
1) We need to be non-judgmental in our interviews and research, as well as use neutral language
-
2) We review contradictory information
-
3) We modify our interview techniques
-
4) We are opinionated about our goals
-
5) We are neutral in capturing results of tools
— I learned from all this:
-
Researchers are designers, and should focus on design rigor
-
Intuition, is a design rigor that we researchers can use
-
Reflection in Action is one framework for us to build on our intuition
— Finally Dewey, proposed two factors for learning activities
-
Learning by Doing
-
Demonstrations by Senior Practitioners, as opposed to verbal instructions
— Students could be coached to absorb lesson, as opposed to taught
— How did these ideas explain the gap between the design and UXR?
-
I found that design activities aligned with an effective learning framework, while UXR was focused on developing the wrong skills in the wrong way
— I then took the next step of building a more effective framework for team, based on all these insights
— Created two frameworks that supports reflection in action 3Rs and DI
— Rs
-
Recount: Describe intuition used in UXR activities, and explaining procedures.
-
Relate: Taking intuition to a particular problem or concept
-
Reframe: Questioning intuition used, and looking at the problem
— Goal to dissect learner’s intuition so that others could learn
— UXRs would synthesize this framework
— Second framework dealt with educating practitioners through demonstrating and imitating, where people reflect on the work they did
— Teams did through this through paired research, and 1 on 1 research sessions
-
Paired Research, where researchers did UXR together
-
1 on 1 had mentors and learners imitate these challenges
— These changes were awkward to begin with, but we got used to it
— My initial hypothesis was off, that stakeholder value on design wasn’t main cause
— Instead the design team, whether they knew it or not, was applying successful learning philosophies
— By introducing 3Rs and DI framework, UXR team quickly caught up, research practice became more effective, and received more positive stakeholder feedback
— I invite you to use your own intuition to absorb this information, glean your own insight, and create frameworks that work for you
— This will help us beat our impostor syndrome, and gatekeeping in design research
-
Much more effective than instruction being thrown at learners
— Confident this reflective learning approach, will help demystify design research for those outside it
-
Our programs need to be broader to those who don’t check all the UXR boxes, and apply to everyone
Q&A
-
Are you putting together an article regarding these thoughts? If not, what do you suggest to read?
—> Reflective Practioner by Donald Schon is good literature to start with