Day 2- Taking Inspiration from Instructional Design for Research
— Hi everyone, thanks for joining me. I’m a staff user researcher for Twilio
-
Previously I was an instructional designer (ID) transitioning into UX
— I saw how similar UX and instructional design principles can be
-
As researchers our job is to explore behaviors, and our insights alter what our users do
—We can follow instructional design principles to do UXR principles
— I define behaviors as the application of knowledge or skills
-
School teachers building curriculum
-
Designers creating UI
— Instructional Design (IS) is the creation of learning experiences that results in new behaviors. This is not just teaching classes, but also applies to UX
-
Every product has a learning experience to it, and ID helps us analyze UX
— UXR is built from pieces of related fields
-
Exploring methodologies in other fields, gives us visual heuristic skills, and lets us take research to next level
— We can borrow many things, but will focus on
-
The basics of ID process and principles
-
Writing goals
-
Analyzing goals
-
Heuristic evaluations
-
And more ways to get inspired
— When done well, the instructional design project is 50% research
-
Stakeholder interviews, analysis, and learning experience
—UXR follows similar process, and you can look-up how instructional designers perform steps for inspiration
— Before ID thinking, you need to grasp the cornerstone
-
To grasp cognitive dimensions of learning from rote memorization to deep expertise
-
From shallow to deep expertise, the learning path is as follows:
-
Remembering
-
Articulating
-
Applying
-
Evaluating
-
Creating new things
-
— How does this apply?
— Imagine if you are becoming a UXR. Here’s how the terms apply:
-
Remember: Define UXR from memory
-
Understanding: Summarizing how to do various methodologies
-
Applying: Using those methodologies
-
Analyzing: Determine which method is optimized for research
-
Evaluating: Critiquing value of insights you get
-
Creating: Running own projects/making new methods
— How to use these lessons?
-
Fastest way is when creating research goals
— Big topic in UXR is get findings that are more actionable (a talk in all conferences)
-
Be intentional with goals, which in ID are very outcome oriented and are executing on their own work
-
Observing how user behavior has changed
-
Getting this right influences everything downstream
-
— Say you are UXR at e-commerce site who wants to have new installment pay feature
-
Installments are option of how users think of budgeting
-
But the real product goal is about changing what user will do with new resource
-
Understanding how people think of budgeting is a different goal
-
— The word ‘understand’ can be pitfall in ID, a word some people say they don’t use that word in project goals
-
Daily conversation especially with stakeholders it’s a general term that doesn’t tell anything about behavior understanding enables
—Remove big cognitive goals from research plans, and focus on enabled behaviors
— Adjustments of goals through ID lens is as follows
-
i.e. Applying knowledge through early shopping process
-
Evaluating which installment option is best
-
Applying rationale
-
Creating a journey map
— Stronger goals show methods and questions to prioritize
-
There are times where you shouldn’t be exact, but this is tactic where you can be more deliberate
— You can also apply Bloom’s taxonomy to analyzing findings. Similar to writing goals.
-
Analyzing with Bloom’s can prevent you from making the wrong conclusions and recommendations
— You can also use cognitive dimensions can drive more appropriate outcomes,
-
Can users articulate why?
-
Can they remember content?
-
Or have they ever heard of installments before?
— We scaffold new knowledge onto existing frameworks.
-
With no framework, it’s a much harder problem to overcome
— Once you know which level of cognition is missing, you can go forward:
-
Asking researchers why, or can you tell me more, is a useful benchmark
-
Best solution is articulated through this way
— We as UXRs now have way for right goals, right methods, stronger analysis, and better change
-
Using ID principles in project planning
-
Using cognitive dimensions to do a heuristic evaluation of an entire UX
-
Reinterpret all behaviors in journey through lens of ID. You can map learning paths, find gaps, and develop appropriate solution
-
— Learning journeys can be complicated, so let’s complicate Bloom’s taxonomy first
— It’s tempting to look at Bloom’s and think there is a linear sequence for the process
— But we can mix it up and even go backwards
-
Design bootcamps start this way, where students create a website from scratch and it’s evaluated
-
Students struggle as they don’t know enough
-
-
This is Productive Failure, where human brain remembers struggle very well, can be effective for stronger learning
— So let’s use this principle to apply it for a platform tool like Squarespace, Notion, which allow custom applications from scratch
-
These platform tools are meant for users to build anything they can imagine
— The platforms require users to be strong creators, and that’s a lot to demand for users
-
More cognitive needs mean more cognitive load and missing crucial learning steps
— How to create proper heuristics review?
— Platforms use reverse bloom’s taxonomy, to help users grasp it
-
By having them evaluate and create, analyze, and work on material
-
Product progressively reveals complexity through lower levels
-
Things broken down and then built back-up
-
-
This helps map out user learning journey step by step
— You can evaluate if user reaches steps and what learning gaps exist, and what issues we should be learning for
-
Be careful about cognitive gaps
— Here’s a few examples where we can capture a cognitive gap
-
Missing level
-
A level in wrong order
-
A level in wrong place
-
Overloaded level
— Level in wrong location
— After finding gaps, see how people are remedying problems
-
When cognitive level isn’t served, people apply other cognitive levels as work arounds
—If an apply level is missing, users shift between evaluating tutorials before applying them
-
Two cognitive load tasks, for one lower cognitive load goal
-
Last thing to do is making suggestion that is too narrow for user to-do
-
Watching how users evaluate and create, we can see what knowledge will be applied
-
-
Suggestion can focus on the level
-
Helps us be more precise in user journey, seeing learning gaps, and how to solve it appropriately
-
— How to incorporate this?
-
Remove vague terms like “understand” terminology from Bloom’s taxonomy
-
Journey map of cognitive levels
-
Recommendations as learning gaps
-
-
Borrowing methods from ID Task analysis during research studies
-
Adapt ID tasks analysis methods for studies
-
Adapt ID recommendations on different formats for teaching insights
-
— For general learning I recommend those linked in the resources list
—> Instructional design is exciting field that can be applied to UXR
-
It’s helped me be more incisive about UX issues, and strengthened the quality of my work
-
I encourage you to learn more about learning science and whether they are applicable or not and reflecting on your own practice as a discipline on new fields
—> Let’s take inspiration from instructional design principles to move our field forward
Q&A
-
Levels of cognition to apply to goals?
—> Useful frameworks for making sure we are using more precise language
-
Usability test seeing what level product is getting users to be at
-
If not a usability test, a foundational understanding can work to grasp level of cognition
—> Bloom’s taxonomy has different terms for each of these levels, and lets us have a more precise vocabulary than ‘understand’
-
If the goal is to create change in behavior? How do you measure it?
—> Depends on research project or ID project at hand
—> If you are doing or evaluating things for an HR training, youmay want to see reports go down
-
Look for behaviors that signal a problem, and whether they are going up/down is indicatior for success
-
Can you give an example of reverse blooms and throwing people to wolves?
—> Comes up with discussions from learning styles
-
People learn by doing in my view
—> When you are being metaphorically thrown to the wolves, start to ask yourself, am I doing this right
-
Going through process, you will remember it well and will proactively figure it out
-
Bloom’s Taxonomy for career paths?
—> Not in position to develop career paths, and how to evaluate your own skills, either with given methodology or incorporating new philosophy into a practice
-
Look at career where you currently are, and how to get to evaluating and to a skill-level
-
Learnability is a usability category. Do you measure learnability of tool using longitudinal studies?
—> Yes, that’s what I’m doing with my work at Twilio
-
Looking at learnability for APIs to make sure improvements impact metrics
-
Clarifying what learning means to a developer working on an API
-
Instructional Designer in org, should they be included in UXR team, or be on their own?
—> Include them in UXR team, and having chance to do field studies together and how to ask questions