Day 2- Taking Inspiration from Instructional Design for Research

— Hi everyone, thanks for joining me. I’m a staff user researcher for Twilio
  • Previously I was an instructional designer (ID) transitioning into UX
— I saw how similar UX and instructional design principles can be
  • As researchers our job is to explore behaviors, and our insights alter what our users do
—We can follow instructional design principles to do UXR principles

— I define behaviors as the application of knowledge or skills
  • School teachers building curriculum
  • Designers creating UI
— Instructional Design (IS) is the creation of learning experiences that results in new behaviors. This is not just teaching classes, but also applies to UX
  • Every product has a learning experience to it, and ID helps us analyze UX
— UXR is built from pieces of related fields
  • Exploring methodologies in other fields, gives us visual heuristic skills, and lets us take research to next level



— We can borrow many things, but will focus on
  • The basics of ID process and principles
  • Writing goals
  • Analyzing goals
  • Heuristic evaluations
  • And more ways to get inspired


— When done well, the instructional design project is 50% research
  • Stakeholder interviews, analysis, and learning experience
—UXR follows similar process, and you can look-up how instructional designers perform steps for inspiration


— Before ID thinking, you need to grasp the cornerstone
  • To grasp cognitive dimensions of learning from rote memorization to deep expertise
  • From shallow to deep expertise, the learning path is as follows:
    • Remembering
    • Articulating
    • Applying
    • Evaluating
    • Creating new things

— How does this apply?


— Imagine if you are becoming a UXR. Here’s how the terms apply:
  • Remember: Define UXR from memory
  • Understanding: Summarizing how to do various methodologies
  • Applying: Using those methodologies
  • Analyzing: Determine which method is optimized for research
  • Evaluating: Critiquing value of insights you get
  • Creating: Running own projects/making new methods


— How to use these lessons?
  • Fastest way is when creating research goals


— Big topic in UXR is get findings that are more actionable (a talk in all conferences)
  • Be intentional with goals, which in ID are very outcome oriented and are executing on their own work
  • Observing how user behavior has changed
    • Getting this right influences everything downstream



— Say you are UXR at e-commerce site who wants to have new installment pay feature
  • Installments are option of how users think of budgeting
  • But the real product goal is about changing what user will do with new resource
    • Understanding how people think of budgeting is a different goal


— The word ‘understand’ can be pitfall in ID, a word some people say they don’t use that word in project goals
  • Daily conversation especially with stakeholders it’s a general term that doesn’t tell anything about behavior understanding enables
—Remove big cognitive goals from research plans, and focus on enabled behaviors


— Adjustments of goals through ID lens is as follows
  • i.e. Applying knowledge through early shopping process
  • Evaluating which installment option is best
  • Applying rationale
  • Creating a journey map
— Stronger goals show methods and questions to prioritize
  • There are times where you shouldn’t be exact, but this is tactic where you can be more deliberate


— You can also apply Bloom’s taxonomy to analyzing findings. Similar to writing goals.
  • Analyzing with Bloom’s can prevent  you from making the wrong conclusions and recommendations


— You can also use cognitive dimensions can drive more appropriate outcomes,
  • Can users articulate why?
  • Can they remember content?
  • Or have they ever heard of installments before?
— We scaffold new knowledge onto existing frameworks.
  • With no framework, it’s a much harder problem to overcome
— Once you know which level of cognition is missing, you can go forward:
  • Asking researchers why, or can you tell me more, is a useful benchmark
  • Best solution is articulated through this way


— We as UXRs now have way for right goals, right methods, stronger analysis, and better change
  • Using ID principles in project planning
  • Using cognitive dimensions to do a heuristic evaluation of an entire UX
    • Reinterpret all behaviors in journey through lens of ID. You can map learning paths, find gaps, and develop appropriate solution


— Learning journeys can be complicated, so let’s complicate Bloom’s taxonomy first


— It’s tempting to look at Bloom’s and think there is a linear sequence for the process


— But we can mix it up and even go backwards
  • Design bootcamps start this way, where students create a website from scratch and it’s evaluated
    • Students struggle as they don’t know enough
  • This is Productive Failure, where human brain remembers struggle very well, can be effective for stronger learning


— So let’s use this principle to apply it for a platform tool like Squarespace, Notion, which allow custom applications from scratch
  • These platform tools are meant for users to build anything they can imagine


— The platforms require users to be strong creators, and that’s a lot to demand for users
  • More cognitive needs mean more cognitive load and missing crucial learning steps
— How to create proper heuristics review?


— Platforms use reverse bloom’s taxonomy, to help users grasp it
  • By having them evaluate and create, analyze, and work on material
  • Product progressively reveals complexity through lower levels
    • Things broken down and then built back-up
  • This helps map out user learning journey step by step
— You can evaluate if user reaches steps and what learning gaps exist, and what issues we should be learning for
  • Be careful about cognitive gaps


— Here’s a few examples where we can capture a cognitive gap
  • Missing level
  • A level in wrong order
  • A level in wrong place
  • Overloaded level
— Level in wrong location


— After finding gaps, see how people are remedying problems
  • When cognitive level isn’t served, people apply other cognitive levels as work arounds


—If an apply level is missing, users shift between evaluating tutorials before applying them
  • Two cognitive load tasks, for one lower cognitive load goal
  • Last thing to do is making suggestion that is too narrow for user to-do
    • Watching how users evaluate and create, we can see what knowledge will be applied
  • Suggestion can focus on the level
    • Helps us be more precise in user journey, seeing learning gaps, and how to solve it appropriately


— How to incorporate this?
  • Remove vague terms like “understand” terminology from Bloom’s taxonomy
  • Journey map of cognitive levels
    • Recommendations as learning gaps
  • Borrowing methods from ID Task analysis during research studies
  • Adapt ID tasks analysis methods for studies
    • Adapt ID recommendations on different formats for teaching insights
— For general learning I recommend those linked in the resources list


—> Instructional design is exciting field that can be applied to UXR
  • It’s helped me be more incisive about UX issues, and strengthened the quality of  my work
  • I encourage you to learn more about learning science and whether they are applicable or not and reflecting on your own practice as a discipline on new fields
—> Let’s take inspiration from instructional design principles to move our field forward


  1. Levels of cognition to apply to goals?
—> Useful frameworks for making sure we are using more precise language
  • Usability test seeing what level product is getting users to be at
  • If not a usability test, a foundational understanding can work to grasp level of cognition
—> Bloom’s taxonomy has different terms for each of these levels, and lets us have a more precise vocabulary than ‘understand’
  1. If the goal is to create change in behavior? How do you measure it?
—> Depends on research project or ID project at hand


—> If you are doing or evaluating things for an HR training,  youmay want to see reports go down
  • Look for behaviors that signal a problem, and whether they are going up/down is indicatior for success
  1. Can you give an example of reverse blooms and throwing people to wolves?
—> Comes up with discussions from learning styles
  • People learn by doing in my view
—> When you are being metaphorically thrown to the wolves, start to ask yourself, am I doing this right
  • Going through process, you will remember it well and will proactively figure it out
  1. Bloom’s Taxonomy for career paths?
—> Not in position to develop career paths, and how to evaluate your own skills, either with given methodology or incorporating new philosophy into a practice
  • Look at career where you currently are, and how to get to evaluating and to a skill-level
  1. Learnability is a usability category. Do you measure learnability of tool using longitudinal studies?
—> Yes, that’s what I’m doing with my work at Twilio
  • Looking at learnability for APIs to make sure improvements impact metrics
  • Clarifying what learning means to a developer working on an API
  1. Instructional Designer in org, should they be included in UXR team, or be on their own?
—> Include them in UXR team, and having chance to do field studies together and how to ask questions