How we Researched and Designed the Advancing Research Conference

January 12, 2022

There are more researchers focused on users, customers, and products than ever before, and those researchers’ needs are changing. Rosenfeld Media has already interacted with researchers through the Advancing Research community, which we curate, but wanted to learn more about who they are, where they work, and how they learn. And—as a conference-production company—we wanted to know what researchers seek from conference experiences.

Our survey ran from July 2-15, 2019, and 721 people responded. We’ve summarized the results below, and have made our quantitative data available for analysis via Tableau.

Why we decided to research researchers

We began our research with the following assumptions about research and researchers involved in learning about how people use products and services:

There are more researchers doing research than ever before.

New technologies enable ever more powerful and complex products and services. They also create yawning gaps between the organizations that create products and the people who use them. Researchers help close those gaps, so the demand for people who can do research is dramatically increasing.

There are more “non-researchers” doing research than ever before.

There are not enough professionally trained researchers to meet the growing demand. And research tools are getting easier to use while simultaneously becoming more powerful. So more people with minimal or no training are being pressed into research roles.

Research teams are getting larger.

Large organizations with complex offerings require greater research capacity than ever before—and are building large research teams to meet internal demand. In turn, those research teams require a level of operational and managerial support than ever before.

Researchers are getting their seat at the table.

As research becomes more valued, research leaders are taking on increasingly strategic roles within their organizations—and often aren’t prepared to lead.

Research is becoming increasingly cross-disciplinary.

As products and services become more complex, more types of researchers, research tools and methods are needed to study problems and synthesize the results from multiple sources. Accordingly, researchers are becoming less anchored to their home disciplines, research methods and tools, and perspective. And they’re becoming more focused on solving complex problems using whatever approaches make the most sense, regardless of provenance. 

These are exciting developments, and are leading to a flowering of ideas, conversations, and activities for and among researchers of all stripes.

Rosenfeld Media is a provider of UX content, education, and conferences, and has a track record of supporting emerging communities that share our goal of creating better experiences for humanity (e.g., design and research operations, and enterprise design). We wanted to explore the opportunity to create a new conference to meet researchers’ needs, especially for the US market. We’ve already been learning about researchers’ needs through informal conversations within Rosenfeld Media’s Advancing Research community. A broad survey seemed to be a logical next step. 

What we set out to learn

We chose a variety of questions, both open and close-ended, to learn about who responded, their experience, and their preferred and actual means of learning. We also drilled deeper into one of the learning approach most interesting to Rosenfeld Media: conferences. 

We were most interested in understanding how job title and organization size relate to how researchers learn new skills and think about conference attendance. The main objectives covered in the survey were:

  • Are there major differences in the way people learn across job titles?
  • How does a person’s approach to conference attendance change as they grow in experience level?
  • What kinds of content would people expect to see at a research conference?
  • Who (speakers and attendees) might people expect to see at a research conference?
  • How much time do people in various roles spend on research related tasks?
  • How much experience do people in various roles have with research related tasks?
  • What factors are most important in determining whether someone will attend a conference?
  • What elements of a conference matter the most to people?

Our questions fell into four main categories:

  1. Your job and organization 
  2. Your research experience
  3. How you learn new skills
  4. Your conference preferences

How the survey was designed

Our survey was created primarily by Lou Rosenfeld, with assistance from Abby Covert, Natalie Hanson, and Steve Portigal. We knew the survey would be long, but hoped that the effort to complete it would leave respondents in a better place; i.e., they would learn something about themselves and their own interests by completing the activity. We also made clear to respondents that we would share the data and our findings publicly.

The survey was set up in Wufoo, and was pilot tested with 12 subjects. Pilot testers required 15 minutes to complete the survey on average. After revisions, the final survey was launched on July 2, 2019, and closed July 15, 2019. It was promoted in a variety of ways—via Rosenfeld Media’s mailing list and social media channels, via the team’s contacts and social media, and to the Advancing Research community. Special efforts were made to socialize the survey to contacts and colleagues who might in turn be able to share it with communities outside the team’s core network of UX people. 

While we expected roughly 200 responses, we received 721—so many that we were tempted to close the survey earlier than we’d planned. That said, we wish more people from outside the UX community would have participated.

How our data informs conference design

The following section takes the qualitative responses and quantitative responses into account and presents the specific findings and recommendations that our team will be using to make decisions as we design the first Advancing Research conference (which will be held March 30-April 1, 2020, in New York City).

As you go through this section you will see some graphs and charts powered by Tableau (instrumented by Sean Oslin). You can click onto various elements in the charts and graphs to pivot the data that you are displaying. We have included specific instructions below each interactive graphic.

Below is a summary of the insights our team got from our analysis of the survey. Each is expanded on below.

  • We are designing this conference for a primarily UX audience. 
  • They want to hear from researchers from outside the UX world. 
  • Organization size doesn’t impact content.
  • Methods aren’t all they are looking for. 
  • There is whitespace in the UX world for a conference about research. 
  • Our audience is boots on the ground doing research. 
  • Our audience is fairly senior in research.
  • UX Designers and UX Researchers share a lot of research responsibility. 
  • Our respondents attend events far less frequently then they read or watch videos. 
  • Our respondents value their network.
  • When we looked at respondents by likelihood and frequency for conference attendance, some clear segments emerged.
  • There are five types of people we should be designing this conference for
  • Our audience vastly prefers small to medium-sized events.
  • We now know what’s most important, and to who. 
  • They come for the content, not as much for the socialization. 

var divElement = document.getElementById(‘viz1568867146395’); var vizElement = divElement.getElementsByTagName(‘object’)[0]; vizElement.style.width=’950px’;vizElement.style.height=’991px’; var scriptElement = document.createElement(‘script’); scriptElement.src = ‘https://public.tableau.com/javascripts/api/viz_v1.js’; vizElement.parentNode.insertBefore(scriptElement, vizElement);

This graphic shows how respondents to our survey compare in terms of job title, organization size and # of employees doing research at their organization. You can click on any job title and see how organization size and # of employees changes. You can click on organization size and see the job title breakdown or # of employees doing research within that organization size. If you have a finding you don’t see in the report, please comment below or write us at advancingresearch@rosenfeldmedia.com. Feel free to share this graphic; here’s a handy link.

We are designing this conference for a primarily UX audience. For now. 

Through this research activity we realized that our access to relevant audiences for a conference about research at this point is primarily UX people. While we tried to broaden the reach of this survey, this exercise reminded us how important building a network is in reaching different types of people. Because the background and primary audience of Rosenfeld Media is those working in the UX space, our team will have to strategize about the best ways to reach outside the UX world when curating and marketing this conference, both now and in the future. We would love to see the audience of this conference diversify over time to include a higher percentage of those practicing research outside UX, but for the first year we anticipate UX people will be our primary audience. Our initial efforts to broaden our coverage will begin with how we design our inaugural program, and who we get to speak. We hope such focused efforts will draw attendees from outside UX over the coming years.

UX people want to hear from researchers from outside the UX world. 

We learned in the qualitative responses that UX Researchers (both by title and by task) want to learn about research methods and best practices outside of the UX space and they want to hear from and meet researchers who are working in adjacent fields. So even though we are designing this conference for a majority UX-audience, the content should not be majority UX-derived.

Research methods are a primary content expectation for our audience. 

Regardless of job type, organization size or other conference preferences, our audience share more similarities than differences when it came to content that they expected to see at a conference about research. 

The top three shared areas of content were:   

  1. Emerging Methods (from both within and outside UX)
  2. Analysis and Synthesis 
  3. Case Studies, especially those that highlight turning research into action or showing measurable results  

But, methods aren’t all they are looking for.

While our respondents are interested in learning about methods, that’s not the only thing they would want to have in terms of content for a research conference. We heard that people wanted a research conference to have content around big topics like diversity, inclusion, ethics and social impact. We also heard about non-research-specific skills that people want to work on such as leadership, management, change management and other soft skills. Also interesting is that there isn’t a huge delta between what people are currently using in terms of methods and what they are looking to explore further. Of the types of subjects of interest they had for a research conference, only a small percentage were related to methods.

research methods

There is whitespace in the UX world for a conference about research. 

Many of our potential audience have not been to a conference dedicated to research (at least not recently). Local meetups like CHI and UXPA are some of the most common research-focused events for our respondents. None was actually the most popular answer to the question “What (if any) events have you attended on the subject of research in the past few years?” 

var divElement = document.getElementById(‘viz1568840363182’); var vizElement = divElement.getElementsByTagName(‘object’)[0]; vizElement.style.width=’950px’;vizElement.style.height=’991px’; var scriptElement = document.createElement(‘script’); scriptElement.src = ‘https://public.tableau.com/javascripts/api/viz_v1.js’; vizElement.parentNode.insertBefore(scriptElement, vizElement);

This graphic shows how respondents to our survey compare in terms how much experience they have and how much their current roles require each main research task. The most interesting view here is to narrow by job role to see how frequency and experience vary. If you have a finding you don’t see in the report, please comment below or write us at advancingresearch@rosenfeldmedia.com. Feel free to share this graphic; here’s a handy link.

 

Our audience is boots on the ground doing research.

There was a surprisingly low number of respondents hiring vendors and/or buying research reports in their current roles. But across all job titles we saw a high number advocating for, planning, managing, conducting, observing and analyzing research as frequent parts of their current roles.

Our audience is fairly senior in research. 

The majority of our respondents reported five or more years experience in analyzing, conducting, planning, observing and advocating for research. We also see a high concentration of people who are currently teaching research, whether that be within their organizations or academically.

UX Designers and UX Researchers share a lot of research responsibility. 

UX Designers reported similar frequency on advocating for, analyzing, conducting, and planning research as UX Researchers. 

Learning Activities

 

var divElement = document.getElementById(‘viz1568840421213’); var vizElement = divElement.getElementsByTagName(‘object’)[0]; vizElement.style.width=’950px’;vizElement.style.height=’991px’; var scriptElement = document.createElement(‘script’); scriptElement.src = ‘https://public.tableau.com/javascripts/api/viz_v1.js’; vizElement.parentNode.insertBefore(scriptElement, vizElement);

This graphic shows how respondents to our survey compare in terms of how often and likely they are to participate in a set of learning activities. You can select which job titles to explore and see how that impacts the size conferences they prefer, the most important factors in deciding to attend a conference and their preferred session formats. If you have a finding you don’t see in the report, please comment below or write us at advancingresearch@rosenfeldmedia.com. Feel free to share this graphic; here’s a handy link.

Our respondents attend events far less frequently than they read or watch videos. 

This data further confirms that our audience is attending very few in-person events. It also confirms the need for long form content to support this audience in addition to in person events (something we at Rosenfeld Media are especially happy to hear)

Our respondents value their network. 

We saw respondents rank “asking a colleague, manager or mentor” as the third most likely place to turn when looking to learn a new skill or concept. 

When we looked at respondents by likelihood and frequency for conference attendance, some clear segments emerged. 

To better focus our qualitative analysis and persona work, we decided to focus on people who were the most likely to attend a conference, and report doing so more than rarely. We observed that the responses from people who were not likely or never attend conferences was less constructive overall for our purposes and also more likely to be throw away responses. By limiting the responses by these two data points, we were able to develop some segments based on job title and organization size for our qualitative researchers to analyze. 

There are five types of people we should be designing this conference for:

  1. Very Experienced Researchers: Individual Contributors and Managers from a variety of organization sizes who are experienced in conducting research.
  2. Somewhat Experienced Researchers: Individual contributors and managers with mid level experience conducting research in a variety of methods in organizations of all sizes. 
  3. Less Experienced Researchers: Individual Contributors and Managers from a variety of organization sizes who are not as experienced in conducting research.
  4. Design Executives and Consultants: Individuals in leadership positions or providing leadership through consulting who are very experienced conducting research and work in organizations of all sizes
  5. Specialists: Individuals pursuing a specialty within design who have a mid level experience conducting research in organizations of all sizes

While they share a lot of qualities, there are balances that we will have to strike to get things right for every attendee. The main similarities and differences we observed can be found in our personas (or watch our team present this work).  

Conference Preferences

 

var divElement = document.getElementById(‘viz1568837571704’); var vizElement = divElement.getElementsByTagName(‘object’)[0]; vizElement.style.width=’950px’;vizElement.style.height=’991px’; var scriptElement = document.createElement(‘script’); scriptElement.src = ‘https://public.tableau.com/javascripts/api/viz_v1.js’; vizElement.parentNode.insertBefore(scriptElement, vizElement);
The above graphic shows how respondents to our survey compare by job title in their responses to questions regarding conference preferences and priorities. You can select which job titles to explore and see how that impacts the size of conferences they prefer, the most important factors in deciding to attend a conference, and their preferred session formats. If you have a finding you don’t see in the report, please comment below or write us at advancingresearch@rosenfeldmedia.com. Feel free to share this graphic; here’s a handy link.

Our audience vastly prefers small to medium sized events. 

Small Conferences (less than 300 people) took the top preference, trailed by retreats/workshops and Mid Size conferences. Large conferences of 500+ was the least preferable of the formats proposed

We now know what’s most important, and to who. 

Topics covered is unsurprisingly the first priority for most, but a more surprising finding in the above data is how the second priority for most is their or their employer’s ability to pay. This confirms something anecdotal that we often talk about in designing RM conferences: “This might be their only conference this year, let’s make it worth it.”

They come for the content, not as much for the socialization. 

While social events might be significant to the experience of attending a conference, very few attendees chose to tell us about their importance in their qualitative responses. In the quantitative data, you can see that content-based formats like full-length talks, keynotes and workshops were by far more preferred on average to things like social events and networking. Probably unsurprisingly the two audiences where social and networking events became almost as important (not more) than content-based sessions was with executives and consultants. Also perhaps unsurprisingly, managers and academics want opportunities to discuss with others, while other folks didn’t find nearly as much importance in those.

Recap of Findings

  • We are designing this conference for a primarily UX audience. 
  • They want to hear from researchers from outside the UX world. 
  • Organization size doesn’t impact content.
  • Methods aren’t all they are looking for. 
  • There is whitespace in the UX world for a conference about research. 
  • Our audience is boots on the ground doing research. 
  • Our audience is fairly senior in research.
  • UX Designers and UX Researchers share a lot of research responsibility. 
  • Our respondents attend events far less frequently than they read or watch videos. 
  • Our respondents value their network.
  • When we looked at respondents by likelihood and frequency for conference attendance, some clear segments emerged.
  • There are five types of people we should be designing this conference for
  • Our audience vastly prefers small to medium-sized events.
  • We now know what’s most important, and to who. 
  • They come for the content, not as much for the socialization. 

What’s Next

We hope that anyone interested in learning more about researchers, and meeting researchers’ information needs, will find our survey results and findings useful. We encourage anyone interested in the data to explore its quantitative components via Tableau

Of course, we’ll be using the data to shape the program of the inaugural Advancing Research conference. It will be held March 30-April 1, 2020, in New York City, and its Call for Submissions, now open, concludes at 11:59pm EDT, Sunday, September 29. Joining the free Advancing Research Community, which offers monthly videoconferences on a variety of topics of interest to researchers, is where we’ll announce details about the conference—including its program—as well as first dibs on discounted tickets.

Additionally, we will share these findings with curators from other conferences that serve researchers.

Acknowledgments

We deeply thank our incredible research team, lead by principal investigator Abby Covert, who volunteered substantial personal time to move the survey from design to data collection to analysis and, ultimately, to the findings contained in this report. Johann Sarmiento took the lead on developing the respondent segments that led to our personas. Rocio Werner, Trisha Thommi, and Neil Mejia developed the insights around each persona. And Sean Oslin helped us understand the meaning of the quantitative data by creating an information visualization tool (via Tableau). As individuals, they are impressive and incredibly professional. As a team, they successfully collaborated despite working remotely across many US states and time zones. We are grateful for their amazing efforts.

Finally, many thanks to the over 700 people who spent 15 minutes on average completing this survey! We are grateful for your generous help, and hope you find the results and our analysis useful.

   —Abby Covert, Natalie Hanson, Steve Portigal, and Lou Rosenfeld (Advancing Research conference program curators)