{"id":220,"date":"2022-01-12T18:55:08","date_gmt":"2022-01-12T18:55:08","guid":{"rendered":"https:\/\/staging.rm.gfolkdev.net\/advancing-research-community\/2022\/01\/12\/how-we-researched-and-designed-the-advancing-research-conference\/"},"modified":"2022-02-20T15:14:57","modified_gmt":"2022-02-20T15:14:57","slug":"how-we-researched-and-designed-the-advancing-research-conference","status":"publish","type":"resources","link":"https:\/\/rosenfeldmedia.com\/advancing-research-community\/resources\/how-we-researched-and-designed-the-advancing-research-conference\/","title":{"rendered":"How we Researched and Designed the Advancing Research Conference"},"content":{"rendered":"
There are more researchers focused on users, customers, and products than ever before, and those researchers\u2019 needs are changing. Rosenfeld Media has already interacted with researchers through the <\/span>Advancing Research community<\/span><\/a>, which we curate, but wanted to learn more about who they are, where they work, and how they learn. And\u2014as a conference-production company\u2014we wanted to know what researchers seek from conference experiences.<\/span><\/p>\n Our survey ran from July 2-15, 2019, and 721 people responded. We\u2019ve summarized the results below, and have made our quantitative data available for analysis via <\/span>Tableau<\/span><\/a>.<\/span><\/p>\n We began our research with the following assumptions about research and researchers involved in learning about how people use products and services:<\/span><\/p>\n New technologies enable ever more powerful and complex products and services. They also create yawning gaps between the organizations that create products and the people who use them. Researchers help close those gaps, so the demand for people who can do research is dramatically increasing.<\/span><\/p>\n There are not enough professionally trained researchers to meet the growing demand. And research tools are getting easier to use while simultaneously becoming more powerful. So more people with minimal or no training are being pressed into research roles.<\/span><\/b><\/p>\n Large organizations with complex offerings require greater research capacity than ever before\u2014and are building large research teams to meet internal demand. In turn, those research teams require a level of operational and managerial support than ever before.<\/span><\/p>\n As research becomes more valued, research leaders are taking on increasingly strategic roles within their organizations\u2014and often aren\u2019t prepared to lead.<\/span><\/p>\n As products and services become more complex, more types of researchers, research tools and methods are needed to study problems and synthesize the results from multiple sources. Accordingly, researchers are becoming less anchored to their home disciplines, research methods and tools, and perspective. And they\u2019re becoming more focused on solving complex problems using whatever approaches make the most sense, regardless of provenance.\u00a0<\/span><\/b><\/p>\n These are exciting developments, and are leading to a flowering of ideas, conversations, and activities for and among researchers of all stripes.<\/p>\n Rosenfeld Media is a provider of UX content, education, and conferences, and has a track record of supporting emerging communities that share our goal of creating better experiences for humanity (e.g., <\/span>design and research operations<\/span><\/a>, and <\/span>enterprise design<\/span><\/a>). We wanted to explore the opportunity to create a new conference to meet researchers\u2019 needs, especially for the US market. We\u2019ve already been learning about researchers\u2019 needs through informal conversations within Rosenfeld Media\u2019s <\/span>Advancing Research community<\/span><\/a>. A broad survey seemed to be a logical next step.\u00a0<\/span><\/p>\n We chose a variety of questions, both open and close-ended, to learn about who responded, their experience, and their preferred and actual means of learning. We also drilled deeper into one of the learning approach most interesting to Rosenfeld Media: conferences.\u00a0<\/span><\/p>\n We were most interested in understanding how job title and organization size relate to how researchers learn new skills and think about conference attendance. The main objectives covered in the survey were:<\/strong><\/span><\/p>\n Our questions fell into four main categories<\/strong>:<\/span><\/p>\n Our survey was created primarily by Lou Rosenfeld, with assistance from Abby Covert, Natalie Hanson, and Steve Portigal. We knew the survey would be long, but hoped that the effort to complete it would leave respondents in a better place; i.e., they would learn something about themselves and their own interests by completing the activity. We also made clear to respondents that we would share the data and our findings publicly.<\/span><\/p>\n The survey was set up in Wufoo, and was pilot tested with 12 subjects. Pilot testers required 15 minutes to complete the survey on average. After revisions, the final survey was launched on July 2, 2019, and closed July 15, 2019. It was promoted in a variety of ways\u2014via Rosenfeld Media\u2019s mailing list and social media channels, via the team\u2019s contacts and social media, and to the Advancing Research community. Special efforts were made to socialize the survey to contacts and colleagues who might in turn be able to share it with communities outside the team\u2019s core network of UX people.\u00a0<\/span><\/p>\n While we expected roughly 200 responses, we received 721\u2014so many that we were tempted to close the survey earlier than we\u2019d planned. That said, we wish more people from outside the UX community would have participated.<\/span><\/p>\n The following section takes the qualitative responses and quantitative responses into account and presents the specific findings and recommendations that our team will be using to make decisions as we design the first <\/span>Advancing<\/span> Research conference<\/span><\/a> (which will be held March 30-April 1, 2020, in New York City)<\/span>.<\/span><\/p>\n As you go through this section you will see some graphs and charts powered by Tableau (instrumented by Sean Oslin). You can click onto various elements in the charts and graphs to pivot the data that you are displaying. We have included specific instructions below each interactive graphic.<\/span><\/p>\n Below is a summary of the insights<\/strong> our team got from our analysis of the survey. Each is expanded on below.<\/span><\/p>\n var divElement = document.getElementById(‘viz1568867146395’); var vizElement = divElement.getElementsByTagName(‘object’)[0]; vizElement.style.width=’950px’;vizElement.style.height=’991px’; var scriptElement = document.createElement(‘script’); scriptElement.src = ‘https:\/\/public.tableau.com\/javascripts\/api\/viz_v1.js’; vizElement.parentNode.insertBefore(scriptElement, vizElement); <\/p>\n This graphic shows how respondents to our survey compare in terms of job title, organization size and # of employees doing research at their organization. You can click on any job title and see how organization size and # of employees changes. You can click on organization size and see the job title breakdown or # of employees doing research within that organization size. If you have a finding you don\u2019t see in the report, please comment below or write us at advancingresearch@rosenfeldmedia.com. Feel free to share this graphic; here’s a handy link.<\/a><\/span><\/i><\/p>\n Through this research activity we realized that our access to relevant audiences for a conference about research at this point is primarily UX people. While we tried to broaden the reach of this survey, this exercise reminded us how important building a network is in reaching different types of people. Because the background and primary audience of Rosenfeld Media is those working in the UX space, our team will have to strategize about the best ways to reach outside the UX world when curating and marketing this conference, both now and in the future. We would love to see the audience of this conference diversify over time to include a higher percentage of those practicing research outside UX, but for the first year we anticipate UX people will be our primary audience. Our initial efforts to broaden our coverage will begin with how we design our inaugural program, and who we get to speak. We hope such focused efforts will draw attendees from outside UX over the coming years.<\/span><\/p>\n We learned in the qualitative responses that UX Researchers (both by title and by task) want to learn about research methods and best practices outside of the UX space and they want to hear from and meet researchers who are working in adjacent fields. So even though we are designing this conference for a majority UX-audience, the content should <\/span>not<\/b> be majority UX-derived.<\/span><\/p>\n Regardless of job type, organization size or other conference preferences, our audience share more similarities than differences when it came to content that they expected to see at a conference about research.\u00a0<\/span><\/p>\n The top three shared areas of content were:\u00a0\u00a0\u00a0<\/span><\/p>\n While our respondents are interested in learning about methods, that\u2019s not the only thing they would want to have in terms of content for a research conference. We heard that people wanted a research conference to have content around big topics like diversity, inclusion, ethics and social impact. We also heard about non-research-specific skills that people want to work on such as leadership, management, change management and other soft skills. Also interesting is that there isn\u2019t a huge delta between what people are currently using in terms of methods and what they are looking to explore further. Of the types of subjects of interest they had for a research conference, only a small percentage were related to methods. <\/p>\n <\/p>\n Many of our potential audience have not been to a conference dedicated to research (at least not recently). Local meetups like CHI and UXPA are some of the most common research-focused events for our respondents. <\/span>None <\/span><\/i>was actually the most popular answer to the question \u201cWhat (if any) events have you attended on the subject of research in the past few years?\u201d\u00a0<\/span><\/p>\n var divElement = document.getElementById(‘viz1568840363182’); var vizElement = divElement.getElementsByTagName(‘object’)[0]; vizElement.style.width=’950px’;vizElement.style.height=’991px’; var scriptElement = document.createElement(‘script’); scriptElement.src = ‘https:\/\/public.tableau.com\/javascripts\/api\/viz_v1.js’; vizElement.parentNode.insertBefore(scriptElement, vizElement); <\/p>\n This graphic shows how respondents to our survey compare in terms how much experience they have and how much their current roles require each main research task. The most interesting view here is to narrow by job role to see how frequency and experience vary. If you have a finding you don\u2019t see in the report, please comment below or write us at advancingresearch@rosenfeldmedia.com. Feel free to share this graphic; here’s a\u00a0handy link<\/a>.<\/span><\/i><\/p>\n <\/p>\n There was a surprisingly low number of respondents hiring vendors and\/or buying research reports in their current roles. But across all job titles we saw a high number advocating for, planning, managing, conducting, observing and analyzing research as frequent parts of their current roles.<\/span><\/p>\n The majority of our respondents reported five or more years experience in analyzing, conducting, planning, observing and advocating for research. We also see a high concentration of people who are currently teaching research, whether that be within their organizations or academically.<\/span><\/p>\n UX Designers reported similar frequency on advocating for, analyzing, conducting, and planning research as UX Researchers.\u00a0<\/span><\/p>\n <\/p>\n var divElement = document.getElementById(‘viz1568840421213’); var vizElement = divElement.getElementsByTagName(‘object’)[0]; vizElement.style.width=’950px’;vizElement.style.height=’991px’; var scriptElement = document.createElement(‘script’); scriptElement.src = ‘https:\/\/public.tableau.com\/javascripts\/api\/viz_v1.js’; vizElement.parentNode.insertBefore(scriptElement, vizElement); <\/p>\n This graphic shows how respondents to our survey compare in terms of how often and likely they are to participate in a set of learning activities. You can select which job titles to explore and see how that impacts the size conferences they prefer, the most important factors in deciding to attend a conference and their preferred session formats. If you have a finding you don\u2019t see in the report, please comment below or write us at advancingresearch@rosenfeldmedia.com.\u00a0Feel free to share this graphic; here’s a\u00a0handy link<\/a>.<\/span><\/i><\/p>\n This data further confirms that our audience is attending very few in-person events. It also confirms the need for long form content to support this audience in addition to in person events (something we at Rosenfeld Media are especially happy to hear)<\/span><\/p>\n We saw respondents rank \u201casking a colleague, manager or mentor\u201d as the third most likely place to turn when looking to learn a new skill or concept.\u00a0<\/span><\/p>\n To better focus our qualitative analysis and persona work, we decided to focus on people who were the most likely to attend a conference, and report doing so more than rarely. We observed that the responses from people who were not likely or never attend conferences was less constructive overall for our purposes and also more likely to be throw away responses. By limiting the responses by these two data points, we were able to develop some segments based on job title and organization size for our qualitative researchers to analyze.\u00a0<\/span><\/p>\n While they share a lot of qualities, there are balances that we will have to strike to get things right for every attendee. The main similarities and differences we observed can be found in <\/span>our personas<\/span><\/a> (or <\/span>watch our team present this work<\/span>).\u00a0\u00a0<\/span><\/p>\n <\/p>\n var divElement = document.getElementById(‘viz1568837571704’); var vizElement = divElement.getElementsByTagName(‘object’)[0]; vizElement.style.width=’950px’;vizElement.style.height=’991px’; var scriptElement = document.createElement(‘script’); scriptElement.src = ‘https:\/\/public.tableau.com\/javascripts\/api\/viz_v1.js’; vizElement.parentNode.insertBefore(scriptElement, vizElement); Small Conferences (less than 300 people) took the top preference, trailed by retreats\/workshops and Mid Size conferences. Large conferences of 500+ was the least preferable of the formats proposed<\/span><\/p>\n Topics covered is unsurprisingly the first priority for most, but a more surprising finding in the above data is how the second priority for most is their or their employer\u2019s ability to pay. This confirms something anecdotal that we often talk about in designing RM conferences: <\/span>\u201cThis might be their only conference this year, let\u2019s make it worth it.\u201d<\/span><\/i><\/p>\nWhy we decided to research researchers<\/span><\/h2>\n
There are more researchers doing research than ever before.<\/strong><\/h5>\n
There are more \u201cnon-researchers\u201d doing research than ever before. <\/b><\/h5>\n
Research teams are getting larger.<\/strong><\/h5>\n
Researchers are getting their seat at the table. <\/b><\/h5>\n
Research is becoming increasingly cross-disciplinary. <\/b><\/h5>\n
What we set out to learn<\/span><\/h2>\n
\n
\n
How the survey was designed<\/span><\/h2>\n
How our data informs conference design<\/span><\/h2>\n
\n
We are designing this conference for a primarily UX audience. For now.\u00a0<\/strong><\/h4>\n
UX people want to hear from researchers from outside the UX world.\u00a0<\/strong><\/h4>\n
Research methods are a primary content expectation for our audience.\u00a0<\/strong><\/h4>\n
\n
But, methods aren\u2019t all<\/i> they are looking for.<\/strong><\/h4>\n
\n<\/span><\/p>\nThere is whitespace in the UX world for a conference about research.\u00a0<\/strong><\/h4>\n
Our audience is boots on the ground doing research.<\/strong><\/h4>\n
Our audience is fairly senior in research.\u00a0<\/strong><\/h4>\n
UX Designers and UX Researchers share a lot of research responsibility.\u00a0<\/strong><\/h4>\n
Learning Activities<\/span><\/h2>\n
Our respondents attend events far less frequently than they read or watch videos.\u00a0<\/strong><\/h4>\n
Our respondents value their network.<\/strong>\u00a0<\/span><\/h4>\n
When we looked at respondents by likelihood and frequency for conference attendance, some clear segments emerged.<\/strong>\u00a0<\/span><\/h4>\n
There are five types of people we should be designing this conference for:<\/strong><\/h4>\n
\n
Conference Preferences<\/span><\/h2>\n
\nThe above graphic shows how respondents to our survey compare by job title in their responses to questions regarding conference preferences and priorities. You can select which job titles to explore and see how that impacts the size of conferences they prefer, the most important factors in deciding to attend a conference, and their preferred session formats. If you have a finding you don\u2019t see in the report, please comment below or write us at advancingresearch@rosenfeldmedia.com.\u00a0Feel free to share this graphic; here’s a\u00a0handy link<\/a>.<\/span><\/i><\/p>\nOur audience vastly prefers small to medium sized events.\u00a0<\/strong><\/h4>\n
We now know what\u2019s most important, and to who.\u00a0<\/strong><\/h4>\n