User experience and the analysts

As part of our ongoing research of the UX environment, we recently took a closer look at the six major analyst firms (Aberdeen, AMR, Forrester, Gartner, IDC, and Yankee). We were hoping to determine if the analysts were paying much attention to user experience, so we searched a variety of UX-related terms (21, to be precise) on their respective web sites. We then looked at which firms paid attention to which UX topics, how these firms stacked up against each other, and how they compared to the web’s overall UX consciousness.

The following table summarizes our general findings (highly recommended: view the full-size version—much easier to read). We used Google searches as our yardstick to gauge our UX terms’ relative strengths across the web. So, for example, of all the UX-related documents that Google knows about, 10.7% mention knowledge management. We did the same thing for each analyst site, using their own local search systems, and then we averaged those numbers:

Which numbers really stood out among the analyst firms?

• Aberdeen is focusing on web analytics, which represents 23.7% of their UX-related documents (versus 3.8% on average for all the analysts).
• AMR pays a great deal of attention to the related areas of content management (38% versus 25.8% on average) and knowledge management (21.8% versus 11.9%).
• Forrester appears relatively strong in areas that are relatively new, such as experience design (2% versus 0.7%), interaction design (2.1% versus 0.2%), interface design (2.5% versus 0.9%), SEO (2.8% versus 0.5%), UCD (2.7% versus 0.2%), and web analytics (8.4% versus 3.8%). User experience, itself a recent term, is the most common term among Forrester’s search results (16.4% versus 11%).
• Gartner’s bread and butter is information management (34% versus 25%); not surprisingly, they also take the top position in information architecture (5.6% versus 2.6%).
• IDC’s numbers are, overall, closest to average; if they have a specific focus, it’s content management (32.6% versus 25.8%).
• Branding seems to dominate Yankee’s mindspace (50.9% versus 13.9%). Perhaps they’re also quite interested in the branding aspects of user experience (23.4% versus 11%).

In terms of the relative strengths of each UX topic, the analysts seem focused on organizing information; the top two topics are content management (25.8%) and information management (25%), with knowledge management coming at #4 (11.9%). So it’s surprising that information architecture checked in at a low 2.6%. And, in a different way, it’s surprising that the user experience, often confused with information architecture, ranks #5 at 11%. Branding took position #3 at 13.7%, which seems reasonable as analysts’ clients typically have large and established marketing budgets. At 3.8%, web analytics is in sixth place; while this may sound low, it’s probably rising quickly. It’ll be interesting to check back in a year.

Our second illustration shows how the analysts, on average, stack up against the Web’s UX consciousness as a whole (as expressed in Google search results):

It’s interesting to look at the discrepancies here. The analysts “lead” the web on these topics:

• Experience design (0.7% versus 0.2%)
• Industrial design (1% versus 0.3%)
• Information architecture (2.6% versus 0.2%)
• Information management (25% versus 11.1%)
• Interaction design (0.9% versus 0.3%)
• Search analytics (0.2% versus 0.02%)
• User experience (11% versus 7.9%)

With the exception of information management, the analysts’ numbers are small, though larger than those witnessed on the web as a whole. So are the analysts leaders here? It’s possible, but quite hard to tell without longitudinal data; we’ll revisit the numbers in a year.

What about topics where the analysts trail the web?

• Ergonomics (2.1% for the web versus 0.7% for the analysts)
• Graphic design (13.4% versus 0.7%)
• Human-computer interaction (2.6% versus 0.2%)
• Search engine optimization (2.4% versus 0.5%)
• Technical communication (0.2% versus 0.05%)
• Usability engineering (0.1% versus 0.04%)
• Web analytics (9.1% versus 3.8%)

Interestingly these topics, with the exception of SEO and web analytics, all represent fairly established fields. Do analysts forgo these areas as insufficiently innovative? If so, many of these field’s practitioners would surely take issue. Conversely, SEO and web analytics are new fields which are not only considered quite innovative, but have accrued legions of software vendors. Given that the mission of many analysts is to help managers understand new technologies, it’s especially strange that the analysts have not paid more attention to these two newer areas.

We’re sure there are plenty of problems with our interpretation of these data, so we’d especially like to hear your thoughts. Let us know if you’d like the raw data and we’ll send you the spreadsheet. We hope to repeat this study again in a year, and we’re considering doing the same thing in other areas where we’d like to gauge UX mindshare, so please let us know how we can do better.