After taking some time off for a new baby boy and traveling to two eMetrics Marketing Optimization Summits in the last month, including speaking for my first time on our book in Madrid at eMetrics Marketing Optimization Summit and I’m glad to be back in NYC and writing again.
So, Lou and I have been talking about this top-down & bottom-up approach to data for awhile now and Lou recently spoke a couple of times on “marrying your data“, but what we haven’t shown you is actually what I call ‘real top-down’ analysis. Given that quantitative data analysis much more my specialty than Lou’s (I am the one with those little Greek letters on my pajamas after all), I thought it was high time I showed everyone what we mean.
Here’s my Keynote presentation for eMetrics Madrid:
The presentation includes a few new things we’ve not discussed previously, such as:
- the 6 components of a search experience and analysis techniques for each
- how to determine the best character length for your search query box
- information foraging and information scent
- search behaviors/patterns (when/why we search)
- how to improve SEO/SEM with search data
- my “scientific method” for determining the quality of a search engine (slide 5)
- and of course some good ‘ol top-down ‘traditional’ search analytics reports & analysis
Please tell us your thoughts and give us some feedback. We’d really love to hear from you on this one.
Earlier today I gave a talk on the need to integrate web analytics and user experience at Janus Boye’s Philadelphia conference. I discussed the differences in perspectives, methods, and use of data among these two communities, as well as an initial start at how they might be integrated over time. Troy Winfrey did a speedy nice job of covering it in CMSwire. You can view and download (in Keynote format, mind you) the slides from Slideshare.
Updated, May 16: Just gave an improved version of this talk as the opening keynote for the IA Konferenz 2009 in Hamburg. I removed the initial slides about the relationship of information architecture and content management, which were there to set context for the J Boye conference, and added some new stuff, mostly a fantastic quote from Tom Chi. (His recent OK/Cancel article is a must-read.)
One of the hardest parts of working on this book has been trying to reconcile WA and UX practices and perspectives. I’m convinced that their sum is far greater than their parts. But how do they fit together?
I’ll be presenting a keynote on this very topic a couple times next month—once in Philadelphia, the other in Hamburg—and we’re covering it in the book. But let me offer a couple simple points for now.
Web Analytics really uncovers the what of a given context. The User-Centered Design methods we UXers rely upon explain that context’s why. Let’s say you’re operating an online store. If 43% of users fail to complete the account creation process, that’s what’s going on. Contextual observation might explain why they’re failing.
What and Why. It’s hard to really justify focusing one without the other. And, in a nutshell, here’s how they fit:
- Web Analytics needs User Experience: It’s not much use to know what is happening if you don’t know why.
- User Experience needs Web Analytics: You can’t know why things are happening if you don’t know what is happening. (Got some good discussion going on this point in Facebook after Tweeting it. Follow all that?)
OK, snarkers, so your reaction might be a big old “DUH!”. Well, how many of your research and design groups have successfully integrated UX and WA? 🙂
These (scroll down to the “SIX MetricsTM Framework”) come from Earley.com’s Jeannine Bartlett, and are really quite good. A nice complement to Lee Romero’s excellent work (here, here, and here) in this area.
Must say, this looks both relevant and fascinating for folks interested in site search analytics: Lorelei Brown and Hallie Wilfert’s pre-conference workshop the morning of March 18 at next week’s IA Summit in Memphis.
Information Archaeology: How to discover the users when talking to them isn’t an option
Design meetings should be about having productive discussions about what our users want, and how we can match the business goals to the users’ needs. Unfortunately, without access to real users, these meetings can be bogged down in details like which business unit is more important, what the CEOs favorite color is, or which corporate brand is hottest.
Too often IAs are brought into projects where they have little or no access to actual users and expected to justify their decisions with real data to elevate themselves out of the “I think”/ “you think” discussion. However, with a little elbow grease and some creativity, you can discover quite a lot about your users without ever talking to them directly!
Most organizations have artifacts on hand that help tell the story of the audiences, but you’ll need to get out your magnifying glass, pith helmet, and your tiny little broom, because we’re going digging for data! We can conduct user research by proxy with information on web traffic, user behavior, and user opinions with free or low-cost tools. Plus, there is also a huge set of Internet usage and user research available to tap into to augment what you can gather.
But it doesn’t just stop there! Learn how to present and justify your data so that you can use your research convincingly in a meeting to support your design choices.
Wish I could make it to SMX Search Analytics later this month. Any event with this much Danny Sullivan on the agenda is well worth considering attending.
Lee Romero is posting an excellent series of articles on search metrics, starting with this one on what he calls basic metrics. For me, these are mostly longitudinal metrics useful for establishing benchmarks and monitoring overall trends over time, rather than basic metrics, but that’s all semantic dithering; they’re quite useful nonetheless.
Very nice article by Hallie Wilfert in the December ’08 FUMSI. (FUMSI. I like to say FUMSI. FUMSI.) Key quotes:
- “…because web analytics data tells what people do on a web site, analytics data should be used to inform and direct more qualitative user research methods such as focus groups or usability tests that tell us why they do what they do. More importantly, you can use the natural interest people have in web metrics to introduce the more qualitative measures into the business overall.”
In effect, the data helps you determine what’s going on and what are the important questions you should use qualitative research methods to answer (what versus why). My only quibble would be mentioning focus groups, which have incredibly narrow value among UX methods, but it’s ultimately immaterial to Hallie’s broader point.
- “Many designers balk at using web analytics because they are intimidated by the numbers and the potential complexity of the analysis. I will share a little secret with you – you do not need to be a statistician to use or interpret web analytics.”
Amen. Leave your T-tests at the door, and Omniture out of your budget. A couple hours with Excel will teach you a lot, even if you’re numbers-averse.
Many thanks for writing it, Hallie.
Avi Rappoport is maintaining a library of (mostly academic) articles on site search analytics. If you want to see what the field’s research minds are thinking about, this is definitely the place to start.
That’s quite a mouthful, but it’s also the title of a very useful presentation given by Erik Verdeyen and David De Block at last September’s EuroIA Conference. Wish I could have been there; looks like this was one of many excellent talks. We need more UX people talking about the combination of UX and web analytics. (I’ve tried.) Anyway, this one is now officially favorited. Hat tip to Victor Lombardi for the link.