In the first survey I ran for the book, someone made this great comment:
“The boss simply went through them going yup, yup, yup, yup, yup, yup, putting them in piles whilst the junior looked on. The boss didn’t really think much, and the junior just nodded.”
Does that sound like something you may have said? If so, please contact me as I’d like to include the quote in the book and would like to attribute it (cardsorting at maadmob dot net)
Recently I talked with Gerry Gaffney, from Information & Design for his UXPod show about card sorting. The interview is now ready. In it I talk about some of my approaches to card sorting, why I don’t do closed card sorts or use dendrograms and what you should do if doing a card sort for the first time.
There are a lot of other great interviews & discussions about UX topics, so go listen to those as well.
I figured out the magic secret to getting great chunks of writing done. Here it is:
- move to somewhere near Canberra
- book a very early morning flight out of Canberra, in the middle of winter
- miss the text message from the airline telling you the flight has been cancelled due to fog
- drive all the way to the airport (45 mins drive), find out that the flight has been cancelled and re-book for a flight 2.5 hours later
- sit in the qantas club lounge for 4.5 hours drinking tea, coffee, hot chocolate and juice, with earphones in to drown out the chatter and where the internet is too expensive to bother
I got so much done. Much more than at home.
After a few days of reading boring articles about statistics, looking at dendrograms, pulling data apart and seeing how things work, I finally understand the basics of cluster analysis. I’m not sure whether my description of it and how to read dendrograms makes sense, but at least I basically get it.
Some conclusions about cluster analysis and card sorting tools:
- Cluster analysis is a particularly poor way to visualise outcomes from a card sort
- It would be possible for card sorting tools to allow sub-grouping and still use cluster analysis in the output (I always figured this is why tools didn’t include sub-groups)
- Current card sorting tools do nothing to help people understand the results they display. There is no information about the method and little transparency
- To do better statistical analysis, I’d need a much deeper understanding of statistical methods than I currently have (or want to have) and some very expensive statistical analysis tools
- Until someone adds good analysis capabilities to card sorting tools, practitioners will continue to be ignorant of better statistical analysis methods
No wonder everyone tells me they find it hard to analyse the outcomes of a card sort!
As I mentioned last week, I’m currently writing about analysis. And in order to do so, I’m putting together an example to demonstrate analysis methods, cluster diagrams and more.
I’m also planning on including a chapter on card sorting tools, and will be critiqueing each available tool.
I’m not fond of doing more work than I have to, so am combining the two – a card sort using software tools, then using the results in the analysis section.
So I read this draft chapter in a really neat card sorting book, and it said I need participants who are end-users of the product you are developing (that sounds like you!). It also had a sample introduction script that goes something like this (details filled in):
“As part of my role of program chair for the IA Summit, I’m currently looking at different ways to organise some of the information on the IA Summit website. An important part of the project is to make old presentations more available and make sure people can easily find presentations that will help them. One way we learn about this is to do an activity called a card sort, and I’d like you to be involved.
In a card sort, I give you a set of ‘cards’ that show information on the website and asking you to sort them into groups. I can then look at how you think about the information groupings and how information could be organised.
You don’t need to have any special knowledge, don’t need to do any preparation and there are no right or wrong answers. It should take around half an hour.
This card sort will be run with a range of software tools – some via the internet, some via installed software. You can use whichever tool you prefer.”
So, there’s the invite. Would you like to be involved? The content is previous presentations from the IA Summit, so I’m looking for people who know about information architecture. You do not need to have been to the summit.
I’ll be using these tools at least:
- USort/EZCalc (an old IBM product, needs to be installed)
- Card Sword (needs to be installed)
- WebSort (via the internet)
- Card Zort (needs to be installed)
- UZ Card Sorting (I think via the internet)
I could also post physical cards to people in Australia.
So, if you are interested in helping out, please email me at cardsorting -at- maadmob -dot- net. Let me know which tool you would prefer to use and I’ll send out instructions and files.
I will also be making the outputs available in full.
I talked to a bunch of author friends at the IA Summit this year, trying to find out just what was so hard about writing and why they all go overboard thanking their partners. I’m going to say naively that I haven’t hit any of those problems so far (so I’m expecting a particularly nasty time some time later this year).
Would you believe it – I have written five chapters of the book – everything from an overview to planning to running a card sort (who’d have thought that would take 5 chapters). Even more amazing is that I’m relatively happy with them. Unfortunately, I am now at the hard point – the point at which I have to describe how to analyse results and use them in a project. Have you ever tried to teach someone to analyse – it is like trying to teach them how to think, an incredibly hard thing to do, and even harder at a distance in writing. But I’m getting there.
Writing has been fun. Really! I have taken a little time from client work to do it so have a block of time every week. It is my most fun time of week. I turn up the music, drink tea and write. Words go out of my fingers, onto the computer, then I fix them all up. It is an amazing feeling and I am enjoying it immensely.
Anyway, sorry for this gratuitously personal post, but a bunch of people have asked me how I’m going, so I thought I’d fill you in. Stay tuned for real card sorting advice in future posts.
Thank you to everyone who filled out the survey about their card sorting experiences. There were 139 responses, which is amazing.
I have summarised the results here. I won’t provide the full unedited results from the survey as many of them were quite specific (and also because I said up front that I wouldn’t do this). But if you are interested in anything below, please email me.
I asked about the number of sorts you had conducted in the last 2 years. The average was 4 (median 2), although some of the high numbers told me that maybe I hadn’t been clear that I was interested in the number of activities you had run, not the number of participants (or some people card sort a lot).
Most people conduct only manual card sorts (60%) and some conduct only software-based sorts (10%). Many people commented that they conduct the sort manually and analyse it with a tool, an option I hadn’t thought about to include in the survey.
Most popular tools were:
- USort/EZCalc (the no-longer-supported program from IBM)
- Joe Lamantia’s analysis spreadsheet
The most rewarding aspects of the activity are (this is my interpretation of detailed answers):
- Face-to-face time with real users
- Observing the group activity – learning as much from the discussions as the data
- Learning unexpected things from users
- Seeing different ways of grouping information
- It is so easy to run
- Being somewhat transparent about the IA design process
The most challenging thing, by far, is analysing the data from the activity. Many responses gave me the impression tthat the other big challenge is designing the information architecture – not directly related to card sorting alone, but raised as a particular problem. Key communication challenges include selling the activity, communicating the results to others and showing how the results informed the design. Some people also mentioned that choosing the content to include is difficult, particularly for large sites.
These are all aspects that will be the focus of the book, so I hope to be able to provide a lot of guidance and tips for these particular challenges.
I also asked if I could talk to you for user research, or for case studies, and many people offerred one or both. Because the response was higher than expected and very detailed, I probably won’t follow up everyone who offerred to be involved in general user research. But I will follow up some individual comments and offers of case studies – within the next few weeks.
Thanks again if you contributed!
The survey about about your card sorting experiences is still open (until Sunday 7 May 2006). If you haven’t filled it in already, please do so – it helps me to make sure I’m answering the right questions. If you have filled it in already, thank you very much.
I’ll post a summary of results next week.
I spent most of last week writing. This surprises even me – to sit down and write for long periods was a very strange experience and actually made me feel a bit like a writer (lucky that!).
I got words down for a substantial chunk of the book – not very good words, but words nonetheless. In doing so, I did three things you may be interested in:
- Found academic publications I had never read before:
These are now in the bibliography, which I’ll continue to build as I go. Many of these are locked in subscription-only journals, which is the perennial problem for practitioners.
- Thought hard about the focus of the book:
Many of the existing resources talk about card sorting as an abstract or research-focused technique. These don’t tend to answer the majority of questions practitioners face. So I’m focusing on card sorting as a practical technique to be used in the design of information environments (not as a psychological technique). Focusing here really helps me to set a direction and pull a set of practical theories and ideas together. It even feels good to me!
- Started preparing an analysis spreadsheet:
I have a spreadsheet I’ve been using for a while to store the master card set (which mail-merges into Word to create labels), record outcomes for each sort, play with the outcomes and show some clustering statistics (not a ‘cluster analysis’ diagram, some basic percentages). I have a lot of work to do on it to explain and document it, but will make it available before the book is out.
I’m very excited to announce that, between now and the end of the year, I will be writing a book on card sorting. Specifically, it will be about how to use card sorting in information architecture and similar projects. I think it will fill an important gap – there is some material about how to run a card sort, but little on how to actually use the outputs.
This book will be one of the first published by Rosenfeld Media. We’ll be incorporating users’ input as much as possible throughout the writing process, primarily via this blog and website for the book. You can keep up to date via the RSS feed:
I’d love to hear from you about your experiences conducting card sorts and have set up a survey to help collect this. It is short – 10 questions – and should take no more than 10 minutes:
Card sorting – your experiences (survey)
I’m really looking forward to working on the book, and hope you’ll consider participating by sharing your suggestions. Send comments and ideas via comment box on the website or to me at cardsorting – at- maadmob -dot- net