First Prototyping Survey Results

Posted on | One comment

  • Back in June 2008, I ran a survey on prototyping. Close to 200 people participated, primarily from the US. The respondents were nearly 60/40 Male/Female and represented a mix of roles in the UX community.

    The goal — challenge some insights I’ve found through interviews, see who’s prototyping, what tools/methods they’re using, find some succss stories, find some failures.

    The result — as expected, a number of assumptions were validated, while others were challenged. I was surprised a bit by the number of people who’s activities were those of a Business Analyst (BAs), which was not an audience I expected to hear from. However, Axure and iRise, two prototyping tools, are relatively common for BAs.

    I’m not shy about the fact that I’m not a big fan of surveys, however, they do have their place. My main dislike for surveys is their lack of ability to properly address the "Why?" question.

    Good survey design is part art, part science. And in hindsight, I would have changed a few things. Something to learn for future surveys:

    • There are a lot of tools used in combination to produce prototypes. Asking participants to weight scale their tools to total 100% might have been helpful.
    • Title is rarely an accurate representation of what people do for a living — at least in the UX field. Close to 20% of people responded perform Business Analyst tasks, but just over 2% self identify as BAs.
    • A large percentage of participants participate in cross-role activities (e.g. design and usability testing). Maybe next time I’ll have role and title as two separate questions, or possibly just dump title all together. I’m really more interested in activities people are performing than what their title is.

    Summary Findings

    • Paper tops the list as the most method, followed by hand-coded HTML is the second most popular method.
    • Paper, Visio, PowerPoint, Dreamweaver, Axure, and OmniGraffle topped the list for most common tools.
    • There were an equal number of people using Dreamweaver and Axure.
    • Visio and PowerPoint are very common tools, especially in larger corporate environments.
    • OmniGraffle and Keynote are very common in mid-small sized companies.
    • Visual designers are more likely to use a tool like Dreamweaver to slap-and-map (link together screens with HTML), or PowerPoint/Keynote to present screens as a prototyping method.
    • Respondants were almost equally split across 1-3 years, 3-5 years, and more than 5 years experience with prototyping.
    • The most important factor for using a tool/method is time and effort to produce a working prototype, followed by a prototype usable for testing, and price.

    I’ll be running some additional surveys in the future, which will be more focused on specific methods and tools.

    Survey Results

    Most Common Methods and Tools for Prototyping  
    Paper 81%
    Hand-coded HTML 58%
    Auto generated from Axure, iRise, Visio, Fireworks, or similar 34%
    Screen shots/comps linked with HTML (slap-and-map) 34%
    Flash, Flex, Blend, or similar 27%
    Keynote or PowerPoint 24%
    Clickable PDF 21%
    Production environment (e.g. Ruby, Java, .Net, PHP, Xcode) 9%
    3D models, cardboard, foam core 1%
    Experience with Paper Prototyping  
    5 or more years 34%
    More than 1, less than 3 years 32%
    More than 3, less than 5 years 17%
    Less than 1 year 10%
    None 7%
    Experience with Digital Prototyping  
    5 or more years 48%
    More than 1, less than 3 years 24%
    More than 3, less than 5 years 21%
    Less than 1 year 5%
    None 2%

    Most Important Factors in a Prototyping Tool/Method

    1. Time and effort to produce a working prototype.
    2. Creating usable prototypes for testing.
    3. Price
    4. Learning curve.
    5. Ability to create own GUI widgets and patterns.
    6. Platform independence.
    7. Collaborative/remote design capabilities.
    8. Built-in solutions/patterns (e.g. AJAX transitions)
    9. Built-in GUI widgets.
    10. Creating usable source code.
    Tools Used in Prototyping  
    Paper 77%
    Visio 59%
    PowerPoint 43%
    Dreamweaver 47%
    Axure 30%
    OmniGraffle 30%
    Illustrator 23%
    Flash 21%
    Acrobat 19%
    Fireworks 18%
    InDesign 12%
    Photoshop 10%
    Other HTML editor 4%
    Keynote 3%
    iRise 0.7%
    Primarily Hi-fi vs. Lo-fi Prototypes  
    Hi-fi 47%
    Lo-fi 42%
    50/50 11%
    Male 62%
    Female 38%
    Interaction Designer/IA 63.2%
    Product Manager 8.3%
    Presentation Layer Developer 6.3%
    Other 6.3%
    Researcher, Design Researcher, Usability Analyst 5.6%
    UX Manager 3.5%
    Application Developer 3.5%
    BA 2.1%
    Visual Designer 1.4%

    One Response to “First Prototyping Survey Results”

    1. Hi there, it would be interesting to have an update on this study… For sure Visio will get out of the top 3.

      Regards, Thomas