Measuring the User Experience
Collecting, Analyzing, and Presenting Usability Metrics
- 2nd Edition - May 23, 2013
- Authors: Bill Albert, Tom Tullis
- Language: English
Measuring the User Experience was the first book that focused on how to quantify the user experience. Now in the second edition, the authors include new material on how recent te… Read more
Measuring the User Experience was the first book that focused on how to quantify the user experience. Now in the second edition, the authors include new material on how recent technologies have made it easier and more effective to collect a broader range of data about the user experience.
As more UX and web professionals need to justify their design decisions with solid, reliable data, Measuring the User Experience provides the quantitative analysis training that these professionals need. The second edition presents new metrics such as emotional engagement, personas, keystroke analysis, and net promoter score. It also examines how new technologies coming from neuro-marketing and online market research can refine user experience measurement, helping usability and user experience practitioners make business cases to stakeholders. The book also contains new research and updated examples, including tips on writing online survey questions, six new case studies, and examples using the most recent version of Excel.
- Learn which metrics to select for every case, including behavioral, physiological, emotional, aesthetic, gestural, verbal, and physical, as well as more specialized metrics such as eye-tracking and clickstream data
- Find a vendor-neutral examination of how to measure the user experience with web sites, digital products, and virtually any other type of product or system
- Discover in-depth global case studies showing how organizations have successfully used metrics and the information they revealed
- Companion site, www.measuringux.com, includes articles, tools, spreadsheets, presentations, and other resources to help you effectively measure the user experience
Usability and User Experience practitioners, human factor professionals, and market researchers, business analysts, interaction and visual designers, information architects, managers within technology companies, students/ instructors in Human Factors/ HCI
Dedication
Preface to the Second Edition
Acknowledgments
Biographies
Chapter 1. Introduction
1.1 What is User Experience
1.2 What are User Experience Metrics?
1.3 The Value of UX Metrics
1.4 Metrics for Everyone
1.5 New Technologies in UX Metrics
1.6 Ten Myths about UX Metrics
Chapter 2. Background
2.1 Independent and Dependent Variables
2.2 Types of Data
2.3 Descriptive Statistics
2.4 Comparing Means
2.5 Relationships Between Variables
2.6 Nonparametric Tests
2.7 Presenting your Data Graphically
2.8 Summary
Chapter 3. Planning
3.1 Study Goals
3.2 User Goals
3.3 Choosing the Right Metrics: Ten Types of Usability Studies
3.4 Evaluation Methods
3.5 Other Study Details
3.6 Summary
Chapter 4. Performance Metrics
4.1 Task Success
4.2 Time on Task
4.3 Errors
4.4 Efficiency
4.5 Learnability
4.6 Summary
Chapter 5. Issue-Based Metrics
5.1 What is a Usability Issue?
5.2 How to Identify an Issue
5.3 Severity Ratings
5.4 Analyzing and Reporting Metrics for Usability Issues
5.5 Consistency in Identifying Usability Issues
5.6 Bias in Identifying Usability Issues
5.7 Number of Participants
5.8 Summary
Chapter 6. Self-Reported Metrics
6.1 Importance of Self-Reported Data
6.2 Rating Scales
6.3 Post-Task Ratings
6.4 Postsession Ratings
6.5 Using Sus to Compare Designs
6.6 Online Services
6.7 Other Types of Self-Reported Metrics
6.8 Summary
Chapter 7. Behavioral and Physiological Metrics
7.1 Observing and Coding Unprompted Verbal Expressions
7.2 Eye Tracking
7.3 Measuring Emotion
7.4 Stress and Other Physiological Measures
7.5 Summary
Chapter 8. Combined and Comparative Metrics
8.1 Single Usability Scores
8.2 Usability Scorecards
8.3 Comparison to Goals and Expert Performance
8.4 Summary
Chapter 9. Special Topics
9.1 Live Website Data
9.2 Card-Sorting Data
9.3 Accessibility Data
9.4 Return-On-Investment Data
9.5 Summary
Chapter 10. Case Studies
10.1 Net Promoter Scores and the Value of a Good User Experience
10.2 Measuring the Effect of Feedback on Fingerprint Capture
Acknowledgment
10.3 Redesign of a Web Experience Management System
10.4 Using Metrics to Help Improve a University Prospectus
Acknowledgments
10.5 Measuring Usability Through Biometrics
Acknowledgments
Chapter 11. Ten Keys to Success
11.1 Make Data Come Alive
11.2 Don’t Wait to be Asked to Measure
11.3 Measurement is Less Expensive than You Think
11.4 Plan Early
11.5 Benchmark Your Products
11.6 Explore Your Data
11.7 Speak the Language of Business
11.8 Show Your Confidence
11.9 Don’t Misuse Metrics
11.10 Simplify Your Presentation
References
Index
"This book discusses various efforts to identify, collect, analyze, improve, and present metrics that can be used to measure usability…It is well made. It is also fun to read. I recommend it to managers interested in or in charge of user experience design for their products."—ComputingReviews.com, January 2014
"…the source of the data in this book is focused more on the data that you collect while performing user research, but the bottom line is that it will help you improve your data analysis skills…this is a great reference book when you feel stuck with a set of user research data, but cannot figure out how to derive value from it."—Actual Insights blog, October 2013
"This is a how-to guide not a theoretical treatise, they say, providing practical advice about what metrics to collect in what situations, how to collect them, how to make sense of the data using various analysis techniques, and how to present the results in the clearest and most compelling way."—Reference & Research Book News, October 2013
"This book is a great resource about the many ways to gather usability metrics without busting your budget. If you're ready to take your company’s user experience process to the next maturity level, Tullis and Albert are here for you and share generously of their vast experience. Highly recommended."—Jakob Nielsen, Principal of Nielsen Norman Group, author of Usability Engineering and Mobile Usability
"A great second edition, with updated content based on new research and completely new case studies. If you work in the field of user experience, you should buy this book, read it, and use it."—James R. (Jim) Lewis, Ph.D., CHFP, Senior Human Factors Engineer, IBM Software Group
"This book is amazing. It's everything you need to know about measuring the user experience. If you are a user experience professional you must read it. Not skim — read. It's well written, and very thorough. I'll be keeping it close at hand as my "go-to"book."—Susan Weinschenk, Ph.D., Founder of the Weinschenk Institute
- Edition: 2
- Published: May 23, 2013
- Language: English
BA
Bill Albert
William (Bill) Albert is Senior Vice President and Global Head of Customer Development at Mach49, a growth incubator for global businesses. Prior to joining Mach49, Bill was Executive Director of the Bentley University User Experience Center (UXC) for almost 13 years. Also, he was Director of User Experience at Fidelity Investments, Senior User Interface Researcher at Lycos, and Post-Doctoral Researcher at Nissan Cambridge Basic Research. He has more than twenty years of experience in user experience research, design, and strategy. Bill has published and presented his research at more than 50 national and international conferences, and published in many peer-reviewed academic journals within the fields of User Experience, Usability, and Human-Computer Interaction. In 2010 he co-authored (with Tom Tullis and Donna Tedesco), “Beyond the Usability Lab: Conducting Large-Scale Online User Experience Studies,” published by Elsevier/Morgan Kauffman.
Affiliations and expertise
Director, Design and Usability Center, Bentley University, USATT
Tom Tullis
Thomas S. (Tom) Tullis retired as Vice President of User Experience Research at Fidelity Investments in 2017. Tom was also an Adjunct Professor in Human Factors in Information Design at Bentley University since 2004. He joined Fidelity in 1993 and was instrumental in the development of the company’s User Research department, whose facilities include state-of-the-art Usability Labs. Prior to joining Fidelity, he held positions at Canon Information Systems, McDonnell Douglas, Unisys Corporation, and Bell Laboratories. He and Fidelity’s usability team have been featured in a number of publications, including Newsweek, Business 2.0, Money, The Boston Globe, The Wall Street Journal, and The New York Times.
Affiliations and expertise
Senior Vice President of User Experience, Fidelity Investments, USARead Measuring the User Experience on ScienceDirect