Measuring the User Experience
Collecting, Analyzing, and Presenting Usability Metrics
- 1st Edition - March 17, 2008
- Imprint: Morgan Kaufmann
- Authors: Bill Albert, Tom Tullis
- Language: English
- eBook ISBN:9 7 8 - 0 - 0 8 - 0 5 5 8 2 6 - 4
Measuring the User Experience provides the first single source of practical information to enable usability professionals and product developers to effectively measure the usabi… Read more
Purchase options
Institutional subscription on ScienceDirect
Request a sales quoteMeasuring the User Experience provides the first single source of practical information to enable usability professionals and product developers to effectively measure the usability of any product by choosing the right metric, applying it, and effectively using the information it reveals.
Authors Tullis and Albert organize dozens of metrics into six categories: performance, issues-based, self-reported, web navigation, derived, and behavioral/physiological. They explore each metric, considering best methods for collecting, analyzing, and presenting the data. They provide step-by-step guidance for measuring the usability of any type of product using any type of technology.
This book is recommended for usability professionals, developers, programmers, information architects, interaction designers, market researchers, and students in an HCI or HFE program.
- Presents criteria for selecting the most appropriate metric for every case
- Takes a product and technology neutral approach
- Presents in-depth case studies to show how organizations have successfully used the metrics and the information they revealed
Usability professionals, developers, programmers, information architects, interaction designers, market researchers, and students in an HCI or HFE program
CHAPTER 1 Introduction
1.1 Organization of This Book
1.2 What Is Usability?
1.3 Why Does Usability Matter?
1.4 What Are Usability Metrics?
1.5 The Value of Usability Metrics
1.6 Ten Common Myths about Usability Metrics
CHAPTER 2 Background
2.1 Designing a Usability Study
2.2 Types of Data
2.3 Metrics and Data
2.4 Descriptive Statistics
2.5 Comparing Means
2.6 Relationships between Variables
2.7 Nonparametric Tests
2.8 Presenting Your Data Graphically
2.9 Summary
CHAPTER 3 Planning a Usability Study
3.1 Study Goals
3.2 User Goals
3.3 Choosing the Right Metrics: Ten Types of Usability Studies
3.4 Other Study Details
3.5 Summary
CHAPTER 4 Performance Metrics
4.1 Task Success
4.2 Time-on-Task
4.3 Errors
4.4 Efficiency
4.5 Learnability
4.6 Summary
CHAPTER 5 Issues-Based Metrics
5.1 Identifying Usability Issues
5.2 What Is a Usability Issue?
5.3 How to Identify an Issue
5.3.1 In-Person Studies
5.4 Severity Ratings
5.5 Analyzing and Reporting Metrics for Usability Issues
5.6 Consistency in Identifying Usability Issues
5.7 Bias in Identifying Usability Issues
5.8 Number of Participants
5.9 Summary
CHAPTER 6 Self-Reported Metrics
6.1 Importance of Self-Reported Data
6.2 Collecting Self-Reported Data
6.3 Post-Task Ratings
6.4 Post-Session Ratings
6.5 Using SUS to Compare Designs
6.6 Online Services
6.7 Other Types of Self-Reported Metrics
6.8 Summary
CHAPTER 7 Behavioral and Physiological Metrics
7.1 Observing and Coding Overt Behaviors
7.2 Behaviors Requiring Equipment to Capture
7.3 Summary
CHAPTER 8 Combined and Comparative Metrics
8.1 Single Usability Scores
8.2 Usability Scorecards
8.3 Comparison to Goals and Expert Performance
8.4 Summary
CHAPTER 9 Special Topics
9.1 Live Website Data
9.2 Card-Sorting Data
9.3 Accessibility Data
9.4 Return-on-Investment Data
9.5 Six Sigma
9.6 Summary
CHAPTER 10 Case Studies
10.1 Redesigning a Website Cheaply and Quickly, Hoa Loranger
10.2 Usability Evaluation of a Speech Recognition IVR, James R. Lewis
10.3 Redesign of the CDC.gov Website Robert Bailey, Cari Wolfson, and Janice Nall
10.4 Usability Benchmarking: Mobile Music and Video, Scott Weiss and Chris Whitby
10.5 Measuring the Effects of Drug Label Design and Similarity on Pharmacists’ Performance, Agnieszka Bojko
10.6 Making Metrics Matter, Todd Zazelenchuk
10.6.7 Acknowledgment
10.6.8 Biography
10.6.9 References
CHAPTER 11 Moving Forward
11.1 Sell Usability and the Power of Metrics
11.2 Start Small and Work Your Way Up
11.3 Make Sure You Have the Time and Money
11.4 Plan Early and Often .
11.5 Benchmark Your Products .
11.6 Explore Your Data
11.7 Speak the Language of Business
11.8 Show Your Confidence
11.9 Don’t Misuse Metrics
11.10 Simplify Your Presentation
1.1 Organization of This Book
1.2 What Is Usability?
1.3 Why Does Usability Matter?
1.4 What Are Usability Metrics?
1.5 The Value of Usability Metrics
1.6 Ten Common Myths about Usability Metrics
CHAPTER 2 Background
2.1 Designing a Usability Study
2.2 Types of Data
2.3 Metrics and Data
2.4 Descriptive Statistics
2.5 Comparing Means
2.6 Relationships between Variables
2.7 Nonparametric Tests
2.8 Presenting Your Data Graphically
2.9 Summary
CHAPTER 3 Planning a Usability Study
3.1 Study Goals
3.2 User Goals
3.3 Choosing the Right Metrics: Ten Types of Usability Studies
3.4 Other Study Details
3.5 Summary
CHAPTER 4 Performance Metrics
4.1 Task Success
4.2 Time-on-Task
4.3 Errors
4.4 Efficiency
4.5 Learnability
4.6 Summary
CHAPTER 5 Issues-Based Metrics
5.1 Identifying Usability Issues
5.2 What Is a Usability Issue?
5.3 How to Identify an Issue
5.3.1 In-Person Studies
5.4 Severity Ratings
5.5 Analyzing and Reporting Metrics for Usability Issues
5.6 Consistency in Identifying Usability Issues
5.7 Bias in Identifying Usability Issues
5.8 Number of Participants
5.9 Summary
CHAPTER 6 Self-Reported Metrics
6.1 Importance of Self-Reported Data
6.2 Collecting Self-Reported Data
6.3 Post-Task Ratings
6.4 Post-Session Ratings
6.5 Using SUS to Compare Designs
6.6 Online Services
6.7 Other Types of Self-Reported Metrics
6.8 Summary
CHAPTER 7 Behavioral and Physiological Metrics
7.1 Observing and Coding Overt Behaviors
7.2 Behaviors Requiring Equipment to Capture
7.3 Summary
CHAPTER 8 Combined and Comparative Metrics
8.1 Single Usability Scores
8.2 Usability Scorecards
8.3 Comparison to Goals and Expert Performance
8.4 Summary
CHAPTER 9 Special Topics
9.1 Live Website Data
9.2 Card-Sorting Data
9.3 Accessibility Data
9.4 Return-on-Investment Data
9.5 Six Sigma
9.6 Summary
CHAPTER 10 Case Studies
10.1 Redesigning a Website Cheaply and Quickly, Hoa Loranger
10.2 Usability Evaluation of a Speech Recognition IVR, James R. Lewis
10.3 Redesign of the CDC.gov Website Robert Bailey, Cari Wolfson, and Janice Nall
10.4 Usability Benchmarking: Mobile Music and Video, Scott Weiss and Chris Whitby
10.5 Measuring the Effects of Drug Label Design and Similarity on Pharmacists’ Performance, Agnieszka Bojko
10.6 Making Metrics Matter, Todd Zazelenchuk
10.6.7 Acknowledgment
10.6.8 Biography
10.6.9 References
CHAPTER 11 Moving Forward
11.1 Sell Usability and the Power of Metrics
11.2 Start Small and Work Your Way Up
11.3 Make Sure You Have the Time and Money
11.4 Plan Early and Often .
11.5 Benchmark Your Products .
11.6 Explore Your Data
11.7 Speak the Language of Business
11.8 Show Your Confidence
11.9 Don’t Misuse Metrics
11.10 Simplify Your Presentation
- Edition: 1
- Published: March 17, 2008
- No. of pages (eBook): 336
- Imprint: Morgan Kaufmann
- Language: English
- eBook ISBN: 9780080558264
BA
Bill Albert
William (Bill) Albert is Senior Vice President and Global Head of Customer Development at Mach49, a growth incubator for global businesses. Prior to joining Mach49, Bill was Executive Director of the Bentley University User Experience Center (UXC) for almost 13 years. Also, he was Director of User Experience at Fidelity Investments, Senior User Interface Researcher at Lycos, and Post-Doctoral Researcher at Nissan Cambridge Basic Research. He has more than twenty years of experience in user experience research, design, and strategy. Bill has published and presented his research at more than 50 national and international conferences, and published in many peer-reviewed academic journals within the fields of User Experience, Usability, and Human-Computer Interaction. In 2010 he co-authored (with Tom Tullis and Donna Tedesco), “Beyond the Usability Lab: Conducting Large-Scale Online User Experience Studies,” published by Elsevier/Morgan Kauffman.
Affiliations and expertise
Director, Design and Usability Center, Bentley University, USATT
Tom Tullis
Thomas S. (Tom) Tullis retired as Vice President of User Experience Research at Fidelity Investments in 2017. Tom was also an Adjunct Professor in Human Factors in Information Design at Bentley University since 2004. He joined Fidelity in 1993 and was instrumental in the development of the company’s User Research department, whose facilities include state-of-the-art Usability Labs. Prior to joining Fidelity, he held positions at Canon Information Systems, McDonnell Douglas, Unisys Corporation, and Bell Laboratories. He and Fidelity’s usability team have been featured in a number of publications, including Newsweek, Business 2.0, Money, The Boston Globe, The Wall Street Journal, and The New York Times.
Affiliations and expertise
Senior Vice President of User Experience, Fidelity Investments, USARead Measuring the User Experience on ScienceDirect