Quantifying the User Experience
Practical Statistics for User Research
- 1st Edition - March 16, 2012
- Imprint: Morgan Kaufmann
- Authors: Jeff Sauro, James R Lewis
- Language: English
- Paperback ISBN:9 7 8 - 0 - 1 2 - 3 8 4 9 6 8 - 7
- eBook ISBN:9 7 8 - 0 - 1 2 - 3 8 4 9 6 9 - 4
Quantifying the User Experience: Practical Statistics for User Research offers a practical guide for using statistics to solve quantitative problems in user research. Many design… Read more
Purchase options
Institutional subscription on ScienceDirect
Request a sales quoteQuantifying the User Experience: Practical Statistics for User Research offers a practical guide for using statistics to solve quantitative problems in user research. Many designers and researchers view usability and design as qualitative activities, which do not require attention to formulas and numbers. However, usability practitioners and user researchers are increasingly expected to quantify the benefits of their efforts. The impact of good and bad designs can be quantified in terms of conversions, completion rates, completion times, perceived satisfaction, recommendations, and sales.The book discusses ways to quantify user research; summarize data and compute margins of error; determine appropriate samples sizes; standardize usability questionnaires; and settle controversies in measurement and statistics. Each chapter concludes with a list of key points and references. Most chapters also include a set of problems and answers that enable readers to test their understanding of the material. This book is a valuable resource for those engaged in measuring the behavior and attitudes of people during their interaction with interfaces.
- Provides practical guidance on solving usability testing problems with statistics for any project, including those using Six Sigma practices
- Show practitioners which test to use, why they work, best practices in application, along with easy-to-use excel formulas and web-calculators for analyzing data
- Recommends ways for practitioners to communicate results to stakeholders in plain English
- Resources and tools available at the authors’ site: http://www.measuringu.com/
Usability and user experience practitioners, software and web-development professionals, marketers, information architects, interaction designers, business analysts, market researchers, and students in these and related fields
Dedication Acknowledgments About the Authors 1. Introduction and How to Use This Book Introduction The Organization of This Book How to Use This Book Key Points from the Chapter Chapter Review Questions References 2. Quantifying User Research What is User Research? Data from User Research Usability Testing A/B Testing Survey Data Requirements Gathering Key Points from the Chapter References 3. How Precise Are Our Estimates? Confidence Intervals Introduction Confidence Interval for a Completion Rate Confidence Interval for Rating Scales and Other Continuous Data Key Points from the Chapter Chapter Review Questions References 4. Did We Meet or Exceed Our Goal? Introduction One-Tailed and Two-Tailed Tests Comparing a Completion Rate to a Benchmark Comparing a Satisfaction Score to a Benchmark Comparing a Task Time to a Benchmark Key Points from the Chapter Chapter Review Questions References 5. Is There a Statistical Difference between Designs? Introduction Comparing Two Means (Rating Scales and Task Times) Comparing Completion Rates, Conversion Rates, and A/B Testing Key Points from the Chapter Chapter Review Questions References 6. What Sample Sizes Do We Need? Introduction Estimating Values Comparing Values What can I Do to Control Variability? Sample Size Estimation for Binomial Confidence Intervals Sample Size Estimation for Chi-Square Tests (Independent Proportions) Sample Size Estimation for McNemar Exact Tests (Matched Proportions) Key Points from the Chapter Chapter Review Questions References 7. What Sample Sizes Do We Need? Introduction Using a Probabilistic Model of Problem Discovery to Estimate Sample Sizes for Formative User Research Assumptions of the Binomial Probability Model Additional Applications of the Model What affects the Value of p? What Is a Reasonable Problem Discovery Goal? Reconciling the “Magic Number 5” with “Eight Is Not Enough” More about the Binomial Probability Formula and Its Small Sample Adjustment Other Statistical Models for Problem Discovery Key Points from the Chapter Chapter Review Questions References 8. Standardized Usability Questionnaires Introduction Poststudy Questionnaires Post-task Questionnaires Questionnaires for Assessing Perceived Usability of Websites Other Questionnaires of Interest Key Points from the Chapter Chapter Review Questions References 9. Six Enduring Controversies in Measurement and Statistics Introduction Is It Okay to Average Data from Multipoint Scales? Do You Need to Test at Least 30 Users? Should You Always Conduct a Two-Tailed Test? Can You Reject the Null Hypothesis When p > 0.05? Can You Combine Usability Metrics into Single Scores? What If You Need to Run More Than One Test? Key Points from the Chapter Chapter Review Questions References 10. Wrapping Up Introduction Getting More Information Good Luck! Key Points from the Chapter References APPENDIX. A Crash Course in Fundamental Statistical Concepts Introduction Types Of Data Populations and Samples Measuring Central Tendency Standard Deviation and Variance The Normal Distribution Area Under the Normal Curve Applying the Normal Curve to User Research Data Central Limit Theorem Standard Error of the Mean Margin of Error t-Distribution Significance Testing and p-Values The Logic of Hypothesis Testing Errors in Statistics Key Points from the Appendix Index
- Edition: 1
- Published: March 16, 2012
- No. of pages (eBook): 312
- Imprint: Morgan Kaufmann
- Language: English
- Paperback ISBN: 9780123849687
- eBook ISBN: 9780123849694
JS
Jeff Sauro
Dr. Jeff Sauro is a six-sigma trained statistical analyst and founding principal of MeasuringU, a customer experience research firm based in Denver. For over fifteen years he’s been conducting usability and statistical analysis for companies such as Google, eBay, Walmart, Autodesk, Lenovo and Drobox or working for companies such as Oracle, Intuit and General Electric.
Jeff has published over twenty peer-reviewed research articles and five books, including Customer Analytics for Dummies. He publishes a weekly article on user experience and measurement online at measuringu.com.
Jeff received his Ph.D in Research Methods and Statistics from the University of Denver, his Masters in Learning, Design and Technology from Stanford University, and B.S. in Information Management & Technology and B.S. in Television, Radio and Film from Syracuse University. He lives with his wife and three children in Denver, CO.
Affiliations and expertise
Usability Metrics and Statistical Analyst, Measuring Usability LLC, CO, USAJL
James R Lewis
Dr. James R. (Jim) Lewis is a senior human factors engineer (at IBM since 1981) with a current focus on the measurement and evaluation of the user experience. He is a Certified Human Factors Professional with a Ph.D. in Experimental Psychology (Psycholinguistics), an M.A. in Engineering Psychology, and an M.M. in Music Theory and Composition. Jim is an internationally recognized expert in usability testing and measurement, contributing (by invitation) the chapter on usability testing for the 3rd and 4th editions of the Handbook of Human Factors and Ergonomics, presenting tutorials on usability testing and metrics at various professional conferences, and serving as the keynote speaker at HCII 2014. He was the lead interaction designer for the product now regarded as the first smart phone, the Simon, and is the author of Practical Speech User Interface Design.
Jim is an IBM Master Inventor Emeritus with 88 patents issued to date by the US Patent Office. He serves on the editorial board of the International Journal of Human-Computer Interaction, is co-editor in chief of the Journal of Usability Studies, and is on the scientific advisory board of the Center for Research and Education on Aging and Technology Enhancement (CREATE). He is a member of the Usability Professionals Association (UPA), the Human Factors and Ergonomics Society (HFES), the ACM Special Interest Group in Computer-Human Interaction (SIGCHI), past-president of the Association for Voice Interaction Design (AVIxD), and is a 5th degree black belt and certified instructor with the American Taekwondo Association (ATA).
Affiliations and expertise
Senior Human Factors Engineer, IBM, FL, USARead Quantifying the User Experience on ScienceDirect