Conformal Prediction for Reliable Machine Learning
Theory, Adaptations and Applications
- 1st Edition - April 29, 2014
- Latest edition
- Editors: Vineeth Balasubramanian, Shen-Shyang Ho, Vladimir Vovk
- Language: English
The conformal predictions framework is a recent development in machine learning that can associate a reliable measure of confidence with a prediction in any real-world pattern… Read more
The conformal predictions framework is a recent development in machine learning that can associate a reliable measure of confidence with a prediction in any real-world pattern recognition application, including risk-sensitive applications such as medical diagnosis, face recognition, and financial risk prediction. Conformal Predictions for Reliable Machine Learning: Theory, Adaptations and Applications captures the basic theory of the framework, demonstrates how to apply it to real-world problems, and presents several adaptations, including active learning, change detection, and anomaly detection. As practitioners and researchers around the world apply and adapt the framework, this edited volume brings together these bodies of work, providing a springboard for further research as well as a handbook for application in real-world problems.
- Understand the theoretical foundations of this important framework that can provide a reliable measure of confidence with predictions in machine learning
- Be able to apply this framework to real-world problems in different machine learning settings, including classification, regression, and clustering
- Learn effective ways of adapting the framework to newer problem settings, such as active learning, model selection, or change detection
Professors, Research Professors, Assistants/Associates, Professional Research/R&D Engineers, Research Scientists, Graduate Students. Clinical researchers, researchers in biomedical informatics, national security, financial analysts, undergraduate students of applied machine learning, pattern recognition, artificial intelligence, clinical informatics
- Contributing Authors
- Foreword
- Preface
- Book Organization
- Part I: Theory
- Part II: Adaptations
- Part III: Applications
- Companion Website
- Contacting Us
- Acknowledgments
- Part 1: Theory
- Chapter 1. The Basic Conformal Prediction Framework
- Abstract
- Acknowledgments
- 1.1 The Basic Setting and Assumptions
- 1.2 Set and Confidence Predictors
- 1.3 Conformal Prediction
- 1.4 Efficiency in the Case of Prediction without Objects
- 1.5 Universality of Conformal Predictors
- 1.6 Structured Case and Classification
- 1.7 Regression
- 1.8 Additional Properties of Validity and Efficiency in the Online Framework
- Chapter 2. Beyond the Basic Conformal Prediction Framework
- Abstract
- Acknowledgments
- 2.1 Conditional Validity
- 2.2 Conditional Conformal Predictors
- 2.3 Inductive Conformal Predictors
- 2.4 Training Conditional Validity of Inductive Conformal Predictors
- 2.5 Classical Tolerance Regions
- 2.6 Object Conditional Validity and Efficiency
- 2.7 Label Conditional Validity and ROC Curves
- 2.8 Venn Predictors
- Chapter 1. The Basic Conformal Prediction Framework
- Part 2: Adaptations
- Chapter 3. Active Learning
- Abstract
- Acknowledgments
- 3.1 Introduction
- 3.2 Background and Related Work
- 3.3 Active Learning Using Conformal Prediction
- 3.4 Experimental Results
- 3.5 Discussion and Conclusions
- Chapter 4. Anomaly Detection
- Abstract
- 4.1 Introduction
- 4.2 Background
- 4.3 Conformal Prediction for Multiclass Anomaly Detection
- 4.4 Conformal Anomaly Detection
- 4.5 Inductive Conformal Anomaly Detection
- 4.6 Nonconformity Measures for Examples Represented as Sets of Points
- 4.7 Sequential Anomaly Detection in Trajectories
- 4.8 Conclusions
- Chapter 5. Online Change Detection
- Abstract
- 5.1 Introduction
- 5.2 Related Work
- 5.3 Background
- 5.4 A Martingale Approach for Change Detection
- 5.5 Experimental Results
- 5.6 Implementation Issues
- 5.7 Conclusions
- Chapter 6. Feature Selection
- Abstract
- 6.1 Introduction
- 6.2 Feature Selection Methods
- 6.3 Issues in Feature Selection
- 6.4 Feature Selection for Conformal Predictors
- 6.5 Discussion and Conclusions
- Chapter 7. Model Selection
- Abstract
- Acknowledgments
- 7.1 Introduction
- 7.2 Background
- 7.3 SVM Model Selection Using Nonconformity Measure
- 7.4 Nonconformity Generalization Error Bound
- 7.5 Experimental Results
- 7.6 Conclusions
- Chapter 8. Prediction Quality Assessment
- Abstract
- Acknowledgments
- 8.1 Introduction
- 8.2 Related Work
- 8.3 Generalized Transductive Reliability Estimation
- 8.4 Experimental Results
- 8.5 Discussion and Conclusions
- Chapter 9. Other Adaptations
- Abstract
- Acknowledgments
- 9.1 Introduction
- 9.2 Metaconformal Predictors
- 9.3 Single-Stacking Conformal Predictors
- 9.4 Conformal Predictors for Time Series Analysis
- 9.5 Conclusions
- Chapter 3. Active Learning
- Part 3: Applications
- Chapter 10. Biometrics and Robust Face Recognition
- Abstract
- 10.1 Introduction
- 10.2 Biometrics and Forensics
- 10.3 Face Recognition
- 10.4 Randomness and Complexity
- 10.5 Transduction
- 10.6 Nonconformity Measures for Face Recognition
- 10.7 Open and Closed Set Face Recognition
- 10.8 Watch List and Surveillance
- 10.9 Score Normalization
- 10.10 Recognition-by-Parts Using Transduction and Boosting
- 10.11 Reidentification Using Sensitivity Analysis and Revision
- 10.12 Conclusions
- Chapter 11. Biomedical Applications: Diagnostic and Prognostic
- Abstract
- Acknowledgments
- 11.1 Introduction
- 11.2 Examples of Medical Diagnostics
- 11.3 Nonconformity Measures for Medical and Biological Applications
- 11.4 Discussion and Conclusions
- Chapter 12. Network Traffic Classification and Demand Prediction
- Abstract
- 12.1 Introduction
- 12.2 Network Traffic Classification
- 12.3 Network Demand Prediction
- 12.4 Experimental Results
- 12.5 Conclusions
- Chapter 13. Other Applications
- Abstract
- 13.1 Nuclear Fusion Device Applications
- 13.2 Sensor Device Applications
- 13.3 Sustainability, Environment, and Civil Engineering
- 13.4 Security Applications
- 13.5 Applications from Other Domains
- Chapter 10. Biometrics and Robust Face Recognition
- Bibliography
- Index
"...captures the basic theory of the framework, demonstrates how to apply it to real-world problems, and presents several adaptations, including active learning, change detection, and anomaly detection."—Zentralblatt MATH, Sep-14
"...the book is highly recommended for people looking for formal machine learning techniques that can guarantee theoretical soundness and reliability."—Computing Reviews,December 4,2014
"This book captures the basic theory of the framework, demonstrates how the framework can be applied to real-world problems, and also presents several adaptations of the framework…" —HPCMagazine.com, August 2014
- Edition: 1
- Latest edition
- Published: April 29, 2014
- Language: English
VB
Vineeth Balasubramanian
Vineeth N Balasubramanian is an Assistant Professor in the Department of Computer Science and Engineering at the Indian Institute of Technology, Hyderabad, India. Until July 2013, he was an Assistant Research Professor at the Center for Cognitive Ubiquitous Computing (CUbiC) at Arizona State University (ASU). He holds dual Masters degrees in Mathematics (2001) and Computer Science (2003) from Sri Sathya Sai University, India, and worked at Oracle Corporation for two years until 2005. His PhD dissertation (2010) on the Conformal Predictions framework was nominated for the Outstanding PhD Dissertation at the Department of Computer Science at ASU, as well as for the annual ACM Doctoral Dissertation Award. He was also a recipient of the Gold Medals for Academic Excellence for his performances in the Bachelors program in Math in 1999, and for his Masters program in Computer Science in 2003. His research interests include pattern recognition, machine learning, computer vision and multimedia computing within assistive and healthcare applications. His current research includes extending the Conformal Predictions framework to real-world problem contexts, and newer machine learning problems such as active learning and transfer learning.
Affiliations and expertise
Department of Computer Science and Engineering, Indian Institute of Technology, Hyderabad, IndiaSH
Shen-Shyang Ho
Shen-Shyang Ho is an Assistant Professor in the School of Computer Engineering at the Nanyang Technological University (NTU), Singapore. Before joining NTU in January 2012, he was an assistant research scientist at the University of Maryland Institute for Advanced Computer Studies (UMIACS). He received the BS degree in mathematics and computational science from the National University of Singapore in 1999, and the MS and PhD degrees in computer science from George Mason University in 2003 and 2007, respectively. He was formerly a NASA Postdoctoral Program (NPP) fellow affiliated to the Jet Propulsion Laboratory (JPL) and a postdoctoral scholar affiliated to the California Institute of Technology working in the Climate, Oceans, and Solid Earth Science section in the science division at JPL. His research interests include learning from data streams/sequences, adaptive learning, pattern recognition, data mining in spatio-temporal domain and moving objects databases, and machine learning/data mining on mobile devices.
Affiliations and expertise
School of Computer Engineering, Nanyang Technological University, SingaporeVV
Vladimir Vovk
Vladimir Vovk is Professor of Computer Science at Royal Holloway, University of London; he also heads the Computer Learning Research Centre. His research interests include machine learning; predictive and Kolmogorov complexity, randomness, and information; the foundations of probability and statistics. He has published numerous research papers in these fields and two books: "Probability and finance: It's only a game" (with Glenn Shafer, Wiley, New York, 2001; Japanese translation: Iwanami Shoten, Tokyo, 2006) and "Algorithmic learning in a random world" (with Alex Gammerman and Glenn Shafer, Springer, New York, 2005), which is a comprehensive book on the Conformal Predictions framework.
Affiliations and expertise
Department of Computer Science, Royal Holloway University of London, UKRead Conformal Prediction for Reliable Machine Learning on ScienceDirect