Dimensionality Reduction in Machine Learning

1st Edition - February 4, 2025
Imprint: Morgan Kaufmann
Editors: Jamal Amani Rad, Snehashish Chakraverty, Kourosh Parand
Language: English
Paperback ISBN:
9 7 8 - 0 - 4 4 3 - 3 2 8 1 8 - 3
eBook ISBN:
9 7 8 - 0 - 4 4 3 - 3 2 8 1 9 - 0

Dimensionality Reduction in Machine Learning covers both the mathematical and programming sides of dimension reduction algorithms, comparing them in various aspects. Part One pr… Read more

Purchase options

World Book Day Celebration!

Save up to 30% on books and eBooks!

Shop now

Institutional subscription on ScienceDirect

Request a sales quote

Dimensionality Reduction in Machine Learning covers both the mathematical and programming sides of dimension reduction algorithms, comparing them in various aspects. Part One provides an introduction to Machine Learning and the Data Life Cycle, with chapters covering the basic concepts of Machine Learning, essential mathematics for Machine Learning, and the methods and concepts of Feature Selection. Part Two covers Linear Methods for Dimension Reduction, with chapters on Principal Component Analysis and Linear Discriminant Analysis. Part Three covers Non-Linear Methods for Dimension Reduction, with chapters on Linear Local Embedding, Multi-dimensional Scaling, and t-distributed Stochastic Neighbor Embedding.

Finally, Part Four covers Deep Learning Methods for Dimension Reduction, with chapters on Feature Extraction and Deep Learning, Autoencoders, and Dimensionality reduction in deep learning through group actions. With this stepwise structure and the applied code examples, readers become able to apply dimension reduction algorithms to different types of data, including tabular, text, and image data.

Title of Book
Cover image
Title page
Table of Contents
Copyright
Dedication
Contributors
About the Editors
Preface
Part 1: Introduction to machine learning and data lifecycle
1: Basics of machine learning
1.1. Data processing in machine learning (ML)
1.1.1. What is data? Feature? Pattern?
1.1.2. Understanding data processing
1.1.3. High-dimensional data
1.2. Types of learning problems
1.2.1. Supervised learning
1.2.2. Unsupervised learning
1.2.3. Semi-supervised learning
1.2.4. Reinforcement learning
1.3. Machine learning algorithms lifecycle
1.4. Python for machine learning
1.4.1. Python and packages installation
2: Essential mathematics for machine learning
2.1. Vectors
2.1.1. Basic concepts
2.1.2. Linear independence
2.1.3. Orthogonality
2.2. Matrices
2.2.1. Basic concepts
2.2.2. Operations
2.2.3. Some definitions
2.2.4. Important matrix properties
2.2.5. Determinant
2.2.6. Row and column spaces
2.2.7. Rank of a matrix
2.3. Vector and matrix norms
2.3.1. Vector norms
2.3.2. Matrix norms
2.4. Eigenvalues and eigenvectors
2.4.1. A system of linear equations
2.4.2. Calculation of eigenvalues and eigenvectors
2.4.3. Cayley–Hamilton theorem
2.5. Matrix centering
2.6. Orthogonal projection
2.7. Definition of gradient
2.8. Definition of the Hessian matrix
2.9. Definition of a Jacobian
2.10. Optimization problem
2.10.1. Feasible solutions
2.10.2. Lagrangian function
2.10.3. Karush–Kuhn–Tucker conditions
Part 2: Linear methods for dimension reduction
3: Principal and independent component analysis methods
3.1. Introduction
3.1.1. History
3.1.2. Intuition
3.2. The PCA algorithm
3.2.1. Projection in one-dimensional space
3.2.2. Projection in two-dimensional space
3.2.3. Projection in r-dimensional space
3.2.4. Example
3.2.5. Additional discussion about PCA
3.3. Implementation
3.3.1. How to implement PCA algorithm in Python?
3.3.2. Parameter options
3.3.3. Attribute options
3.4. Advantages and limitations
3.5. Unveiling hidden dimensions in data
3.5.1. The need for kernel PCA
3.5.2. Discovering nonlinear relationships
3.5.3. Dimensionality reduction and disclosing hidden dimensions
3.6. The Kernel PCA algorithm
3.6.1. Data preprocessing
3.6.2. Kernel selection
3.6.3. Kernel matrix calculation
3.6.4. Centering data points in feature space
3.6.5. Example
3.7. Implementation of Kernel PCA
3.8. Independent Component Analysis
3.8.1. The Cocktail Party Problem
3.8.2. A comparison between PCA and ICA
3.8.3. Theoretical background of ICA
3.8.4. The ICA model
3.8.5. Algorithms for ICA
3.8.6. Ambiguity in ICA
3.8.7. Example
3.8.8. Example of implementing ICA
3.9. Conclusion
4: Linear discriminant analysis
4.1. Introduction to linear discriminant analysis
4.1.1. What is linear discriminant analysis?
4.1.2. How does linear discriminant analysis work?
4.1.3. Application of linear discriminant analysis
4.2. Understanding the LDA algorithm
4.2.1. Prerequisite
4.2.2. Fisher's linear discriminant analysis
4.2.3. Linear algebra explanation
4.3. The advanced linear discriminant analysis algorithm
4.3.1. Statistical explanation
4.3.2. Linear discriminant analysis compared to principal component analysis
4.3.3. Quadratic discriminant analysis
4.4. Implementing the linear discriminant analysis algorithm
4.4.1. Using LDA with Scikit-Learn
4.5. LDA parameters and attributes in Scikit-Learn
4.5.1. Parameter options
4.5.2. Attributes option
4.5.3. Worked example of linear discriminant analysis algorithm for dimensionality
4.5.4. Fitting LDA algorithm on MNIST dataset
4.5.5. LDA advantages and limitations
4.6. Conclusion
Part 3: Nonlinear methods for dimension reduction
5: Linear local embedding
5.1. Introduction
5.1.1. What is nonlinear dimensionality reduction?
5.1.2. Why do we need nonlinear dimensionality reduction?
5.1.3. What is embedding?
5.2. Locally linear embedding
5.2.1. k nearest neighbors
5.2.2. Distance metrics
5.2.3. Weights
5.2.4. Coordinates
5.3. Variations of LLE
5.3.1. Inverse LLE
5.3.2. Kernel LLE
5.3.3. Incremental LLE
5.3.4. Robust LLE
5.3.5. Weighted LLE
5.3.6. Landmark LLE for big data
5.3.7. Supervised and semi-supervised LLE
5.3.8. LLE with other manifold learning methods
5.4. Implementation and use cases
5.4.1. How to use LLE in Python?
5.4.2. Using LLE in MNIST
5.5. Conclusion
6: Multi-dimensional scaling
6.1. Basics
6.1.1. Introduction to multi-dimensional scaling
6.1.2. Data in MDS
6.1.3. Proximity and distance
6.2. MDS models
6.2.1. Metric MDS
6.2.2. Torgerson's method
6.2.3. Least square model
6.2.4. Non-metric MDS
6.2.5. The goodness of fit
6.2.6. Individual differences models
6.2.7. INDSCAL
6.2.8. Tucker–Messick model
6.2.9. PINDIS
6.2.10. Unfolding models
6.2.11. Non-metric uni-dimensional scaling
6.3. Kernel-based MDS
6.4. MDS in practice
6.4.1. MDS in Python
6.4.2. Conclusion
7: t-Distributed stochastic neighbor embedding
7.1. Introduction to t-SNE
7.1.1. What is t-SNE?
7.1.2. Why is t-SNE useful?
7.1.3. Prerequisite
7.1.4. Applications of t-SNE
7.2. Understanding the t-SNE algorithm
7.2.1. The t-SNE perplexity parameter
7.2.2. The t-SNE objective function
7.2.3. The t-SNE learning rate
7.2.4. Implementing t-SNE in practice
7.3. Visualizing high-dimensional data with t-SNE
7.3.1. Choosing the right number of dimensions
7.3.2. Interpreting t-SNE plots
7.4. Advanced t-SNE techniques
7.4.1. Using t-SNE for data clustering
7.4.2. Combining t-SNE with other dimensionality reduction methods
7.5. Conclusion and future directions
Part 4: Deep learning methods for dimensionality reduction
8: Feature extraction and deep learning
8.1. The revolutionary history of deep learning: from biology to simple perceptrons and beyond
8.1.1. A brief history
8.1.2. Biological neurons
8.1.3. Artificial neurons: the perceptron
8.2. Deep neural networks
8.2.1. Deep feedforward networks
8.2.2. Convolutional networks
8.3. Learned features
8.3.1. Visualizing learned features
8.3.2. Deep feature extraction
8.3.3. Deep feature extraction applications
8.4. Conclusion
9: Autoencoders
9.1. Introduction to autoencoders
9.1.1. Generative modeling
9.2. Autoencoders for feature extraction
9.2.1. Latent variable
9.2.2. Representation learning
9.2.3. Feature learning approaches
9.3. Types of autoencoders
9.3.1. Denoising autoencoder (DAE)
9.3.2. Sparse autoencoder (SAE)
9.3.3. Contractive autoencoder (CAE)
9.3.4. Variational autoencoder (VAE)
9.4. Autoencoder and learned features applications
9.4.1. Language encoding
9.4.2. Vision models
9.4.3. Convolutional autoencoder
9.5. Conclusion
10: Dimensionality reduction in deep learning through group actions
10.1. Introduction
10.2. Geometric context of deep learning
10.3. Group actions, invariant and equivariant maps
10.4. Equivariant neural networks
10.4.1. Group equivariant neural networks
10.4.2. The general theory of group equivariant neural networks
10.5. Implementation of equivariant neural networks
10.5.1. Implementing groups and actions
10.5.2. Implementing equivariant convolution layers
10.6. Conclusion
Index

Jamal Amani Rad

Dr. Jamal Amani Rad currently works in Choice Modelling Centre and Institute for Transport Studies, University of Leeds, Leeds LS2 9JT, UK He obtained his PhD in Mathematics at the Department of Mathematics at University of Shahid Beheshti. His research interests include modelling, numerics, and analysis of partial differential equations by using meshless methods, with an emphasis on applications from finance.

Affiliations and expertise

Choice Modelling Centre and Institute for Transport Studies, University of Leeds, Leeds LS2 9JT, UK

Snehashish Chakraverty

Snehashish Chakraverty has thirty-one years of experience as a researcher and teacher. Presently, he is working in the Department of Mathematics (Applied Mathematics Group), National Institute of Technology Rourkela, Odisha, as a senior (Higher Administrative Grade) professor. Dr Chakraverty received his PhD in Mathematics from IIT-Roorkee in 1993. Thereafter, he did his post-doctoral research at the Institute of Sound and Vibration Research (ISVR), University of Southampton, UK, and at the Faculty of Engineering and Computer Science, Concordia University, Canada. He was also a visiting professor at Concordia and McGill Universities, Canada, during 1997–1999 and visiting professor at the University of Johannesburg, Johannesburg, South Africa, during 2011–2014. He has authored/co-authored/edited 33 books, published 482 research papers (till date) in journals and conferences. He was the president of the section of mathematical sciences of Indian Science Congress (2015–2016) and was the vice president of Orissa Mathematical Society (2011–2013). Prof. Chakraverty is a recipient of prestigious awards, viz. “Careers360 2nd Faculty Research Award” for the Most Outstanding Researcher in the country in the field of Mathematics, Indian National Science Academy (INSA) nomination under International Collaboration/Bilateral Exchange Program (with the Czech Republic), Platinum Jubilee ISCA Lecture Award (2014), CSIR Young Scientist Award (1997), BOYSCAST Fellow. (DST), UCOST Young Scientist Award (2007, 2008), Golden Jubilee Director’s (CBRI) Award (2001), INSA International Bilateral Exchange Award (2015), Roorkee University Gold Medals (1987, 1988) for first positions in MSc and MPhil (Computer Application). He is in the list of 2% world scientists (2020 to 2024) in the Artificial Intelligence and Image Processing category based on an independent study done by Stanford University scientists.

Affiliations and expertise

HAG Professor, Department of Mathematics, Applied Mathematics Group, National Institute of Technology Rourkela, Rourkela, Odisha, India. Differential Equations (ordinary, partial, and fractional), Numerical Analysis, Computational Methods, Structural Dynamics (FGM, Nano), Fluid Dynamics, Mathematical and Uncertainty Modelling, Soft Computing and Machine Intelligence (Artificial Neural Network, Fuzzy, Interval, and Affine Computations).

Kourosh Parand

Dr. Kourosh Parand is a Professor in International Business University, Toronto, Canada . His main research field is Scientific Computing, Spectral Methods, Meshless methods, Ordinary Differential Equations (ODEs), Partial Differential Equations(PDEs) and Computational Neuroscience Modeling.

Affiliations and expertise

Professor, International Business University, Toronto, Canada

Life Sciences

Physical Sciences & Engineering

Social Sciences & Humanities

Health

Dimensionality Reduction in Machine Learning

Purchase options

World Book Day Celebration!

Institutional subscription on ScienceDirect

Jamal Amani Rad

Snehashish Chakraverty

Kourosh Parand

Related books