Chemometrics in Spectroscopy

1. A New Beginning

Section 1 Elementary Matrix Algebra

2. Elementary Matrix Algebra, Part 1: Primitive operations: Addition, Subtraction, Multiplication, Division, Inverse, Transpose

3. Elementary Matrix Algebra, Part 2: Elementary Operations,Inverse of a Matrix

Section 2 Matrix Algebra and Multiple Linear Regression

4. Matrix Algebra and Multiple Linear Regression: Part 1 Quasi-Algebraic Operations, Multiple Linear Regression, The Least Squares Method

5. Matrix Algebra and Multiple Linear Regression: Part 2 When There Are More Equations Than Unknowns, The Power of Matrix Mathematics

6. Matrix Algebra and Multiple Linear Regression: Part 3 The Concept of Determinants

7. Matrix Algebra and Multiple Linear Regression: Part 4 Concluding Remarks, and A Word of Caution

Section 3 Experimental Designs

8. Experimental Designs, Part 1: Introduction

9. Experimental Designs, Part 2: One-way ANOVA

10. Experimental Designs, Part 3: Two-factor Designs

11. Experimental Designs, Part 4: Varying Parameters to Expand the Design

12. Experimental Designs Part 5: One-at-a-time Designs

13. Experimental Designs, Part 6: Sequential designs

14. Experimental Designs, Part 7: β, the Power of a Test

15. Experimental Designs, Part 8: β, the Power of a Test (continued)

16. Experimental Designs, Part 9: Sequential Designs (concluded)

Section 4 Analytic Geometry

17. Analytic Geometry: Part 1: The Basics in Two and Three Dimensions

18. Analytic Geometry: Part 2: Geometric Representation of Vectors and Algebraic Operation

19. Analytic Geometry: Part 3: Reducing Dimensionality

20. Analytic Geometry: Part 4: The Geometry of Vectors and Matrices

Section 5 Regression Techniques

21. Calculating the Solution for Regression Techniques: Part 1: Multivariate Regression Made Simple

22. Calculating the Solution for Regression Techniques: Part 2: Principal Component(s) Regression Made Simple

23. Calculating the Solution for Regression Techniques: Part 3: Partial Least Squares Made Simple

24. Calculating the Solution for Regression Techniques: Part 4: Singular Value Decomposition

25. Interlude: Looking Behind and Ahead

26. A Simple Question

27. Challenges: Unsolved Problems in Chemometrics

Section 6 Linearity in Calibration

28. Linearity in Calibration, Act I: A Thought Experiment Carried Out by Computer Simulation

29. Linearity in Calibration, Act II Scene I: A Firestorm Erupts and A Theoretical Explanation of Linearity

30. Linearity in Calibration, Act II Scene II: Details of Reader Responses

31. Linearity in Calibration, Act II Scene III: Summary of Reader Responses, and Our Commentary on Those Responses

32. Linearity in Calibration, Act II Scene IV: A Summary of Findings and Recommendations for Future Explorations

33. Linearity in Calibration, Act II Scene V: Effect of (Non) Linearity on PLS Algorithm

Section 7 Collaborative Laboratory Studies

34. Collaborative Laboratory Studies: Part 1 - A Blueprint

35. Collaborative Laboratory Studies: Part 2 - Using ANOVA

36. Collaborative Laboratory Studies: Part 3 - Testing for Systematic Error

37. Collaborative Laboratory Studies: Part 4 - Ranking Test

38. Collaborative Laboratory Studies: Part 5 - Efficient Comparison of Two Methods

39. Collaborative Laboratory Studies: Part 6 - MathCad Worksheet Text

Section 8 Analysis of Noise

40. Is Noise Brought by the Stork? Analysis of Noise - Part 1—A Listing of the Sources of Spectroscopic Noise and Their Characteristics

41. Analysis of Noise - Part 2—The analysis of the effect of ‘constant’ detector noise on a transmission measurement

42. Analysis of Noise - Part 3—The Analysis of the Effect of ‘constant’ Detector Noise on the Absorbance, the Relative Absorbance (ΔA/A) and the Optimum Absorbance Value

43. Analysis of Noise - Part 4—The Analysis of the Effect of ‘constant’ Gaussian Detector Noise When the Noise Is Not Negligible Compared to the Signal

44. Analysis of Noise - Part 5—The Analysis of the Effect of ‘constant’ Gaussian Detector Noise When the Reference Energy Approaches Zero

45. Analysis of Noise - Part 6—The Analysis of the Effect of ‘constant’ Gaussian Detector Noise: Comparing the Effect of Noise in the Sample Channel Versus Noise in the Reference Channel

46. Analysis of Noise - Part 7—The Analysis of ‘constant’ Detector Noise on the Kubelka-Munk Function

47. Analysis of Noise - Part 8—Effect of Noise on the Computed Transmittance, Analysis of Uniformly Distributed Noise for Transmittance and Absorbance Values

48. Analysis of Noise - Part 9—Analysis of Poisson-Distributed Noise, Effects on Transmittance and Absorbance Values

49. Analysis of Noise - Part 10—Analysis of Poisson-Distributed Noise, Effects on Relative Absorbance

50. Analysis of Noise - Part 11—Analysis of Poisson-Distributed Noise, When the Noise Is Not Small Compared to the Reference Signal

51. Analysis of Noise - Part 12—Analysis of Poisson-Distributed Noise: Computation of the Transmittance Noise

52. Analysis of Noise - Part 13—Analysis of Poisson-Distributed Noise: Computation of the Absorbance Noise

53. Analysis of Noise - Part 14—Analysis of Noise Proportional to the Signal, Small-Noise Case

54. Analysis of Noise - Part 15—Analysis of Noise Proportional to the Signal, Large-Noise Case

Section 9 - Derivatives

55. Derivatives in Spectroscopy, Part 1 - The Behavior of the Theoretical Derivative

56. Derivatives in Spectroscopy, Part 2 - The "True" Derivative

57. Derivatives in Spectroscopy, Part 3 - Computing the Derivative (the Savitzky-Golay method)

58. Derivatives in Spectroscopy, Part 4 - Calibrating with Derivatives

59. Corrections and Discussion Regarding Derivatives

Section 10 - Goodness of Fit Statistics

60. Comparison of Goodness-of-Fit Statistics for Linear Regression: Part 1 - Introduction

61. Comparison of Goodness-of-Fit Statistics for Linear Regression: Part 2 - The Correlation Coefficient

62. Comparison of Goodness-of-Fit Statistics for Linear Regression: Part 3 - Computing Confidence Limits for the Correlation Coefficient

63. Comparison of Goodness-of-Fit Statistics for Linear Regression: Part 4 - Confidence Limits for Slope and Intercept

Section 11 - More About Linearity in Calibration

64. Linearity in Calibration, Act III Scene I: Importance of (non)Linearity

65. Linearity in Calibration, Act III Scene II: A Discussion of the Durbin-Watson Statistic, a Step in the Right Direction

66. Linearity in Calibration, Act III Scene III: Other Tests for non-Linearity

67. Linearity in Calibration, Act III Scene IV: How Test For non-Linearity

68. Linearity in Calibration: Act III Scene V: Quantifying Non-linearity

69. Linearity in Calibration, Act III Scene VI: Quantifying Non-linearity, Part II, Calculus-Based Approach, and a News Flash

Section 12 - Connecting Chemometrics to Statistics

70. Connecting Chemometrics to Statistics: Part 1--The Chemometrics Side

71. Connecting Chemometrics to Statistics: Part 2--the Statistics Side

Section 13 - Limitations in Analytical Accuracy

72. Limitations in Analytical Accuracy: Part 1 - Horwitz's Trumpet

73. Limitations in Analytical Accuracy: Part 2 - Theories to Describe the Limits in Analytical Accuracy

74. Limitations in Analytical Accuracy: Part 3 - Comparing Test Results for Analytical Uncertainty

75. The Statistics of Spectral Searches

76. The Chemometrics of Imaging Spectroscopy

77. Corrections to Analysis of Noise - Part 1: Alternate Analysis of Transmittance Noise in the ‘large noise’ Regime

78. Corrections to Analysis of Noise - Part 2: Alternate Analysis of Absorbance noise in the ‘large noise’ Regime

79. What can NIR predict?

Section 14 - Derivations of Principal Components

80. The Long, Complicated, Tedious and Difficult Route to Principal Components (or, when you’re through reading this set you’ll know why it’s always done with matrices) - Part 1, Introduction and Review

81. The Long, Complicated, Tedious and Difficult Route to Principal Components (or, when you’re through reading this set you’ll know why it’s always done with matrices) - Part 2, our first attempt

82. The Long, Complicated, Tedious and Difficult Route to Principal Components (or, when you’re through reading this set you’ll know why it’s always done with matrices) - Part 3, multivariate curve fitting

83. The Long, Complicated, Tedious and Difficult Route to Principal Components (or, when you’re through reading this set you’ll know why it’s always done with matrices) - Part 4, the Lagrange Multiplier

84. The Long, Complicated, Tedious and Difficult Route to Principal Components (or, when you’re through reading this set you’ll know why it’s always done with matrices) - Part 5, Solving the Equations with Determinants

85. The Long, Complicated, Tedious and Difficult Route to Principal Components (or, when you’re through reading this set you’ll know why it’s always done with matrices) - Part 6: Solving the Equations Without Determinants

86. The Long, Complicated, Tedious and Difficult Route to Principal Components (or, when you’re through reading this set you’ll know why it’s always done with matrices) - Coda: Applying Constrained Univariate Calculations

Section 15 - Clinical Data Reporting

87. Statistics and Chemometrics for Clinical Data Reporting - Part 1: Fundamentals

88. Statistics and Chemometrics for Clinical Data Reporting - Part 2: Using Excel for Computations

89. Statistics and Chemometrics for Clinical Data Reporting - Part 3: Using Excel for Data Plotting

Section 16 - Classical Least Squares (CLS)

90. Classical Least Squares, Part 1: Mathematical theory

91. Classical Least Squares, Part 2: Mathematical Theory Continued

92. Classical Least Squares, Part 3: Spectroscopic Theory

93. Classical Least Squares, Part 4: Spectroscopic Theory Continued

94. Classical Least Squares, Part 5: Experimental Results

95. Classical Least Squares, Part 6: Spectral Results

96. Classical Least Squares, Part 7: Spectral Reconstruction of Mixtures

97. Classical Least Squares, Part 8: Comparison of CLS Values with Known Values

98. Classical Least Squares, Part 9: Spectral Results from a Second Laboratory

99. Classical Least Squares, Part 10: Numerical Results from the Second Laboratory

100. Classical Least Squares, Part 11: Comparison of Results from the Two Laboratories (Continued)

Section 17 - Transfer of Calibrations

101. Transfer of Calibrations - Part 1: An Overview

102. Calibration Transfer - Part 2: The Instrumentation Aspects

103. Calibration Transfer - Part 3: The Mathematical Aspects

104. Calibration Transfer - Part 4: Measuring the Agreement Between Instruments Following Calibration Transfer

105. Calibration Transfer - Part 5: The Mathematics of Wavelength Standards Used for Spectroscopy

106. Calibration Transfer - Part 6: The Mathematics of Photometric Standards Used for Spectroscopy

Section 18 - The Importance of Units of Measure

107. Units of Measure in Spectroscopy, Part 1: ... and Then The Light Dawned

108. Units of Measure in Spectroscopy, Part 2: It's the VOLUME, Folks!

109. Units of Measure in Spectroscopy, Part 3: What Does it all Mean

110. Units of Measure in Spectroscopy, Part IV: Summary of our Findings

111. Units of Measure in Spectroscopy, Part V: The "Mythbusters" and Spectral Reconstruction

Section 19 - The Best Calibration Model

112. Choosing the Best Calibration Model

113. Optimizing the Regression Model: The Challenge of Intercept/Bias and Slope “Correction"

Section 20 - Statistics

114. Statistics, Part 1: First Foundation: Probability Theory

115. STATISTICS, Part 2: Second Foundation: Analysis of Variance

116. STATISTICS, Part 3: Third Foundation: Least Squares

117. How to Select the Appropriate Degrees of Freedom for Multivariate Calibration

118. Bias and Slope Correction

Section 21 - Outliers

119. Outliers—Part 1: What Are Outliers?

120. Outliers—Part 2: Pitfalls in Detecting Outliers

121. Outliers—Part 3: Dealing With Outliers

Section 22 - Spectral Transfer: Making Instruments Agree

122. Calibration Transfer Chemometrics, Part 1: Review of the Subject

123. Calibration Transfer Chemometrics, Part 2: Review of the Subject

Section 23 - Applying Standard Reference Materials

124. Using Reference Materials, Part 1: Standards for Aligning the X-Axis

125. Using Reference Materials, Part 2: Aligning the Y-Axis

Section 24 - More About CLS

126. More About CLS, Part 1: Expanding the Concept

127. More About CLS, Part 2: Spectral Results & CLS (not requiring constituent values)

128. More About CLS, Part 3: Expanding the Analysis to Include Concentration Information (PCR & PLS)