Probability and Statistics

1 INTRODUCTION TO PROBABILITY

1.0 Introduction

1.1 Interpretations of Probability
Objectivists
Classical (a priori)
Empirical or Relative Frequency (a posteriori)
Mathematical or Axiomatic

1.2 Sets
Set Algebra

1.3 Probability Parlance

1.4 Probability Theorems

1.5 Conditional Probability and Independence

1.6 Bayes’s Rule

1.7 Counting the Ways
Two Fundamental Principles of Counting (FPC) and the Pigeonhole Principle
Tree Diagrams
Permutations
Combinations
Match Problem Revisited

1.8 Summary
Problems
Miscellaneous
Software Exercises
Self Quiz 1A: Conditional Probability
Self Quiz 1B: Poker Probability


2 RANDOM VARIABLES, MOMENTS,
AND DISTRIBUTIONS

2.0 Introduction

2.1 Random Variables

2.2 Distributions

2.3 Moments
Information Content (Entropy)
Higher Moments

2.4 Standardized Random Variables

2.5 Jointly Distributed Random Variables
Discrete Joint Density Functions

2.6 Independence of Jointly Distributed
Random Variables

2.7 Covariance and Correlation

2.8 Conditional Densities Functions

2.9 Moment Generating Functions

2.10 Transformation of Variables
Transformation of 2 or More Variables

2.11 Summary
Problems
Review
Paradoxes
Software Exercises
Self Quiz 2: Moments


3 SPECIAL DISCRETE DISTRIBUTIONS 158

3.0 Introduction

3.1 Discrete Uniform

3.2 Bernoulli Distribution

3.3 Binomial Distribution

3.4 Multinomial Distribution

3.5 Hypergeometric Distribution

3.6 Geometric Distribution

3.7 Negative Binomial Distribution

3.8 Poisson Distribution

3.9 Summary
Problems
Review
Software Exercises
Self Quiz 3: Discrete Distributions


4 SPECIAL CONTINUOUS DISTRIBUTIONS

4.0 Introduction

4.1 Continuous Uniform Distribution

4.2 Gamma Function

4.3 Gamma Family (Exponential, Chi-Square, Gamma)

4.4 Exponential Distribution

4.5 Chi-Square Distribution

4.6 Normal Distribution

4.7 Student t Distribution

4.8 Beta Distribution

4.9 Weibull Distribution

4.10 F Distribution

4.11 Summary
Problems
Miscellaneous
Review
Software Exercises
Self Quiz 4: Continuous Distributions



5 SAMPLING, DATA DISPLAYS, MEASURES
OF CENTRAL TENDENCIES, MEASURES
OF DISPERSION, AND SIMULATION

5.0 Introduction

5.1 Data Displays
Boxplots
Frequency Distributions and Histograms

5.2 Measures of Location
Mean
Median
Mode
Trimmed Mean
Robustness

5.3 Measures of Dispersion
Sample Variance and Standard Deviation
Interquartile Range (IQR)
Median Absolute Deviation from the Median
(MAD)
Outliers
Coefficient of Variation
Skewness
Kurtosis

5.4 Joint Distribution of X and S2

5.5 Simulation of Random Variables
Rejection Method

5.6 Using Monte Carlo for Integration

5.7 Order Statistics

5.8 Summary
Problems
Software Exercises
Self Quiz 5: Sampling and Data Displays



6 POINT AND INTERVAL ESTIMATION

6.0 Introduction

6.1 Unbiased Estimators and Point Estimates
Cramér-Rao Inequality

6.2 Methods of Finding Point Estimates
Method of Moments Estimators (MME)
Maximum Likelihood Estimators (MLE)

6.3 Interval Estimates (Confidence Intervals)
Trade-Off: Sample Size
Confidence Interval When s Is Not Known
Confidence Interval for the Difference between
Two Means (m1 - m2 )
Confidence Interval for s2 of a Normal Distribution
Confidence Interval for a Proportion
Confidence Interval for the Difference between
Two Proportions
Confidence Interval for the Paired T-Test
Confidence Intervals for Ratio of Variances s2

2/s21

6.4 Prediction Intervals

6.5 Central Limit Theorem (Revisited)

6.6 Parametric Bootstrap Estimation

6.7 Summary
Problems
Confidence Intervals
Miscellaneous
Software Exercises
Self Quiz 6: Estimation and Confidence Intervals



7 HYPOTHESIS TESTING

7.0 Introduction

7.1 Terminology in Statistical Tests of Hypotheses

7.2 Hypothesis Tests: Means
P-value
Directional Tests

7.3 Hypothesis Tests: Proportions
Fisher-Irwin Test

7.4 Hypothesis Tests for Difference between
Two Means: Small Samples (n £ 30) s2 Known
n < 30; s2 Unknown

7.5 Hypothesis Test with Paired Samples
Paired vs. Unpaired
Statistically Significant vs. Practically
Significant

7.6 Hypothesis Tests: Variances
Hypothesis Tests for the Equality of Two Variances

7.7 Hypothesis Tests for Independence, Homogeneity,
and Goodness of Fit
R ¥ C Contingency Tables Test for Homogeneity
and Independence
Goodness of Fit
Probability Plots

7.8 Summary
Problems
Miscellaneous
Software Exercises
Self Test 7: Hypothesis Testing



8 REGRESSION

8.0 Introduction

8.1 Review of Joint and Conditional Densities

8.2 Simple Linear Regression
Least Squares Estimation
Other Models of Simple Linear Regression

8.3 Distribution of Estimators with Inference
on Parameters
Distribution of RV E
Distribution of RV Yi
Distribution of RV B
Inference on the Slope b
Distribution of RV A
Inference on the Intercept a
Distribution of RV

8.4 Variation
Coefficient of Determination

8.5 Residual Analysis
Lack of Fit F-Test

8.6 Convertible Nonlinear Forms for Linear
Regression

8.7 Polynomial Regression

8.8 Multiple Linear Regression
Multiple Linear Regression with Matrices

8.9 Multiple Regression Techniques
Forward Selection
Backward Elimination
Model Variables Selection Criteria
Stepwise Regression

8.10 Correlation Analysis

8.11 Summary
Problems
Miscellaneous
Software Exercises
Self Test 8: Regression



9 ANALYSIS OF VARIANCE

9.0 Introduction

9.1 Single-Factor Analysis
The Bartlett Test for Homogeneity of Variances

9.2 Two-Way ANOVA without Replication
To Block or Not to Block

9.3 Two-Way ANOVA with Replication

9.4 Multiple Comparisons of Treatment Means
Contrasts
Contrast Confidence Intervals
Least Significant Difference (LSD), Fisher LSD,
and Scheffe Procedures
Tukey Method
Bonferroni Method
Tukey Method vs. Bonferroni Method

9.5 ANOVA and Regression

9.6 Analysis of Means (ANOM)
Graphical Analysis of Treament Means

9.7 Summary
Problems
Review
Software Exercises
Self Quiz 9: Analysis of Variance



10 NONPARAMETRIC STATISTICS

10.0 Introduction

10.1 The Sign Test

10.2 Nonparametric Bootstrap Estimation

10.3 The Sign Test for Paired Data
Type II Beta Error for the Sign-Test

10.4 The Wilcoxon Signed-Rank Test

10.5 Wilcoxon-Mann-Whitney (WMW) Rank Test for
Two Samples

10.6 Spearman Rank Order Correlation Coefficient

10.7 Kendall’s Rank Correlation Coefficient (t)

10.8 Nonparametric Tests for Regression

10.9 Nonparametric Tests for ANOVA
Kruskal-Wallis
Friedman Test

10.10 Runs Test

10.11 Randomization Tests

10.12 Summary
Problems
Software Exercises