1. Descriptive Statistics

Descriptive statistics summarize and describe the main features of a dataset. They form the foundation for all statistical analysis and inference.

Measures of Central Tendency

Arithmetic Mean

x̄ = (1/n) ∑_i=1ⁿ x_i

Median

Median = x_((n+1)/2) for odd n
Median = (x_(n/2) + x_(n/2+1)) / 2 for even n

Mode: The most frequently occurring value in the dataset.

Measures of Dispersion

Sample Variance

s² = (1/(n-1)) ∑_i=1ⁿ (x_i - x̄)²

The n-1 divisor (Bessel's correction) provides an unbiased estimate of population variance.

Standard Deviation

s = √s²

Coefficient of Variation

CV = (s / x̄) × 100%

Dimensionless measure useful for comparing variability across different scales.

Measures of Shape

Skewness

g₁ = (1/n) ∑ [(x_i - x̄) / s]³

Positive skewness: tail extends to the right. Negative skewness: tail extends to the left.

Kurtosis

g₂ = (1/n) ∑ [(x_i - x̄) / s]⁴ - 3

Excess kurtosis (subtracting 3) measures deviation from normal distribution. Positive = heavier tails.

2. Probability Theory

Axioms of Probability (Kolmogorov)

P(A) ≥ 0 for any event A
P(S) = 1 where S is the sample space
For mutually exclusive events: P(A ∪ B) = P(A) + P(B)

Conditional Probability

P(A|B) = P(A ∩ B) / P(B), provided P(B) > 0

Bayes' Theorem

P(A|B) = P(B|A) P(A) / P(B)

Relates conditional probabilities and allows updating beliefs based on new evidence.

Law of Total Probability

P(B) = ∑_i P(B|A_i) P(A_i)

Where {A_i} forms a partition of the sample space.

3. Probability Distributions

Discrete Distributions

Binomial Distribution

P(X = k) = C(n,k) p^k (1-p)^n-k

E[X] = np, Var(X) = np(1-p)
Models the number of successes in n independent Bernoulli trials.

Poisson Distribution

P(X = k) = e^-λ λ^k / k!

E[X] = Var(X) = λ
Models rare events over time or space.

Continuous Distributions

Normal Distribution

Probability Density Function

f(x) = (1 / σ√(2π)) exp[-(x - μ)² / (2σ²)]

The most important distribution due to the Central Limit Theorem.

Standard Normal Distribution

Z = (X - μ) / σ ~ N(0, 1)

t-Distribution

Used when population variance is unknown and estimated from sample. Has heavier tails than normal; approaches normal as df → ∞.

Chi-Square Distribution

If Z₁, ..., Z_k are independent standard normal, then ∑Z_i² ~ χ²(k)

F-Distribution

Ratio of two independent chi-square variables divided by their degrees of freedom. Used in ANOVA and regression.

4. Point Estimation

Properties of Estimators

Unbiasedness: E[θ̂] = θ
Efficiency: Minimum variance among unbiased estimators
Consistency: θ̂ → θ as n → ∞
Sufficiency: Captures all information about θ in the sample

Maximum Likelihood Estimation

Likelihood Function

L(θ) = ∏_i=1ⁿ f(x_i; θ)

MLE finds θ̂ that maximizes L(θ), typically by setting ∂log L / ∂θ = 0

5. Confidence Intervals

A (1-α) confidence interval provides a range that, in repeated sampling, would contain the true parameter value (1-α)×100% of the time.

CI for Mean (σ known)

x̄ ± z_α/2 (σ / √n)

CI for Mean (σ unknown)

x̄ ± t_{α/2, n-1} (s / √n)

CI for Proportion

p̂ ± z_α/2 √(p̂(1-p̂) / n)

6. Hypothesis Testing Framework

The Testing Process

State null (H₀) and alternative (H₁) hypotheses
Choose significance level α
Select appropriate test statistic
Determine critical region or compute p-value
Make decision: reject or fail to reject H₀

Types of Errors

	H₀ True	H₀ False
Reject H₀	Type I Error (α)	Correct Decision (Power)
Fail to Reject	Correct Decision	Type II Error (β)

Statistical Power

Power = 1 - β = P(Reject H₀ | H₀ is false)

7. t-Tests

One-Sample t-Test

t = (x̄ - μ₀) / (s / √n)

Tests whether the sample mean differs from a hypothesized value μ₀. df = n - 1

Independent Two-Sample t-Test

t = (x̄₁ - x̄₂) / √(s_p²(1/n₁ + 1/n₂))

Where s_p² is the pooled variance (assuming equal variances).

Paired t-Test

t = d̄ / (s_d / √n)

Where d̄ is the mean of paired differences. Used for before-after designs or matched pairs.

8. Analysis of Variance (ANOVA)

One-Way ANOVA

Tests whether means of k groups are equal: H₀: μ₁ = μ₂ = ... = μ_k

F-Statistic

F = MS_Between / MS_Within = (SS_B/(k-1)) / (SS_W/(N-k))

Sum of Squares

SS_Total = SS_Between + SS_Within

SS_B = ∑_j n_j(x̄_j - x̄)²

SS_W = ∑_j∑_i (x_ij - x̄_j)²

Assumptions of ANOVA

Independence of observations
Normality within groups
Homogeneity of variances (homoscedasticity)

Post-Hoc Tests

When ANOVA rejects H₀, post-hoc tests identify which means differ:

Tukey's HSD: Controls family-wise error rate for all pairwise comparisons
Bonferroni: Adjusts α for multiple comparisons (α/m)
Scheffé: Most conservative; allows all contrasts

9. Nonparametric Tests

Nonparametric tests make fewer assumptions about the underlying distribution and are appropriate when normality cannot be assumed or with ordinal data.

Mann-Whitney U Test

Nonparametric alternative to independent two-sample t-test. Tests whether one distribution is stochastically greater than the other.

Wilcoxon Signed-Rank Test

Nonparametric alternative to paired t-test. Uses ranks of absolute differences.

Kruskal-Wallis Test

Nonparametric alternative to one-way ANOVA. Extends Mann-Whitney to k groups.

10. Correlation Analysis

Pearson Correlation Coefficient

r = ∑(x_i - x̄)(y_i - ȳ) / √[∑(x_i - x̄)² ∑(y_i - ȳ)²]

Measures linear association. -1 ≤ r ≤ 1

Testing Correlation

t = r √(n-2) / √(1-r²)

Tests H₀: ρ = 0. df = n - 2

Spearman Rank Correlation

Nonparametric correlation based on ranks. Measures monotonic (not necessarily linear) relationships.

11. Regression Analysis

Simple Linear Regression

Model

Y = β₀ + β₁X + ε

Least Squares Estimates

b₁ = ∑(x_i - x̄)(y_i - ȳ) / ∑(x_i - x̄)²

b₀ = ȳ - b₁x̄

Coefficient of Determination

R² = 1 - SS_Residual / SS_Total = SS_Regression / SS_Total

Proportion of variance in Y explained by the model. 0 ≤ R² ≤ 1

Regression Inference

t-Test for Slope

t = b₁ / SE(b₁)

Tests H₀: β₁ = 0 (no linear relationship)

12. Categorical Data Analysis

Chi-Square Test of Independence

χ² = ∑ (O_ij - E_ij)² / E_ij

Where E_ij = (row total)(column total) / grand total
df = (r-1)(c-1)

Chi-Square Goodness of Fit

Tests whether observed frequencies match expected frequencies from a hypothesized distribution.

Fisher's Exact Test

Exact test for 2×2 tables when expected frequencies are small (any E_ij < 5).

References and Further Reading

Casella, G. & Berger, R.L. (2002). Statistical Inference, 2nd Edition. Cengage Learning.
Wackerly, D.D., Mendenhall, W., & Scheaffer, R.L. (2014). Mathematical Statistics with Applications, 7th Edition. Cengage.
Agresti, A. (2018). Statistical Methods for the Social Sciences, 5th Edition. Pearson.
Kutner, M.H., et al. (2004). Applied Linear Statistical Models, 5th Edition. McGraw-Hill.
Hollander, M., Wolfe, D.A., & Chicken, E. (2013). Nonparametric Statistical Methods, 3rd Edition. Wiley.

← Previous: DOE Next: Reliability Analysis →

Statistical Analysis