1
Introduction
2
Multiple Linear Regression
2.1
Assumptions of the linear model
2.2
Geometric interpretation
2.3
Hat matrix
2.4
Multiple regression vs. simple regression
2.5
Properties
2.6
Tests
2.7
Diagnostics
2.8
Generalized least squares
2.9
Model Selection
2.9.1
Mallow’s cp statistic
2.9.2
Search strategies
3
Non-parametric Density Estimation
3.1
The Histogramm
3.2
Kernels
3.2.1
The naive Estimator
3.2.2
Other Kernels
3.3
The Bandwidth
3.4
Bringing it all together
3.5
Other Density Estimators
3.6
Higher Dimensions
3.6.1
The Curse of Dimensionality
4
Non-parametric Regression
4.1
Alternative Interpretation
4.2
The Bandwidth
4.3
Hat Matrix
4.4
Degrees of Freedom
4.4.1
Applications
4.5
Inference
4.6
Local Polynomial Estimator
4.7
Smoothing splines
5
Cross Validation
5.1
Motivation and Core Idea
5.2
Loss Function
5.3
Implementations
5.3.1
Leave-one-out
5.4
K-fold Cross-Validation
5.4.1
Random Division into test and training data set
5.5
Properties of the different schemes
5.6
Shortcuts for (some) linear fitting operators
5.7
Isolation of each cross validation sample
5.8
Examples with R
5.8.1
Application 1: Estimating the generalization error
5.8.2
Application 2: Parameter Tuning
5.8.3
Application 3: Stochastic approximation for leave d out
6
Bootstrap
6.1
Motivation
6.2
The Bootstrap Distribution
6.3
Bootstrap Consistency
6.4
Boostrap Confidence Intervals
6.5
Boostrap Estimator of the Generalization Error
6.6
Out-of-Boostrap sample for estimating the GE
6.7
Double Boostrap Confidence Intervals
6.8
Three Versions of Boostrap
6.8.1
Non-parametric Regression
6.8.2
Parametric Boostrap
6.8.3
Model-Based Bootstrap
6.9
Conclusion
7
Classification
7.1
Indirect Classification - The Bayes Classifier
7.2
Direct Classification - The Discriminant View
7.2.1
LDA
7.2.2
QDA
7.3
Indirect Classification - The View of Logistic Regression
7.4
Discriminant Analysis or Logistic Regression?
7.5
Multiclass case (J > 2)
8
Flexible regression and classification methods
8.1
Additive Models
8.1.1
Structure
8.1.2
Fitting Procedure
8.1.3
Additive Models in R
8.2
MARS
8.2.1
Details for Dummies
8.2.2
Example
8.3
Neural Networks
8.3.1
Fitting Neural Networks (in R)
8.4
Projection Pursuit Regression
8.4.1
Proejction Pursuit Example
8.5
Classification and Regression Trees
8.5.1
Prediction given Partitioning
8.5.2
Assumptions on the Patritions
8.5.3
Algorithm
8.5.4
Backward Deletion / Pruning
8.5.5
Pros and Cons of Trees
8.5.6
Random Forests
9
Variable Selection - Ridge Regression and Lasso
9.1
Ridge Regression
9.2
Lasso
9.3
Extensions
9.3.1
Elastic Net
9.3.2
Adaptive Lasso
9.3.3
Relaxed Lasso
9.3.4
(Sparse) Group Lasso
9.3.5
Oracle Properties
10
Bagging and Boosting
10.1
Bagging
10.2
Subagging
10.3
\(L_2\)
-Boosting
10.4
Some unfinished stuff
11
Round up
11.1
Comparing models
11.2
Exercises Take-aways
11.3
Cheatsheet
12
Introduction
References
Computational Statistics - Summary
References