Introduction to linear regression analysis

1. INTRODUCTION

1.1 Regression and Model Building

1.2 Data Collection

1.3 Uses of Regression

1.4 Role of the Computer

2. SIMPLE LINEAR REGRESSION

2.1 Simple Linear Regression Model

2.2 Least-Squares Estimation of the Parameters

2.3 Hypothesis Testing on the Slope and Intercept

2.4 Interval Estimation in Simple Linear Regression

2.5 Prediction of New Observations

2.6 Coefficient of Determination

2.7 A Service Industry Application of Regression

2.8 Using SAS and R for Simple Linear Regression

2.9 Some Considerations in the Use of Regression

2.10 Regression Through the Origin

2.11 Estimation by Maximum Likelihood

2.12 Case Where the Regressor x is Random

3. MULTIPLE LINEAR REGRESSION

3.1 Multiple Regression Models

3.2 Estimation of the Model Parameters

3.3 Hypothesis Testing in Multiple Linear Regression

3.4 Confidence Intervals in Multiple Regression

3.5 Prediction of New Observations

3.6 A Multiple Regression Model for the Patient Satisfaction Data

3.7 Using SAS and R for Basic Multiple Linear Regression

3.8 Hidden Extrapolation in Multiple Regression

3.9 Standardized Regression Coeffi cients

3.10 Multicollinearity

3.11 Why Do Regression Coeffi cients Have the Wrong Sign? 4. MODEL ADEQUACY CHECKING

4.1 Introduction

4.2 Residual Analysis

4.3 PRESS Statistic

4.4 Detection and Treatment of Outliers

4.5 Lack of Fit of the Regression Model

5. TRANSFORMATIONS AND WEIGHTING TO CORRECT MODEL INADEQUACIES

5.1 Introduction

5.2 Variance-Stabilizing Transformations

5.3 Transformations to Linearize the Model

5.4 Analytical Methods for Selecting a Transformation

5.5 Generalized and Weighted Least Squares

5.6 Regression Models with Random Effect

6. DIAGNOSTICS FOR LEVERAGE AND INFLUENCE

6.1 Importance of Detecting Infl uential Observations

6.2 Leverage

6.3 Measures of Infl uence: Cook's D

6.4 Measures of Infl uence: DFFITS and DFBETAS

6.5 A Measure of Model Performance

6.6 Detecting Groups of Infl uential Observations

6.7 Treatment of Infl uential Observations

7. POLYNOMIAL REGRESSION MODELS

7.1 Introduction

7.2 Polynomial Models in One Variable

7.3 Nonparametric Regression

7.4 Polynomial Models in Two or More Variables

7.5 Orthogonal Polynomials. 8. INDICATOR VARIABLES

8.1 General Concept of Indicator Variables

8.2 Comments on the Use of Indicator Variables

8.3 Regression Approach to Analysis of Variance

9. MULTICOLLINEARITY

9.1 Introduction

9.2 Sources of Multicollinearity

9.3 Effects of Multicollinearity

9.4 Multicollinearity Diagnostics

9.5 Methods for Dealing with Multicollinearity

9.6 Using SAS to Perform Ridge and Principal-Component Regression

10. VARIABLE SELECTION AND MODEL BUILDING

10.1 Introduction

10.2 Computational Techniques for Variable Selection

10.3 Strategy for Variable Selection and Model Building

10.4 Case Study: Gorman and Toman Asphalt Data Using SAS

11. VALIDATION OF REGRESSION MODELS

11.1 Introduction 372 11.2 Validation Techniques

11.3 Data from Planned Experiments

12. INTRODUCTION TO NONLINEAR REGRESSION

12.1 Linear and Nonlinear Regression Models

12.2 Origins of Nonlinear Models

12.3 Nonlinear Least Squares

12.4 Transformation to a Linear Model

12.5 Parameter Estimation in a Nonlinear System

12.6 Statistical Inference in Nonlinear Regression

12.7 Examples of Nonlinear Regression Models

12.8 Using SAS and R. 13. GENERALIZED LINEAR MODELS

13.1 Introduction

13.2 Logistic Regression Models

13.3 Poisson Regression

13.4 The Generalized Linear Model

14. REGRESSION ANALYSIS OF TIME SERIES DATA

14.1 Introduction to Regression Models for Time Series Data

14.2 Detecting Autocorrelation: The Durbin-Watson Test

14.3 Estimating the Parameters in Time Series Regression Models

15. OTHER TOPICS IN THE USE OF REGRESSION ANALYSIS

15.1 Robust Regression

15.2 Effect of Measurement Errors in the Regressors

15.3 Inverse Estimation

The Calibration Problem

15.4 Bootstrapping in Regression

15.5 Classifi cation and Regression Trees (CART)

15.6 Neural Networks

15.7 Designed Experiments for Regression

APPENDIX A. STATISTICAL TABLES

APPENDIX B. DATA SETS FOR EXERCISES

APPENDIX C. SUPPLEMENTAL TECHNICAL MATERIAL

C.1 Background on Basic Test Statistics

C.2 Background from the Theory of Linear Models

C.3 Important Results on SSR and SSRes

C.4 Gauss-Markov Theorem, Var(epsilon) = sigma2I. C.5 Computational Aspects of Multiple Regression

C.6 Result on the Inverse of a Matrix

C.7 Development of the PRESS Statistic

C.8 Development of S2 (i)

C.9 Outlier Test Based on R-Student

C.10 Independence of Residuals and Fitted Values

C.11 Gauss

Markov Theorem, Var(epsilon) = V

C.12 Bias in MSRes When the Model Is Underspecified

C.13 Computation of Infl uence Diagnostics

C.14 Generalized Linear Models

APPENDIX D. INTRODUCTION TO SAS

D.1 Basic Data Entry

D.2 Creating Permanent SAS Data Sets

D.3 Importing Data from an EXCEL File

D.4 Output Command

D.5 Log File

D.6 Adding Variables to an Existing SAS Data Set

APPENDIX E. INTRODUCTION TO R TO PERFORM LINEAR REGRESSION ANALYSIS

E.1 Basic Background on R

E.2 Basic Data Entry

E.3 Brief Comments on Other Functionality in R

E.4 R Commander. Contents note continued: Machine generated contents note