第六讲多元线性回归

第六讲 多元回归分析 Multiple Regression Analysis MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLE GRAPHING A RELATIONSHIP IN A MULTIPLE REGRESSION MODEL PROPERTIES OF THE MULTIPLE REGRESSION COEFFICIENTSSIB,BFSU ECONOMETRICS1LECTURE 6MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLEEARNINGSASVABCSb1EARNINGS = b1 + b2S + b3ASVABC + uThis sequence provides a geometrical interpretation of a multiple regression model with two explanatory variables. Specifically, we will look at an earnings function model where hourly earnings, EARNINGS, depend on years of schooling (highest grade completed), S, and a measure of cognitive ability, ASVABC.SIB,BFSU ECONOMETRICS2LECTURE 6MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLEEARNINGSASVABCSb1EARNINGS = b1 + b2S + b3ASVABC + uThe model has three dimensions, one each for EARNINGS, S, and ASVABC. The starting point for investigating the determination of EARNINGS is the intercept, b1. SIB,BFSU ECONOMETRICS3LECTURE 6MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLEEARNINGSASVABCSb1Literally the intercept gives EARNINGS for those respondents who have no schooling and who scored zero on the ability test. However, the ability score is scaled in such a way as to make it impossible to score zero. Hence a literal interpretation of b1 would be unwise.EARNINGS = b1 + b2S + b3ASVABC + uSIB,BFSU ECONOMETRICS4LECTURE 6MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLEEARNINGSASVABCThe next term on the right side of the equation gives the effect of variations in S. A one year increase in S causes EARNINGS to increase by b2 dollars, holding ASVABC constant.Sb1pure S effect b1 + b2SEARNINGS = b1 + b2S + b3ASVABC + uSIB,BFSU ECONOMETRICS5LECTURE 6pure ASVABC effectMULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLESb1b1 + b3ASVABCEARNINGS = b1 + b2S + b3ASVABC + uEARNINGSASVABCSimilarly, the third term gives the effect of variations in ASVABC. A one point increase in ASVABC causes earnings to increase by b3 dollars, holding S constant.SIB,BFSU ECONOMETRICS6LECTURE 6pure ASVABC effectpure S effectMULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLESb1b1 + b3ASVABCb1 + b2S + b3ASVABCEARNINGS = b1 + b2S + b3ASVABC + uEARNINGSASVABCb1 + b2Scombined effect of S and ASVABCDifferent combinations of S and ASVABC give rise to values of EARNINGS which lie on the plane shown in the diagram, defined by the equation EARNINGS = b1 + b2S + b3ASVABC. This is the nonstochastic (nonrandom) component of the model.SIB,BFSU ECONOMETRICS 7LECTURE 6pure ASVABC effectpure S effectMULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLESb1b1 + b3ASVABCb1 + b2S + b3ASVABCb1 + b2S + b3ASVABC + uEARNINGSASVABCcombined effect of S and ASVABCuThe final element of the model is the disturbance term, u. This causes the actual values of EARNINGS to deviate from the plane. In this observation, u happens to have a positive value. EARNINGS = b1 + b2S + b3ASVABC + ub1 + b2SSIB,BFSU ECONOMETRICS8LECTURE 6pure ASVABC effectpure S effectMULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLESb1b1 + b3ASVABCb1 + b2S + b3ASVABCb1 + b2S + b3ASVABC + uEARNINGSASVABCb1 + b2Scombined effect of S and ASVABCuWe assume that the effects of S and ASVABC on EARNINGS are additive. The impact of a difference in S on EARNINGS is not affected by the value of ASVABC, or vice versa.EARNINGS = b1 + b2S + b3ASVABC + uSIB,BFSU ECONOMETRICS9LECTURE 6MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLEThe regression coefficients are derived using the same least squares principle used in simple regression analysis. The fitted value of Y in observation i depends on our choice of b1, b2, and b3.SIB,BFSU ECONOMETRICS10LECTURE 6MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLEThe residual ei in observation i is the difference between the actual and fitted values of Y.SIB,BFSU ECONOMETRICS11LECTURE 6MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLEFirst we expand RSS as shown, and then we use the first order conditions for minimizing it.SIB,BFSU ECONOMETRICS12LECTURE 6MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLE Then SIB,BFSU ECONOMETRICS13LECTURE 6MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLEWe thus obtain three equations in three unknowns. Solving for b1, b2, and b3, we obtain the expressions shown above.SIB,BFSU ECONOMETRICS14LECTURE 6MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLEThe expressions for the slope coefficients are considerably more complex than that for the slope coefficient in simple regression analysis. For the general case when there are many explanatory variables, ordinary algebra is inadequate. It is necessary to switch to matrix algebra.SIB,BFSU ECONOMETRICS15LECTURE 6Dependent Variable: EARNINGS Method: Least Squares Date: 03/26/04 Time: 15:39 Sample: 1 570 Included observations: 570 Variable CoefficientStd. Error t-Statistic Prob. C -4.624749 2.013200 -2.297213 0.0220 S 0.739037 0.160622 4.601103 0.0000 ASVABC 0.154534 0.042949 3.598121 0.0003 R-squared0.123597 Mean dependent var 13.11782 Adjusted R-squared 0.120505 S.D. dependent var 8.214719 S.E. of regression 7.703877 Akaike info criterion 6.926574 Sum squared resid 33651.29 Schwarz criterion 6.949445 Log likelihood -1971.073 F-statistic 39.98123 Durbin-Watson stat 1.962011 Prob(F-statistic) 0.000000 Here is the regression output for the earnings function. It indicates that earnings increase by $0.74 for every extra year of schooling and by $0.15 for every extra point increase in ASVABC.MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLESIB,BFSU ECONOMETRICS16LECTURE 6Dependent Variable: EARNINGS Method: Least Squares Date: 03/26/04 Time: 15:39 Sample: 1 570 Included observations: 570 Variable CoefficientStd. Error t-Statistic Prob. C -4.624749 2.013200 -2.297213 0.0220 S 0.739037 0.160622 4.601103 0.0000 ASVABC 0.154534 0.042949 3.598121 0.0003 R-squared0.123597 Mean dependent var 13.11782 Adjusted R-squared 0.120505 S.D. dependent var 8.214719 S.E. of regression 7.703877 Akaike info criterion 6.926574 Sum squared resid 33651.29 Schwarz criterion 6.949445 Log likelihood -1971.073 F-statistic 39.98123 Durbin-Watson stat 1.962011 Prob(F-statistic) 0.000000 Literally, the intercept indicates that an individual who had no schooling and an ASVABC score of zero would have hourly earnings of -$4.62. Obviously, this is impossible. The lowest value of S in the sample was 6, and the lowest ASVABC score was 22. We have obtained a nonsense estimate because we have extrapolated too far from the data range.MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLESIB,BFSU ECONOMETRICS17LECTURE 6GRAPHING A RELATIONSHIP IN A MULTIPLE REGRESSION MODELSuppose that you were particularly interested in the relationship between EARNINGS and S and wished to represent it graphically, using the sample data.SIB,BFSU ECONOMETRICS18LECTURE 6GRAPHING A RELATIONSHIP IN A MULTIPLE REGRESSION MODELA simple plot, like the one above, would be misleading.SIB,BFSU ECONOMETRICS19LECTURE 6GRAPHING A RELATIONSHIP IN A MULTIPLE REGRESSION MODELThere appears to be a strong positive relationship, but it is distorted by the fact that S is positively correlated with ASVABC, which also has a positive effect on EARNINGS. correlationship(obs=570)| S ASVABC-+-S| 1.0000ASVABC| 0.5779 1.0000SIB,BFSU ECONOMETRICS20LECTURE 6Dependent Variable: EARNINGS Method: Least Squares Date: 03/26/04 Time: 18:01 Sample: 1 570 Included observations: 570 Variable Coefficient Std. Error t-Statistic Prob. C -0.359883 1.818571 -0.197893 0.8432 ASVABC 0.268743 0.035666 7.534995 0.0000 R-squared 0.090874 Mean dependent var 13.11782 Adjusted R-squared 0.089274 S.D. dependent var 8.214719 S.E. of regression 7.839469 Akaike info criterion 6.959722 Sum squared resid 34907.73 Schwarz criterion 6.974970 Log likelihood -1981.521 F-statistic 56.77615 Durbin-Watson stat1.987808 Prob(F-statistic) 0.000000 EEARN=RESIDUAL IN THIS REGRESSIONGRAPHING A RELATIONSHIP IN A MULTIPLE REGRESSION MODELTo eliminate the distortion, you purge both EARNINGS and S of their components related to ASVABC and then draw a scatter diagram using the purged variables. SIB,BFSU ECONOMETRICS21LECTURE 6Dependent Variable: S Method: Least Squares Date: 03/26/04 Time: 18:07 Sample: 1 570 Included observations: 570 Variable CoefficientStd. Error t-Statistic Prob. C 5.770845 0.466847 12.36131 0.0000 ASVABC 0.154538 0.009156 16.87857 0.0000 R-squared0.334026 Mean dependent var 13.52105 Adjusted R-squared 0.332854 S.D. dependent var 2.463886 S.E. of regression 2.012478 Akaike info criterion 4.240113 Sum squared resid 2300.439 Schwarz criterion 4.255361 Log likelihood -1206.432 F-statistic 284.8862 Durbin-Watson stat 2.032864 Prob(F-statistic) 0.000000 We do the same with S. We regress it on ASVABC and save the residuals as ES.ES=RESIDUAL IN THIS REGRESSIONSIB,BFSU ECONOMETRICS22LECTURE 6Dependent Variable: EEARN Method: Least Squares Date: 03/26/04 Time: 18:05 Sample: 1 570 Included observations: 570 Variable CoefficientStd. Error t-Statistic Prob. C -5.34E-15 0.322396 -1.66E-14 1.0000 ES 0.739037 0.160480 4.605159 0.0000 R-squared0.035993 Mean dependent var-4.11E-15 Adjusted R-squared 0.034296 S.D. dependent var 7.832577 S.E. of regression 7.697092 Akaike info criterion 6.923065 Sum squared resid 33651.29 Schwarz criterion 6.938313 Log likelihood -1971.073 F-statistic 21.20749 Durbin-Watson stat 1.962011 Prob(F-statistic) 0.000005 Here is the regression of EEARN on ES.SIB,BFSU ECONOMETRICS23LECTURE 6GRAPHING A RELATIONSHIP IN A MULTIPLE REGRESSION MODELNow we plot EEARN on ES and the scatter is a faithful representation of the relationship, both in terms of the slope of the trend line (the black line) and in terms of the variation about that line.SIB,BFSU ECONOMETRICS24LECTURE 6GRAPHING A RELATIONSHIP IN A MULTIPLE REGRESSION MODEL10As you would expect, the trend line is flatter than in scatter diagram which did not control for ASVABC (reproduced here as the gray line).SIB,BFSU ECONOMETRICS25LECTURE 6PROPERTIES OF THE MULTIPLE REGRESSION COEFFICIENTSProvided that the model is correctly specifi