Estimating the Variance of the Least Squares Estimator.ppt

1、Applied Econometrics,William Greene Department of Economics Stern School of Business,Applied Econometrics,7. Estimating the Variance of the Least Squares Estimator,Context,The true variance of b is 2E(XX)-1 We consider how to use the sample data to estimate this matrix. The ultimate objectives are t

2、o form interval estimates for regression slopes and to test hypotheses about them. Both require estimates of the variability of the distribution. We then examine a factor which affects how large this variance is, multicollinearity.,Estimating 2,Using the residuals instead of the disturbances: The na

3、tural estimator: ee/n as a sample surrogate for /n Imperfect observation of i = ei + ( - b)xi Downward bias of ee/n. We obtain the result Eee = (n-K)2,Expectation of ee,Method 1:,Estimating 2,The unbiased estimator is s2 = ee/(n-K). “Degrees of freedom.” Therefore, the unbiased estimator is s2 = ee/

4、(n-K) = M/(n-K).,Method 2: Some Matrix Algebra,Decomposing M,Example: Characteristic Roots of a Correlation Matrix,Varb|X,Estimating the Covariance Matrix for b|X The true covariance matrix is 2 (XX)-1 The natural estimator is s2(XX)-1 “Standard errors” of the individual coefficients. How does the c

5、onditional variance 2(XX)-1 differ from the unconditional one, 2E(XX)-1?,Regression Results,XX,(XX)-1,s2(XX)-1,Bootstrapping,Some assumptions that underlie it - the sampling mechanism Method: 1. Estimate using full sample: - b 2. Repeat R times: Draw n observations from the n, with replacement Estim

6、ate with b(r). 3. Estimate variance with V = (1/R)r b(r) - bb(r) - b,Bootstrap Application,- matr;bboot=init(3,21,0.)$ - name;x=one,y,pg$ - regr;lhs=g;rhs=x$ - calc;i=0$ - proc - regr;lhs=g;rhs=x$ - matr;i=i+1;bboot(*,i)=b$ - endproc - exec;n=20;bootstrap=b$ - matr;list;bboot $,+-+-+-+-+-+-+ |Variab

7、le| Coefficient | Standard Error |t-ratio |P|T|t| Mean of X| +-+-+-+-+-+-+ Constant| 7.96106957 6.23310198 1.277 .2104 Y | .01226831 .00094696 12.955 .0000 9232.86111 PG | -8.86289207 1.35142882 -6.558 .0000 2.31661111 Completed 20 bootstrap iterations. +-+ | Results of bootstrap estimation of model

8、.| | Model has been reestimated 20 times. | | Means shown below are the means of the | | bootstrap estimates. Coefficients shown | | below are the original estimates based | | on the full sample. | | bootstrap samples have 36 observations.| +-+ +-+-+-+-+-+-+ |Variable| Coefficient | Standard Error |

9、b/St.Er.|P|Z|z| Mean of X| +-+-+-+-+-+-+ B001 | 7.96106957 6.72472102 1.184 .2365 4.22761200 B002 | .01226831 .00112983 10.859 .0000 .01285738 B003 | -8.86289207 1.94703780 -4.552 .0000 -9.45989851,Bootstrap Replications,Full sample result,Bootstrapped sample results,Multicollinearity,Not “short ran

10、k,” which is a deficiency in the model. A characteristic of the data set which affects the covariance matrix. Regardless, is unbiased. Consider one of the unbiased coefficient estimators of k. Ebk = k Varb = 2(XX)-1 . The variance of bk is the kth diagonal element of 2(XX)-1 . We can isolate this wi

11、th the result in your text. Let X,z be Other xs, xk = X1,x2 (a convenient notation for the results in the text). We need the residual maker, M1. The general result is that the diagonal element we seek is x2M1x2-1 , which we know is the reciprocal of the sum of squared residuals in the regression of

12、x2 on X1. The remainder of the derivation of the result we seek is in your text. The estimated variance of bk is,Variance of a Least Squares Coefficient Estimator,Estimated Varbk =,Multicollinearity,Clearly, the greater the fit of the regression in the regression of x2 on X1, the greater is the vari

13、ance. In the limit, a perfect fit produces an infinite variance. There is no “cure” for collinearity. Estimating something else is not helpful (principal components, for example). There are “measures” of multicollinearity, such as the condition number of X. Best approach: Be cognizant of it. Understand its implications for estimation. A familiar application: The Longley data. See data table F4.2 Very prevalent in macroeconomic data, especially low frequency (e.g., yearly). Made