多元线性回归模型：假设检验

1、多元线性回归模型：假设检验1 Multiple Regression Analysis w y = b0 + b1x1 + b2x2 + . . . bkxk + u w 2. Inference 多元线性回归模型：假设检验2 Assumptions of the Classical Linear Model (CLM) So far, we know that given the Gauss- Markov assumptions, OLS is BLUE, In order to do classical hypothesis testing, we need to add another

2、 assumption (beyond the Gauss-Markov assumptions) Assume that u is independent of x1, x2, xk and u is normally distributed with zero mean and variance s2: u Normal(0,s2) 多元线性回归模型：假设检验3 CLM Assumptions (cont) Under CLM, OLS is not only BLUE, but is the minimum variance unbiased estimator We can summa

3、rize the population assumptions of CLM as follows y|x Normal(b0 + b1x1 + bkxk, s2) While for now we just assume normality, clear that sometimes not the case Large samples will let us drop normality 多元线性回归模型：假设检验4 . . x1x2 The homoskedastic normal distribution with a single explanatory variable E(y|x

4、) = b0 + b1x y f(y|x) Normal distributions 多元线性回归模型：假设检验5 Normal Sampling Distributions errors theofn combinatiolinear a is it becausenormally ddistribute is 0,1Normal thatso , ,Normal st variableindependen theof valuessample the on lconditiona s,assumption CLM Under the j b b bb bbb j jj jjj sd Var

5、 多元线性回归模型：假设检验6 The t Test 1:freedom of degrees theNote by estimate tohave webecause normal) (vson distributi a is thisNote sassumption CLM Under the 22 1 j kn t t se kn j j ss b bb 多元线性回归模型：假设检验7 The t Test (cont) Knowing the sampling distribution for the standardized estimator allows us to carry o

6、ut hypothesis tests Start with a null hypothesis For example, H0: bj=0 If accept null, then accept that xj has no effect on y, controlling for other xs 多元线性回归模型：假设检验8 The t Test (cont) 0 j H ,hypothesis null accept the o whether tdetermine torulerejection a withalong statistic our use then willWe :

7、for statistic the form toneedfirst eour test w perform To t se tt j j j b b b b 多元线性回归模型：假设检验9 t Test: One-Sided Alternatives Besides our null, H0, we need an alternative hypothesis, H1, and a significance level H1 may be one-sided, or two-sided H1: bj 0 and H1: bj 0 c0 a 1 a One-Sided Alternatives

8、(cont) Fail to reject reject 多元线性回归模型：假设检验12 Examples 1 Hourly Wage Equation H0: bexper = 0 H1: bexper 0 316. 0526 )003. 0()0017. 0()007. 0()104. 0( 022. 0exp0041. 0092. 0284. 0)log( 2 Rn tenureereducgeaw 多元线性回归模型：假设检验13 One-sided vs Two-sided Because the t distribution is symmetric, testing H1: bj

9、0 is straightforward. The critical value is just the negative of before We can reject the null if the t statistic than c then we fail to reject the null For a two-sided test, we set the critical value based on a/2 and reject H1: bj 0 if the absolute value of the t statistic c 多元线性回归模型：假设检验14 yi = b0

10、 + b1Xi1 + + bkXik + ui H0: bj = 0 H1: bj 0 c0 a/2 1 a -c a/2 Two-Sided Alternatives reject reject fail to reject 多元线性回归模型：假设检验15 Summary for H0: bj = 0 Unless otherwise stated, the alternative is assumed to be two-sided If we reject the null, we typically say “xj is statistically significant at the

11、 a % level” If we fail to reject the null, we typically say “xj is statistically insignificant at the a % level” 多元线性回归模型：假设检验16 Examples 2 Determinants of College GPA colGPAcollege GPA(great point average), hsGPAhigh school GPA skippedaverage numbers of letures missed per week. 234. 0141 )026. 0()0

12、11. 0()094. 0()33. 0( 083. 0015. 0412. 039. 1 2 Rn skippedACThsGPAAPcolG 多元线性回归模型：假设检验17 Testing other hypotheses A more general form of the t statistic recognizes that we may want to test something like H0: bj = aj In this case, the appropriate t statistic is teststandard for the 0 where, j j jj a

13、se a t b b 多元线性回归模型：假设检验18 Examples 3 Campus Crime and Enrollment H0: benroll = 1 H1: benroll 1 585. 0)11. 0()03. 1 ( 97)log(27. 163. 6) log( 2 R nenrollmeicr uenrollcrime)log()log( 10 bb 多元线性回归模型：假设检验19 Examples 4 Housing Prices and Air Pollution H0: blog(nox) = -1 H1: blog(nox) - 1 581. 0506 )006.

14、 0()019. 0()043. 0()117. 0()32. 0( 052. 0255. 0)log(134. 0)log(954. 008.11)log( 2 Rn stratioroomsdistnoxprice ustratioroomsdistnoxprice 43210 )log()log()log(bbbbb 多元线性回归模型：假设检验20 Confidence Intervals Another way to use classical statistical testing is to construct a confidence interval using the sam

15、e critical value as was used for a two-sided test A (1 - a) % confidence interval is defined as ondistributi ain percentile 2 -1 theis c where, 1 kn jj t sec a bb 多元线性回归模型：假设检验21 Computing p-values for t tests An alternative to the classical approach is to ask, “what is the smallest significance lev

16、el at which the null would be rejected?” So, compute the t statistic, and then look up what percentile it is in the appropriate t distribution this is the p-value p-value is the probability we would observe the t statistic we did, if the null were true 多元线性回归模型：假设检验22 Most computer packages will com

17、pute the p-value for you, assuming a two-sided test If you really want a one-sided alternative, just divide the two-sided p-value by 2 Many software,such as Stata or Eviews provides the t statistic, p-value, and 95% confidence interval for H0: bj = 0 for you 多元线性回归模型：假设检验23 Testing a Linear Combinat

18、ion Suppose instead of testing whether b1 is equal to a constant, you want to test if it is equal to another parameter, that is H0 : b1 = b2 Use same basic procedure for forming a t statistic 21 21 bb bb se t 多元线性回归模型：假设检验24 Testing Linear Combo (cont) 2112 2 1 12 2 2 2 121 212121 2121 , of estimate

19、an is where 2 , 2 then, Since bb bbbb bbbbbb bbbb Covs ssesese CovVarVarVar Varse 多元线性回归模型：假设检验25 Testing a Linear Combo (cont) So, to use formula, need s12, which standard output does not have Many packages will have an option to get it, or will just perform the test for you More generally, you can

20、 always restate the problem to get the test you want 多元线性回归模型：假设检验26 Examples 5 Suppose you are interested in the effect of campaign expenditures on outcomes Model is voteA = b0 + b1log(expendA) + b2log(expendB) + b3prtystrA + u H0: b1 = - b2, or H0: q1 = b1 + b2 = 0 b1 = q1 b2, so substitute in and

21、 rearrange voteA = b0 + q1log(expendA) + b2log(expendB - expendA) + b3prtystrA + u 多元线性回归模型：假设检验27 Example (cont): This is the same model as originally, but now you get a standard error for b1 b2 = q1 directly from the basic regression Any linear combination of parameters could be tested in a simila

22、r manner Other examples of hypotheses about a single linear combination of parameters: nb1 = 1 + b2 ; b1 = 5b2 ; b1 = -1/2b2 ; etc 多元线性回归模型：假设检验28 Multiple Linear Restrictions Everything weve done so far has involved testing a single linear restriction, (e.g. b1 = 0 or b1 = b2 ) However, we may want

23、 to jointly test multiple hypotheses about our parameters A typical example is testing “exclusion restrictions” we want to know if a group of parameters are all equal to zero 多元线性回归模型：假设检验29 Testing Exclusion Restrictions Now the null hypothesis might be something like H0: bk-q+1 = 0, . , bk = 0 The

24、 alternative is just H1: H0 is not true Cant just check each t statistic separately, because we want to know if the q parameters are jointly significant at a given level it is possible for none to be individually significant at that level 多元线性回归模型：假设检验30 Exclusion Restrictions (cont) To do the test

25、we need to estimate the “restricted model” without xk-q+1, , xk included, as well as the “unrestricted model” with all xs included Intuitively, we want to know if the change in SSR is big enough to warrant inclusion of xk-q+1, , xk edunrestrict isur and restricted isr where, 1 knSSR qSSRSSR F ur urr

26、 多元线性回归模型：假设检验31 The F statistic The F statistic is always positive, since the SSR from the restricted model cant be less than the SSR from the unrestricted Essentially the F statistic is measuring the relative increase in SSR when moving from the unrestricted to restricted model q = number of restr

27、ictions, or dfr dfur n k 1 = dfur 多元线性回归模型：假设检验32 The F statistic (cont) To decide if the increase in SSR when we move to a restricted model is “big enough” to reject the exclusions, we need to know about the sampling distribution of our F stat Not surprisingly, F Fq,n-k-1, where q is referred to as

28、 the numerator degrees of freedom and n k 1 as the denominator degrees of freedom 多元线性回归模型：假设检验33 0c a1 a f(F) F The F statistic (cont) reject fail to reject Reject H0 at a significance level if F c 多元线性回归模型：假设检验34 Example:Major League Baseball Players Salary urbisyrhrunsyrbavggamesyryearssalary 543

29、210 )log(bbbbbb 6278. 0186.183353 )0072. 0()0161. 0()00110. 0( 0108. 00144. 000098. 0 )0026. 0()0121. 0()29. 0( 0126. 00689. 010.11)log( 2 RSSRn rbisyrhrunsyrbavg gamesyryearssalary ugamesyryearssalary 210 )log(bbb 5971. 0311.198353 )0013. 0()0125. 0()11. 0( 0202. 00713. 022.11)log( 2 RSSRn gamesyry

30、earssalary 多元线性回归模型：假设检验35 Relationship between F and t Stat The F statistic is intended to detect whether any combination of a set of coefficients is different from zero, The t test is best suited for testing a single hypothesis. Group a bunch of insignificant varialbes with a significant variable,

31、 it is possible conclude that the entire set of variables is jointly insignificant. Often, when a variable is very statistically significant and it is tested jointly with another set of variables, the set will be jointly significant. 多元线性回归模型：假设检验36 The R2 form of the F statistic Because the SSRs may be large and unwieldy, an alternative form of the f