简单回归模型

第一章简朴回归模型y=b0+b1x+u要求：1、一般最小二乘估计措施(OLS)2、OLS旳统计特征1ContentsWhatisthesimpleregressionmodel?Howtoderivetheordinaryleastsquares(OLS)estimates?PropertiesofOLSstatisticsandR2UnbiasednessofOLS

andVariancesoftheOLSestimators2Whatisthesimpleregressionmodel?y=b0+b1

x+u3SomeTerminologyInthesimplelinearregressionmodel,wherey=b0+b1x+u,wetypicallyrefertoyastheDependentVariable,orLeft-HandSideVariable,orExplainedVariable,orResponseVariable,orRegressand4SomeTerminology,cont.y=b0+b1x+u

Inthesimplelinearregressionofyonx,wetypicallyrefertoxastheIndependentVariable,orRight-HandSideVariable,orExplanatoryVariable,orRegressor,orCovariate,orControlVariables5ASimpleAssumption y=b0+b1x+uTheaveragevalueofu,theerrorterm,inthepopulationis0.Thatis,E(u)=0Thisisnotarestrictiveassumption,sincewecanalwaysuseb0

tonormalizeE(u)to0wage=b0+b1educ+u6ZeroConditionalMean y=b0+b1x+uWeneedtomakeacrucialassumptionabouthowuandxarerelatedWewantittobethecasethatknowingsomethingaboutxdoesnotgiveusanyinformationaboutu,sothattheyarecompletelyunrelated.Thatis,thatE(u|x)=E(u)=0,whichimpliesE(y|x)=b0+b1x,whichisoftencalled

PopulationRegressionFunction(PRF)7..x1x2E(y|x)asalinearfunctionofx,whereforanyx

thedistributionofyiscenteredaboutE(y|x)E(y|x)=b0+b1xyf(y)PopulationRegressionFunctionHowtoestimatetheparameters

b0andb1?8Howtoderivetheordinaryleastsquares(OLS)estimates?9OrdinaryLeastSquaresBasicideaofregressionistoestimatethepopulationparametersfromasampleLet{(xi,yi):i=1,…,n}denotearandomsampleofsizenfromthepopulationForeachobservationinthissample,itwillbethecasethat

yi=b0+b1xi+ui10....y4y1y2y3x1x2x3x4}}{{u1u2u3u4xyPopulationregressionline,sampledatapointsandtheassociatederrortermsE(y|x)=b0+b1x11DerivingOLSEstimatesToderivetheOLSestimatesweneedtorealizethatourmainassumptionofE(u|x)=E(u)=0alsoimpliesthatCov(x,u)=E(xu)=0Why?RememberfrombasicprobabilitythatCov(X,Y)=E(XY)–E(X)E(Y)12DerivingOLScontinuedWecanwriteour2restrictionsjustintermsofx,y,b0andb1,sincey=b0+b1x+u,u=y–b0–b1xE(y–b0–b1x)=0E[x(y–b0–b1x)]=0Thesearecalledmomentrestrictions13DerivingOLSusingM.O.M.ThemethodofmomentsapproachtoestimationimpliesimposingthepopulationmomentrestrictionsonthesamplemomentsWhatdoesthismean?RecallthatforE(X),themeanofapopulationdistribution,asampleestimatorofE(X)issimplythearithmeticmeanofthesampleSinxi/n14MoreDerivationofOLSWewanttochoosevaluesoftheparametersthatwillensurethatthesampleversionsofourmomentrestrictionsaretrueThesampleversionsareasfollows:15MoreDerivationofOLSGiventhedefinitionofasamplemean,andpropertiesofsummation,wecanrewritethefirstconditionasfollows16MoreDerivationofOLS17SotheOLSestimatedslopeis18SummaryofOLSslopeestimateTheslopeestimateisthesamplecovariancebetweenxandydividedbythesamplevarianceofxIfxandyarepositivelycorrelated,theslopewillbepositiveIfxandyarenegativelycorrelated,theslopewillbenegativeOnlyneedxtovaryinoursample19Whyordinaryleastsquares(OLS)?Intuitively,OLSisfittingalinethroughthesamplepointssuchthatthesumofsquaredresidualsisassmallaspossible,hencethetermleastsquaresTheresidual,û,isanestimateoftheerrorterm,u,andisthedifferencebetweenthefittedline(sampleregressionfunction)andthesamplepoint20.y4y1y2y3xySampleregressionline,sampledatapointsandtheassociatedestimatederrortermsE(y|x)=b0+b1x}{.x1x2x3x4}û2û3û4.û1•{21AlternateapproachtoderivationGiventheintuitiveideaoffittingaline,wecansetupaformalminimizationproblemThatis,wewanttochooseourparameterssuchthatweminimizethefollowing:22Alternateapproach,continuedIfoneusescalculustosolvetheminimizationproblemforthetwoparametersyouobtainthefollowingfirstorderconditions,whicharethesameasweobtainedbefore,multipliedbyn23AnSimpleExample(PRtextbook)(1)y(grade-pointaverage)(2)x(incomeofparentsin$1,000)(3)xi-x(4)yi-y(5)(xi-x)(yi-y)(6)(xi-x)

24.021.07.51.07.556.253.015.01.5.0.02.253.515.01.5.5.752.252.09.0-4.5-1.04.520.253.012.0-1.5.0.02.253.518.04.5.52.2520.252.56.0-7.5-.53.7556.252.512.0-1.5-.5.752.25y=3.0x=13.5S(xi-x)=0S(yi-y)=0S(xi-x)(yi-y)=19.50S(xi-x)2=162.00b1=0.120b0=y-b1x=1.37524PropertiesofOLSstatisticsandR225AlgebraicPropertiesofOLSThesumoftheOLSresidualsiszeroThus,thesampleaverageoftheOLSresidualsiszeroaswellThesamplecovariancebetweentheregressorsandtheOLSresidualsiszeroTheOLSregressionlinealwaysgoesthroughthemeanofthesample26AlgebraicProperties(precise)27Moreterminology28ProofthatSST=SSE+SSR29Goodness-of-FitHowdowethinkabouthowwelloursampleregressionlinefitsoursampledata?Cancomputethefractionofthetotalsumofsquares(SST)thatisexplainedbythemodel,callthistheR-squaredofregressionR2=SSE/SST=1–SSR/SSTWhere,0≤R2≤1....ûixy30UsingStataforOLSregressionsNowthatwe’vederivedtheformulaforcalculatingtheOLSestimatesofourparameters,you’llbehappytoknowyoudon’thavetocomputethembyhandRegressionsinStataareverysimple,toruntheregressionofyonx,justtyperegyx31UnbiasednessofOLSandVariancesoftheOLSestimators32UnbiasednessofOLSUnbiasedness:Assumethepopulationmodelislinearinparametersasy=b0+b1x+uAssumewecanusearandomsampleofsizen,{(xi,yi):i=1,2,…,n},fromthepopulationmodel.Thuswecanwritethesamplemodelyi=b0+b1xi+ui

AssumeE(u|x)=0andthusE(ui|xi)=0

AssumethereisvariationinthexiRandomsamplemeansCov(ui,uj)=033UnbiasednessofOLS(cont)Inordertothinkaboutunbiasedness,weneedtorewriteourestimatorintermsofthepopulationparameterStartwiththeformula34UnbiasednessofOLS(cont)35UnbiasednessofOLS(cont)Then,36UnbiasednessSummary

TheOLSestimatesofb1andb0areunbiasedProofofunbiasednessdependsonour4assumptions–ifanyassumptionfails,thenOLSisnotnecessarilyunbiasedRememberunbiasednessisadescriptionoftheestimator–inagivensamplewemaybe“near”or“far”fromthetrueparameter37VarianceoftheOLSEstimators

NowweknowthatthesamplingdistributionofourestimateiscenteredaroundthetrueparameterWanttothinkabouthowspreadoutthisdistributionisMucheasiertothinkaboutthisvarianceunderanadditionalassumption,soAssumeVar(u|x)=s2(Homoskedasticity)38VarianceofOLS(cont)Var(u|x)=E(u2|x)-[E(u|x)]2E(u|x)=0,sos2

=E(u2|x)=E(u2)=Var(u)Thuss2isalsotheunconditionalvariance,calledtheerrorvariances,thesquarerootoftheerrorvarianceiscalledthestandarddeviationoftheerrorCansay:E(y|x)=b0+b1xandVar(y|x)=s2Mayintroducethepropertiesofvariance,var(cx)=c2var(x),Var(c+x)=var(x)39..x1x2HomoskedasticCaseE(y|x)=b0+b1xyf(y|x)40.x

x1x2yf(y|x)HeteroskedasticCasex3..E(y|x)=b0+b1x41VarianceofOLS(cont)42VarianceofOLSSummaryThelargertheerrorvariance,s2,thelargerthevarianceoftheslopeestimat