多元线性回归模型：估计.ppt

1,MultipleRegressionAnalysis,y=b0+b1x1+b2x2+.bkxk+u1.Estimation,2,ParallelswithSimpleRegression,b0isstilltheinterceptb1tobkallcalledslopeparametersuisstilltheerrorterm(ordisturbance)Stillneedtomakeazeroconditionalmeanassumption,sonowassumethatE(u|x1,x2,xk)=0Stillminimizingthesumofsquaredresiduals,sohavek+1firstorderconditions,3,theOLSregressionlineorthesampleregressionfunction(SRF).istheOLSinterceptestimateandaretheOLSslopeestimates.Stilluseordinaryleastsquarestogettheestimates:,4,OLSFirstOrderConditions,Thisminimizationproblemcanbesolvedusingmultivariablecalculas.Thisleadstok+1linearequationink+1unknown:,5,AFittedorPredictedValue,Forobservationi,thefittedvalueisTheresidualforobservationiisdefinedasinthesimpleregressioncase,Theproperties123Thepoint()isalwaysontheOLSregressionline:,6,InterpretingMultipleRegression,7,A“PartiallingOut”Interpretation,8,“PartiallingOut”continued,Previousequationimpliesthatregressingyonx1andx2givessameeffectofx1asregressingyonresidualsfromaregressionofx1onx2Thismeansonlythepartofxi1thatisuncorrelatedwithxi2arebeingrelatedtoyisowereestimatingtheeffectofx1onyafterx2hasbeen“partialledout”,9,SimplevsMultipleRegEstimate,10,Goodness-of-Fit,11,Goodness-of-Fit(continued),Howdowethinkabouthowwelloursampleregressionlinefitsoursampledata?Cancomputethefractionofthetotalsumofsquares(SST)thatisexplainedbythemodel,callthistheR-squaredofregressionR2=SSE/SST=1SSR/SST,12,Goodness-of-Fit(continued),13,MoreaboutR-squared,R2canneverdecreasewhenanotherindependentvariableisaddedtoaregression,andusuallywillincreaseBecauseR2willusuallyincreasewiththenumberofindependentvariables,itisnotagoodwaytocomparemodels,14,AssumptionsforUnbiasedness,Populationmodelislinearinparameters:y=b0+b1x1+b2x2+bkxk+uWecanusearandomsampleofsizen,(xi1,xi2,xik,yi):i=1,2,n,fromthepopulationmodel,sothatthesamplemodelisyi=b0+b1xi1+b2xi2+bkxik+uiE(u|x1,x2,xk)=0,implyingthatalloftheexplanatoryvariablesareexogenousNoneofthexsisconstant,andtherearenoexactlinearrelationshipsamongthem,15,TooManyorTooFewVariables,Whathappensifweincludevariablesinourspecificationthatdontbelong?Thereisnoeffectonourparameterestimate,andOLSremainsunbiasedWhatifweexcludeavariablefromourspecificationthatdoesbelong?OLSwillusuallybebiased,16,OmittedVariableBias,17,OmittedVariableBias(cont),18,OmittedVariableBias(cont),19,OmittedVariableBias(cont),20,SummaryofDirectionofBias,21,OmittedVariableBiasSummary,Twocaseswherebiasisequaltozerob2=0,thatisx2doesntreallybelonginmodelx1andx2areuncorrelatedinthesampleIfcorrelationbetweenx2,x1andx2,yisthesamedirection,biaswillbepositiveIfcorrelationbetweenx2,x1andx2,yistheoppositedirection,biaswillbenegative,22,TheMoreGeneralCase,Technically,canonlysignthebiasforthemoregeneralcaseifalloftheincludedxsareuncorrelatedTypically,then,weworkthroughthebiasassumingthexsareuncorrelated,asausefulguideevenifthisassumptionisnotstrictlytrue,23,VarianceoftheOLSEstimators,NowweknowthatthesamplingdistributionofourestimateiscenteredaroundthetrueparameterWanttothinkabouthowspreadoutthisdistributionisMucheasiertothinkaboutthisvarianceunderanadditionalassumption,soAssumeVar(u|x1,x2,xk)=s2(Homoskedasticity),24,VarianceofOLS(cont),Letxstandfor(x1,x2,xk)AssumingthatVar(u|x)=s2alsoimpliesthatVar(y|x)=s2The4assumptionsforunbiasedness,plusthishomoskedasticityassumptionareknownastheGauss-Markovassumptions,25,VarianceofOLS(cont),26,ComponentsofOLSVariances,Theerrorvariance:alargers2impliesalargervariancefortheOLSestimatorsThetotalsamplevariation:alargerSSTjimpliesasmallervariancefortheestimatorsLinearrelationshipsamongtheindependentvariables:alargerRj2impliesalargervariancefortheestimators,27,MisspecifiedModels,28,MisspecifiedModels(cont),Whilethevarianceoftheestimatorissmallerforthemisspecifiedmodel,unlessb2=0themisspecifiedmodelisbiasedAsthesamplesizegrows,thevarianceofeachestimatorshrinkstozero,makingthevariancedifferencelessimportant,29,EstimatingtheErrorVariance,Wedontknowwhattheerrorvariance,s2,is,becausewedontobservetheerrors,uiWhatweobservearetheresiduals,iWecanusetheresidualstoformanestimateoftheerrorvariance,30,ErrorVarianceEstimate(cont),df=n(k+1),ordf=nk1df(i.e.degr