简单线性回归模型.ppt

1 TheSimpleRegressionModel y b0 b1x u 2 SomeTerminology Inthesimplelinearregressionmodel wherey b0 b1x u wetypicallyrefertoyastheDependentVariable orLeft HandSideVariable orExplainedVariable orRegressand orResponseVariables 3 SomeTerminology cont Inthesimplelinearregressionofyonx wetypicallyrefertoxastheIndependentVariable orRight HandSideVariable orExplanatoryVariable orRegressor orCovariate orControlVariables 4 TheSimpleRegressionModel y b0 b1x uIftheotherfactorsinuareheldfixed sothatthechangeinuiszero u 0 thenxhasalineareffectony y b1 xif u 0 theerrorterm ordisturbance 5 Example2 1SoybeanYieldandFertilizeryield b0 b1fertilizer uThecoefficientb1measurestheeffectoffertilizeronyield holdingotherfactorsfixed yield b1 fertilizerExample2 2ASimpleWageEquationwage b0 b1educ ub1measuresthechangeinhourlywagegivenanotheryearofeducation holdingallotherfactorsfixed 6 ZeroConditionalMean WeneedtomakeacrucialassumptionabouthowuandxarerelatedWewantittobethecasethatknowingsomethingaboutxdoesnotgiveusanyinformationaboutu sothattheyarecompletelyunrelated Thatis thatE u x E u 0 whichimpliesE y x b0 b1xPopulationRegressionFunction PRF 7 x1 x2 E y x asalinearfunctionofx whereforanyxthedistributionofyiscenteredaboutE y x PRF E y x b0 b1x y f y 8 Themeaningof linear regression Whatdoes linear meanhere Lookingatequationy b0 b1x uThekeyisthatthisequationislinearintheparameters b0andb1 Therearenorestrictionsonhowyandxrelatetotheoriginalexplainedandexplanatoryvariablesofinterest 9 Themeaningof linear regression Arethesemodelslinearregressionmodel Why 10 OrdinaryLeastSquares BasicideaofregressionistoestimatethepopulationparametersfromasampleLet xi yi i 1 n denotearandomsampleofsizenfromthepopulationForeachobservationinthissample itwillbethecasethatyi b0 b1xi ui 11 y4 y1 y2 y3 x1 x2 x3 x4 u1 u2 u3 u4 x y Populationregressionline sampledatapointsandtheassociatederrorterms E y x b0 b1x 12 DerivingOLSEstimates Giventheintuitiveideaoffittingaline wecansetupaformalminimizationproblemThatis wewanttochooseourparameterssuchthatweminimizethefollowing 13 DerivingOLSEstimates continued Ifoneusescalculustosolvetheminimizationproblemforthetwoparametersyouobtainthefollowingfirstorderconditions 14 DerivationofOLS Giventhedefinitionofasamplemean andpropertiesofsummation wecanrewritethefirstconditionasfollows 15 MoreDerivationofOLS 16 SotheOLSestimatedslopeis 17 SummaryofOLSslopeestimate TheslopeestimateisthesamplecovariancebetweenxandydividedbythesamplevarianceofxIfxandyarepositivelycorrelated theslopewillbepositiveIfxandyarenegativelycorrelated theslopewillbenegativeOnlyneedxtovaryinoursample 18 SampleRegressionFunction SRF SampleRegressionFunction SRF Wecanwritetheslopeestimateas 19 ANoteonTerminology Weruntheregressionofyonx orsimplythatweregressyonx Wealwaysregressthedependentvariableontheindependentvariable 20 MoreOLS Intuitively OLSisfittingalinethroughthesamplepointssuchthatthesumofsquaredresidualsisassmallaspossible hencethetermleastsquaresTheresidual isanestimateoftheerrorterm u andisthedifferencebetweenthefittedline sampleregressionfunction andthesamplepoint 21 y4 y1 y2 y3 x1 x2 x3 x4 1 2 3 4 x y Sampleregressionline sampledatapointsandtheassociatedestimatederrorterms 22 AlgebraicPropertiesofOLS ThesumoftheOLSresidualsiszeroThus thesampleaverageoftheOLSresidualsiszeroaswellThesamplecovariancebetweentheregressorsandtheOLSresidualsiszeroTheOLSregressionlinealwaysgoesthroughthemeanofthesample 23 AlgebraicProperties precise 24 Example2 3 CEOSalaryandReturnonEquitySalary b0 b1roe u 25 Example2 4 WageandEducation 26 Example Thedemandfunctionofbooks 27 Moreterminology Wecanthinkofeachobservationasbeingmadeupofanexplainedpart andanunexplainedpart Then wedefinethefollowing ThenSST SSE SSR 28 ProofthatSST SSE SSR 29 Goodness of Fit Howdowethinkabouthowwelloursampleregressionlinefitsoursampledata Cancomputethefractionofthetotalsumofsquares SST thatisexplainedbythemodel callthistheR squaredofregressionR2 SSE SST 1 SSR SST 30 AssumptionsofSimpleLinearRegression Assumethepopulationmodelislinearinparametersasy b0 b1x uAssumewecanusearandomsampleofsizen xi yi i 1 2 n fromthepopulationmodel Thuswecanwritethesamplemodelyi b0 b1xi uiAssumeE u x 0andthusE ui xi 0AssumethereisvariationinthexiAssumeVar u x s2 Homoskedasticity 31 x1 xi u1 ui x y PRFE y x b0 b1x y1 yi 32 Homoskedasticity Var u x E u2 x E u x 2E u x 0 sos2 E u2 x E u2 Var u Thuss2isalsotheunconditionalvariance calledtheerrorvariances thesquarerootoftheerrorvarianceiscalledthestandarddeviationoftheerrorCansay E y x b0 b1xandVar y x s2 33 x1 x2 HomoskedasticCase E y x b0 b1x y f y x 34 x x1 x2 y f y x HeteroskedasticCase x3 E y x b0 b1x 35 SampleVarianceoftheOLSEstimators 36 VarianceofOLSSummary Thelargertheerrorvariance s2 thelargerthevarianceoftheslopeestimateThelargerthevariabilityinthexi thesmallerthevarianceoftheslopeestimateAsaresult alargersamplesizeshoulddecreasethevarianceoftheslopeestimateProblemthattheerrorvarianceisunknown 37 EstimatingtheErrorVariance Wedon tknowwhattheerrorvariance s2 is becausewedon tobservetheerrors uiWhatweobservearetheresiduals iWecanusetheresidualstoformanestimateoftheerrorvariance 38 ErrorVarianceEstimate cont 39 ErrorVarianceEstimate cont 40 TheGauss MarkovTheorem Givenour5Gauss MarkovAssumptionsitcanbeshownthatOLSis BLUE BestLinearUnbiasedThus iftheassumptionshold useOLS 41 LinearityofOLS 42 UnbiasednessofOLS Inordertothinkaboutunbiasedness weneedtorewriteourestimatorintermsofthepopulationparameterStartwithasimplerewriteoftheformulaas 43 UnbiasednessofOLS cont 44 UnbiasednessofOLS cont 45 UnbiasednessofOLS cont 46 UnbiasednessSummary TheOLSestimatesofb1andb0areunbiasedProofofunbiasednessdependsonour4assumptions ifanyassumptionfails thenOLSisnotnecessarilyunbiasedRememberunbiasednessisadescriptionoftheestimator inagivensamplewemaybe near or far fromthetrueparameter 47 EfficiencyofOLS Assumeb1 isanotherlinearunbiasedestimatorofb1isanunbiasedestimator so 48 EfficiencyofOLS cont 49 RegressionThroughTheOrigin ObtaintheequationiscalledregressionthrouththeoriginToobtaintheslopeestimate westillrelyonthemethodofordinaryleastsquares thatismininizingthesumofsquaresresiduals Usingcalculus mustsolvethefirstordercondition 50 ChangingUnitsofMeasurementonOLSStatistics InExample2 3 wemeasureannualsalaryinthousandsofdollars thereturnonequitywasmeasuredasapercent ratherthanadecimal LetsalarydolbesalaryindollarsDefineroedec roe 100tobethedecimalequivalentofroe 51 NonlinearitiesinSimpleRegression Recallthewage educationexample whereweregressionhourlywageonyearsofeducation WeusealogarithmicformInparticular if u 0 then wage 100b1 educThisequationimpliesanincreasingreturntoeducation 52 AConstantElasticityModel ExampleCEOSalaryandFirmSalesWecanestimateaconstantelasticitymodelrelatingCEOsalarytofirmsales Letsalesbeannualfirmsales measuredinmillionsofdollars log salary b0 b1log sales uLetthedependentvariabletobey log salary andtheindependen