大学的英文线性回归讲义linearregression

Chapter17SimpleLinearRegressionandCorrelation17.1LinearRegressionAnalysis…Regressionanalysisisusedtopredictthevalueofonevariable(thedependentvariable)onthebasisofothervariables(theindependentvariables).Dependentvariable:denotedYIndependentvariables:denotedX1,X2,…,XkIfweonlyhaveONEindependentvariable,themodeliswhichisreferredtoassimplelinearregression.Wewouldbeinterestedinestimatingβ0andβ1fromthedatawecollect.2LinearRegressionAnalysisVariables: X=IndependentVariable(weprovidethis) Y=DependentVariable(weobservethis)Parameters:

β0=Y-Intercept

β1=Slope

ε~NormalRandomVariable(με=0,σε=???)[Noise]3EffectofLargerValuesofσε

HousesizeHousePrice25K$Samesquarefootage,butdifferentpricepoints(e.g.décoroptions,cabinetupgrades,lotlocation…)Lowervs.HigherVariabilityHousePrice=25,000+75(Size)+4TheoreticalLinearModel51.BuildingtheModel–CollectDataTest2Grade=β0+β1*(Test1Grade)FromData: Estimateβ0

Estimateβ1 Estimateσε6LinearRegressionAnalysis…7CorrelationAnalysis…“-1<

<1”Ifweareinterestedonlyindeterminingwhetherarelationshipexists,weemploycorrelationanalysis.Example:Student’sheightandweight.8CorrelationAnalysis…“-1<

<1”Ifthecorrelationcoefficientiscloseto+1thatmeansyouhaveastrongpositiverelationship.Ifthecorrelationcoefficientiscloseto-1thatmeansyouhaveastrongnegativerelationship.Ifthecorrelationcoefficientiscloseto0thatmeansyouhavenocorrelation.WEHAVETHEABILITYTOTESTTHEHYPOTHESIS H0:=09Regression:ModelTypes…X=sizeofhouse,Y=costofhouseDeterministicModel:anequationorsetofequationsthatallowustofullydeterminethevalueofthedependentvariablefromthevaluesoftheindependentvariables.

y=$25,000+(75$/ft2)(x)Areaofacircle:A=*r2ProbabilisticModel:amethodusedtocapturetherandomnessthatispartofareal-lifeprocess.y=25,000+75x+εE.g.doallhousesofthesamesize(measuredinsquarefeet)sellforexactlythesameprice?10SimpleLinearRegressionModel…Meaningofand>0[positiveslope]<0[negativeslope]

yxrunrise=slope(=rise/run)=y-intercept11Whichlinehasthebest“fit”tothedata????12EstimatingtheCoefficients…Inmuchthesamewaywebaseestimatesofon,weestimatewithb0andwithb1,they-interceptandslope(respectively)oftheleastsquaresorregressionlinegivenby:(Thisisanapplicationoftheleastsquaresmethodanditproducesastraightlinethatminimizesthesumofthesquareddifferencesbetweenthepointsandtheline)13LeastSquaresLine…Thislineminimizesthesumofthesquareddifferencesbetweenthepointsandtheline……butwheredidthelineequationcomefrom?Howdidweget.934foray-interceptand2.114forslope??thesedifferencesarecalledresidualsorerrors14LeastSquaresLine…[suregladwehavecomputersnow!]Thecoefficientsb1andb0fortheleastsquaresline……arecalculatedas:15DataStatisticsInformationDataPoints:xy16213945517612y=.934+2.114xLeastSquaresLine…SeeifyoucanestimateY-interceptandslopefromthisdataRecall…16LeastSquaresLine…SeeifyoucanestimateY-interceptandslopefromthisdata17Excel:DataAnalysis-Regression18Excel:PlottedRegressionModel–Youwillneedtoplayaroundwiththistogettheplottolook“Good”

19RequiredConditions…Fortheseregressionmethodstobevalidthefollowingfourconditionsfortheerrorvariable()mustbemet:•Theprobabilitydistributionofisnormal.•Themeanofthedistributionis0;thatis,E()=0.•Thestandarddeviationofis,whichisaconstantregardlessofthevalueofx.•Thevalueofassociatedwithanyparticularvalueofyisindependentofassociatedwithanyothervalueofy.20AssessingtheModel…Theleastsquaresmethodwillalwaysproduceastraightline,evenifthereisnorelationshipbetweenthevariables,oriftherelationshipissomethingotherthanlinear.Hence,inadditiontodeterminingthecoefficientsoftheleastsquaresline,weneedtoassessittoseehowwellit“fits”thedata.We’llseetheseevaluationmethodsnow.They’rebasedonthewhatiscalledsumofsquaresforerrors(SSE).21SumofSquaresforError(SSE–anotherthingtocalculate)…Thesumofsquaresforerroriscalculatedas:andisusedinthecalculationofthestandarderrorofestimate:Ifiszero,allthepointsfallontheregressionline.22StandardError…Ifissmall,thefitisexcellentandthelinearmodelshouldbeusedforforecasting.Ifislarge,themodelispoor…Butwhatissmallandwhatislarge?23StandardError…Judgethevalueofbycomparingittothesamplemeanofthedependentvariable().Inthisexample, =.3265and =14.841so(relativelyspeaking)itappearstobe“small”,henceourlinearregressionmodelofcarpriceasafunctionofodometerreadingis“good”.24TestingtheSlope…Exceloutputdoesthisforyou.Ifnolinearrelationshipexistsbetweenthetwovariables,wewouldexpecttheregressionlinetobehorizontal,thatis,tohaveaslopeofzero.Wewanttoseeifthereisalinearrelationship,i.e.wewanttoseeiftheslope()issomethingotherthanzero.Ourresearchhypothesisbecomes: H1:≠0Thusthenullhypothesisbecomes: H0:=0Alreadydiscussed!25TestingtheSlope…Wecanimplementthisteststatistictotryourhypotheses:H0:β1=0whereisthestandarddeviationofb1,definedas:Iftheerrorvariable()isnormallydistributed,theteststatistichasaStudentt-distributionwithn–2degreesoffreedom.Therejectionregiondependsonwhetherornotwe’redoingaone-ortwo-tailtest(two-tailtestismosttypical).26Example17.4…Testtodetermineiftheslopeissignificantlydifferentfrom“0”(at5%significancelevel)Wewanttotest: H1:≠0 H0:=0(ifthenullhypothesisistrue,nolinearrelationshipexists)Therejectionregionis:ORcheckthep-value.27Example17.4…WecancomputetmanuallyorrefertoourExceloutput…Weseethatthetstatisticfor“odometer”(i.e.theslope,b1)is–13.49whichisgreaterthantCritical=–1.984.Wealsonotethatthep-valueis0.000.Thereisoverwhelmingevidencetoinferthatalinearrelationshipbetweenodometerreadingandpriceexists.COMPUTEComparep-value28TestingtheSlope…Wecanalsoestimate(tosomelevelofconfidence)andintervalfortheslopeparameter,.Recallthatyourestimateforisb1.Theconfidenceintervalestimatorisgivenas:Hence:Thatis,weestimatethattheslopecoefficientliesbetween–.0768and–.057029CoefficientofDetermination…Teststhusfarhaveshownifalinearrelationshipexists;itisalsousefultomeasurethestrengthoftherelationship.Thisisdonebycalculatingthecoefficientofdetermination–R2.Thecoefficientofdeterminationisthesquareofthecoefficientofcorrelation(r),henceR2=(r)2rwillbecomputedshortlyandthisistrueformodelswithonly1indepenentvariable30CoefficientofDeterminationR2hasavalueof.6483.Thismeans64.83%ofthevariationintheauctionsellingprices(y)isexplainedbyyourregressionmodel.Theremaining35.17%isunexplained,i.e.duetoerror.Unlikethevalueofateststatistic,thecoefficientofdeterminationdoesnothaveacriticalvaluethatenablesustodrawconclusions.IngeneralthehigherthevalueofR2,thebetterthemodelfitsthedata.R2=1:Perfectmatchbetweenthelineandthedatapoints.R2=0:Therearenolinearrelationshipbetweenxandy.31RememberExcel’sOutput…Ananalysisofvariance(ANOVA)tableforthe

simplelinearregressionmodelcanbegiveby:SourcedegreesoffreedomSumsofSquaresMeanSquaresF-StatisticRegression1SSRMSR=SSR/1F=MSR/MSEErrorn–2SSEMSE=SSE/(n–2)Totaln–1Variation

iny(SST)32UsingtheRegressionEquation…Wecoulduseourregressionequation: y=17.250–.0669xtopredictthesellingpriceofacarwith40(40,000)milesonit: y=17.250–.0669x=17.250–.0669(40)=14,574Wecallthisvalue($14,574)apointprediction(estimate).Chancesarethoughtheactualsellingpricewillbedifferent,hencewecanestimatethesellingpriceintermsofaconfidence

interval.33PredictionIntervalThepredictionintervalisusedwhenwewanttopredictoneparticularvalueofthedependentvariable,givenaspecificvalueoftheindependentvariable:(xgisthegivenvalueofxwe’reinterestedin)34ConfidenceIntervalEstimatorforMeanofY…Theconfidenceintervalestimatefortheexpectedvalueofy(MeanofY)isusedwhenwewanttopredictanintervalweareprettysurecontainsthetrue“regressionline”.Inthiscase,weareestimatingthemeanofygivenavalueofx:(Technicallythisformulaisusedforinfinitelylargepopulations.However,wecaninterpretourproblemasattemptingtodeterminetheaveragesellingpriceofallFordTauruses,allwith40,000milesontheodometer)35What’stheDifference?PredictionIntervalConfidenceInterval1no1Usedtoestimatethevalueofonevalueofy(atgivenx)Usedtoestimatethemeanvalueofy(atgivenx)Theconfidenceintervalestimateoftheexpectedvalueofywillbenarrowerthanthepredictionintervalforthesamegivenvalueofxandconfidencelevel.Thisisbecausethereislesserrorinestimatingameanvalueasopposedtopredictinganindividualvalue.36RegressionDiagnostics…Therearethreeconditionsthatarerequiredinordertoperformaregressionanalysis.Theseare: •Theerrorvariablemustbenormallydistributed, •Theerrorvariablemusthaveaconstantvariance,& •Theerrorsmustbeindependentofeachother.Howcanwediagnoseviolationsoftheseconditions?ResidualAnalysis,thatis,examinethedifferencesbetweentheactualdatapointsandthosepredictedbythelinearequation…37Nonnormality…Wecantaketheresidualsandputthemintoahistogramtovisuallycheckfornormality……we’relookingforabellshapedhistogramwiththemeanclosetozero[ourold“testfornormality].38Heteroscedasticity…Whentherequirementofaconstantvarianceisviolated,wehaveaconditionofheteroscedasticity.Wecandiagnoseheteroscedasticitybyplottingtheresidualagainstthepredictedy.39Heteroscedasticity…Ifthevarianceoftheerrorvariable()isnotconstant,thenwehave“heteroscedasticity”.Here’stheplotoftheresidualagainstthepredictedvalueofy:theredoesn’tappeartobeachangeinthespreadoftheplottedpoints,thereforenoheteroscedasticity40NonindependenceoftheErrorVariableIfweweretoobservetheauctionpriceofcarseveryweekfor,say,ayear,thatwouldconstituteatimeseries.Whenthedataaretimeseries,theerrorsoftenarecorrelated.Errortermsthatarecorrelatedovertimearesaidtobeautocorrelatedorseriallycorrelated.Wecanoftendetectautocorrelationbygraphingtheresiduals

against

thetimeperiods.Ifapatternemerges,itislikelythattheindependencerequirementisviolated.41NonindependenceoftheErrorVariablePatternsintheappearanceoftheresidualsovertimeindicatesthatautocorrelationexists:Notetherunsofpositiveresiduals,replacedbyrunsofnegativeresidualsNotetheoscillatingbehavioroftheres