北大暑期课程《回归分析》(Linear-Regression-Analysis)讲义

Class1,PageClass1,PageClass1:Expectations,variances,andbasicsofestimationBasicsofmatrix(1)OrganizationalMatters⑴Courserequirements:Exercises:Therewillbeseven⑺exercises,thelastofwhichisoptional.Eachexercisewillbegradedonascaleof0-10.Inadditiontothegradedexercise,ananswerhandoutwillbegiventoyouinlabsections.Examination:Therewillbeonein-class,open-bookexamination.)Computersoftware:StataII.TeachingStrategiesEmphasisonconceptualunderstanding.Yes,wewilldealwithmathematicalformulas,actuallyalotofmathematicalformulas.But,Idonotwantyoutomemorizethem.WhatIhopeyouwilldo,istounderstandthelogicbehindthemathematicalformulas.Emphasisonhands-onresearchexperience.Yes,wewillusecomputersformostofourwork.ButIdonotwantyoutobecomeacomputerprogrammer.Manypeoplethinktheyknowstatisticsoncetheyknowhowtorunastatisticalpackage.Thisiswrong.Doingstatisticsismorethanrunningcomputerprograms.WhatIwillemphasizeistousecomputerprogramstoyouradvantageinresearchsettings.Computerprogramsarelikeautomobiles.Thebestautomobileisuselessunlesssomeonedrivesit.Youwillbethedriverofstatisticalcomputerprograms.Emphasisonstudent-instructorcommunication.Ihappentobelieveinstudents'judgmentabouttheirowneducation.EventhoughIwillbeultimatelyresponsibleiftheclassshouldnotgowell,Ihopethatyouwillfeelpartoftheclassandcontributetothequalityofthecourse.Ifyouhavequestions,donothesitatetoaskinclass.Ifyouhavesuggestions,pleasecomeforwardwiththem.Theclassisasmuchyoursasmine.Nowletusgettotherealbusiness.III(1)ExpectationandVarianceRandomVariable:Arandomvariableisavariablewhosenumericalvalueisdeterminedbytheoutcomeofarandomtrial.Twoproperties:randomandvariable.Arandomvariableassignsnumericvaluestouncertainoutcomes.Inacommonlanguage,"giveanumber".Forexample,incomecanbearandomvariable.Therearemanywaystodoit.Youcanusetheactualdollaramounts.Inthiscase,youhaveacontinuousrandomvariable.Oryoucanuselevelsofincome,suchashigh,median,andlow.Inthiscase,youhaveanordinalrandomvariable[1=high,2=median,3=low].Orifyouareinterestedintheissueofpoverty,youcanhaveadichotomousvariable:1=inpoverty,0=notinpoverty.

Insum,themappingofnumericvaluestooutcomesofeventsinthiswayistheessenceofarandomvariable.ProbabilityDistribution:TheprobabilitydistributionforadiscreterandomvariableXassociateswitheachofthedistinctoutcomesxi(i=1,2,…，k)aprobabilityP(X=xi).CumulativeProbabilityDistribution:ThecumulativeprobabilitydistributionforadiscreterandomvariableXprovidesthecumulativeprobabilitiesP(Xx)forallvaluesx.ExpectedValueofRandomVariable:TheexpectedvalueofadiscreterandomvariableXisdenotedbyE{X}anddefined:E{X}=P(xi)where:P(xi)denotes P(X=xi).ThenotationE{}(readexpectationof)iscalledtheexpectationoperator.Incommonlanguage,expectationisthemean.Butthedifferenceisthatexpectationisaconceptfortheentirepopulationthatyouneverobserve.Itistheresultoftheinfinitenumberofrepetitions.Forexample,ifyoutossacoin,theproportionoftailsshouldbe.5inthelimit.Ortheexpectationis.5.Mostofthetimesyoudonotgettheexact.5,butanumberclosetoit.ConditionalExpectationItisthemeanofavariableconditionalonthevalueofanotherrandomvariable.Notethenotation:E(Y|X).In1996,per-capitaaveragewagesinthreeChinesecitieswere(inRMB):Shanghai: 3,778Wuhan:where:P(xi)denotes P(X=xi).ThenotationE{}(readexpectationof)iscalledtheexpectationoperator.Incommonlanguage,expectationisthemean.Butthedifferenceisthatexpectationisaconceptfortheentirepopulationthatyouneverobserve.Itistheresultoftheinfinitenumberofrepetitions.Forexample,ifyoutossacoin,theproportionoftailsshouldbe.5inthelimit.Ortheexpectationis.5.Mostofthetimesyoudonotgettheexact.5,butanumberclosetoit.ConditionalExpectationItisthemeanofavariableconditionalonthevalueofanotherrandomvariable.Notethenotation:E(Y|X).In1996,per-capitaaveragewagesinthreeChinesecitieswere(inRMB):Shanghai: 3,778Wuhan: 1,709Xi'an: 1,155VarianceofRandomVariable:Theanddefined:varianceofadiscreterandomvariable XisdenotedbyV{X}V{X}=(xi-E{X})2P(xi)where:P(xi)denotesP(X=xi).ThenotationV{}(readoperator.varianceof")iscalledthvarianceSincethevarianceofarandomvariable Xisaweightedaverageofthesquareddeviations,(X-E{X})2,itmaybedefinedequivalentlyasanexpectedvalue: V{X}=E{(X-E{X})2}.Analgebraicallyidenticalexpressionis: V{X}=E{X2}-(E{X})2.StandardDeviationofRandomVariable:ThepositivesquarerootofthevarianceofXiscalledthestandarddeviationofXandisdenotedby{X}:{X}=Thenotation{}(readstandarddeviationof")iscalledandarddeviationoperator.StandardizedRandomVariables:IfXisarandomvariablewithexpectedvalueE{X}andstandarddeviation{X},then:Y=X E{X}{X}isknownasthestandardizedformofrandomvariableThenotation{}(readstandarddeviationof")iscalledandarddeviationoperator.StandardizedRandomVariables:IfXisarandomvariablewithexpectedvalueE{X}andstandarddeviation{X},then:Y=X E{X}{X}isknownasthestandardizedformofrandomvariableX.Covariance:Theanddefined:covarianceoftwodiscreterandomvariablesXandYisdenotedbyCov{X,Y}Cov{X,Y}=where:P(xi,yj)denotes)ThenotationofCov{,}(read"covarianceof")iscalledcovarianceoperator.WhenXandYareindependent,Cov{X,Y}=0.Cov{X,Y}=E{(X-E{X})(Y-E{Y})};Cov{X,Y}=E{XY}-E{X}E{Y}(Varianceisaspecialcaseofcovariance.)CoefficientofCorrelation:ThecoefficientofcorrelationoftworandomvariablesXandYisdenotedby{X,Y}(Greekrho)anddefined:X;{Y}isthestandarddeviationofY;Covisthewhere:{X}isthestandarddeviationX;{Y}isthestandarddeviationofY;CovisthecovarianceofXandY.SumandDifferenceofTwoRandomVariables:IfXandYaretworandomvariables,thentheexpectedvalueandthevarianceofX+Yareasfollows:ExpectedValue:E{X+Y}=E{X}+E{Y};Variance:V{X+Y}=V{X}+V{Y}+2Cov(X,Y).IfXandYaretworandomvariables,thentheexpectedvalueandthevarianceofareasfollows:ExpectedValue:E{X-Y}=E{X}-E{Y};Variance:V{X-Y}=V{X}+V{Y}-2Cov(X,Y).SumofMoreThanTwoIndependentRandomVariables:IfT=X1+X2+...+Xsisthesumofsindependentrandomvariables,thentheexpectedvalueandthevarianceofTareasfollows:ExpectedValue:Variance:III(2).PropertiesofExpectationsandCovariances:⑴PropertiesofExpectationsunderSimpleAlgebraicOperationsE(abX)abE(x)Thissaysthatalineartransformationisretainedaftertakinganexpectation.X*abXiscalledrescaling:aisthelocationparameter,bisthescaleparameter.Specialcasesare:Foraconstant:E(a)aForadifferentscale:E(bX)bE(X),e.g.,transformingthescaleofdollarsintothescaleofcents.PropertiesofVariancesunderSimpleAlgebraicOperations2V(abX)bV(X)Thissaystwothings:(1)Addingaconstanttoavariabledoesnotchangethevarianceofthevariable;reason:thedefinitionofvariancecontrolsforthemeanofthevariable[graphics].(2)Multiplyingaconstanttoavariablechangesthevarianceofthevariablebyafactoroftheconstantsquared;thisistoeasyprove,andIwillleaveittoyou.Thisisthereasonwhyweoftenusestandarddeviationinsteadofvariance2xxisofthesamescaleasx.PropertiesofCovarianceunderSimpleAlgebraicOperationsCov(a+bX,c+dY)=bdCov(X,Y).Again,onlyscalematters,locationdoesnot.PropertiesofCorrelationunderSimpleAlgebraicOperationsIwillleavethisaspartofyourfirstexercise:(abX,cdY)(X,Y)Thatis,neitherscalenorlocationaffectscorrelation.IV:Basicsofmatrix.DefinitionsMatricesToday,Iwouldliketointroducethebasicsofmatrixalgebra.Amatrixisarectangulararrayofelementsarrangedinrowsandcolumns:XiiXi2.…x〔mX21XnmIndex:rowindex,columnindex.Dimension:numberofrowsxnumberofcolumns(nxm)Elements:aredenotedinsmallletterswithsubscripts.Anexampleisthespreadsheetthatrecordsthegradesforyourhomeworkinthefollowingway:Name1st2nd6thA7109B658...Z...89……8Thisisamatrix.Notation:IwilluseCapitalLettersforMatrices.VectorsVectorsarespecialcasesofmatrices:Ifthedimensionofamatrixisnx1,itisacolumnvector:xix2x...xnIfthedimensionis1xm,itisarowvector:y'=|y〔yYm|Notation:smallunderlinedlettersforcolumnvectors(inlecturenotes)TransposeThetransposeofamatrixisanothermatrixwithpositionsofrowsandcolumnsbeingexchangedsymmetrically.Forexample:ifX11X12.…XimX(X(nm)X21X11XnmX11XnmX21.…Xn1X'(mn)X'(mn)x12xlmXxlmXnmItiseasytoseethatarowvectorandacolumnvectoraretransposesofeachother.2.MatrixAdditionandSubtractionAdditionsandsubtractionoftwomatricesarepossibleonlywhenthematriceshavethesamedimension.Inthiscase,additionorsubtractionofmatricesformsanothermatrixwhoseelementsconsistofthesum,ordifference,ofthecorrespondingelementsofthetwomatrices.X11X11 y11X21 y21X1m y1mXn1Xn1 yn1Xnm ymnExamples:12(22) 34(22)MatrixMultiplicationMultiplicationofascalarandamatrixMultiplyingascalartoamatrixisequivalenttomultiplyingthescalartoeachoftheelementsofthematrix.cxilCX12.…CX1mCX21cXcxnicxnmMultiplicationofaMatrixbyaMatrix(InnerProduct)TheinnerproductofmatrixX(axb)andmatrixY(cxdexistsifbisequaltoc.Theinnerproductisanewmatrixwiththedimension(axd).TheelementofthenewmatrixZis:cZijxikYkjk=1NotethatXYandYXareverydifferent.Veryoften,onlyoneoftheinnerproducts(XYandYX)exists.Example:A(2x2)B(2x1)BAdoesnotexist.ABhasthedimension2x12AB4Otherexamples:IfA(3x5),B(5x3),whatisthedimensionofAB?(3x3)IfA(3x5),B(5x3),whatisthedimensionofBA?(5x5)IfA(1x5),B(5x1),whatisthedimensionofAB?(1x1,scalar)IfA(3x5),B(5x1),whatisthedimensionofBA?(nonexistent)SpecialMatricesSquareMatrixAC(nn)SymmetricMatrixAspecialcaseofsquarematrix.F