统计学习[TheElementsofStatisticalLearning]第二章习题

1、 # TheElementofStatisticalLearningChapter2 HYPERLINK mailto:oxstarS.JTUoxstarS.JTUJcinuciry3?2011Ex.2.1SupposeeachofclasseshasanassociatedtargetwhichisavectorofallzerosexceptnoneintheA：thpositionShowthatclassifyingtothelargestekunentofynmountstochoosingtheclosesttarget,mi】以训，iftheelementsofysumtoone

2、.ProofOurgoalisprovingarginaxfc（?/）=arginin|/人一d|Infact,arginin|一训=nrgmin|t*：-y|2/|如-训ispositiveK=argminZ（如）：一y,）2/expandingArt=lK=arginin5?（）2一2g除+yi2）kt=iK=argininy（A：）i22（力）皿）/itemyt2doesiftcontainvariablekkt=i=arginin（若（如）/_2g（如）皿）=argininI2=arginax=mgininI1/definitionof=zgmax（呱）/wheni=k、以=1.els

3、e“=0Ex.2.2ShowhowtocomputetheBayesdecisionboundaryforthesimulationexampleinFigure2.5.AnswerThebookshowshowtogeneratethetrainingdtita.ThismodelcontainstwoclasseswhicharelabeledBLUEandORANGEr（）（tively.L4kGBLUE,ORANGE,andw（useGtorepresentclasskEachdasarecodedto心viadummyvariables,whereTOC o 1-5 h z“BLUE

4、=（1,0）丁（1）“ORANGE=（,1）丁Firstly10ineaiisinfromabivariateGaussiandistribution丫（“人；1）aregeneratedSoweha-eP（nu|gj=/（nujgl）（3） where1千一(“)2/(2异)(4)Thenforeacliclass100oljservatioiisaregeneratedfollows:foreachobservation,ail(m)iatraiKloinwithprobability1/10ispicked,andthengeneratedaN(mJ:1/5)：thusleadingto

5、amixtureofGaussianclustersforeachclassSowehaveP(X=工|(nu),GJ=召/(眄(111小1/5)101P(X=|m皿)=/(z;(mfc)f,I/5)(5)(6)Thevaluesofnuisunknown,soweshouldmarginalizethemout.P(x=xgk)=IP(X=xmk.Qk)P(mkQk)dmkFromeuation(2.23)intextbook,theBayes-optiinal(ixisionbouiKiarycanbecalculateclbythevaluesofxthatsatisfy“P(张ue|X

6、=z)=P(Grange|X=x)(8)Becauseeven-classescontainthesameaniountoftrainingdata,soP(0BLUE)=P(GoRANGE)Baves?formulatollusr畑)P($)(10)Accordingtoequation(8)-(10),wehaveP(X=x|/BLUE)=P(X=r|ORANGE)(11)ThuswocallcalculateBavos-optiinaldecisionboundarybyequation(1)(11)Ex.2.4Theedgeeffectproblemdiscussedoni)age23

7、isnotpeculiartouniformsamplingfromboundeddomainsConsiderinputsdrawnfromasphericalinultinorinaldistributionXAr(0.Ip).Fhesejuareddistancefromanysamplepointtotheoriginhasajtdistributionwithmeanp.Considerapredictionpointr()drawnfromthisdistribution,andleta=o/llollboanassociatedunitvector.LetZi=betheproj

8、ectionofeachofthetrainingpointsonthisdirection.Showthatthec,aredistributedA(0.1)withexpectedsejuareddistancefromtheorigin1.whilethetargetpointhasexpectedscpiareddistancepfromtheoriginHenceforp=10,randomlydrawntestpointisabout3.1standarddeviationsfromtheorigin,whileallthetrainingpointsareonaverageone

9、standarddeviationalongdirectiona.Somostpredictionpointsseethemselvesaslyingonthecxlgeofthetrainingset.ProofzN(ai“)(Xt)jN(0.1),wherej1.2p幻(珀jN(0.町)益=刀雋勺(工小冷)=N(M)/.SciuttreddistiincefromtheoriginE(d0)2)=Var(zi)=1/Thesquareddistancedfromanysamplepointtotheoriginhasadistributionwithmeanp.Pargetpointhas

10、expectedsquarecidistanceE(d)fromtheorigin,andE(J)=p/Thereisarandomlydrawntestpointxtand(xt0)2=dthatVar(zt)=E(以一0)2)=E(d)=p .Forp=10,Std(xt)=/Var(xJ=彳=v/TOa3.1andStd(Ct)=/ar(C|)=1Ex.2.6()nsi(leraregressionproblemwithinputsz,anrloutputsandaparameterizcKlmodeltobefitbyleastsquares.Showthatifthereareobs

11、ervationswithtiedoridenticalvaluesoftlientlielitcanbeobtiiinedfromareducedweightedleastsquaresproblem.MMarginin2(yf7-/tf(xt)2=arginin-2曲小(珀+feM2)t/=1t=arg严工(出,-2N呂严=argmin(疇一2M耐)+Nifs(xi)2/MProofuIftherearcmultipleobservationpairsXi.yit,f=1,.,Arititeachvalueof航,theriskisliinited.(Page32)IVoshouldest

12、imatetheparameters6infebyliiininiizingtheresidualsuin-of-squares,i.e.calculatearginin0“尸，while=arginin牙(扫必-N质2)+N示-2Nfe(Xi)+Nifj=arginin一勿丁(眄)+0i=argming(砺-%(曲)尸0i=argininRSS(0)Inthiscase.thesolutionspasstliroughtheaveragevaluesofthey*ateach4：Sowecanuseleastweightedsquarestoestimatethei)araineters0inbEx.AdditionalDeriveequation(2.17)k22EPE%)=+/(%)_?工/他)+壬hz=iAnswerSupposethedataarisefromamodelY=JX)+J(X)andeareindependentSo(12)E(/(X)=E(/(X)E(f)ErrorsareonlyineandE(若)=0,Var(e)=a2,sowehaveVar(V)=Var(e)=E(e一E(f)2=E(e2),andVar(V)=E(e2)=a2(13