苏良军高等数理统计课后习题答案

PAGEPAGE1AdvancedMathematicalChapter1ProvetheIsthereanyeventwhichisindependentofitself?Ifso,giveanThestatedresultisimmediatefromP(Ac∩Bc)=P((A∪B)c)=1−P(A∪B)=1−P(A) P(B)+P(A∩B)=1−P(A)−P(B)+P(A)P(B)=(1−P(A))(1−P(B))=P(A)P(B) ForanyeventA,wehaveP(A∩Ω)=P(A)=P(A)P(Ω),andP(A∩∅)=P(∅)P(A)P(∅)=0,asYes.Fromthedefinitionofindependence,aneventAisindependentofitselfifP(A∩A)P(A)P(A),i.e.,P(A)=1,or0.Thus,Ωand∅areobviousexamplesofsuchan3LetAandBbetwoeventswithprobabilitiesP(A)=,P(B)=,andP(A∩B)=DefinerandomvariablesX=1(A)andY=1(B),where1(A)=1ifAistrueand0FindthejointdistributionofXandObtaintheconditionaldistributionofXgivenY.ThejointdistributionofXandYisgiven1P(X=1,Y=1)=P(A∩B)=3 236P(X=1,Y=0)=P(A∩B)=P(A)−P(A∩B) 236333P(X=0,Y=1)=P(Ac∩B)=P(B)−P(A∩B)=2−1=1333P(X=0,Y=0)=P(Ac∩Bc)=1−P(A∪B)=1−P(A)−P(B)+P(A∩ =1−2−3+3=6TheconditionaldistributionofXgivenYfollowsP(X=1,Y=P(X=1|Y=1) P(Y=

P(A∩P

==32 ==323 ∩P(X=0|Y=1)=1−P(X=1|Y=1)=1−2=2∩P(X=1|Y=0)

P(X=1,Y=P(Y=

P P

==62 ==623 P(X=0|Y=0)=1−P(X=1|Y=0)=1−2=204.LetY≥0bearandomvariableandp>0.ProveE(Yp)=R∞pyp−1P(Y>y)0LetY˜FY.

Z∞pyp−1P(Y>y) =Z∞Z∞

(Y)Y YZ=py−Z∞ZZ=py−0000 ∞YpdF=E(Yp)Y0Y6ThejointprobabilitydensityfunctionofXandYisgivenf(x,y)=3(x+y)1{0≤x+y≤1,0≤x,y≤1}FindthemarginaldensityofFindP{ω|X(ω)+Y(ω)<}FindE(Y|X)FindCov(X,Y)ThemarginaldensityofXisgivenf(x)02.(R1−x3(x+y)dy=3¡1−x2¢,iff(x)02.P{XP{X+Y<}223(x+y)dydx

Z1Z1 00TheconditionaldensityofYgivenX=x2for0≤x+y≤1,0≤x,y≤1.2

3(x+y)f(y|x)=3(1−x2)=

2(x+y)(1−x2)ZZE(Y|X=x)0

2(x+y(1−x2)dy

(1−x)(x+2),3(1+x)(d)E(X)=R13x¡1−(d)E(X)=R13x¡1−x2¢dx=3.Byasymmetricargument,E(Y)=3.0 Z1Z 1

E(XY)

3xy(x+y)dydx Cov(X,Y)=E(XY)−E(X)E(Y)=10−88=−3201.7LetXandYhavethejointdensitygivenf(x,y)=8xy1(0≤x≤y≤1)(a)ComputeE(X),E(Y),andE(X/Y)(b)NoticethatE(X/Y)=E(X)/E(Y).Isittrueingeneral?Ifnot,explainbrieflywhythisholds(a)(a)ThemarginaldensityofXisf(x)=R18xydy=4¡x−x3¢for0≤x≤1,and0otherwise.E(X)=R14x¡x−x0Similarly,themarginaldensityof0Similarly,themarginaldensityofYisf(y)=Ry8xydx=4y3for0≤y≤1,and0otherwise.R E(Y)=04ydy=5

EµX

Z1Z1

=2=8xydydx (b)Itistrue

EE(Y

==5

x2 =3

YµX¶Ywhich,however,isnottrueingeneral.ThereasonwhythisholdshereisthatthetworandomX/YandYareuncorrelated,YYYCovµX,Y¶=EµXY¶−EµX¶E(Y)=YYY9AmedianofadistributionisavaluemsuchthatP(X≤m)≥2andP(X≥m)≥.Ifmiscontinuous,msatisfiesRmf(x)dx=R∞f(x)dx=,wheref(x)isthepdfofm2Findthemedianoff(x)=3x,0<x<2ShowthatisXisacontinuousrandomvariable,aminE|X−a|=E|X−m|awheremisthemedianofX0SinceXiscontinuoushere,themedianmZm3x2dx=1Zm That2Sothemedianism=¡1¢1/32

m3=12

ZE|X−Z (x−a)f(x)dx

∞Z− x)f(x)Z−aTakingderivativewithrespecttoa,we−f(x)dxf−f(x)dxf(x)dx=aRThat R

f(x)dx=R∞f(x)dx.Consequently,a=msolvestheminimizationproblem.(Oneaa11ShowthatE|X|p≤(E|X|q)p/qforall0≤p≤q(e.g.,E|X|≤³EX22E|X|p=Eh(|X|q)p/qi≤(E|X|q)p/q 16Letπ(Z)=E(X|Z)whereZ=(Z1,Z2)andXisarandomvectorwith EXXT|Z1≥Eπ(Z)π(Z)T|Z1Answer.Letε=X−π(Z).ThenE(ε|Z)=XXT =(π(Z)+ε)(π(Z)+ =π(Z)π(Z)T+εεT+π(Z)εT+επ £ Consequently,EXXT|Z=π(Z)π(Z)T+EεεT £ EXXT =EEXXT|Z=Enhπ(Z)π(Z)T+E¡εεT|Z¢i=E³π(Z)π(Z)T1´+E¡εεT≥E³π(Z)π(Z)T117Provethevariancedecompositionformula.Thatis,var(X)=E(var(X|Y))+var(E(X|Y))providedvar(X)<∞.Answer.BythelawofiteratedE(var(X|Y))+var(E(X|Y=nE£E¡X2|Y¢¤−E³[E(X|Y)]2´o+nEh[E(X|Y)]2i−[E(E(X|Y=E¡X2¢−[E(X)]2=var(X)19.ForanytworandomvariablesX,YandZwithfinitevariances,proveinthespiritofvariancedecompositionthecovarianceidentityCov(X,Y)=E(Cov(X,Y|Z))+Cov(E(X|Z),E(Y|Z))whereCov(X,Y|Z)isthecovarianceofXandYundertheconditionalpdff(x,y|z)of(X,Y)Z=Bythepropertiesof(conditional)covariancesandlawofiteratedE[Cov(X,Y|Z)]+Cov(E(X|Z),E(Y={E[E(XY|Z)−E(X|Z)E(Y|Z)]}+{E[E(X|Z)E(Y|Z)]−E[E(X|Z)]E[E(Y=E(XY)−E(X)E(Y)=Cov(X,Y)21.LetXbeexponentiallydistributedwithpdff(x)=θexp(−θx)1(x≥0),θ>(a)FindthemomentgeneratingfunctionMX(t)forX,wheret<θ(b)VerifyE(X)=1andE¡X2¢=2θ(a)Them.g.f.ofXisgivenXM(t)=E¡etX¢=Z∞etxθe−θxdx=Z∞X θ θ−tprovidedθ−t>(Ift≥θ,MX(t)=∞.) (Alternatively,usingintegrationbyparts,wehaveE(X)=R∞xθe−θxdx=−R∞xde−θx=(Alternatively,usingintegrationbyparts,wehaveE(X)=R∞xθe−θxdx=−R∞xde−θx=0R∞ 0 dx=θ

¡

EX=dt2MXEX=dt2MX(t)|t=0=(θ−t)3|t=0=θ223Findthemean,variance,andm.g.f.offortheGamma(α,β)distributionwithp.d.f.givenf(x;α,β)0(Hint.Γ(z)=R∞0

Γ

xα−1e−βxdx,0<x<∞,α>0,β>E¡Xk¢

∞ZZ0ZZ

Γ

=Γ

∞0=Γ

Γ(α+k)

Γ(α+βkΓIn

α(α+1)...(α+k−1)αE(X) ,Var(X)β

α(α+

µαββ

=β2Next,them.g.f.

M(t)=

¡etX¢

Γ

∞Z0ZZZ=Γ

∞0 Γ

µβWhent≥β,M(t)=

β−

fort<25.LetXandYberandomvariableswithfinitesecondmoments.Show minE m(X))2=E E(Y|X)]2 wherem(x)rangesoverallfunctions.(Remark.E(Y|X)iscalledtheregressionofYonX.Itisthe“best”predictorofYconditionalonX.)Bythelawofiterated E[Y−m =E{[Y−E(Y|X)]+[E(Y|X)−m =E[Y−E(Y|X)]2+E[E(Y|X)−m +2E{[Y−E(Y|X)][E(Y|X)−m =E[Y−E(Y|X)]2+E[E(Y|X)−m2≥E[Y−E(Y|X)]LetXi˜Poisson(λi),i=1,2,andX1areindependentofShowthatX1+X2is˜Poisson(λ1+λ2)ShowthedistributionofX1|X1+X2isbinomialwithsuccessprobabilityλ1/(λ1+λ2)BytheindependenceofX1andX2,weXxXP(X1+X2=x)=

P(X1=x1,X2=x−XxXP(X1=x1)P(X2=x−x =x =

(x−xλ21=exp(−(λ1+λ2))xλ21

x1x1=0x1!(x−,=(λ1+λ2)xexp(−(λ1+λ2)),=wherethelastequalityfollowsfromthefact1+2x=

Ãx

λx1λx−x1,Ãx! x1 x1

x1!(x−LetT=X1+X2.TheconditionalpdfofX1givenT=tP[X1=x1|X1+X2= P[X1=x1andX2=t−P[X1+X2=11

2×2

à !

¶x1

x1! (t−x1)!

λ1+

λ1+Thatis,X1|T=t˜Binomial³t,λ1´.Inotherwords,X1|X1+X2isBinomial(X1+X2,λ1/(λ1+PAGEPAGE1AdvancedMathematicalChapter1.LetXbearandomvariablewith2 (2 ff(x)

if1<x<Findamonotonefunctionu(x)suchthattherandomvariableY=u(X)hasauniform(0,1)Bythepropertyofprobabilityintegraltransform,weknowthatthecdfofacontinuousrandomvariableisuniformlydistributedon(0,1).ItsufficestofindthecdfofX:FX(x)1(x−FX(x)1(x− if1≤x≤.4⎩ ifx>4SoY=FX(X)hasauniform(0,1)LetXbearandomvariablewithdistributionfunctionF.DefineX+=max(X,0)andX−=max(−X,0).ObtainthedistributionfunctionofX+andX−.ItfollowsdirectlyfromthedefinitionsofX+andX−

FX+(x)=

≤xª

P{∅}forx< (P{X≤x}=F(x)forx≥(FX−(x)=

≤xª

P{∅}forx<(P{−X≤x}=1−P{X<−x}=1−(

−

.forx≥ whereF(−x)−meansthatwehavetosubtracttheprobabilitymassat−xifanyfromF(−x). FisrightcontinuousLetX1andX2betwoindependentUniform(0,2)randomFindthepdfofY=X1+FindthepdfofZ=NotethatthejointpdfofX1andX21f(x1,x2)=41(0≤x1≤2)1(0≤x2≤2)Ã Ã ! X1+X2 ThenX1=Y1−Y2andX2=Y2.ItiseasytoverifytheJacobianoftheabovetransformationis1.SothejointpdfofY1andY2isfY1,Y2(y1,y2)=Integratingy2out,weobtainthepdfofZ2

41(0≤y1−y2≤2)1(0≤y2≤141(y1−2≤y2≤y1)1(0≤y2≤2)0fY1(y1)0=

41(y1−2≤y2≤y1)1(0≤y2≤2)41 if0≤y1≤441(4− if2<y1≤4Thisisalsothepdfof

⎩ Ã ! Ã !ThenX1=Z1Z2andX2=Z2.ItiseasytoverifytheJacobianoftheabovetransformationisz2.SothejointpdfofZ1andZ2is12412 (z,z) z21(0≤zz≤2)1(012412

≤2 z21(0≤z≤2/z)1(0≤2

Integratingz2out,weobtainthepdfoffZ1(z1)

20 41(0≤z2≤2/z1)1(0≤z2≤0 =

2111

if0≤z1≤1ifz1>1Thisisalsothepdfof

⎩ LetX˜N(0,1).LetY˜χ2(k)independentofX.ObtainthedensityofarandomvariabledefinedbyqXqU Yk(Hint.LetV=YandfindthejointdensityofUandVAnswer.LetV=Y.FromthedefinitionofUandV,we1X=

1UV2andY=V,where−∞<U<∞,V≥0.SotheJacobianofthetransformationisgiven=.Ã=.J=J=

∂x

áv ¡1¢1/2u= k=k 2

³vkkConsequently,thejointdensityofUandVis,for−∞<u<∞,v>f(u,v)=f(x(u,v))f(y(u,v))2vk−1exph−1³1+u2´2 2 22(k+1)/2(kπ)1/2Γ¡kBysomecalculation,itfollowsthatthedensityofUΓ¡k+1

¡¢f(u)¡¢

1+22(kπ)1/2Γ 22

,−∞<u<whichisthedensityofthestudentt-distributionwithkdegreesofZ(b)LetZi,i=1,...,n,beindependentnormalrandomvariables.ShowthatX=2i(a)LetZbeastandardnormal.ShowthatX=Z2isdistributedaccordingZ(b)LetZi,i=1,...,n,beindependentnormalrandomvariables.ShowthatX=2idistributedaccordingtoχ2(n).(Hint:Usethemomentgeneratingfunctiontechniqueandrecallthatthem.g.f.ofχ2(n)is(1−2t)−n/2.)Method1:Usingthetransformationtechnique:ThetransformationX=Z2isnotone-to-one.Butfirstwecanconsidertheone-sidedone-to-onetransformation:X∗=Z2,Z≥0.Here,Z=√X∗.SotheJacobian andthep.d.f.ofZ,Z≥0,isf(z)=

|J|=dx∗=2√x∗2exp³−z2´1{z≥0}.Consequentlythep.d.f.ofX∗2

³xfX∗(x)=2√x√2π−1{x≥0}TheoriginaltransformationX=Z2issymmetric,sothep.d.f.fX∗(x)=2√x√2π−1{x≥0}1 ³x´fX(x)=2fX∗(x)=√2πexp− √x,x≥Thatis,X=Z2isdistributedaccordingtoMethod2:Usingthedistributiontechnique.ThedistributionfunctionofX=FZ¡√x¢−FZ¡−√x¢FX(x)=P(X≤x=FZ¡√x¢−FZ¡−√x¢DifferentiatingwithrespecttoxandusingthechainrulegivesfX(x)√2π−−√2πfX(x)√2π−−√2π

³x

11 ³x´2√x−√2π−=−which2√x−√2π−=−Frompart(a),weknowthatZ2isdistributedasχ2(1).Them.g.f.ofZ2 Z2ii2 (t)=Eexp¡tZ2¢=(1−2t)−1Z2ii2iNotethattheindependenceofZi,i=1,...,n,impliestheindependenceofZ2,i=1,...,n.Sothei!2ofX=!2

Z2isgivenitMX(t)=Eit

Ã

n=nn=n

ininZiThusX˜χ2(n)Zi11.LetX1andX2beindependentrandomvariables.LetX1andY=X1+X2havePoissondistributionwithmeansλ1andλ>λ1,respectively.FindthedistributionofX2.(Hint.Themgftechniqueishandyhere.)tSinceX1isPossion(λ1),itsmgfisgivenbyMX1(t)=exp((e−1)λ1).Similarly,themgfYisgivenbyMY(t)=exp((et−1)λ).Ontheotherhand,bytheindependenceofX1andX2,MY(t)=MX1(t)MX2(t)andhenceMX2(t)=exp((eMX2(t)=exp((et−1)λ)=1

−

(λ−λ1)¢Thatis,X2∼Possion(λ− LetX1,X2andX3beiidNμ,σ2. Y1=X1+aX3,andY2=X2+FindthemeansandvariancesofY1andY2andtheircorrelationFindthemgfofFindthejointmgfofY1andE(Y1)=(1+a)μ,E(Y2)=(1+a)μ,Var(Y1)=(1+a2)σ2,Var(Y2)=(1+a2)σ2,Cov(Y1,

Corr(Y1,Y2)=(1+a2)σ2=(1+a2)σ2=1+a2ThemgfofY1MY1(t)=Eexp[t(X1+=E[exp(tX1)]E[exp 12

122===

tμ+t2tt(1+a)μ

t1+12t1+22

taμ2σσ

ta..ThejointmgfofY1andY2MY1,Y2(t)=E[exp(t1Y1+=E[exp(t1X1)]E[exp(t2X2)]E[exp(a(t1+t2)=t1μ+2=t1μ+2

12t2μ+2×at2μ+2×a(t1+t2)μ+2(t1+σ1221212=expµ(t+t)(1+a)μ+1³t2+t2+a2(t+t21221212122121=expµ(t+t)(1+a)μ+1©¡1+a2¢t2+¡1+a2¢t2+2a2ttªσ2¶122121LetXandYhaveabivariatenormaldistribution.ShowthatX+YandX−YareindependentifandonlyifVar(X)=Var(Y).Since(X,Y)followsabivariatenormaldistribution,thenX+YandY−YarejointlyHencetheyareindependentifandonlyifCov(X+Y,X−Y)=0.ThatVar(X)−Cov(X,Y)−Var(Y)+Cov(X,Y)=LetX=(X1,...,Xn)TbearandomvariablewithmeanμιandvarianceΣ,whereμisscalar,ιisthen-vectorofones,andΣisann×nsymmetricmatrix.WedefineX=1 Xi,iS2=1i

¡X−X¢2.Considerthefollowing

A1.Xhasmultivariatenormaldistribution;A2.Σ=σ2In;A3.μ=WeXandS2are(c)E¡S2¢=(c)E¡S2¢= X˜Nμ,σ2/n√ (n−1)S2/σ2˜χ2(n−1)√ nX−μ/S˜t(n−1)youspecified.AssumptionsA1andA2.ThisisshowninthelecturenNoneofAssumptionsA1-A3isrequired.E¡X¢=E¡1n

Xi¢=1

E(Xi)=1 =nnAssumptionA2.E¡S2¢=1E=nn

¡X−X2=1E

X2−X2=1(nσ2nnσ2)=n

¡ ¡AssumptionsA1-A2.Alinearcombinationofnormalrandomvariablesisnormal.ItsufficestoverifythatEX=μandVarX=σ2¡ ¡AssumptionsA1-A2.ShowninthelectureAssumptionsA1-A2.Showninthelectureà !LetXandYhavebivariatenormaldistributionwithmeanà ! Findaconstantβ∗suchthatY−β∗XisindependentofX.ShowthatVar(Y−βX)≥Var(Y−β∗X)foranyconstantβ.ObtainE(X|X+Y) Forthefirstpart,notingthatbothY−β∗XandXarenormallydistributed,itsufficestofindβ∗suchthatCov(Y−β∗X,X)=0.Cov(Y−β∗X,X)=1−β∗1=0,soβ∗=1.Var(Y−βX)=2−2β+β=(β−1)+1≥1wheretheequalityholdsifandonlyifβ=β=1 Ã!!First,itiseasytofindthejointdistributionofX+YandX−YÃ!!˜X+ ˜X−

-

!!SotheconditionaldistributionofX+YgivenX−YisN(μ1.2,Σ11.2),μ1.2=3+(−1)1−1(X−Y−(−1))=Y−X+

ÃÃ

NÃÃ

!!

!!3˜SotheconditionaldistributionofXgivenX+YisN(μ1.2,Σ11.2),3˜ =1+2×5−1(X+Y−3)=2(X+Y)−1

1 =1−2×155Consequently,E(X|X+Y)=2(X+Y)−155

×2 5entiabledensityfX(x).SupposeY1hastheLaplacedistributionin{X1,...,Xn},thatisY1=Xiwithprobability1/n.SupposeY2iscontinuouslydistributedwithp.d.f.givenbyk(y2)withsupportonR,andY1andY2areindependent.Conditionalon{X1,...,Xn},findthep.d.f.fn(y)ofY≡Y1+bnY2,wherebnisapositivesequencethatconvergesto0asn→LetFn(y1)denotetheempiricaldistributionoftherandomsample{X1,...,Xn}.ThenY1˜Fn(y1)NotingthatY1andY2areindependent,sothejointdistributionF(y1,y2)ofY1andY2dF(y1,y2)=k(y2)dy2dFn(y1)andthedistributionfunctionofYisgivenFn(y)=P{Y1+bnY2≤Z∞Z k(y2)dy2dFn−∞n=Z∞Kµy−y1¶dF(yn 11 nnn

y−Xt whereK(x)=R

=1

k³y−Xt´PAGEPAGE5AdvancedMathematicalChapterLetXandYbetworandomvariablesdefinedonacommonprobabilityspace(Ω,F,P).IsitpossiblethatXandYhavethesamedistribution,andyetX=Ya.s.,i.e.,P{ω|X(ω)=Y(ω)}=1.Ifso,giveanexample.Otherwise,brieflyexplainwhynot.Yes.Forexample,letΩ={H,T},F={{H},{T},∅,Ω},andP{H}=.Definetworan-domvariables:X=1(H),andY=1(T).Obviously,XandYhavethesamedistributionbutP{ω|X(ω)=Y(ω)}=1.Alternatively,letX˜N(0,1)andY=−X.ItiseasytoseethatXandYhavethesamedistributionbutP{ω|X(ω)=Y(ω)}=1.3.SupposeX1,...,Xnarei.i.d.Poisson(λ).LetXn=1

(a)Findthem.g.f.ofYnnXn(a)Findthem.g.f.ofYnnXn−/.(Hint.(b)(b)Findthelimitingdistributionof

dYifdYifthem.g.f.(t)ofm.g.f.MY(t)ofYintheneighborhoodof0asn→SinceX1,...,Xnarei.i.d.Poisson(λ),nXnisPoisson(nλ).For−∞<t<MYn(t)=E[exp=Ehexp³t(nXn)/√nλ´iexp∞=∞

etx/√nλe−nλ∞ =e−t==exp³−t√nλ+nλ³et/√nλ−1´´2TofindthelimitingdistributionofYn,noticingthatex=1+x+1x2+o¡x2¢asx→0,we2nlimMY(t) limexp³−t√nλ+nλ³et/√nλ−nnλ+nλ+

Ã

1µt2,2,2√√ √√

µt222nwhichisthem.g.f.ofthestandardnormaldistribution.Consequently,Yn

N(0,1)SupposeX1,...,Xnarei.i.d.Exponential(1)withp.d.f.f(x)=x<x<∞}.LetXn1

Findthem.g.f.of Findthem.g.f.ofYn=√nXn−1FindthelimitingdistributionofThem.g.f.ofX1

(t)=E[exp(tX)]=Z∞Them.g.f.ofYn

1=(1−t)−11nMYn(t)=E[exp(tYn)]=nnMYn(t)=E[exp(tYn)]=nn11=0n=n1n½Z =0n=n1n

√wheret<

=het/√n¡1−¡t/√n¢¢i−n2Sinceln(1−x)=−x−1x2+o¡x2¢asx→0,we2lim (t) limhet/√n¡1−

exp©−n£t/√n+ln¡1−

"√√√−"√√√−

(−nt−

1µt = 2

nwhichisthem.g.f.ofthestandardnormaldistribution.Consequently,Yn

N(0,1)AnswertheShowthatif

L2X,then

L1Suppose

n nd→N(0,1)andY=d

(1).FindthelimitingdistributionofX2+XY

nBytheJensen’sE|Xn−X|

³E

´1´222− ItisthenimmediatethatXLXimpliesXL n nnnX2+ =X2+Op(1)op dn=X2+op(1)→χ2(1)nLetX1,...,Xnbei.i.d.withmeanμandvarianceσ2.FindtheprobabilitylimitPZn= PZn= −nnnwhereX=1 nni→1nipByWLLNfori.i.d.data,1 E¡X2¢=μ2+σ2,andX=1 ni→1nip

pμ.By ¡X−X¢2=1

X2−X2

¡μ2+σ2¢−μ2=σ2.Consequently,ntheSlutsky’s

pZn

μ2+ 1Remark.TheoriginalquestionaskstofindthelimitingdistributionofZnbuttheconditionisnotenough.AssumingthatE¡X4¢<∞,youcandothisbutitiscomplicated:1¶√nµZn¶

μ2+σ2i√ 1 i

μ2+ i i

nPnP

−X¢2

1

¡X−ni¡√ μ2+ μ2+σ2ni¡P+P

−X¢2 nn nn

£ ¡

1.1.

Xi

μ+

+opItiseasytosee

+√n¡μ2+σ2¢nn

− Pn X£ ¡Pn

2¢¤

X¢

11

Xi

μ+

→ hb2d1

−μ)2i´b2 ¡X−X¢2.Sobythedelta 1√¡1

2¢µ

1¶

¡μ2+σ2¢2Varh(X−nμ+

σ2−

→N

⎠bOnecanconjecturethat√nb

−μ2+σ2´

N(0,Σ),butitisnoteasytogiveanexplicitformulaXSuppose(X1,Y1),...,(Xn,Yn)arei.i.d.randomvectorswithVar(X1)=X

<∞andY=Y

<∞.

1X¡X−X

¢2nn

n−

1X¡Y−Y

¢2

n−

n−inin 1X¡X−X¢¡Y−Yn−ininShowthat pσ2,and p

asn→X,n→p Y,n→ShowthatSXY,n→σXY≡Cov(X1,Y1)FindastatisticthatconvergesinprobabilitytoρXY=Corr(X,Y).ShowthatthestatisticconvergesinprobabilitytoρXY.ppWeshowonly

Xasn→∞because

σ2 Ybyanalogousargument.Byσ2σni→1X2pWLLNfori.i.d.sample,1 E¡X2¢=σni→1X2p

X+σ2,andX

=1

nni μ.Asn→nnin/(n−1)→1.SobytheSlutsky’snn−1in−n n 1XX2nn−1in−n →(σx+μX)−xnn=σ2xnn(b)BytheCauchy-Schwartzinequality,E((b)BytheCauchy-Schwartzinequality,E(XY)≤©E¡X2¢E¡Y2¢ª1/2<∞.BytheWLLNnBytheSlutsky’s

ii11X E(XY),ii11

=1

i1E(X),andi1

=1

ni1E(Y)ni1nn 1X¡X−nn

¢¡Y−Y

n−

n

n n 1XXY−nXnn−1p

i

n−→E(X1Y1)−E(X1)E=σXY≡Cov(X1,Y1)Giventheresultsinparts(a)-(b),thedesiredstatisticρ=q2q2XY,nXYX,nXY,nXY,nXYX,nXY,nY

p ,

pσ2,and p

asn→∞.Itfollows Cov(X,YSupposeasn→

ρ→pVar(X)pVar(Y)=ρXY90√n90

bbb

!−Ã

!!d

à Ã

!!õqb Findthelimitingdistributionof αnβn−õqb bn

−1=

³n−1´and

−9=

qb

b

nnaround(α,β)tonn2nn2βp2nqb2nn2βp2n

=pαβ+1rβ

−α)+1rα

−β´+o

−α)+o³b

−22αn2αn=pαβ+1rβ22αn2αnLetLetα=1andβ=9.We

−α)+1rα

−β´+

³n−1´2βnnµqb2βnn

−3¶ 1√9√n(b

−1)+1r1√n

−´+obn2n29n 3√nbn2n29n6nd6n

−1)+1√n³b

−´+

→N(0,3.5)92(b92

−α)+1√n³b

−1n ×1+1n

×9+2n3612n361

×2=6LetXn˜χ2(n) FindthelimitingdistributionofYn=Xn− FindthelimitingdistributionofZn Xn SinceXn∼χ2(n),wecanwriteXnXnXn=XZ2iiiwhereUiareiidN(0,1).NotingthatE¡U2−1¢=0andVar¡U2−1¢=2,wehavebytheCLTiin√nYn=Ui−→N(0,n√nYn=Ui−→N(0,2)BytheWLLN,

pE¡Z2¢=1.BytheSlutskytheorem,wendnd

n

1+ n

r→rXn−

1nZn=n

q → 0, 1+n1+n

nnnpShowthat√n¡X−p¢dN(0,p(1−p))nnnpShowthatforp=,theestimateofvarianceX¡1−X¢√n¡X¡1−X¢−p(1−p)¢dN³0,(1−2p)2p(1−p´Showthatforp=

nµX¡1−X¢−1¶

1χ2(1) 4→−(Hint.NotethatX1−X≤,sothelefthandsideisalwaysnegative. 4→−ThethefollowsfromtheLindeberg-LevyCLTLetφ(p)=p(1−p).Thenφ0(p)=1−2p,whichisnonzeroprovidedp=.TheresultthenfollowsbythefirstorderDeltamethod.Whenp=,φ0(p)=0,φ”(p)=−2,24√nµX−1¶dNµ0,124X¡1−X¢=1+−2¡X−1/2¢2+o³¡XX¡1−X¢=1+−2¡X−1/2¢2+o³¡X−2 4→−nX¡1−X¢− =−£√n¡X−1/2¢¤24→−

1χ2(1) Let{X1,...,Xn}denotearandomsamplefromadistributionwithatwicecontinuouslydif-ferentiabledensityfX(x)whosederivativesareboundedoveritssupport.SupposeY1isuniformlydistributedon{X1,...,Xn},Y2iscontinuouslydistributedwithp.d.f.givenbyk(y2)withsupportonR,andY1andY2areindependent.FromQuestion1.17,wehaveknownthatthep.d.f.fn(y)ofY≡Y1+bnY2conditionalon{X1,...,Xn}is fn(y)

11nbn

y−Xt wherebnisapositivesequencethatconvergesto0asn→Notethatfn(y)dependsontherandomsample{X1,...,Xn}.FindtheprobabilitylimitoffnForthefollowingquestions,assumethatRRuk(u)du=0,RRu2k(u)du<∞,RRk du<forsomeδ>0,nbn→∞andnb5→ Findthelimit nbn[E(fn(y))−f(y)] (c)Findthelimitof(c)Findthelimitofk. (d)Findthelimitingdistributionof

nbn[n(y)−f(y)](a)Bythefn

kµy−Xtn∙nbn n∙→ 1→

µy−Xt Z

1kµy−x¶

(x)X Zk(u)fX(y−bnu)XX Zk(u)[X(y)−bnufX

(y+cbnu)]X fX(y)Zk(u)du−bnZuk(u)fX

(y+cbnu)→fX(y)asn→wherec∈[0,1].Thatisf(y)=fX(y) (b)SincefX(x)istwicecontinuouslydifferentiablewithboundedderivatives,RRuk(u)du=0,Ru2k(u)du<∞,weE(fn(y))−fX=Z∞1kµy−x¶X

(x)dx−

Z∙−∞ Z∙kk

fX(y)+bfX(y)+bnufX(y)+2bnufX(y+

−

XnXuk(u)du+2Xn=bf0(y) 12Zk(u)uXnXuk(u)du+2Xn

u)2nXnn 1b2Zk(u)u2f”(y+cbu)du=O¡b2¢2nXnnnnnSo√nbn[E(fn(y))−f(y)]=√nbnO¡b2¢=O³pnb5´=o(1)asnb5→∞.Thatnnn

pnbn[E(fn(y))−f(y)]→0asn→∙∙

µ√kµ√

µy−Xtµ√µ√Z Z

√Ek√Z∙Z∙

µy−Xt ∞ −∞

y−x

−fX(x)dx−Z∙Z∙

√k√

1n k(u)fX(y+bnu)du−n

¸k(u)fX(y+bnu)¸ ¸ Zk(u)2duf(y)+ ¸

)−1O(b4X→Zk(u)2dufX(y)

(d)Bytheresultsinparts(b)and(c)andtheCLTfori.i.d.random11µy−Xtµµy−Xtpnbn[n(y)−f(y)]=pnbn[n(y)−E(fn(y))]+11µy−Xtµµy−Xtnn√ √

−E

+dn→Nµ0,Zk(u)2dufX(y)¶dn 15.TheBox-Mullermethodforgeneratingnormalpseudo-randomvariablesisbasedonthetrans- X1=cos −2lnU2,X2=sin −2lnwhereU1andU2areiidUniform(0,1).ProvethatX1andX2areindependentN(0,1)random⎨u1=1arctan³x2Notethatthetransformationfrom(U1,U2)⎨u1=1arctan³x2⎩u=exp³−x2+x2´Thus,theJacobianJJ=det

2³2³

221x1+´³1 221x1+´³1´⎠−.−1− 11−

−x2 12 Notingthatthepdfof(U1,U2)Tf(u1,u2)=1(0<u1<1)1(0<u2<1)weobtainthepdfof(X1,X2)Tf(x1,x2)−12,−∞<f(x1,x2)−12,−∞<x1,x2< 2Thatis,X1andX2areindependentN(0,1)randomAdvancedMathematicalChapter ³b Letθnbeanestimatorofaparameterθ0.Provethefollowingclaim:Iflimn Eθn=θ0andlimn→∞Varθn=0 ³b Notethat

θ

´=Eθ

−θ2θ

´+hE

´−θi2→0asn→∞.Sofor2°ε>0, n−2°ε>0, n−θ0°>εn0→0asn→∞.Thatis,→pb

−θ00nn00nnLetT1andT2betwoindependentunbiasedestimatorsforθsuchthatVar(T1)=2Var(T2).T=c1T1+FindtheconditionunderwhichE(T)=Findthevaluesc1andc2suchthatTisanunbiasedestimatorofθwiththesmallestpossiblevarianceforsuchalinearcombination.SinceE(T1)=E(T2)=θ,E(T)=c1E(T1)+c2E(T2)=(c1+c2)θ=θprovidedc1+c2=LetVar(T1)=σ2,thenVar(T2)=2σ2.Considertheclassofunbiasedestimatorsofθ:TcT1+(1−c)T2.WeneedtochoosectoVar(cT1+(1−c)T2)=c2σ2+2(1−c)2Itiseasytofindthesolutionisgivenbyc∗=2/3.Soweneedtochoosec1=2/3andc2= LetX1,...,Xnbearandomsamplefromtheunderlyingdistributiongivenbythedensity f(x;θ) 1 θ}FindtheMLEofShowthatT=max{X1,...,Xn}is =(max{X1,...,Xm},max{Xm+1,..., =(max{X1,...,Xm},min{Xm+1,...,where1<m<n.DiscussthesufficiencyofS1and f(x;θ)

θ2nΠi=1xi1{0≤xi≤≤≤=θ2n ≤≤=θ2n 1max ,itfollowsthatθML=max{X1,...,Xn}Writethejointp.d.f.asf(x;θ)=g(τ(x);θ)h(x),τ(x)=max{x1,...,½g(τ;θ)= 1{τ≤θ}½h(x)=

xi1

{xi}¾ItfollowsfromtheFisher-NeymanFactorizationTheoremthatT=τ(X)=max{X1,...,Xn}isareasonisasfollows:θMLmaximizesf(x;θ)withrespectto(w.r.t.)θ,sothatitmaximizesf(τ(x);θ) w.r.t.θ.Therefore,theMLEθMLisalwaysafunctionofT=τ(x),whichimpliesthatθMLisminimalifθMLissufficient.Generallyspeaking,iftheMLEisasufficientstatistic,thenitisaminimal NotethatSissufficientifT=ϕ(S)forsomefunctionϕissufficient.ForS1,wehaveT=max{X1,...,Xn}=max{S1}.ThereforeS1isasufficientstatistic.ForS2tobesufficient,thereissomefunctionϕsuchthatT=ϕ(S2)forminimalsufficientstatisticT=max{X1,...,Xn},whichisobviouslyimpossible.LetX1,...,Xnbei.i.d.Bernoulli(p)FindastatisticthatiscompleteandFindaUMVUEforFindaUMVUEforg(p)=p(1−p)ThejointlikelihoodfunctionofX1,...,Xnf(x;p)=

pxi(1−p)1−xi1(xi=0or区nnµp

n=(1−

1−

Πi=11(xi=0or1)BytheFisher-NeymanFactorizationTheorem,T=

Xiissufficient.SinceBenoullibelongstotheexponentialfamily,itiscompletenLetX=1 Xi.ThenE¡X¢=E(X1)=p.SinceXisacompleteandsufficientnXistheUMVUEbytheLehmann-ScheffeTofindaUMVUEforg(p),itsufficestofindanunbiasedestimatorforg(p)thatisafunctionofT.ItfollowsfromE(nT)=n2p,E¡T

=V

i+

=np(1−p)+thatE£nT−T2¤=n(n−1)p(1−p).ThisimpliesT∗

nT−Tn(n−isunbiasedforp(1−p).ItisalsotheUMVUEbecauseitisafunctionofthecompleteandsufficientstatisticT.SupposethatXhasaPoissondistributionwithparameterλ.Nowlettheparameterof(a)Showthatθisθ=e−2λbeestimatedby(a)Showthatθis(b)ShowthatθisUMVUE.Butthisestimatorisabsurd.Why?CommentonthetheoryofUMVUEinlightofthisexample.as

³θ´

=

=e−λe−λ=∞∞θ(b)Thep.d.f.ofXisf(x;λ)=λxe−λ,whichbelongstotheexponentialfamily.SoitiseasytoseethatXisacompleteandsufficientstatistic,implyingthatθ∞∞θθisanabsurdestimatorbecauseitcantakenegativevalues(onlytakevalue-1or1)despitethefactthatθ=e−2λisalwayspositiveandliesbetween0and1.TheUMVUEhastodowiththeadoptionofaprinciple(unbiasednessandminimumvariance).Aprincipleisnotatheoremandseldomdoesaprincipleyieldsatisfactoryresultsinallcases.AUMVUEmaybe“bad”insomecase.Inparticularcase,onemayconsiderthemaximumlikelihoodestimatore−2Xofe−2λ,whichisprobablyamuchbetterestimatorinpracticethanistheunbiasedestimator(−1)X.Xi,X2¢isasufficientXi,X2¢isasufficientstatisticfor i(b)ShowthatTisnotcomplete.Thejointp.d.f.ofX1,...,Xn¡

¢−

1 f(x;θ)=

2n2

nnn!Ã−nnn!

(xi−2+1

nxi

iBytheFisher-NeymanFactorizationTheorem,T= Xi, X2¢isasufficientstatisticforiToshowthatTisnotcomplete,itsufficestofindafunctionofTthatisunbiasedfor0.S=2 X)2−(n+1) X2,afunctionofT.Thenforallθ∈

n⎨ n

Eθ(S)= ⎩

−(n+

X1i X1i=

X2´−(n+1)nE==

µn+ −(n+1)µ

+Thatis,Tisnot13.Let13.LetX1,...,XnbearandomsamplefromU¡θ−1,θ+1¢.LetX(1),···,X(n)betheθθ¡Find FindEθ¡X(n)¢ IsT=X(1),X(n)acompletesufficientstatisticforθ?LetF(x)andf(x)denotethecdfandpdfofX1,respectively.Thatf(x)202,( ifθ−1≤f(x)202,

2ifx<θ−2 222F(x) x−θ ifθ ≤x≤θ 2222First,weneedtofindthecdfof⎩2

ifx>θ+SothepdfofX(1)

F(1)(x)≡PX(1)≤x=1−PX(1)> =1−P(X1>x,···,X =1−(1−F(x))nnf(1)(x)=n(1−F(x))n−1f(x)nE¡X(1)¢

ZZ

tn(1−F

ZZ

=−t(1−F

n

ZnZn

(1−F(t))2n+1=µθ−1¶+12n+1 First,weneedtofindthecdfofX(n) F(n)(x)≡PX(n)≤x=P(X1≤x,···,Xn≤x)=F(x)nSothepdfofX(n)E¡X(n)¢

ZZ

tnF

f(t)dt

ZZ

tdF=tF

n

−Z−Z

F(t)n=µθ+1¶−1 n+1f(x;θ)=

1

−2≤

≤θ+1 =1

−2

22

¶1

≤θ+1¶2Clearly,T=X(1),X(n)asufficientstatisticforθ.Itisnotcompletebecauseonecanverifybaseduponthesolutiontoparts(a)and(b)that2E

−

n− = n Remark.LetX1,...,XnbearandomsamplefromU(0,θ),θ>0.T∗=X(n)iscompleteandLetX1,...,Xnbei.i.d.U[α−β,α+β],whereβ>0.Letθ=(α,β)FindaminimalsufficientstatisticT=τ(X)forFindtheMLEθMLofθ.(Hint:Graphtheregionforθsuchthatthejointdensityf(x;θ)>GiventhefactthatTin(a)iscomplete,findtheUMVUEofα.(Hint:NotethatEθ(X1)=Thejointp.d.f.isgivenf(x;θ) 1

i1{α−β≤i

≤α+ii 11½α−β≤minx¾1½maxxiiLetT1=min{X1,...,Xn},andT2=max{X1,...,Xn}.ThenT=(T1,T2)issufficientforNowwewillshowthat(T1,T2)isminimalsufficient.Generally,twostatistics,T=τ(X)andT∗=τ∗(X)willbesaidtobeequiva