信息论与编码标准答案

1、第二章信源及信源熵2-1JU 36P11 = i1(3,5) = -Leg - -4175丿Ijj = -Log 二5A73HE ,埶瞋陶小敖彳号4337(?) + Lg十一36X)3.2742-2(1)-Log991000.014-Log Log100Log(% = 22-325%75%门“=0.375 -Log(0.375) = L41350%u 丿2-438111LogLog(4) Log(4) Log (8) =1.906834460 1.906 =114.362-5(1,2) (2,1)共两种 Log多= 4.172-6二 2.585(1,6) (6,1) (2,5) (5,2) (

2、3,4) (4,3)0 14 个 1-13 个 2-12个 3-6 个2-7 Log(2) =1 Log(4) =2 Log(8) =32-8”用三个脉冲 “”用一个脉冲(1) I(14ioj 白/白)=210；黑/白)=-P(0331H(Y/(149 T (14+ Log 丿141 9 丿0.94(4) P(白）=nLog 黑）二厂p（白）二二H(Y)二2-10 (1) H(1 亠 2 丫 （3-+ - Log| -色）諾叫刃烷叫划烷网3H(Y)= 1811 - 1 = O.gll(2)I) H(YZJ=H(XY)-H(Y)= l.gu - 1 = O.gll(3) I :一一 -H . -

3、.：二- :2-13H(IJ)=4- LogfS) + 2-Lo+ 2- Log( 15) 4- 3Lo臥+ Log 12) = 3.4158101536122-14(1)n i(31 f 6 2、4 4i i7 91 7P(ij)=1 1P(i/j)=1 7U sj卫 115 ；J 5PQ/i)=pCyCD17 一16方法1：mm 卜“=0/03 + 0.236 = 0.3111616方法2:+ Log 1+ Log 16 12-15P(j/i)=pCl) = pCbZ)=p(a1/b1)=pGDp】逋1p(tl)1砂1)=胡霁卜胡干 k P3) )1I 2 )=Lo2(l - ejp(sd

4、/b2)=p(al) p(b2 al)P他=Log(2e)2-16(1)(2) 设最后平稳概率为W1,W2(0.9143 0.0857 I 0.20.3 )TP -Ws W得 W=07 W2=0.3wr + w0 = 1H(Y/黑)=-0.9143Log(0.9143 -0.0857Log(0.0857) =0.422H(Y/白)= -0.2Log(0.2) 0.8Log (0.8) =0.722H(Y/X)=W H(Y/ 黑)+ W2 H(Y/ 白)=2-170.8(1)I =-Log=3 105Log(128) = 2.1 x 106I =-Log-lOOOLogWOO = 1 32P x

5、 1042-24(1) H(X)二=0.31113(A-Logf4) 4- - Log 一44LuJ4 .IP(1.X2.X100l)= _LOg(3) I/.:：:：, I/.!： -r门二i 工 小2-25FW(0.25lo5解方程组PT W 二 WW1 化二10.25 0.5 W10.75 0.5 丿 Wc 2解得W1=0.4 W2=0.6 2-262 2UJ2 33232311r 0.4、P(j/i)=-03解方程组-02W2=W2求得W=0.361 200 20 30.24；3 3J1 3)1/22-27求平稳概率0000rO.E 0.2 00符号条件概率10.5 2状态转移概率01

6、0 C 0.5 0 5100.5 0.51005 D5 00110.2 2丿11(000.2咔丿解方程组Tp1 w = W得到W=广 03J7、0.143w2 + w2 + w3 + w4 =:1山2巩2-28(1) 求平稳概率P(j/i)=解方程组得到H(S/s1)=-3Log(3) =0.918H(S/s2)= 0信源熵为:3 1H(S)=Wd H(Ss1 )4W H(S/s2)= -0 918 + - 0 = OjfiSS4 42-29P(j/i)=113331113331102-解方程组331wnW2=W2得到W1 =扌,W2=-晝03 3 3P(i/j)=1 1 13 3 31 1

7、1解方程组丿口i得 W=W=W= -H(X/0Log(3)H(X/1Log(3)H(MLog(3)信源熵为-2-31i iq(111 r22 444S3P(Xi)=14P(j/i)=2 D313P(X 1X2)=212011212 10)21IQa H(X1 )= i LogfZ) + Egg(関 + -Log(4) = 1.5 a-2444-0 + -Lq) - 0918b.求H(X/Xi)有两种方法:方法1：Hpy%F - LO0G9 + - Loat-9 + - Lo的244Hpyf戸a21Z Z / lZ12.c.求 H(X3/X2)7_12P(X2)=245_方法1：P(X/X2)=

8、524H缶嗣)24丄 I d 4- 0 518 4- 018 k 1 257 1224方法 2： P(X3/X2)=7一一-Lo + -Lo + -LoLog43364-LagfJ) +Log /JJod.最后H(X1X2HCX1)+H(X2W1)(X/1X2)=Hp1)-HH(X2W1)-HH(X3ZX2)=2 一 I -；-(2) 首先求解稳定情况下的概率解方程组PT w = wWj 4- W2 4- W3 = 1j -W : )+ W L + W I、； ：! 1.5 4- 0.P13+ 0.918 = 1 2511414142-32(1)不做P(j/i)=号J 22求解方程组(i-p)

9、wi + m + Ew3 = wi 2 2W1 +(1 - p) W2 + Ew3 W22 2W1 + E W2 + (1 - p) W3 = W32 2Wl + W2 + W3 = 1e1得 p(0)=p(1)=p(2)=-HpA)=H(X/1)=Hp)= (1 - p)log(p) - 5,21oHco(X)= -(1 - p)-Log(l - p) - pLoH(X)=log(3)=1.58d .In.=_i 1 卩. t：P=当p二二时-厂：达到最大值1.58当 p 二 0 时 H(X)二 0当 p 二 1 时 H(X)二 12-33(1)- B0p、P(j/i)=p1 - p0I 0

10、 P 1卩丿解方程组：(1 - ?)wi + p m = wi(1 - p) W2 + p W3 = W2p W1 十(1 - p) W3 = W3WI + W2 + W3 s 1口1得 p(0)=p(1)=p(2)=-HH(X/1 )= H(XZ2)= -(1 - pjLo痰 1 pj pLa取p)-p Log(p)Hco(X)= i*H(X/D)+ 凶1)十 i*H(X/2)= -(1 - p) Log(l -当p=0或p=1时 信源熵为01第二章无失真信源编码3-1(1)C1.C2.C3.C6C1tC3,C6HM= 扣瞋耳+ Lq的 4斗話Logfl = 2K1=3R1 =顷& - 2

11、K3K2=1 , 1 -1 + -2 + 3+ 4 + 5 +16 =2.125R2=比 -2=0.9412416 16 16162.125K3=L I1 H2 + *3 + -4 + 5 +16 =2.125R3=HE -2=輛412416 16 16圧2.125K6=-2 + 3 +*3 + a3 + b3 +13 = 2.5R6二H旳=.2=0.82416 16 1616K62.33-2(1)因为A,B,C,D四个字母，每个字母用两个码，每个码为0.5ms,所以每个字母用10ms当信源等概率分布时平均信息传递速率为，信源熵为 H(X)=log(4)=2H约10bit/ms=200bit/

12、s103J信源熵为H(X)=- - - 1- - 1 .-节=0.198bit/ms=198bit/s3-3与上题相同 3-511111(1) 111(1)2 4 8 16 32 64 128 128H(U)=1Log(2)1Log(4) !Log(8)丄Log(16) - Log (32) 丄Log(64)1Log(128) 丄Log(128) = 1.984248163264128128(2) 每个信源使用3个二进制符号，出现0的次数为3 - + 2- + 2-i + + 2 + + = 2.39224216326412S出现1的次数为P(0)=却82.398 + 0.5S6=0,304P

13、(1)=2.35*8 + 0.586信源符号Xi符号概率pi累加概率Pi-L ogp(xi)码长Ki码字x11/20110x21/40.52210x31/80.7533110x41/160.875441110x51/320.9385511110x61/640.96966111110x71/1280.984771111110x81/1280.99277111111101”9843(4)相应的香农编码相应的费诺码=0.661信源符号xi符号概率Pi第一次分组第二次分组第三次分组第四 次分 组第五次分组第六次分 组第七次分 组二元码x11/200x21/41010x31/810110x41/1610

14、1110x51/321011110x61/6410111110x71/128101111110x81/128111111110(5)香农码和费诺码相同平均码长为71 + 724j+ i 4+ 存讣缶八占 7= 1W4编码效率为：一二IPS4IP843-71i-1(1) Pi= i累加概率为Pi二二ti-0累加概率分别为符号x1x1x2x3x4x5x6x7概率1/21/41/81/161/321/641/1281/256累加概率00.50.750.8750.9380.9690.9840.992码长12345678元码010110111011110111110111111011111110(2)

15、信源的信息量为H(X)= - Log(2) + - Log0.0.-0.V0.t114444x20.18七0.1*0.f0.70.Y0.7 0.4001381819/2327x30.1*0.1丫o.0.h0.0113卜3181923x40.1*0.10.o.0 u0.0000411318x50.07X0.07.b0.01004911x60.06 0.0 +0.01014709x70.05勺o.o0001056x80.04000115K = 0.4-1 + 0.18 3 + 0.1 3 4- 0.1-4 +0.07-4 + 0.06-4+ D.05-3 4- 0.04-5 = 2.61 戸 H(

16、X) 2352.R =亠丄=0 972K 泅香农编码信源符号xi符号概率pi累加概率Pi-Logp(xi)码长Ki码字x10.401.322200x20.180.42.4743011x30.10.583.32241001x40.10.683.32241010x50.070.783.83741100x60.060.854.059511011x70.050.914.322511101x80.040.964.644511110平均码长:K = 0.4 2 + 013 3 + 01 4+ 0.1 4 + O.Q7 4 + 0 06 5 4 0 05 5 + 0 045 = 3.17H（邇=25520.

17、305信源符号xi符号概率Pi码码长x10.400002x20.181012x30.11001003x40.111013x50.0710011004x60.06111014x70.051011104x80.04111114(4) 费诺编码:K = 0 4 2 + 0.18-2 + 01 3 十（H-3 + 0.07-4 + 0.06 4 4- 0 03 4 + 0 04 4 = 2.643-14信源符 号xi符号概率pi编码过程编码码长x11/3* 1/3*1/*1/ 产 1/ 字/ 勺3333002x21/3x31/9x41/9x51/27x61/27x71/271/31/91/92/21/

18、21/1/1/1/1/1/1/ I1/1/0121003101311131100411014H(X)= 2 Log(3) + 2 i Lo9) + 3Log(27) - 2J2S9K =-2 +-2+-3+-3 + 4 4- 4+ 4-2.44433 P 3272727口 H(X) _ 2.289 卄”R= = = 0 937K 2 444第四章限失真信源编码4-1失真矩阵为-4-2101111011 11信源熵为H(x) =Log(4)3333Dmax =min：, 4, ;, 42R(Dmax)=0Dmi n=0R(Dmi n)二R(0)=H(X)=log(4)=2p(y1) p(y2)p

19、(y3)p(y4)只 要满足 p(y1)+p(y2)+p(y3)+p(y4)=1 在0,1区间可以任意取值。第五章信道编码5-1(1) p(x0)=3/4p(x1 )=1/4P(y/x=2 P3 31 2 4612H(XY Ilo/D + 丄 Log(母 + 丄 Log(l + -Log(6) = 1 73 24126H(XHCXY)-H(Y)= 1.73 -0 93 - 0.75H(Y/X)=H(XY)-H(X)= 1.73-0.811 = 0.S19IptYFHgH紳=0.511 - 0.75 - 0 061C=log(Q)+EPijLog(卩沪tog(2) 4- Log=0.032PW

20、= P(少5-2(1)Pxl :=oc Px2 := 1 -1 1“0-aO卩（坷出戶2 2-(1 - a) -(1 - a) - (1 - a)244_Py1)二丄就十 1(1 - x) t 2 p(y2)F 1& +_ a) _ 1 a + 1p(y3)= 1(1 - a)2 2224444/*X0十护)珂1接收端的不确定度为：H()= Log(2)-(扌 a + Log-1 Log(l + a-) - -_ Log(l - a)H(Y/X)二ia Lagj + ccLcg(2) 4- 0 + (1 - ac) Log) 十(1 - ac) Lo(4)十-(1 -(X)Log(4)22G4

21、43 1=：2 2(1)l(X;Y)=H(Y)-H(Y/X)=齐宁说* 宁说)C=maxIX,Y)p(xi)31. + at, log(l + a) 1 - a, log(l - a.)( 3124 log(7)4- ：=0dot得到1 -ln(l + ct) + ln(l - a) + 21n(2)4lnTZlC=maxl(X.Y)=0 1615-3C = LogfQ) + s Log(巧 I =1 + O99Lo0(O9?) + 001 LogJOl) = 0.919 i-10919*1000=919bit/s 5-501212jf Lo0?3 + V (Py Log(P1J I =Lo0

22、t4)+ - Lo氏-5-6(1)0302条件概率0.40.30.3,联合概率I091151021151013301010,后验概率1-LogfJ) + - Lo g(2) + 丄 Lcg(6) = 1A593 26(2)H(Y/X)二右L瞋另十LoS T13+ 一 Ln酊ln + 一 Loe3010（少1.175当接收为y2，发为x1时正确，如果发的是x1和X3为错误,(3)各自的概率为:113P(x1/y2)= 5, P(x2/y2)= 5, P(x3/y2)=-其中错误概率为：13Pe=P(x1/y2)+P(x3/y2)= 5 2 =855(4) 平均错误概率为Z* 丄 4丄 + + +

23、 丄=0.733153010101510(5) 仍为 0.733(6)此信道不好原因是信源等概率分布，从转移信道来看正确发送的概率x1-y1的概率0.5有一半失真x2-y2的概率0.3有失真严重x3-y3的概率0完全失真(7)H(X)- Log(3) = 1JS5H(X/Y)=1 Log(2)611 5+ Log (5) + Log -1015匕丿+ Log 5152)111 5+ Log (5) + Log -101013丿13 5+ Log (10)+ Log | -=1.301301013 丿课外习题1. 设某信道，其信道矩阵为r0.5 03 0.202 03 05bl b3若信道的输入

24、符号a1,a2,a3先验等概,(1)若使平均错误译码概率最小，请选择译码函数。(2)求出此错误译码概率 Pemir。解：(1)因为先验等概，所以选择最大似然译码准则F(b1)=a1 F(b2)=a3 F(b3)=a2(2) Pemi n=(0 2 + 0.3 + 0 3 + 0 3 + 0.2 4- 0.4)=0.5672. 有二进制对称信道p=0.01=0.99p(1) 采用最大似然译码准则确定译码函数，(2) 求出最小平均错误译码概率。(3) 对该信道进行扩展，采用简单重复编码，000,111,采用最大似然 译码准则确定译码规则。(4) 求出扩展后的最小平均错误译码概率。(5) 求出扩展后

25、的信道传输率解:5(1) P(j/i)=引0蝕 讪) 译码函数为 F(b1)=a1, F(b2)=a210.01 0.9?(2) P emin = (0.01+0.01)/2 = 0.01001 010 100 011 110 101 111P2卩3 04卩5 Pfi 衍塚_2 -2 -2_ 2 _ 2 _ 2 3P P P P P P P P PP PP P- 2 - 2222 ,2 -3P P P-P PP PP PP PPPJ000= 000=111_3 P译码函数 F( B 1)= F( B 2)= F( B 3)= F( B 4)=000= a 1F( B 5)= F( B 6)= F( B 7)= F( B 8)=000=(4) 平均错误最小概率为(5) R=L- = 13. a i , B j是两个码符号0,1组成的符号序列-，、二m门求a i , pj之间的汉明距离N解：D(a i，p j)=!_ -.：=k = 14. W:000,001,010,100,011,110,101,111的最小汉明距离解：Dnin = 15. 设有一离散信道，其信道矩阵为*1 b2 b3n i i_ _ _軻2 4 4o 1 1 IP= - - -4 2 4111U