第四章：信道与信道容量(英文)

Chapter4：ChannelandChannelCapacity

ThegoaltoachieveTheunderstandingofthegoalandcontenthowInfo.TheorystudiesthechannelUnderstandthebasicclassificationofthechannelandgraspthebasicdescriptionmethodofthechannelGrasptheconceptofchannelcapacity/Channelcapacitypricefunction,aswellasrelationsbetweenthisconceptandmutualinfo.,channelinputprobabilitydistribution,channeltransferfunctionCancalculatethechannelcapacity/Thechannelcapacitypricefunction(symmetricaldiscretechannel,additiveGaussnoisechannelwithoutmemory)ofsimplechannelUnderstandthefunctionofthechannelcapacity/CapacitypricefunctionincommunicationssystemresearchDefinitionquestionEntropyEntropyratefunctioninlosslesssourcecodingtheoremMutualinfo.ChannelCapacityfunctioninsourcecodingtheoremReview–natureofmutualinfo.function:Nature.1Relationshipbetweenmutualinfo.andchannelinputprobabilitydistributionNature1:I(X;Y)isaconvexfunctionofthechannelinputprobabilitydistributionp(x).I(X;Y)p(x)Review–natureofmutualinfo.function:Nature.2

RelationshipbetweenInfo.contentandchanneltransitionprobabilitydistribution

Nature2:I(X;Y)isaconcavefunctionofchanneltransitionprobabilitydistributesp(y/X).I(X;Y)p(y/x)Review–natureofmutualinfo.function:Nature.3RelationshipbetweenInfo.contentandchannelinputmarkrelevancy

Nature3:ThechannelinputisdiscreteandwithoutmemoryReview–natureofmutualinfo.function:Nature.4

RelationshipbetweenInfo.contentandchannelinputmarkrelevancyNature4:ThechannelhasnomemoryReview–natureofmutualinfo.function:Nature.5Deductionofnature3andnature4:

Thechannelinputandthechannelthemselvesarealldiscreteandwithoutmemory.ChannelandChannelCapacityOutlineclassificationanddescriptionofthechannelThediscretechannelwithoutmemoryanditsthecapacityContinualchannelanditscapacityCapacitypricefunctionC(F)

§4.1:OutlineThecontentofInfo.TheoryresearchingonthechannelWhat’schannel？ThefunctionofchannelThegoalofresearchingthechannel§5.1:Outline－1ContentthattheInfo.Theorystudiesonthechannel:Channelmodeling:describewithtwoappropriateinput/outputsstochasticprocessesChannelcapacityvariousmeansoftakingfullyadvantageofthechannelcapacityunderdissimilarcondition§5.1:Outline－2Whatischannel? Thechannelisacarrierwhichtransmitsmessages——apassagewhichsignalpassesthrough. Theinformationisabstract,thechannelthenisconcrete.Forinstance:Iftwopeopleconverse,theairisthechannel;Ifthetwocalleachother,thetelephonelineisthechannel;Ifwewatchthetelevision,listentotheradio,thespacetoreceiveandsendisthechannel.§5.1:Outline－3Thefunctionofchannel Thechannelmainlybeusedintransmittingandstoringdatatheinfo.system,butincommunicationssystemmainlybeusedintransmitting.§5.1:Outline－4thegoaltoresearchchannelRealizevalidityandreliabilityofintelligencetransmissionValidity:FullyusethechannelcapacityReliability:Reducestheerrorratethroughthechannelcoding

Studyingthechannelinthecommunicationssystemismainlytodescribe,measure,analyzedifferenttypesofchannel,calculatetheircapacity,namely,thelimittransmissioncapacity,andanalyzetheircharacteristic.Thecommunicationtechnologymainlyresearches－－Thephysicalrulewhichthesignalfollowswhiletransmittinginthechannel,namelytransmissioncharacteristicTheInfo.Theorymainlyresearches－－info.transmissionquestion(hypothesisisthatthetransmissioncharacteristicsareknown)§4.2:ClassificationanddescriptionofthechannelClassificationDescription§4.2:Classificationanddescriptionofchannel－1classification projectphysics——transmissionmediumtype; mathematicsdescriptionway——descriptionwayofsignalanddisturbance;parametertypeofthechannelitself——changeableandpermanentparameter;usertype——singleuserandmulti-user; §4.2:Classificationanddescriptionofchannel－2§4.2:Classificationanddescriptionofchannel－3§4.2:Classificationanddescriptionofchannel－4§4.2:Classificationanddescriptionofchannel－5§4.2:Classificationanddescriptionofchannel－6DescriptionofchannelThechannelmayquotethreegroupsofvariablestodescribe:Channelinputprobabilityspace：Channeloutputprobabilityspace：Channelprobabilityshiftmatrix：PNamely：{P}，Itcanbesimplified：.§4.2:Classificationanddescriptionofchannel－7其中：而 而§4.2:Classificationanddescriptionofchannel－8When

K=1，degeneratestothe

singlemessage(mark)channel;Furtherwhen

n=m=2，degeneratestothebinarysinglemessagechannel.Ifitsatisfiedsymmetry,namelyconstitutesmostcommonlyusedbinarysinglemessagesymmetricalchannelBSC:And:,,§4.3:DiscretechannelwithoutmemoryanditschannelcapacityDiscretechannelwithoutmemoryanditschannelcapacityCalculationofchannelcapacityofthediscretechannelwithoutmemoryChannelcapacitytheoremofthediscretechannelwithoutmemoryChannelcapacityoftheSymmetricdiscretechannelwithoutmemoryPhysicalsignificanceofShannonFirstTheorem§4.3:Discretechannelwithoutmemoryanditschannelcapacity-1Channelofdiscretemessagesequence§4.3:Discretechannelwithoutmemoryanditschannelcapacity-2Discretechannelwithoutmemoryanditschannelcapacity

Accordingtothecharacteristicofthemutualinfo.ofthemessagesequence,tothediscretechannelwithoutmemory:（Nature4）So:

Onlywhenthesourcewithoutmemory,thepreviousformulais“equivalence”（Cor.ofNature3、4）

§4.3:Discretechannelwithoutmemoryanditschannelcapacity-3FurtherunderstandingExistenceofCmaxMutualinfo.nature1，thelimitvalueoftheconvexfunctionexisted.TwoconditionstoreachCmax：ThesourceisdiscreteandwithoutmemoryTheprobabilitydistributionofthechannelinputisthedistributiontomakeI(X,Y)maximum.ThevalueofCisnotdeterminedbytheP(x)ofthesource,butdeterminedbyP.Cistheperformancemeasurementofthechannelwhichisthepassageofinfo.transmission.OnlywhenthesourceX（x1x2…xn)satisfiedcertainconditions，itcanfullyusetheabilityoftheinfo.transmission.§4.3:Discretechannelwithoutmemoryanditschannelcapacity-4ComputationofthechannelcapacityofthediscretechannelwithoutmemoryMentality:Thequestioncanbetransformedto:togettherestraintextremevalueonaclosedregionMethod:1st,togettheextremevalueintheregion2nd,togetextremevalueoftheboundary3rd,togetthemaximumvaluesofprevioustwoConcreterealization：1、solvesinthesimplesituation(forexamplesinglemarkchannel,symmetricalchannel)2、soluteequation3、iterate4、others§4.3:Discretechannelwithoutmemoryanditschannelcapacity-5Channelcapacitytheoremofdiscretechannelwithoutmemory

Theorem5.1：Tothediscretechannelwithoutmemorywhosepre-transitionprobabilitymatrixisQ,theabundantandnecessaryconditionthatitsinputletterprobabilitydistributionp*cancausethemutualinfo.I(p,Q)takethemaximumvalueis:

Note：

istheaveragemutualinfo.whichisthesourceletteraktransmits，Cisthechannelcapacityofthischannel.

§4.3:Discretechannelwithoutmemoryanditschannelcapacity-6UnderstandingofthetheoremUnderthiskindofdistribution,eachletterwhoseprobability>0providesmutualinformation=C,eachletterwhoseprobability=0providesmutuallyinformation≤COnlywhenunderthiskindofdistribution,maycauseI(p,Q)obtainthemaximumvalueCI(X,Y)istheaverageofI(x=ak;Y).Namely:WantstoenhanceI(X,Y)，mayenhancep(ak)Butifenhancep(ak)，mayreduceI(x=ak;Y)Adjustrepeatedlyp(ak),makeI(x=ak;Y)allequaltoCThistime:I(X,Y)＝CThetheoremonlyprovidestheabundantandnecessaryconditionofp(x)tomakeI(X,Y)＝C.ItdoesnothavetheconcretedistributionandtheCvalue,butmayhelptogettheCvalueofpartialchannelsinsimplesituation§4.3:Discretechannelwithoutmemoryanditschannelcapacity-7ChannelcapacityofthesymmetricaldiscretechannelwithoutmemorysymmetricaldiscretechannelwithoutmemoryTheoutputlettersetmaybedividedintocertainsubsets,toeachsubset:Inthematrix,eachlineistherearrangementofthefirstline;

Inthematrix,eachrowistherearrangementofthefirstrow.Theorem5.2：Asforthesymmetricaldiscretechannelwithoutmemory,whenthechannelinputlettersatisfiedequalprobabilitydistribution,itwillachievethechannelcapacity.§4.3:Discretechannelwithoutmemoryanditschannelcapacity-8Symmetricalchannel§4.3:Discretechannelwithoutmemoryanditschannelcapacity-9a1a2b1b2b30。70。10。10。20。7a1a2b1b2b30。20。70。70。10。10。2§4.3:Discretechannelwithoutmemoryanditschannelcapacity-10ComputationoftheBSCchannelcapacitya1a2b1b21-ε1-εεε§4.3:Discretechannelwithoutmemoryanditschannelcapacity-11AccordingtoTheorem5.2，whentheinputsatisfiedequalprobabilitydistribution,themutualinfo.willachievethechannelcapacitynamely：whenp(a1)=p(a2)=1/2；

so：

note：ApplicationExample3.2(18)、3.6(23)§4.3:Discretechannelwithoutmemoryanditschannelcapacity-12computationofthechannelcapacityofdualdeletionchannel

a1a2b1b21-ε1-εεεb3§4.3:Discretechannelwithoutmemoryanditschannelcapacity-13AccordingtoTheorem5.2，whentheinputsatisfiedequalprobabilitydistribution,themutualinfo.willachievethechannelcapacitynamely：whenp(a1)=p(a2)=1/2；

so：

§4.3:Discretechannelwithoutmemoryanditschannelcapacity-140.51.000.51.0cεbaCa=Cb=a:BSC信道的信道容量曲线b:二进制删除信道的信道容量曲线

TogetthechannelcapacityofsymmetricaldiscretematrixP:1/21/31/6P=1/61/21/31/31/61/2

C=logs-H(p1,p2,p3)=log3-H(1/2,1/3,1/6)=log3+1/2log1/2+1/3log1/3+1/6log1/6=1.126bit/sTheresultindicatedthat,Onlywhentheinputsatisfiedequalprobabilitydistribution,thechannelcapacityachievesmaximumvalue.Theaveragegreatestinfo.contentofeachmarktransmissionis1.126bit.1losslesschannel一个输入对多个互不相交的输出,因为损失熵H(X/Y)=0故I(X,Y)=H(X)C=logrr为输入个数2definitechannel一个输出对多个互不相交的输入,因为噪声熵H(Y/X)=0故I(X,Y)=H(Y)C=logss为输出个数3losslessdefinitechannel一个输入对一个输出,因为损失熵H(X/Y)=0噪声熵H(Y/X)=0故I(X,Y)=H(X)=H(Y)C=logrr为输入个数Discretenoiselesschannel§4.3:Discretechannelwithoutmemoryanditschannelcapacity-15physicssignificanceoftheShannonfirsttheorem:（lengthuncertainlosslesssourcecoding）

（equalsignalestablisheswhenachievedlimit）

astothechannel,thechannelinformationtransmissibility:

（equalsignalestablisheswhenachievedlimit）§4.3:Discretechannelwithoutmemoryanditschannelcapacity-16physicssignificanceoftheShannonfirsttheorem:（lengthuncertainlosslesssourcecoding）Channelcapacityofnoiselessandlosslesschannel：C=logMNowwhentheaveragecodelengthhasextremevalue:ChannelinformationtransmissibilityR=channelcapacityCofthenoiselesschanneltheessenceoflosslesssourcecoding：对离散信源进行适当变换，使变换后新的码符号信源（信道的输入信源）尽可能为等概分布，以使新信源的每个码符号平均所含的信息量达到最大，从而使信道的信息传输率R达到信道容量C，实现信源与信道理想的统计匹配。Alsocalled：noiselesschannelcodingtheorem若信道的信息传输率R不大于信道容量C，总能对信源的输出进行适当的编码，使得在无噪无损信道上能无差错地以最大信息传输率C传输信息；但要使信道的信息传输率R大于C而无差错地传输信息则是不可能的。§4.5:信道容量InformationsourceEncoderSXDiscreteCommunicationChannelX={x1,x2,…,xr}DecoderXSInformationreceiverContinuouschannel－－analogchannelContinuouschannel：Char1：thetimeisdiscrete、thescopeiscontinuousChar2：ateachmoment,itisthesinglerandomvariablewhosevalueiscontinuous(vsdiscretesequence)Researchmethod：randomvariableofNfreedomdegree,takestoresearchConaverageeachfreedomdegreeanalogchannel:Char1：thetimeiscontinuous、thescopeiscontinuousChar2：Agroupoftimesamplefunctions,ateachmoment,thetimeandthescopevalueofthesamplefunctionareallcontinualResearchmethod：1、discretewhenthefrequencyandthetimearelimited,canbequantifiedtothediscreterandomvector2、toavoidthedifficultyofresearchingtherandomvectorhavingmemory,findagroupofcompletelyorthogonalfunctionsets,launchesfortheprogression,enabletomaketherandomvectorcomposedbythecoefficientbeindependentorbelinearindependent.Note：1、timelimit--frequencyspectruminfinite,frequencylimit--timeinfinite.2、assumethatthefunctionvalueisverysmalloutsideForT,andthatthetimeandthefrequencyislimitedwillnotcauseseriousdistortionAnalogsourceAnalogcommunicationsystemSourcecodingChannelcodingAnalogchannelChanneldecodingSourcedecodingDestinationA/DconverterModulationDemodulationD/Aconverter01101…01110010…01111010…01101…§5.4:Continuouschannelanditscapacity－1－ReviewEntropyofcontinuousrandomvariable－differentialentropy（VSdiscreterandomvariable）Thecontinuousrandomvariablebiggestentropydistribution--reliesontheconstraintcondition(VSdiscreterandomvariable)Thepeakpowerislimited--theuniformdistributionrandomvariablewillhavethebiggestdifferentialentropyTheaveragepowerislimited--theGaussdistributionrandomvariablewillhavethebiggestdifferentialentropyThevalueterritoryofthecontinualchannelinputisinsufficienttoexpressthelimittothechannelalsotheconstraintconditionC＝max[h(Y)-h(n)]C取决于信道的统计特性（加性信道即噪声的统计特性）输入随机矢量X所受的限制条件（一般考虑平均功率受限时）Unit：bit/N个自由度Channelcapacityofthecontinualchannel--capacityexpensefunctiondescription§5.4:Continuouschannelanditscapacity－2C.FWu&Zhu&Fu－－channelcapacityWu：Channelcapacity：discrete、continuousCapacitycostfunction：discretechannel、continuouschannelZhu：Channelcapacity：discreteCapacitycostfunction：continuouschannel&continuouschannelFu：Channelcapacity：discrete、continuous§5.4:Continuouschannelanditscapacity－3Methodstoresearchcontinuouschannelcapacitybasic、simplechannel：addablenoisechannelwithoutmemoryWhenthechannelnoiseisGaussWhichdistributioninputwilltakefullyadvantageofthechannelWhenthechannelinputisGaussWhichdistributionnoisewillaffectthechannelinfo.transmissionmost§5.4:Continuouschannelanditscapacity－4Basicknowledge：toaddablechannelY=X+NX：channelinputN：channelnoiseY：channeloutputThechanneltransitionprobabilitydistributionfunctionistheNdistributionfunctionb(x)isthecorrespondexpensewhenthechannelinputisxIftwoofX,Y,NistheGaussdistribution,thenthenotheralsoistheGaussdistributionDifferentialentropyoftherandomvariablesatisfiesGaussdistributionh(XG)＝Thevalueofdifferentialentropyh(XG)onlyconcernswiththevariance ,hasnothingtodowiththeaveragevalueAddablechannelErrorSource+EXOutputInput§5.5:AnalogchannelanditscapacityChannelcapacityexpensefunctionoftheanalogchannelanditscomputation:Generalizedsteadylimitedfrequency(F),limitedtime(T),limitedpower(P)whiteGausschannelanditscapacityCShannon

formula

PhysicalsignificanceofShannon

formulaUsageofShannon

formula§5.5:Analogchannelanditscapacity－4Generalizedsteadylimitedfrequency(F),limitedtime(T),limitedpower(P)whiteGausschannelanditscapacityC

Tothecontinuousprocesssourcewhichhaslimitedfrequency(F),limitedtime(T)maybeunfoldedthefollowingsamplingfunctionsequence:Nowletthese2FTsamplesvaluesequencepassthroughthewhiteGausschannelwhichhaspowerlimited(P)andgetitscapacityC.

§5.5:Analogchannelanditscapacity－Shannon

formula1ShannonformulaTheorem5.3：WhenageneralsteadyrandomprocessX(t,w)sourcewhichsatisfieslimitedfrequency(F)andtime(T)passesthroughawhiteGausschannelwhichhaslimitedpower(P),itscapacityis:ThisisthefamousShannonformula.WhenT=1,thecapacityis:§5.5:Analogchannelanditscapacity－Shannonformula2prove：wehadgotthecapacityofthesinglecontinuousmessage(NO.k)whenpassingthroughtheGausschannel:Meanwhile，wehadprovedthatwhenthesourceandthechannelhasnomemoryinthemutualinfo.,thefollowingformulawillbetenable:Accordingtothedef.ofthechannelcapacity:§5.5:Analogchannelanditscapacity－Shannon

formula3PhysicalsignificanceoftheShannonformulaItpresentsthedialecticalrelationsamongthethreesignalphysicsparameterswhichdeterminedthechannelcapacityC:F、T、。Productofthethreeisa“moldable"volume(threedimensional).Thethreemayexchange.§5.5:Analogchannelanditscapacity－Shannonformula4－usageofShannonformula:1

Usethefrequencyband

inexchangeforthesignalnoiseratio:Widenfrequencycorrespondenceprinciple.Intheradarsignaldesign,thereislinearlyfrequencymodulatedpulse.Intheanalogcommunications,thefrequencymodulationsurpassestheamplitudemodulation,andthewiderthefrequencybandis,thestrongertheanti-disturbanceis.Inthedigitalcommunication,thepseudo-code(PN)straightlyexpandswiththetimefrequencycode.Thewiderthebandwidthis,themorethewidenfrequencyincreases,andthestrongertheanti-disturbanceis.Inthedeepspacecorrespondence(powerenergyislimited,frequencyspectrumresourcesisrelativelyrich),weusetwolevelsdigitalcommunicationwaytoeffectivelyusethechannelcapacity.

Attention：thereislimit归一化信道容量关于带宽W的关系图归一化信道带宽关于信噪比SNR的关系图

－anotherformofShannonformula：

其中，为噪声密度，即单位带宽的噪声强度，σ2=N0F;

Eb

表示单位符号信号的能量，Eb=STb=S/F；Eb/N0

称为归一化信噪比.也称为能量信噪比.WhenEb/N0<<1，

≈Eb/N0

(nat)＝(bit)Conclusion：whenthesignalnoiseratioislow,thechannelcapacityisapproximatelydeterminedbythepowersignalnoiseratio.§5.5:Analogchannelanditscapacity－Shannonformula

5－usageofShannonformula:2usethesignalnoiseratioinexchangeforthefrequencybrandbasicprincipleofmultiplesystem,multi-levels,multi-dimensionalconstellationmodulationwayThatbeusedInthesatellite,thedigitalmicrowaveoftenincluding:

themulti-levelsmodulation,theheterogeneitymodulation,thehighdimensionalconstellationmodulation(M-QAM)andsoon.Itusethewealthysignalnoiseratiointhehighqualitychanneltoexchangeforthefrequencyband,thustoenhancethetransmissionvalidity.

§5.5:Analogchannelanditscapacity－Shannonformula6usageofShannonformula:3usethetimetoexchangeforsignalnoiseratio重传、弱信号累积接收基于这一原理。t=T0istheboundary.信号功率S有规律随时间线性增长，噪声功率σ2无规律，随时间呈均方根增长。§5.5:Analogchannelanditscapacity－Shannonformula7usageofShannonformula:4

usethetimetoexchangeforthefrequencybandorusethefrequencybandtoexchangeforthetime:

Thewidenfrequency--reducesthetime:electroniccountermeasureofcorrespondence,submarinecorrespondenceThenarrowband--increasesthetime:Thetelephonechannelpassesontheaccuratemovingpicture§5.5:Analogchannelanditscapacity－Shannonformula

8Thegoaltodiscusschannelcapacityandthechannelexpensefunction：Nottorealizethereliabletransmission（whichisthegoalofthechannelcoding）OnlytorealizethemosttransmissionabilityofthechannelMaypresenttheboundaryofthechannelcodingAbouttheShannon

formulathechannelisaddablewhiteGaussnoise（AWGN)PresenttherelationshipoftheS、N、Wandthechannelcapacity（thebiggesttransmitspeed）NotpresenttherelationshipofS、N、Wandthedifferenceprobability－1.59dB例2、在图片传输中，每帧约为2.25×106个像素，为了能很好地重现图像，需分16个亮度电平，并假设亮度电平等概率分布。试计算每秒钟传送30帧图片所需信道的带宽(信噪功率比为30dB)。信噪比（S/N）通常用分贝（dB）表示，分贝数=10×log10（S/N）

解：高斯白噪声加性信道单位时间的信道容量：(比特/秒）

要求的信息传输率为：Ct=2.25×106×log16×30=2.7×108（bit/s)=Wlog(1+S/N)而：10lg(S/N)=30dBS/N=103W=(2.7×108)/log(1+103)≈2.7×107(HZ)

猫速度和宽带的解释举例3.8(26)曹志刚《现代通信原理》关于香农公式的一些结论Channelcapacity：Thegreatestinformationcontentthatcanbetransmittedin