Advanced-Digital-Signal-Processing-ch1-fundamentals.ppt

AdvancedDigitalSignalProcessing ModernDigitalSignalProcessing Chapter1FundamentalsforDiscreteRandomSignalAnalysisandProcessing 1 1DiscreteRandomSignalandItsRepresentation RandomSignalSignalwhosevaluesarerandomSignalvaluevarieswithtime butitcannotberepresentedbyadeterministicfunctionoftimeForacertaininstant signalvalueisarandomvariableAsampleofarandomsignaliscalledarealization afunctionoftime TheClassificationofRandomSignalContinuousrandomsignal SignalwhichiscontinuousintimeandamplitudedomainDiscreterandomsignal randomsequence SignalwhichiscontinuousinamplitudebutdiscreteintimeContinuoustime discreteamplituderandomsignalDigitalrandomsignal Signalwhichisdiscreteinbothamplitudeandtime Itcanbetreatedasrandomsequencebyignoringquantizationeffectsorfiniteword lengtheffects ExamplesofRandomSignalContinuousrandomsignal 3realizations Randomsequence 3realizations TheRepresentationofRandomSequenceArandomsequencecanbemodeledbyadiscrete timestochasticprocessisarandomvariable denotedasTimeisfixedandvalueisvariableorisarealization samplesequence isvariablebutisfixedisanumberBothandisfixedisastochasticprocessBothandisvariable brieflydenotedas TheCompleteDescriptionofaStochasticProcess RandomSequence TheNthprobabilitydistributionfunctionTheNthprobabilitydensityfunction PDF If thenisarandomvariableanditsProbabilitydistributionfunction PDF TheNumericalCharacteristics Statistics ofRandomSequenceExpectationormeanvalue 1st ordermoment Meansquarevalue 2nd orderoriginmoment Variance 2nd ordercentralmoment ThecorrelationofthesamerandomsequenceindifferenttimeAutocorrelationfunction 2nd ordermixedoriginmoment Itisarelevancemeasureoftheoutcomesoftherandomsequenceattimeinstantsn1andn2Autocovariancefunction 2nd ordermixedcentralmoment ascatterordispersionmeasure representscomplexconjugate ThecorrelationofdifferentrandomsequencesindifferenttimeCross correlationfunctionisthejointPDFofrandomvariableandCross covariancefunction High ordermomentsThemomentswhoseorderishigherthan2ndTheyareusuallyusedtoanalyzethenon Gaussianrandomsequences 1 2StationaryRandomSequence Strict SenseStationaryRandomSequenceTherandomsequencewithtime invariantprobabilitydistributionfunctionorPDFThenumericalcharacteristics statistics ofstrict sensestationaryrandomsequencearealsotime invariant Weakly Wide Sense StationaryRandomSequenceTherandomsequencesatisfiesBothits1st orderand2nd ordermomentsexistForanyintegerm Foranyintegern1 n2andm Theweaklystationaryrandomsequenceisusuallycalled stationaryrandomsequence forshortorabbreviatedasWSS wide sensestationary randomsequence Noticethatastrict sensestationarysignalisnotalwaysweaklystationaryoneandmostweaklystationarysignalsarenotstrict sensestationary Propertiesof Weakly StationaryRandomSequenceMeansquareandvarianceareirrelevanttotimeAutocovarianceistime shift invariantAutocorrelationisconjugatesymmetrical AutocorrelationmatrixisanHermitematrix Ifx n isrealsignal thenRxxisrealsymmetricalandnonnegativedefinitematrix Ifx n andy n arestationaryrandomsequencesrespectivelyandtheyarejointlystationary thenIfrxy m 0foranyintegerm thenx n andy n aremutuallyorthogonalIfrxy m mxmy i e covxy m 0foranyintegerm thenx n andy n aremutuallyuncorrelated PropertiesofReal ValuedStationaryRandomSequenceIfrandomsequencesx n andy n arestationaryandreal valued then 1 3Frequency DomainDescriptionofStationaryRandomSequence Wiener KhintchineTheoremTherelationbetweentheautocorrelationfunctionandpowerspectrumdensity PSD ofstationaryrandomsequencex n ifmx 0 thenrxx 0 istheaveragepowerofx n PropertiesofthePSDofReal ValuedStationaryRandomSequencePSDisevenabout PSDisrealandnonnegativeTheCross PSDofReal ValuedStationaryRandomSequencesx n andy n 1 4TheErgodicityofStationaryRandomSequence DefinitionLetxs n beasample realization ofastationarystochasticsequencex n thenx n issaidtobeergodicif TheUnderstandingofErgodicityThestatisticalexpectationalongthetimeofonerealizationissameasthestatisticalexpectationacrossthespace orensemble ofdifferentrealizationsoftherandomsequence itcanbedenotedbrieflyasTheergodicityofrandomsequencefacilitatestheanalysisofstationaryrandomsequencebyexaminingonerealizationofthissignalinsteadofthesetofsignalsamples 1 5SomeUsefulClassesofRandomSequencesorStochasticProcesses Gaussian Normal RandomSequenceArandomsequencexs n withNthPDF TheAdvantagesofGaussianModelGaussianPDFonlydependsonits1st orderand2nd ordermoments Awide sensestationaryGaussianprocessisalsoastrict sensestationaryprocessandviceversa GaussianPDFscanmodelthedistributionofmanyprocessesincludingsomeimportantclassesofsignalsandnoise ThesumofmanyindependentrandomprocesseshasaGaussiandistribution centrallimittheorem Non Gaussianprocessescanbeapproximatedbyaweightedcombination i e amixture ofanumberofGaussianpdfsofappropriatemeansandvariances OptimalestimationmethodsbasedonGaussianmodelsoftenresultinlinearandmathematicallytractablesolutions MarkovProcessAstochasticprocesswhoseconditionalPDFsatisfiesisa1st orderMarkovprocess orMarkovprocessforshort ThestateoftheMarkovprocessattimendependsonlyonitsstateattimen 1andisindependentoftheprocesshistorybeforen 1 Gauss MarkovProcessstochasticprocessesthatsatisfytherequirementsforbothGaussianprocessesandMarkovprocessesIftheinputofaMarkovprocessisaGaussianrandomsequence thentheoutputisaGauss Markovrandomsequence stochasticprocess Gauss Markovprocessisnotalwaysstationary AscalarstationaryGauss Markovprocessx n withvarianceandtimeconstanthasautocorrelationfunctionandPSDasfollowingforms WhiteNoiseSequenceAwhitenoisesequenceisarandomprocessofrandomvariablesthatareuncorrelatedandhaveafinitevariance i e ThemeanvalueofwhitenoisesequenceiszeroAstationarywhitenoisesequencehasaconstantPSD ThedifferencebetweenirrelevanceandindependenceThattworandomvariables orsignals xandyareirrelevant uncorrelated impliesthatforanymThattworandomvariables orsignals xandyarestatisticalindependentimpliesthattheirjointPDFIndependent uncorrelated butuncorrelatedindependentForthewhitenoisesequencewithGaussiandistribution uncorrelated independent SomeexamplesoftherealizationofwhitenoisesequenceStandardnormaldistributionUniformdistributionbetween 1 1 HarmonicProcesswhereHarmonicprocessx n isastationaryprocess Independentandidenticallydistributed ExampleofHarmonicProcess 3Realizations 1 6LTISystemswithStationaryRandomInputs LTIsystemwithunitpulseresponseh n Stationaryrandomsequencex n systemresponsey n H z Y z X z my n ryy n n m andtheStationaryofy n Therefore likethex n y n isalsoastationaryrandomsequence Irrelevantwithn ThePSDofy n TheCross Correlation Cross PSDofx n andy n TheCorrelation ConvolutionTheorem Proof TimeSeriesModelofStationaryRandomSequences linearsystemwithsystemfunctionH z Whitenoisew n x n Astationaryrandomsequencecanbedenotedasanoutputofalinearsystemwithawhitenoiseinput 1 7TimeSeriesModel Ifthelinearsystemhassystemfunction ThenH z iscalledthetimeseriesmodelofx n and IfallthezerosandpolesofH z areinsidetheunitcircle thentheH z iscalledaminimumphasesystem Ithasminimumphaselagamongallthesystemswithsameamplitude frequencyresponse Aminimumphasesystemusuallyhasastableinvertiblesystem ThreekindsoftimeseriesmodelsMoving average MA model Ifqisfinite thentheH z isaFIRfilter TheMAmodelissuitablefordescribingthesequenceswhosePSDhasvalesbuthasnopeak canbemodeledwithlessorders Auto regressive AR model TheARmodelisstableiffallitspolesareinsidetheunitcircle ItissuitableforthedescriptionofthesequenceswhosePSDhaspeaksbuthasnovale Auto regressivemovingaverage ARMA model TheARMAmodelissuitableforsequenceswhosePSDwithvalesandpeaks Wold sdecompositiontheoremAnyreal valuedstationaryrandomsequencex n canbedecomposedas whereu n andv n areuncorrelated Theu n isdeterministicandcanbeexactlydescribedbyalinearcombinationofitsownpast Itissometimesignoredinpractice Thev n isastationaryMAsequenceoftenwithfiniteorder EachoftheMA ARandARMAmodelsisuniversalandanyofthesethreemodelscanbearbitrarilyconvertedfromonetoanother oftenwithinfiniteorder 1 8TheIntroductiontoStatisticalEstimationTheory Estimation StatisticalSignalProcessingEstimationisthecentralprobleminrandomsignalprocessingAfterestimation thesignalsorparameterscanbetreatedasdeterministiconesAnexampleofcommunication Transmitter Receiver Noise Deterministicsignal Encoding Estimating Decoding Randomsignal bothofthesignalandnoisearerandom Deterministicsignal EstimationModelThegeneralstatisticalestimationmodel Parameterorstatespace unknown Observationspace Parameterorstateestimation Probabi listicmapping Estima tionrule Observationmodel Predictivemodel Systemnoiseprocess Observationnoiseprocess Randomsignalanditsobservationmodel Excitationprocess Parameterprocess Estimatingmethods estimators Knownsystemandobservationnoisemodels Estimationmodel Observationsfromtime0 k Theestimationofparameterorstateatk Iff isalinearfunction thentheestimatoriscalledalinearestimator TwoClassesofEstimationProblemsParameterestimationSignalmodelestimation estimatingthemodelofrandomsignalornoise suchasthenumericalcharacteristicestimation PSDestimationSystemmodelestimation systemidentification estimatingthemodelofthesystemwitharandomsignalinput Stateestimation waveestimation Estimatingstate wave basedonobservations alsoreferredasfiltering suchasWienerfilterandKalmanfilterFilteringproblemPredictionproblemSmoothingproblem BayesianEstimationTheoreticalbasis Bayes ruleCostfunctionandriskfunctionCostfunctionThecostwillbepaidwhentheestimationiswhileitstruevalueisRiskfunction theaveragecostwillbepaidforwiththeobservations Bayesianestimation theestimationwithminimumriskfunctionBayesianestimationisageneralframeworkfortheformulationofstatisticalinferenceproblems SomeEstimatorsMaximumaposteriori MAP estimation theestimationwithmaximum Maximum likelihood ML estimation theestimationthatmaximizesthelikelihoodfunction itcorrespondstotheMAPestimationwithuniformlydistributed i e thepriorPDFof isTherefore Forcaseswheretheisunknown theMMSEestimationcanbeobtainedby Minimummeansquareerror MMSE estimation PerformanceMeasuresandDesirablePropertiesofEstimatorsPerformancemeasuresExpectedvalueBiasCovariancematrix DesirablepropertiesUnbiasedestimatorAnestimatorisasymptoticallyunbiasedifEfficientestimator anunbiasedestimatorof isefficientthanotherunbiasedestimatorsifConsistentestimator anestimatorisconsistentifforany 0 i e theestimationerrorisreducedgraduallyalongwiththeincrementofsamples TheoremforthejudgmentofconsistenceIftheestimationof satisfiesThenisaconsistentestimationof Cramer RaoinequationTheCramer Raolowerboundonthevarianceofestimator MostefficientestimatorAnunbiasedestimatorthatachievestheCramer Raolowerboundiscalledtheminimumvariance orthemostefficient estimator i e amostefficientestimationfor ihas 1 9TheEstimationoftheStatisticsofErgodicStationaryRandomSequence EstimatorforMeanValueMLestimatorwithsamplexi i 0 1 N 1 Gaussianprocessisoftenassumed PerformanceBiashenceisanunbiasedestimator Variance Ifxiandxjareuncorrelatedfori j i e thenwhenN henceisaconsistentestimator Ifxiandxjarecorrelatedfori j thenwhere EstimatorforVarianceMLestimatorwithsamplexi i 0 1 N 1 Gaussianprocessisoftenassumed Suchaestimationisunbiasedandconsiste