语音信号和前端处理

7.0SpeechSignalandFront-endProcessing语音信号和前端处理

数字语音处理概论IntroductiontoDigitalSpeechProcessingWaveformplotsoftypicalvowelsounds-Voiced（浊音）tone1tone2tone4t

(音高)SpeechProductionandSourceModelHumanvocalmechanismSpeechSourceModelVocaltractu(t)x(t)excitationVoicedandUnvoicedSpeech

u(t)x(t)pitchpitchvoicedunvoicedWaveformplotsoftypicalconsonantsoundsUnvoiced（清音）Voiced（浊音）WaveformplotofasentenceTimeandFrequencyDomains

(P.12of2.0)X[k]timedomain1-1

mappingFourierTransformFastFourierTransform(FFT)

frequencydomain

FrequencydomainspectraofspeechsignalsVoicedUnvoicedFrequencyDomainformantfrequenciesformantfrequenciesexcitationexcitationFormantStructureFormantStructureVoicedUnvoicedInput/OutputRelationshipforTime/FrequencyDomainsFFFtimedomain:convolutionfrequencydomain:product

excitationformantstructureSpectrogramSpectrogramFormantFrequenciesFormantfrequencycontoursHewillallowararelie.Reference:6.1ofHuang,or2.2,2.3ofRabinerandJuangSpeechSignalsVoiced/unvoiced 浊音、清音Pitch/tone 音高、声调Vocaltract 声道Frequencydomain/formantfrequencySpectrogramrepresentationSpeechSourceModeldigitizationandtransmissionoftheparameterswillbeadequateatreceivertheparameterscanproducex[n]withthemodelmuchlessparameterswithmuchslowervariationintimeleadtomuchlessbitsrequiredthekeyforlowbitratespeechcoding

x[n]u[n]parametersparametersExcitationGeneratorVocalTractModelExG(

),G(z),g[n]x[n]=u[n]

g[n]X()=U()G()X(z)=U(z)G(z)U(

)U(z)SpeechSourceModelx(t)a[n]tnSpeechSourceModelSophisticatedmodelforspeechproductionSimplifiedmodelforspeechproductionUnvoicedPeriodicImpulseTrainGeneratorUncorrelatedNoiseGeneratorG(z)GlottalfilterH(z)VocalTractFilterR(z)LipRadiationFilterVUGGx(n)SpeechSignalVoicedPitchperiod,NUnvoicedPeriodicImpulseTrainGeneratorRandomSequenceGeneratorCombinedFilterVUGx(n)SpeechSignalVoiced/unvoicedswitchPitchperiod,NVoicedSimplifiedSpeechSourceModelExcitationparametersv/u:voiced/unvoicedN:pitchforvoicedG:signalgain

excitationsignalu[n]unvoiced

voiced

randomsequencegeneratorperiodicpulsetraingenerator

x[n]G(z)=11

akz-kPk=1ExcitationG(z),G(

),g[n]VocalTractModelu[n]Gv/uNVocalTractparameters{ak}:LPCcoefficients

formantstructureofspeechsignalsAgoodapproximation,thoughnotpreciseenoughReference:3.3.1-3.3.6ofRabinerandJuang,or6.3ofHuangSpeechSourceModelu[n]x[n]

FeatureExtraction-MFCCMel-FrequencyCepstralCoefficients(MFCC)MostwidelyusedinthespeechrecognitionHasgenerallyobtainedabetteraccuracyatrelativelylowcomputationalcomplexityTheprocessofMFCCextraction:SpeechsignalPre-emphasisWindowDFTMel

filter-bankLog(||2)IDFTMFCCenergyderivativesx(n)x’(n)xt(n)Xt(k)Yt(m)Yt’(m)yt

(j)etPre-emphasisTheprocessofPre-emphasis:ahigh-passfilterH(z)=1-a•z-10<a≤1Speechsignalx(n)x’(n)=x(n)-ax(n-1)Whypre-emphasis?Reason:Voicedsectionsofthespeechsignalnaturallyhaveanegativespectralslope(attenuation)ofapproximately20dBperdecadeduetothephysiologicalcharacteristicsofthespeechproductionsystemHighfrequencyformantshavesmallamplitudewithrespecttolowfrequencyformants.Apre-emphasisofhighfrequenciesisthereforehelpfultoobtainsimilaramplitudeforallformantsWhyWindowing?Whydividingthespeechsignalintosuccessiveandoverlappingframes?Voicesignalschangetheircharacteristicsfromtimetotime.Thecharacteristicsremainunchangedonlyinshorttimeintervals(short-timestationary,short-timeFouriertransform)FramesFrameLength

:thelengthoftimeoverwhichasetofparameterscanbeobtainedandisvalid.Framelengthrangesbetween20~10msFrameShift:thelengthoftimebetweensuccessiveparametercalculationsFrameRate:numberofframespersecondWaveformplotofasentenceHammingRectangularx[n]x[n]w[n]FFEffectofWindowing(1)Windowing:xt(n)=w(n)•x’(n),w(n):theshapeofthewindow(productintimedomain)Xt(

)=W(

)*X’(

),*:convolution

(convolutioninfrequencydomain)

Rectangularwindow(w(n)=1for0≤n≤L-1):

simplyextractasegmentofthesignalwhosefrequencyresponsehashighsidelobesMainlobe:spreadsoutthenarrowbandpowerofthesignal(thataroundtheformantfrequency)inawiderfrequencyrange,

andthusreducesthelocalfrequency

resolutioninformantallocationSidelobe:swapenergyfromdifferent

anddistantfrequencies

(dB)RectangularHammingInput/OutputRelationshipforTime/FrequencyDomains(P.10of7.0)FFFtimedomain:convolutionfrequencydomain:product

excitationformantstructureWindowingmainlobesidelobesMainlobe:spreadsoutthenarrowbandpowerofthesignal(thataroundtheformantfrequency)inawiderfrequencyrange,andthusreducesthelocalfrequencyresolutioninformantallocationSidelobe:swapenergyfromdifferent

anddistantfrequenciesEffectofWindowing(2)Windowing(Cont.):

Foradesignedwindow,wewishthatthemainlobeisasnarrowaspossiblethesidelobeisaslowaspossibleHowever,itisimpossibletoachievebothsimultaneously.Sometrade-offisneededThemostwidelyusedwindowshapeistheHammingwindowDFTandMel-filter-bankProcessingForeachframeofsignal(Lpoints,e.g.,L=512),

theDiscreteFourierTransform(DFT)isfirstperformedtoobtainitsspectrum(Lpoints,forexampleL=512)ThebankoffiltersbasedonMelscaleisthenapplied,andeachfilteroutputisthesumofitsfilteredspectralcomponents(Mfilters,andthusMoutputs,forexampleM=24)DFTt

Timedomainsignalspectrumsumsumsum

Yt(0)Yt(1)Yt(M-1)xt(n)Xt(k)n=0,1,....L-1k=0,1,....-12LPeripheralProcessingforHumanPerceptionMel-scaleFilterBank

WhyFilter-bankProcessing?Thefilter-bankprocessingsimulateshumanearperceptionFrequenciesofacomplexsoundwithinacertainfrequencybandcannotbeindividuallyidentified.Whenoneofthecomponentsofthissoundfallsoutsidethisfrequencyband,itcanbeindividuallydistinguished.Thisfrequencybandisreferredtoasthecriticalband.Thesecriticalbandssomehowoverlapwitheachother.Thecriticalbandsareroughlydistributedlinearlyinthelogarithmfrequencyscale(includingthecenterfrequenciesandthebandwidths),speciallyathigherfrequencies.Humanperceptionforpitchofsignalsisproportionaltothelogarithmofthefrequencies(relativeratiosbetweenthefrequencies)

FeatureExtraction-MFCCMel-FrequencyCepstralCoefficients(MFCC)MostwidelyusedinthespeechrecognitionHasgenerallyobtainedabetteraccuracyatrelativelylowcomputationalcomplexityTheprocessofMFCCextraction:SpeechsignalPre-emphasisWindowDFTMel

filter-bankLog(||2)IDFTMFCCenergyderivativesx(n)x’(n)xt(n)Xt(k)Yt(m)Yt’(m)yt

(j)etLogarithmicOperationandIDFTThefinalprocessofMFCCevaluation:logarithmoperationandIDFTMel-filteroutput

Yt(m)Filterindex(m)Filterindex(m)Log(||2)Y’t(m)IDFTquefrency(j)MFCCvectoryt(j)yt=CY’t00M-10M-1J-1WhyLogEnergyComputation?Usingthemagnitude(orenergy)onlyPhaseinformationisnotveryhelpfulinspeechrecognitionReplacingthephasepartoftheoriginalspeechsignalwithcontinuousrandomphaseusuallywon’tbeperceivedbyhumanears

UsingtheLogarithmicoperationHumanperceptionsensitivityisproportionaltosignalenergyinlogarithmicscale(relativeratiosbetweensignalenergyvalues)Thelogarithmcompresseslargervalueswhileexpandssmallervalues,whichisacharacteristicofthehumanhearingsystemThedynamiccompressionalsomakesfeatureextractionlesssensitivetovariationsinsignaldynamicsTomakeaconvolvednoisyprocessadditiveSpeechsignalx(n),excitationu(n)andtheimpulseresponseofvocaltractg(n)

x(n)=u(n)*g(n)X(

)=U(

)G(

)

|X(

)|=|U(

)||G(

)|log|X(

)|=log|U(

)|+log|G(

)|

WhyInverseDFT?FinalprocedureforMFCC:performingtheinverseDFTonthelog-spectralpowerAdvantages:Sincethelog-powerspectrumisrealandsymmetric,theinverseDFTreducestoaDiscreteCosineTransform(DCT).TheDCThasthepropertytoproducehighlyuncorrelatedfeaturesytdiagonalratherthanfullcovariancematricescanbeusedintheGaussiandistributionsinmanycasesEasiertoremovetheinterferenceofexcitationonformantstructuresthephonemeforasegmentofspeechsignalisprimarilybasedontheformantstructure(orvocaltractshape)onthefrequencyscaletheformantstructurechangesslowlyoverfrequency,whiletheexcitationchangesmuchfasterSpeechProductionandSourceModel(P.3of7.0)HumanvocalmechanismSpeechSourceModelVocaltractu(t)x(t)excitationVoicedandUnvoicedSpeech

(P.4of7.0)

u(t)x(t)pitchpitchvoicedunvoicedFrequencydomainspectraofspeechsignals(P.8of7.0)VoicedUnvoicedFrequencyDomain

(P.9of7.0)formantfrequenciesformantfrequenciesexcitationexcitationFormantStructureFormantStructureVoicedUnvoicedInput/OutputRelationshipforTime/FrequencyDomains(P.10of7.0)FFFtimedomain:convolutionfrequencydomain:product

excitationformantstructureLogarithmicOperation

u[n]g[n]x[n]=u[n]*g[n]

DerivativesDerivativeoperation:toobtainthechangeofthefeaturevectorswithtime

MFCCstreamyt(j)quefrency(j)Frameindex

t-1tt+1t+2quefrency(j)Frameindex

MFCCstream

yt(j)quefrency(j)Frameindex

2MFCCstream

2yt(j)LinearRegression(xi,yi)y=ax+bfinda,bWhyDeltaCoefficients?TocapturethedynamiccharactersofthespeechsignalSuchinformationcarriesrelevantinformationforspeechrecognitionThevalueofpshouldbeproperlychosenThedynamicc