说话人差异：适应与识别

13.0SpeakerVariabilities:AdaptionandRecognition说话人差异：适应与识别

数字语音处理概论IntroductiontoDigitalSpeechProcessingReferences:1.9.6ofHuang 2.“MaximumAPosterioriEstimationforMultivariateGaussianMixtureObservationsofMarkovChains”,IEEETrans.onSpeechandAudioProcessing,April1994 3.“MaximumLikelihoodLinearRegressionforSpeakerAdaptationofContinuousDensityHiddenMarkovModels”,ComputerSpeechandLanguage,Vol.9,1995 4.Jolliffe,“PrincipalComponentAnalysis”,Springer-Verlag,1986 5.“RapidSpeakerAdaptationinEigenvoiceSpace”,IEEETrans.onSpeechandAudioProcessing,Nov2000 6.“ClusterAdaptiveTrainingofHiddenMarkovModels”,IEEETrans.onSpeechandAudioProcessing,July2000 7.“ACompactModelforSpeaker-adaptiveTraining”,InternationalConferenceonSpokenLanguageProcessing,1996 8.“ATutorialonText-independentSpeakerVerification”,EURASIPJournalonAppliedSignalProcessing20049.“AnOverviewofText-independentSpeakerRecognition:fromFeaturestoSupervectors”,SpeechCommunication,Jan2010SpeakerDependent/Independent/AdaptationSpeakerDependent(SD)

trainedwithandusedfor1speakeronly,requiringhugequantityoftrainingdata,bestaccuracypracticallyinfeasibleMulti-speakertrainedfora(small)groupofspeakersSpeakerIndependent(SI)trainedfromlargenumberofspeakers,eachspeakerwithlimitedquantityofdatagoodforallspeakers,butwithrelativelyloweraccuracySpeakerAdaptation(SA)startedwithspeakerindependentmodels,adaptedtoaspecificuserwithlimitedquantityofdata(adaptationdata)technicallyachievableandpracticallyfeasibleSupervised/UnsupervisedAdaptationsupervised:text(transcription)oftheadaptationdataisknownunsupervised:text(transcription)oftheadaptationdataisunknown,basedonrecognitionresultswithspeaker-independentmodels,maybeperformediterativelyBatch/Incremental/On-lineAdaptationbatch:basedonawholesetofadaptationdataincremental/on-line:adaptedstep-by-stepwithiterativere-estimationofmodels e.g.firstadaptedbasedonfirst3utterances,thenadaptedbasedonnext3utterancesorfirst6utterances,...SASASDSD/i//a/SpeakerDependent/Independent/AdaptationMAP(MaximumAPosteriori)AdaptationGivenSpeaker-independentModelsetΛ={λi=(Ai,Bi,πi),i=1,2,...M}andAsetofAdaptationDataO=(o1,o2,...ot,...oT)forASpecificSpeakerWithSomeAssumptionsonthePriorKnowledgeProb[Λ]andsomeDerivation(EMTheory)exampleadaptationformulaaweightedsumshiftingμjktowardsthosedirectionsofot(inj-thstateandk-thGaussian)

largerτjkimplieslessshiftOnlyThoseModelswithAdaptationDatawillbeModified,UnseenModelsremainUnchanged—MAPPrinciplegoodwithlargerquantityofadaptationdatapoorperformancewithlimitedquantityofadaptationdataargmaxΛτjk:aparameterhavingtodothepriorknowledgeaboutμjkmayhavetodowithnumberofsamplesusedtotrainμjkMAPAdaptation

AccuracyAdaptationDataSDSIBlock-diagonalMLLRMLLRMAPMAPAdaptationMaximumLikelihoodLinearRegression(MLLR)DividetheGaussians(orModels)intoClassesC1,C2,...CL,andDefineTransformation-basedAdaptationforeachClasslinearregressionwithparametersA,bestimatedbymaximumlikelihoodcriterionAllGaussiansinthesameclassup-datedwiththesameAi,bi:parametersharing,adaptationdatasharingunseenGaussians(ormodels)canbeadaptedaswellAicanbefullmatrices,orreducedtodiagonalorblock-diagonaltohavelessparameterstobeestimatedfasteradaptationwithmuchlessadaptationdataneeded,butsaturatedatloweraccuracywithmoreadaptationdataduetothelessprecisemodelingClusteringtheGaussians(orModels)intoLClassestoomanyclassesrequiresmoreadaptationdata,toolessclassesbecomeslessaccuratebasicprinciple:Gaussian(ormodels)withsimilarpropertiesand“justenough”dataformaclassdata-driven(e.g.byGaussiandistances)primarily,knowledgedrivenhelpfulTree-structuredClassesthenodeincludingminimumnumberofGaussians(ormodels)butwithadequateadaptationdataisaclassdynamicallyadjustingtheclassesasmoreadaptationdataareobserved(A2,b2)(A1,b1)(A3,b3)MLLRMLLRDiagonalBlock-diagonalFull

MLLRPrincipalComponentAnalysis(PCA)ProblemDefinition:

forazeromeanrandomvectorxwithdimensionalityN,x∈RN,E(x)=0,iterativelyfindasetofk(k

N)orthonormalbasisvectors{e1,e2,…,ek}sothat

(1)var(e1Tx)=max(xhasmaximumvariancewhenprojectedone1)

(2)var(eiTx)=max,subjecttoei

ei-1……e1,2

i

k

(xhasnextmaximumvariancewhenprojectedone2,etc.)Solution:{e1,e2,…,ek}aretheeigenvectorsofthecovariancematrix

forxcorrespondingtothelargestkeigenvaluesnewrandomvectory

Rk:theprojectionofxontothesubspacespannedbyA=[e1

e2……

ek],y=ATxasubspacewithdimensionalityk≤Nsuchthatwhenprojectedontothissubspace,yis“closest”toxintermsofits“randomness”foragivenkvar(eiT

x)istheeigenvalueassociatedwithei

Proofvar(e1T

x)=e1TE(x

xT)e1=e1TΣe1=max,subjectto|e1|2=1usingLagrangemultiplier J(e1)=e1TE(xxT)e1-λ(|e1|2-1),

⇒E(xxT)e1=λ1e1,var(e1T

x)=λ1=maxsimilarfor

e2withanextraconstrainte2Te1=0,etc.=0

J(e1)

e1

eigenvectoreigenvaluePCA

PCA

BasicProblem3(P.35of4.0)

PCA

eigenvectoreigenvalue

eigenvectoreigenvalue

N-dimk-dimPCA

N=3k=2PCAEigenvoiceASupervectorx

constructedbyconcatenatingallrelevantparametersforthespeakerspecificmodelofatrainingspeaker

concatenatingthemeanvectorsofGaussiansinthespeaker-dependentphonemodelsconcatenatingthecolumnsofA,binMLLRapproachxhasdimensionalityN(N=5,000×3×8×40=4,800,000forexample)SDmodelmeanparameters(m)transformationparameters(A,b)AtotalofL(L=1,000forexample)trainingspeakersgivesLsupervectorsx1,x2,...xLx1,x2,

x3.....xLaresamplesoftherandomvectorxeachtrainingspeakerisapoint(orvector)inthespaceofdimensionalityNPrincipalComponentAnalysis(PCA)x'=x-E(x),Σ=E(x'x'T), {e1,e2,.....ek}:eigenvectorswithmaximumeigenvaluesλ1>λ2...>

λk

kischosensuchthatλj,j>kissmallenough(k=50forexample)EigenvoicePrincipalComponentAnalysis(PCA)x'=x-E(x),Σ=E(x'x'T),

{e1,e2,.....ek}:eigenvectorswithmaximumeigenvaluesλ1>λ2...>

λk

kischosensuchthatλj,j>kissmallenough(k=50forexample)EigenvoiceSpace:spannedby{e1,e2,.....ek}eachpoint(orvector)inthisspacerepresentsawholesetoftri-phonemodelparameters{e1,e2,.....ek}representsthemostimportantcharacteristicsofspeakersextractedfromhugequantityoftrainingdatabylargenumberoftrainingspeakerseachnewspeakerasapoint(orvector) inthisspace,aiestimatedbymaximumlikelihood principle(EMalgorithm)FeaturesandLimitationsonlyasmallnumberofparametersa1...akisneededtospecifythecharacteristicsofanewspeakerrapidadaptationrequiringonlyverylimitedquantityoftrainingdataperformancesaturatedatloweraccuracy(becausetoofewfreeparameters)highcomputation/memory/trainingdatarequirementsTrainingSpeaker1TrainingSpeaker2NewSpeakerNewSpeakerSpeakerAdaptiveTraining(SAT)andClusterAdaptiveTraining(CAT)SpeakerAdaptiveTraining(SAT)

tryingtodecomposethephoneticvariationandspeakervariationremovingthespeakervariationamongtrainingspeakersasmuchaspossibleobtaininga“compact”speaker-independentmodelforfurtheradaptationy=Ax+binMLLRcanbeusedinremovingthespeakervariationClusteringAdaptiveTraining(CAT)dividingtrainingspeakersintoRclustersbyspeakerclusteringtechniquesobtainingmeanmodelsforallclusters(mayincludeamean-biasforthe“compact”modelinSAT)modelsforanewspeakerisinterpolatedfromthemeanvectorsSpeakerAdaptiveTraining(SAT) TrainingSpeakersSpeaker1Speaker2SpeakerL“Compact”Speaker-independentmodelMAPMLLRA1,b1A2,b2AL,bLClusterAdaptiveTraining(CAT)

clustermean1

clustermean2

clustermeanRLTrainingSpeakersa1a2aRmean-bias1Σ

meanforanewspeaker,mi:clustermeani,mb:mean-bias

SDSI/a//i/SAT

SpeakerRecognition/VerificationTorecognizethespeakersratherthanthecontentofthespeech

phoneticvariation/speakervariationspeakeridentification:toidentifythespeakerfromagroupofspeakersspeakerverification:toverifyifthespeakerisasclaimedGaussianMixtureModel(GMM)

λi={(wj,μj,Σj,),j=1,2,...M}forspeakeri forO=o1o2...ot...oT,maximumlikelihoodprincipleFeatureParametersthosecarryingspeakercharacteristicspreferredMFCCMLLRcoefficientsAi,bi,eigenvoicecoefficientsai,CATcoefficientsaiSpeakerVerificat