数字语音处理概论

DigitalSpeechProcessing数字语音处理概论第一讲IntroductiontoDigitalSpeechProcessing数字语音处理简介SpeechSignalProcessingMajorApplicationAreasSpeechCoding:DigitizationandCompression

Considerations:1)bitrate(bps) 2)recoveredquality 3)computation complexity/feasibilityVoice-basedNetworkAccess—UserInterface,ContentAnalysis,User-contentInteractionLPFoutputProcessingAlgorithmsx(t)x[n]Processingxk110101…InverseProcessingx[n]x[n]^Storage/transmissionSpeechSignalsCarryingLinguisticKnowledgeandHumanInformation:Characters,Words,Phrases,Sentences,Concepts,etc.DoubleLevelsofInformation:AcousticSignalLevel/SymbolicorLinguisticLevelProcessingandInteractionoftheDouble-levelInformationSamplingofSignalsX(t)X[n]tnDoubleLevelsofInformation字(Character)词(Word)句(Sentence)人人用电脑计算机SpeechSignalProcessing–ProcessingofDouble-LevelInformation

SpeechSignal

Sampling

Processing

LinguisticStructure

LinguisticKnowledge

今天的

常好LexiconGrammar今天的

今天的天气非常好AlgorithmChipsorComputers

天气非Well-KnownApplicationExamplesofSpeechandLanguageTechnologies–SpeakingPersonalAssistantExamplesWeatherinNewYorknextweek?WhoisthepresidentofUS?Whatdidhesaytoday?HowcanIgotoNationalTaiwanUniversity?Shortmessaging,personalscheduling,etc.Examples:Siri(Apple),GoogleNow(Google),Cortana(Microsoft)OutputSpeechSignalsLanguageGenerationInputSpeechSignalsLanguageUnderstandingSpeechSynthesisDialogueManagerInformationRetrievalKnowledgeGraphMachineTranslationSpeechRecognitionQuestionAnsweringWikipediaSpecialQuestions:唐诗宋词,出师表…说个笑话…Voice-basedNetworkAccessContentAnalysisUserInterfaceInternetUser-ContentInteractionUserInterface

—whenkeyboards/miceinadequateContentAnalysis

—helpinbrowsing/retrievalofmultimediacontentUser-ContentInteraction

—alltext-basedinteractioncanbeaccomplishedbyspokenlanguageUserInterface—WirelessCommunicationsTechnologieshaveCreatedaWholeVarietyofUserTerminalsatAnyTime,fromAnywhereSmartphones,Hand-heldDevices,Notebooks,VehicularElectronics,Hands-freeInterfaces,HomeAppliances,WearableDevices…SmallinSize,LightinWeight,Ubiquitous,Invisible…Post-PCEraKeyboard/MouseMostConvenientforPC’snotConvenientanylonger—humanfingersnevershrink,andapplicationenvironmentischangedServiceRequirementsGrowingExponentiallyVoiceistheOnlyInterfaceConvenientforALLUserTerminalsatAnyTime,fromAnywhere,andtothepointinoneutteranceSpeechProcessingistheonlylessmaturepartintheTechnologyChain

InternetTextContentMultimediaContentContentAnalysis—MultimediaTechnologieshaveCreatedaWorldofMultimediaContentMostAttractiveFormoftheNetworkContentisMultimedia,whichusuallyIncludesSpeechInformation(butProbablynotText)MultimediaContentDifficulttobeSummarizedandShownontheScreen,thusDifficulttoBrowseTheSpeechInformation,ifIncluded,usuallyTellstheSubjects,TopicsandConceptsoftheMultimediaContent,thusBecomestheKeyforBrowsingandRetrievalMultimediaContentAnalysisbasedonSpeechInformationInternetReal–timeInformation

weather,trafficflightschedulestockpricesportsscoresSpecialServicesGoogleFaceBookYouTubeAmazonKnowledgeArchieves

digitallibrariesvirtualmuseumsIntelligentWorkingEnvironment

e–mailprocessorsintelligentagentsteleconferencingdistantlearningelectriccommercePrivateServices

personalnotebookbusinessdatabaseshomeappliancesnetworkentertainmentsUser-ContentInteraction

—WirelessandMultimediaTechnologiesareCreatingAnEraofNetworkAccessbySpokenLanguageProcessingvoiceinformationMultimediaContentInternetvoiceinput/outputtextinformationHand-heldDeviceswithMultimediaFunctionalitiesCommonlyusedTodayNetworkAccessisPrimarilyText-basedtoday,butalmostallRolesofTextscanbeAccomplishedbySpeechUser-ContentInteractioncanbeAccomplishedbySpokenandMulti-modalDialoguesUsingSpeechInstructionstoAccessMultimediaContentwhoseKeyConceptsSpecifiedbySpeechInformationMultimedia

ContentAnalysisTextInformationRetrievalTextContentVoice-basedInformationRetrievalText-to-SpeechSynthesisSpokenandmulti-modalDialogueVoice-basedInformationRetrievalVoiceInstructions请问鼎泰丰的地址？TextInstructionsd1TextInformationd2d3d1d2d3VoiceInformation

鼎泰丰台北101分店在…BoththeUserInstructionsandNetworkContentCanbeinformofSpeechSpokenandMulti-modalDialoguesAlmostAllUser-ContentInteractioncanbeAccomplishedbySpokenorMulti-modalDialoguesDatabasesSentenceGenerationandSpeechSynthesisOutputSpeechInputSpeechDialogueManagerSpeechRecognitionandUnderstandingUser’sIntentionDiscourseContextResponsetotheuserInternetWirelessNetworksUsersDialogueServerOutlineBothTheoreticalIssuesandPracticalProblemswillbeDiscussedStartingwithFundamentals,butEnteringResearchTopicsintheSecondHalfPartI:FundamentalTopics1.0IntroductiontoDigitalSpeechProcessing2.0FundamentalsofSpeechRecognition3.0MapofSubjectAreas4.0MoreaboutHiddenMarkovModels5.0AcousticModeling6.0LanguageModeling7.0SpeechSignalsandFront-endProcessing8.0SearchAlgorithmsforSpeechRecognitionPartII:AdvancedTopics

9.0SpeechRecognitionUpdates10.0Speech-basedInformationRetrieval11.0SpokenDocumentUnderstandingandOrganizationforUser-contentInteraction12.0Computer-assistedLanguageLearning(Call)13.0SpeakerVariabilities:AdaptionandRecognition14.0LatentTopicAnalysis15.0RobustnessforAcousticEnvironment16.0SomeFundamentalProblem-solvingApproaches17.0SpokenDialogues18.0Conclusion教科书：无主要参考书：X.Huang,A.Acero,H.Hon,“SpokenLanguageProcessing”,PrenticeHall,2001,松瑞F.Jelinek,“StatisticalMethodsforSpeechRecognition”,MITPress,1999L.Rabiner,B.H.Juang,“FundamentalsofSpeechRecognition”,PrenticeHall,1993,民全C.Becchetti,L.PrinaRicotti,“SpeechRecognition-TheoryandC++implementation”,JohyWileyandSons,1999,民全D.Jurafsky,J.Martin,“SpeechandLanguageProcessing-AnIntroductiontoNaturalLanguageProcessing,SpeechRecognition,andComputationalLinguistics,2ndedition”,Prentice-Hall,2009G.Tur,R.DeMori,“SpokenLanguageUnderstanding-SystemsforExtractingSemanticInformationfromSpeech”,JohnWiley&Sons,2011其他参考文献课堂上提供References教材：availableonwebbeforethedayofclass(.tw)适合年级：三、四（电机系、资工系）成绩评量方式MidtermExam25%Homeworks(I)(II)(Ⅲ)15%、5%、15％FinalExam10%TermProject30%OtherInformationGoals课程目的：提供同学进入此一充满机会与挑战的新领域所需的基本知识，体验数学模型与软件程序如何相辅相成，学习进入一个新领域由基础进入研究的历程，体会吸收非结构性知识(UnstructuredKnowledge)的经验UnstructuredKnowledgeABCDMath&ProgrammingMathematicalModelsProgrammingHardware1.0Introduction—ABriefSummaryofCoreTechnologiesandExampleApplicationSeenariosReferencesfor1.01.“SpeechandLanguageProcessingovertheWeb”,IEEESignalProcessingMagazine,May2008FeatureExtractionunknownspeechsignalPatternMatchingDecisionMakingx(t)WXoutputwordfeaturevectorsequenceReferencePatternsFeatureExtractiony(t)YtrainingspeechSpeechRecognitionasapatternrecognitionproblemASimplifiedBlockDiagramExampleInputSentence

thisisspeechAcousticModels(声学模型)

(th-ih-s-ih-z-s-p-ih-ch)Lexicon(th-ih-s)→this(ih-z)→is(s-p-iy-ch)→speechLanguageModel

(语言模型)

(this)–(is)–(speech) P(this)P(is|this)P(speech|thisis) P(wi|wi-1)bi-gram

languagemodel P(wi|wi-1,wi-2)tri-gramlanguagemodel,etcBasicApproachforLargeVocabularySpeechRecognitionFront-endSignalProcessingAcousticModelsLexiconFeatureVectorsLinguisticDecodingandSearchAlgorithmOutputSentenceSpeechCorporaAcousticModelTrainingLanguageModelConstructionTextCorporaLanguageModelInputSpeechSpeechRecognitionTechnologies,ApplicationsandProblemsWordRecognitionvoicecommand/instructionsKeywordSpottingidentifyingthekeywordsoutofapre-definedkeywordsetfrominputvoiceutterancesLargeVocabularyContinuousSpeechRecognitionenteringlongertextsremotedictation/automatictranscriptionSpeakerDependent/Independent/AdaptiveAcousticReception/BackgroundNoise/ChannelDistortionRead/Spontaneous/ConversationalSpeechText-to-speechSynthesisTextAnalysisandLetter-to-soundConversionProsodyGenerationSignalProcessingandConcatenationLexiconandRulesProsodicModelVoiceUnitDatabaseInputTextOutputSpeechSignalTransforminganyinputtextintocorrespondingspeechsignalsE-mail/WebpagereadingProsodicmodelingBasicvoiceunits/rule-based,non-uniformunits/corpus-based,model-basedSpeechUnderstandingUnderstandingSpeaker’sIntentionratherthanTranscribingintoWordStringsLimitedDomains/FiniteTasksacousticmodelsphraselexiconSyllableRecognitionKeyPhraseMatchinginpututterancesyllablelatticephrasegraphconceptgraphconceptsetphrase/conceptlanguagemodelSemanticDecodingunderstandingresultsAnExample

utterance:请帮我查一下

我省银行的电话号码是几号?keyphrases:(查一下)-(我省银行)-(电话号码)concept:(inquiry)-(target)-(phonenumber)SpeakerVerificationFeatureExtractionVerificationinputspeechyes/noVerifyingthespeakerasclaimedApplicationsrequiringverificationTextdependent/independentIntegratedwithotherverificationschemesSpeakerModelsVoice-basedInformationRetrievalSpeechInstructionsSpeechDocuments(orMulti-mediaDocumentsincludingSpeechInformation)speechinstruction请问鼎泰丰的地址？textinstructiond1textdocumentsd2d3d1d2d3speechdocuments鼎泰丰台北101分店在…LocateexactlythedesiredutterancesTextdescriptionsnotneededforindexing/retrievingpuporsesSpokenDialogueSystemsAlmostallhuman-networkinteractionscanbeaccomplishedbyspokendialogueSpeechunderstanding,speechsynthesis,dialoguemanagementSystem/user/mixedinitiativesReliability/efficiency,dialoguemodeling/flowcontrolTransactionsuccessrate/averagenumberofdialogueturnsDatabasesSentenceGenerationandSpeechSynthesisOutputSpeechInputSpeechDialogueManagerSpeechRecognitionandUnderstandingUser’sIntentionDiscourseContextResponsetotheuserInternetNetworksUsersDialogueServerSpokenDocumentUnderstandingandOrganizationUnliketheWrittenDocumentswhichareeasilyshownonthescreenforusertobrowseandselect,SpokenDocumentsarejustAudioSignals

—theusercan’tlisteneachonefromthebeginningtotheendduringbrowsing

—betterapproachesforunderstanding/organizationofspokendocumentsbecomesnecessarySpokenDocumentSegmentation

—automaticallysegmentingaspokendocumentintoshortparagraphs,eachwith acentraltopicSpokenDocumentSummarization

—automaticallygeneratingasummary(intextorspeechform)foreachshortparagraphTitleGenerationforSpokenDocuments

—automaticallygeneratingatitle(intextorspeechform)foreachshortparagraphKeyTermExtractionandKeyTermGraphConstructionforSpokenDocuments

—automaticallyextractingasetofkeytermsforeachspokendocument,andconstructingkeytermgraphsforacollectionofspokendocumentsSemanticStructuringofSpokenDocuments