基于语音的信息检索70

10.0Speech-basedInformationRetrieval基于语音的信息检索

数字语音处理概论IntroductiontoDigitalSpeechProcessingText-basedinformationretrievalextremelysuccessfulinformationdesiredbytheuserscanbeobtainedveryefficientlyalluserslikeitproducingverysuccessfulindustryAllrolesoftextscanbeaccomplishedbyvoicespokencontentormultimediacontentwithvoiceinaudiopartvoiceinstructions/queriesviahandhelddevicesSpeech-basedinformationretrievaluserinstructions/queriesInternetServerServerDocuments/InformationText/Speech-basedInformationRetrievalSpeech-basedInformationRetrievalSpokenInstructions/QueriesSpokencontent(multimediacontentincludingaudiopart)USpresident？TextInstructions/QueriesTextContentBarackObama….BarackObama….Userinstructionsand/ornetworkcontentcanbeinformofvoicetextqueries/spokencontent:spokendocumentretrieval,spokentermdetectionspokenqueries/textcontent:voicesearchspokenqueries/spokencontent:querybyexample

Manyhand-helddeviceswithmultimediafunctionalitiesavailableUnlimitedquantitiesofmultimediacontentfastgrowingovertheInternetUser-contentinteractionnecessaryforretrievalcanbeaccomplishedbyspokenandmulti-modaldialoguesNetworkaccessisprimarilytext-basedtoday,butalmostallrolesoftextscanbeaccomplishedbyvoiceWirelessandMultimediaTechnologiesareCreatingAnEnvironmentforSpeech-basedInformationRetrievalMultimedia

ContentAnalysisvoiceinformationMultimediaandSpokenContentvoiceinput/outputTextContentRetrievalTextContentSpokenContentRetrievalSpokenandmulti-modalDialogueInternettextinformationText-to-SpeechSynthesisBasicApproachforSpokenContentRetrievalLexiconSpokenContentRecognitionEngineLanguageModelTranscriptionsRetrievalResults(listofspokendocuments/utterances)Text-basedSearchEngineuserQueryQ(textortranscribedifinvoice)AcousticModelsTranscribethespokencontentSearchoverthetranscriptionsastheyaretextsRecognitionerrorscauseseriousperformancedegradationLowrecognitionaccuraciesforspontaneousspeechincludingOut-of-Vocabulary(OOV)wordsunderadverseenvironmentconsideringlatticeswithmultiplealternativesratherthan1-bestoutputhigherprobabilityofincludingcorrectwords,butalsoincludingmorenoisywordscorrectwordsmaystillbeexcluded(OOVandothers)hugememoryandcomputationrequirementsLatticesforSpokenContentRetrievalW6W8W4W1W8W7W9W3W2W5W10StartnodeEndnodeTimeindexWi:wordhypothesesConfusionMatricesuseofconfusionmatricestomodelrecognitionerrorsandexpandthequery/document,etc.PronunciationModelinguseofpronunciationmodelstoexpandthequery,etc.FuzzyMatchingquery/contentmatchingnotnecessarilyexactOtherApproachExamplesinadditiontoLatticesOOVorRareWordsHandledbySubwordUnitsOOVWordW=w1w2w3w4can’tberecognizedandneverappearsinlattice

wi

:subwordunits:phonemes,syllables…a,b,c,d,e:othersubwordunitsW=w1w2w3w4hiddenatsubwordlevelcanbematchedatsubwordlevelwithoutbeingrecognizedFrequentlyUsedSubwordUnitsLinguisticallymotivatedunits:phonemes,syllables/characters,morphemes,etc.Data-drivenunits:particles,wordfragments,phonemultigrams,morphs,etc.w1w2w2w3bcdew3w4baLattice:Timeindexw1w2w3w4w3w4PerformanceMeasures(1/2)RecallandPrecisionRatesB A

CretrieveddocumentsrelevantdocumentsPrecisionrate=Recallrate= AA+Crecallratemaybedifficulttoevaluate,whileprecisionrateisdirectlyperceivedbyusersrecall-precisionplotwithvaryingthresholdsAA+BMAP(meanaverageprecision)areaunderrecall-precisioncurveaperformancemeasurefrequentlyusedforinformationretrievalPerformanceMeasures(2/2)MAP=0.484MAP=0.586ReferencesGeneralorbasicSpokenContentRetrieval/asru2011/lecture.php?lang=en&id=5SpokenContentRetrieval-LatticesandBeyond(Lin-shanLee’stalkatASRU2011)Chelba,C.,Hazen,T.J.,Saraclar,M.,"Retrievalandbrowsingofspokencontent,"

SignalProcessingMagazine,IEEE

,vol.25,no.3,pp.39-49,May2008MarthaLarsonandGarethJ.F.Jones(2012)"SpokenContentRetrieval:ASurveyofTechniquesandTechnologies",FoundationsandTrendsinInformationRetrieval:Vol.5:No4-5,pp235-422“AnIntroductiontoVoiceSearch”,SignalProcessingMagazine,IEEE,Vol.25,2008Text-basedInformationRetrieval/IR-book/ChristopherD.

Manning,

PrabhakarRaghavan,

HinrichSchütze,IntroductiontoInformationRetrieval,CambridgeUniversityPress.2008.VectorSpaceModelVectorRepresentationsofqueryQanddocumentdforeachtypejofindexingfeature(e.g.syllable,word,etc.)avectorisgeneratedeachcomponentinthisvectoristheweightedstatisticszjtofaspecificindexingtermt(e.g.syllablesi)ct:frequencycountsfortheindexingtermtpresentinthequeryqordocumentd(fortext),orsumofnormalizedrecognitionscoresorconfidencemeasuresfortheindexingtermt(forspeech)N:totalnumberofdocumentsinthedatabaseNt:totalnumberofdocumentsinthedatabasewhichincludetheindexingtermtIDF:thesignificance(orimportance)orindexingpowerfortheindexingtermtTheOverallRelevanceScoreistheWeightedSumoftheRelevanceScoresforallTypesofIndexingFeaturesInverseDocumentFrequency(IDF)TermFrequency(TF)VectorSpaceModel

赛

德

克

•

巴

莱ㄙ

ㄉ

ㄎ

ㄅ

ㄌㄞ

ㄜ

ㄜ

ㄚ

ㄞ

DifficultiesinSpeech-basedInformationRetrievalforChineseLanguageEvenforText-basedInformationRetrieval,FlexibleWordingStructureMakesitDifficulttoSearchbyComparingtheCharacterStringsAlone

name/title李登辉→李前总统登辉，李前主席登辉(PresidentT.HLee)arbitraryabbreviation北二高→北部第二高速公路(SecondNorthernFreeway)

华航→中华航空公司(ChinaAirline)similarphrases 中华文化→中国文化(Chineseculture)translatedterms 巴塞隆那→巴瑟隆纳(Barcelona)WordSegmentationAmbiguityEvenforText-basedInformationRetrieval脑科(humanbrainstudies)→电脑科学(computerscience)土地公(Godofearth)→土地公有政策(policyofpublicsharingoftheland)UncertaintiesinSpeechRecognitionerrors(deletion,substitution,insertion)outofvocabulary(OOV)words,etc.veryoftenthekeyphrasesforretrievalareOOVAWholeClassofSyllable-LevelIndexingFeaturesforBetterDiscriminationOverlappingsyllablesegmentswithlengthNSyllablepairsseparatedbyMsyllablesCharacter-orWord-LevelFeaturescanbeSimilarlyDefinedSyllable-LevelIndexingFeaturesforChineseLanguageS(N),N=1

N=2

N=3P(M),M=1

M=2S1S2S3S4S5………S10SyllablePairSeparatedbyMsyllablesExamplesP(M),M=1(s1s3)(s2s4)……(s8s10)P(M),M=2(s1s4)(s2s5)……(s7s10)P(M),M=3(s1s5)(s2s6)……(s6s10)P(M),M=4(s1s6)(s2s7)……(s5s10)Syllable-LevelStatisticalFeaturesSingleSyllablesallwordsarecomposedbysyllables,thuspartiallyhandletheOOVproblemveryoftenrelevantwordshavesomesyllablesincommoneachsyllableusuallysharedbymorethanonecharacterswithdifferentmeanings,thuscausingambiguityOverlappingSyllableSegmentswithLengthNcapturingtheinformationofpolysyllabicwordsorphraseswithflexiblewordingstructuresmajorityofChinesewordsarebi-syllabicnottoomanypolysyllabicwordssharethesamepronunciationSyllablePairsSeparatedbyMSyllablestacklingtheproblemsarisingfromtheflexiblewordingstructure,abbreviations,anddeletion,insertion,substitutionerrorsinspeechrecognitionImprovedSyllable-levelIndexingFeaturesSyllable-alignedLatticesandsyllable-levelutteranceverificationIncludingmultiplesyllablehypothesistoconstructsyllable-alignedlatticesforbothqueryanddocumentsGeneratingmultiplesyllable-levelindexingfeaturesfromsyllablelatticesfilteringoutindexingtermswithloweracousticconfidencescoresInfrequenttermdeletion(ITD)Syllable-levelstatisticstrainedwithtextcorpususedtopruneinfrequentindexingtermsStopterms(ST)IndexingtermswiththelowestIDFscoresaretakenasthestoptermssyllableswithhigheracousticconfidencescoressyllableswithloweracousticconfidencescoressyllablepairsS(N),N=2prunedbyITDsyllablepairsS(N),N=2prunedbySTExpectedTermFrequenciesE(t,x):expectedtermfrequencyfortermtinthelatticeofanutterancexlatticeofutterancexL(x)uL(x):allthewordsequences(paths)inthelatticeofanutterancexN(t,u):theoccurrencecountoftermtinwordsequenceuu:awordsequence(path)inthelatticeofanutterancexP(u|x):posteriorprobabilityofthewordsequenceugivenxWFSTforRetrieval(1/4)FactorAutomataThefinitestatemachinesacceptingallsubstringsoftheoriginalmachineretrievalistohaveallsubstringsconsideredababaAcceptaAcceptabWFSTforRetrieval(2/4)TheindextransduceroftextdocumentEverysubstringofthedocumentistransducedintothecorrespondingdocumentID(e.g.,3014)Forspokendocuments,theindextransducersaregeneratedfromlatticesdirectlyTheindextransducerofthewholecorpusUnionofalltransducersofallutterancesWFSTforRetrieval(3/4)QueryTransducerSplitthequerystringinto

words,characters,syllables,etc.GeneratethequerytransducerFactorizetheautomatonDistributeweightsoverdifferenttransitionsEx:花莲县「花」「莲」Accept-2-2+6=2「花莲县」Accept-6+6=0

WFSTforRetrieval(4/4)QueryTransducer:花莲县花莲县:2033/0.7

莲:737/5.6CompositionDocument5034Document1Document2…

IndexTransducerUser

ImprovedRetrievalbyTrainingImprovetheretrievalwithsometrainingdataTrainingdata:asetofqueriesandassociatedrelevant/irrelevantutterancesCanbecollectedfromuserdatae.g.click-throughdataImprovetext-basedsearchenginee.g.learnweightsfordifferentclues(suchasdifferentrecognizers,differentsubwordunits…)OptimizetherecognitionmodelsforretrievalperformanceConsideringretrievalandrecognitionprocessesasawholeRe-estimateHMMparameters

time1:10Ttime2:01Ftime3:04Ttime5:31T

time1:10Ttime2:01Ftime3:04Ttime5:31F

time1:10Ftime2:01Ftime3:04Ttime5:31TQueryQ1QueryQ2QueryQn……RetrievalconsideredontopofrecognitionoutputinthepastrecognitionandretrievalastwocascadedstagesretrievalperformancerelyingonrecognitionaccuracyConsideringretrievalandrecognitionprocessesasawholeacousticmodelsre-estimatedbyoptimizingretrievalperformanceacousticmodelsbettermatchedtoeachrespectivedatasetHMMParameterRe-estimationSpokenArchiveRecognitionEngineAcousticModelslatticesSearchEngineRetrievalModeluserQueryQRetrievalOutputRecognitionRetrievalObjectiveFunctionforre-estimatingHMMHMMParameterRe-estimationFindnewHMMparametersforrecognitionsuchthattherelevancescoresofpositiveandnegativeexamplesarebetterseparated.xt,xf:positive/negativeexamplesforqueryQQtrain:trainingqueryset:relevancescoreofutterancexgivenqueryQandmodelparameterssetλ

(SinceS(Q,x)isobtainedfromlattice,itdependsonHMMparametersλ.)λ:setofHMMparameters,:re-estimatedparametersforretrievalReferencesWFSTforRetrievalCyrilAllauzen,MehryarMohri,andMuratSaraclar,“Generalindexationofweightedautomata:applicationtospokenutteranceretrieval,”inProceedingsoftheWorkshoponInterdisciplinaryApproachestoSpeechIndexingandRetrievalatHLT-NAACL,Stroudsburg,PA,USA,2004,SpeechIR’04,pp.33–40,AssociationforComputationalLinguistics.D.CanandM.Saraclar,“Latticeindexingforspokentermdetection,”IEEETransactionsonAudio,Speech,andLanguageProcessing,vol.19,no.8,pp.2338–2347,2011.SpokenContentinMandarinChinese“DiscriminatingCapabilitiesofSyllable-basedFeaturesandApproachesofUtilizingThemforVoiceRetrievalofSpeechInformationinMandarinChinese”,IEEETransactionsonSpeechandAudioProcessing,Vol.10,No.5,July2002,pp.303-314.TrainingRetrievalSystemsClick-throughdataThorstenJoachims.2002.Optimizingsearchenginesusingclickthroughdata.In

ProceedingsoftheeighthACMSIGKDDinternationalconferenceonKnowledgediscoveryanddatamining

(KDD'02)Improvetext-basedsearchengine“ImprovedLattice-basedSpokenDocumentRetrievalbyDirectlyLearningfromtheevaluationMeasures”,IEEEInternationalConferenceonAcoustics,SpeechandSignalProcessing,2009Re-estimateHMMparameters"IntegratingRecognitionandRetrievalWithRelevanceFeedbackforSpokenTermDetection,"

Audio,Speech,andLanguageProcessing,IEEETransactionson

,vol.20,no.7,pp.2095-2110,Sept.2012ReferencesPseudo-relevanceFeedback(PRF)(1/3)CollectingtrainingdatacanbeexpensivePseudo-relevancefeedback(PRF):GeneratetrainingdataautomaticallyProcedure:Generatefirst-passretrievalresultsassumethetopNobjectsonthefirst-passretrievalresultsarerelevant(pseudorelevant)assumethebottomMobjectsonthefirst-passretrievalresultsareirrelevant(pseudoirrelevant)Re-ranking:scoresofobjectssimilartothepseudo-relevant/irrelevantobjectsincreased/decreasedtime1:01time2:05time1:45…time2:16time7:22time9:01SearchEngineSpokenarchiveTopN“assumed”relevanttime1:01time2:05BottomN“assumed”irrelevanttime7:22time9:01QueryQComputeacousticsimilarityRe-rank:increase/decreasethescoreofutteranceshavinghigheracousticsimilaritywithpseudo-relevant/-irrelevantutterancestime1:01time2:16time7:22…time2:05time1:45time9:01FinalResultsRe-rankPseudo-relevanceFeedback(PRF)(2/3)(pseudo-relevant)(pseudo-irrelevant)First-passRetrievalResultsAcousticsimilaritybetweentwoutterancesxiandxjPseudo-relevanceFeedback(PRF)(3/3)CQAABBBlatticeforutterance

xiCAFQDElatticeforutterance

xjDynamicTimeWarping(DTW)similaritybetweenutterancexiandxjacousticfeaturesequencehypothesizedregionforqueryQhypothesizedregionforqueryQacousticfeaturesequenceImprovedPRF–Graph-basedApproach(1/4)Graph-basedapproachonlythetopN/bottomNutterancesaretakenasreferencesinPRFnotnecessarilyreliableconsideringtheacousticsimilaritystructureofallutterancesinthefirst-passretrievalresultsgloballyusingagraphConstructagraphforallutterancesinthefirst-passretrievalresultsnodes:utterancesedgeweights:acousticsimilaritiesbetweenutterancesFirst-passRetrievalResultsx1x3x2x4x5x3x1x2x5x4…..ImprovedPRF–Graph-basedApproach(2/4)Utterancesstronglyconnectedto(similarto)utteranceswithhighrelevancescoresshouldhaverelevancescoresincreased

?x1x3x2x4x5x3x1x2x5x4…..highhighhighfirst-passretrievalresultsImprovedPRF–Graph-basedApproach(3/4)?x1x3x2x4x5x3x1x2x5x4…..lowlowlowfirst-passretrievalresultsUtterancesstronglyconnectedto(similarto)utteranceswithlowrelevancescoresshouldhaverelevancescoresreduced

ImprovedPRF–Graph-basedApproach(3/4)Relevancescorespropagateonthegraphrelevancescoressmoothedamongstronglyconnectednodesx1x3x2x4x5x3x1x2x5x4…..x3x1x2x5x4…..first-passretrievalresultsRe-rankedImprovedPRF–Graph-basedApproach(4/4)PageRankandRandomWalk(1/2)ObjectrankingbytheirrelationsRankwebpagesforGooglesearchBasicIdeaObjectshavinghighconnectivitytootherhigh-scoreobjectsarepopular(givenhigherscores)v1v2v3v41/21/211/31/31/31/21/2fromtoTransitionmatrixPageRankandRandomWalk(2/2)

ScorepropagationPriorscorefinalscoreinterpolationweight

ForGraphandRandomwalkKurtBryan1,TanyaLeise,“The$25,000,000,000eigenvector:thelinearalgebrabehindgoogle”Amy.N.Langville,Carl.D.Meyer,“DeeperinsidePageRank”,InternetMathematics,Vol.1“ImprovedSpokenTermDetectionwithGraph-BasedRe-RankinginFeatureSpace”,in

ICASSP2011“Open-VocabularyRetrievalofSpokenContentwithShorter/LongerQueriesConsideringWord/Subword-basedAcousticFeatureSimilarity”,Interspeech,2012ReferencesSupportVectorMachine(SVM)(1/2)ABProblemdefinitionsupposetherearetwoclassesofobjects(positiveandnegative)goal:classifynewobjectsgiventrainingexamplesRepresenteachobjectasanN-dimensionalfeaturevectoro:positiveexamplex:negativeexampleFindahyperplaneseparatingpositiveandnegativeexamplesClassifynewobjectsbythishyperplanepointAispositive,pointBisnegativeSupportVectorMachine(SVM)(2/2)SupportvectorsMaximizedmarginManyhyperplanescanseparatepositiveandnegativeexamplesChoosetheonemaximizingthe“margin”margin:theminimumdistancebetweentheexamplesandthehyperplaneSomenoisemaychangethefeaturevectorsofthetestingobjectslargemarginmayminimizethechanceofmisclassificationSVM–SoftMarginHardMargin:Ifsometrainingexamplesareoutliers,separatingallpositive/negativeexamplesmaynotbethebestsolutionSoftMargin:Toleratesomenon-separablecases(outliers)HardMarginSoftMarginoutlierIgnoretheoutlierSVM–FeatureMappingA(1,1,1)B(1,1,-1)D(1,1,-1)C(1,1,1)Ifpositiveandnegativeexamplesarenotlinearlyseparableintheoriginalfeaturevectorform,maptheirfeaturevectorsontoahigher-dimensionalspacewheretheymaybecomeseparable(Canbeseparatedbyhyperplanez=xy=0)A(1,1)B(-1,1)D(1,-1)C(-1,-1)Originalfeaturevectors(Non-separable)Maporiginalfeaturevectorsontoahigher-dimensionalspaceTrainanSVMforeachquerytime1:01time2:05time1:45…time2:16time7:22time9:01time1:01time2:16time7:22…time2:05time1:45time9:01SVMRe-rankingFeatureExtractionSearchEngineSpokenarchiveFinalResultsFirst-passretrievalresultsNegativeexamplesPositiveexamplesFeatureExtractionImprovedPRF–SVM(1/3)TopN“assumed”relevanttime1:01time2:05BottomN“assumed”irrelevanttime7:22time9:01QueryQRepresentingeachutterancebyitshypothesizedregionsegmentedbyHMMstates,withfeaturevectorsineachstateaveragedandconcatenatedQABCBDBEFFHypothesizedRegionFeatureVectorSequence...…DStateBoundaries………afeaturevectoraveragedImprovedPRF–SVM(2/3)QQABCBDBEF0.20.30.20.50.20.20.30.40.20.1D0.1F0.9ABCD…Q0.20.60.50.3…0.4ABCD…Q0.00.30.00.0…0.0ABCD…Q0.20.00.50.0…0.0ImmediateleftcontextImmediaterightcontextwholesegmentConcatenatedintoa3V-dimensionalfeaturevectorV-dimensionalvector(V:lexiconsize)ImprovedPRF–SVM(3/3)Contextconsistencythesametermusuallyhavesimilarcontext;whilequitedifferentcontextusuallyimpliesthetermsaredifferentFeatureExtractionReferencesSVM/materials.html

(Lecturenotes3)"ATutorialonSupportVectorMachinesforPatternRecognition,"DataMiningandKnowledgeDiscovery,vol.2,no.2,pp.121-167,1998.Bishop,C.M.<http://library.wur.nl/WebQuery/clc?achternaam==Bishop>,"Patternrecognitionandmachinelearning."Chapter7.NelloCristianiniandJohnShawe-Taylor."AnIntroductiontoSupportVectorMachines:AndOtherKernel-BasedLearningMethods."SVMToolkit.tw/~cjlin/libsvm/LibSVM/SVMlightPseudo-relevanceFeedback(PRF)“ImprovedSpokenTermDetectionbyFeatureSpacePseudo-RelevanceFeedback”,AnnualConferenceoftheInternationalSpeechCommunicationAssociation,2010SVM-basedReranking“ImprovedSpokenTermDetectionUsingSupportVectorMachinesBasedonLatticeContextConsistency”,InternationalConferenceonAcoustics,SpeechandSignalProcessing,Prague,CzechRepublic,May2011,pp.5648-5651.“ImprovedSpokenTermDetectionUsingSupportVectorMachineswithAcousticandContextFeaturesFromPseudo-RelevanceFeedback”,IEEEWorkshoponAutomaticSpeechRecognitionandUnderstanding,Hawaii,Dec2011,pp.383-388.“EnhancedSpokenTermDetectionUsingSupportVectorMachinesandWeightedPseudoExamples”,IEEETransactionsonAudio,SpeechandLanguageProcessing,Vol.21,No.6,Jun2013,pp.1272-1284ReferencesLanguageModelingRetrievalApproach(TextorSpeech)BothqueryQandspokendocumentdarerepresentedaslanguagemodelsθQandθd(considerunigramonlybelow,maybesmoothed(orinterpolated)byabackgroundmodelθb)GivenqueryQ,rankspokendocumentsdaccordingtoSLM(Q,d)InverseofKLdivergence(KLdistance)betweenθQandθdThedocumentswithdocumentmodelsθdsimilartoquerymodelθQaremorelikelytoberelevantDocumentmodelN(t,Q):

OccurrencecountorexpectedtermfrequencyfortermtinqueryQQuerymodelN(t,d):OccurrencecountorexpectedtermfrequencyfortermtindocumentdE(t,x):Expectedtermfrequencyfortermtinthelatticeofutterancex(forspeech)SemanticRetrievalbyQueryExpansionConceptmatchingratherthanLiteralmatchingReturningutterances/documentssemanticallyrelatedtothequery(e.g.Obama)notnecessarilycontainingthequery(e.g.includingUSandWhiteHouse,butnotObama)Expandthequery(Obama)withsemanticallyrelatedterms(USandWhiteHouse)QueryexpansionwithlanguagemodelingretrievalapproachRealizedbyPRFFindcommontermdistributioninpseudo-relevantdocumentsanduseittoconstructanewqueryfor2nd-phaseretrievalRetrievalEnginedoc101doc205doc145……Documentmodelfordoc101w1w2w3w4w5……w1w2w3w4w5……TextQueryQTopNdocumentsaspseudo-relevantdocumentsDocumentmodelfordoc205w1w2w3w4w5QuerymodelArchiveofDocumentModel’sθdFirst-passRetrievalResultsSemanticRetrievalbyQueryExpansiondoc101doc205doc145……w1w2w3w4w5……w1w2w3w4w5……TextQueryQTopNdocumentsaspseudo-relevantdocumentsw1w2w3w4w5commonpatternsestimatedfromthepseudo-relevantdocumentmodelsandtheoriginalquerymodelNewQueryModelQuerymodelw1w2w3w4w5……RetrievalEngineArchiveofDocumentModel’sθdFirst-passRetrievalResultsSemanticRetrievalbyQueryExpansiondoc101doc205doc145……w1w2w3w4w5……w1w2w3w4w5……TextQueryQTopNdocumentsaspseudo-relevantdocumentsw1w2w3w4w5QuerymodelRetrievalEngineFinalResultw1w2w3w4w5……RetrievalEngineNewQueryModelArchiveofDocumentModel’sθdFirst-passRetrievalResultsSemanticRetrievalbyQueryExpansionSemanticRetrievalbyDocumentExpansionDocumentexpansionConsideradocumentonlyhastermsUSandWhiteHouseAddsomesemanticallyrelatedterms(Obama)intothedocumentmodelDocumentexpansionforlanguagemodelingretrievalapproachP(Ti|d):probabilityofobservingtopicTigivendocumentdP(t|Ti):probabilityofobservingtermtgiventopicTiObtainedbylatenttopicanalysis(e.g.PLSA)θd:originaldocumentmodelα:interpolationweightθd':expandeddocumentmodelLatentTopicAnalysisAnexample:ProbabilisticLatentSemanticAnalysis(PLSA)CreatingasetoflatenttopicsbetweenasetoftermsandasetofdocumentsmodelingtherelationshipsbyprobabilisticmodelstrainedwithEMalgorithmOtherwell-knownapproaches:LatentSemanticAnalysis(LSA),Non-negativeMatrixFactorization(NMF),LatentDirichletAllocation(LDA)……SemanticRetrievalofSpokenContent“ImprovedSemanticRetrievalofSpokenContentbyLanguagemodelsEnhancedwithAcousticSimilarityGraph”,IEEEWorkshoponSpokenLanguageTechnology,2012T.K.Chia,K.C.Sim,H.Li,andH.T.Ng,“Statisticallattice-basedspokendocumentretrieval,”ACMTrans.Inf.Syst.,vol.28,pp.2:1–2:30,2010.ReferencesSearchspeechbyspeech–noneedtoknowwhichwordisspokenNorecognition,withoutannotateddata,withoutknowledgeaboutthelanguageBypassthedifficultiesofrecognition:annotateddataforthetargetdomain,OOVwords,recognitionerrors,noiseconditions,etc.UnsupervisedSpokenTermDetection(STD)withSpokenQueriesTwomajorapproachesforUnsupervisedSTDTemplatematching(signal-to-signalmatching)DynamicTimeWarping(DTW)based,matchingthesignalsdirectlyPrecisebutlesscompatibletosignalvariations(bydifferentspeakers,differentacousticconditions,etc.)withhighercomputationrequirementsModel-basedapproachwithautomaticallydiscoveredpatternsRepresentingsignalsbymodelsandmatchingwiththesemodelsDiscoveringacousticpatternsandtrainingcorrespondingmodelswithoutannotateddataTemplateMatchingDynami