版权说明:本文档由用户提供并上传,收益归属内容提供方,若内容存在侵权,请进行举报或认领
文档简介
【5A文】迁移学习算法研究【5A文】迁移学习算法研究TrainingDataClassifierUnseenData(…,long,T)good!Whatif…2022/12/302传统监督机器学习(1/2)[fromProf.QiangYang]TrainingClassifierUnseenData(传统监督机器学习(2/2)2022/12/303传统监督学习同源、独立同分布两个基本假设标注足够多的训练样本在实际应用中通常不能满足!训练集测试集分类器训练集测试集分类器传统监督机器学习(2/2)2022/12/303传统监督学习迁移学习2022/12/304实际应用学习场景HP新闻Lenovo新闻不同源、分布不一致人工标记训练样本,费时耗力迁移学习运用已有的知识对不同但相关领域问题进行求解的一种新的机器学习方法放宽了传统机器学习的两个基本假设迁移学习2022/12/304实际应用学习场景HP新闻Le迁移学习场景(1/4)2022/12/305迁移学习场景无处不在迁移知识迁移知识图像分类HP新闻Lenovo新闻新闻网页分类迁移学习场景(1/4)2022/12/305迁移学习场景无处迁移学习场景(2/4)异构特征空间2022/12/306Theappleisthepomaceousfruitoftheappletree,speciesMalusdomesticaintherosefamilyRosaceae...BananaisthecommonnameforatypeoffruitandalsotheherbaceousplantsofthegenusMusawhichproducethiscommonlyeatenfruit...Training:TextFuture:ImagesApplesBananas[fromProf.QiangYang]XinJin,FuzhenZhuang,SinnoJialinPan,ChangyingDu,PingLuo,QingHe:HeterogeneousMulti-taskSemanticFeatureLearningforClassification.CIKM2015:1847-1850.迁移学习场景(2/4)异构特征空间2022/12/306ThTestTestTrainingTrainingClassifierClassifier72.65%DVDElectronicsElectronics84.60%ElectronicsDrop!迁移学习场景(3/4)2022/12/307[fromProf.QiangYang]TestTestTrain迁移学习场景(4/4)2022/12/308DVDElectronicsBookKitchenClothesVideogameFruitHotelTeaImpractical![fromProf.QiangYang]迁移学习场景(4/4)2022/12/308DVDElectOutlineConceptLearningforTransferLearningConceptLearningbasedonNon-negativeMatrixTri-factorizationforTransferLearningConceptLearningbasedonProbabilisticLatentSemanticAnalysisforTransferLearningTransferLearningusingAuto-encodersTransferLearningfromMultipleSourceswithAutoencoderRegularizationSupervisedRepresentationLearning:TransferLearningwithDeepAuto-encoders2022/12/309OutlineConceptLearningforTrConceptLearningbasedonNon-negativeMatrixTri-factorizationforTransferLearning2022/12/30ConceptLearningforTransferLearning10ConceptLearningbasedonNon-Introduction2022/12/30ConceptLearningforTransferLearning11Manytraditionallearningtechniquesworkwellonlyundertheassumption:Trainingandtestdatafollowthesamedistribution
Training(labeled)ClassifierTest(unlabeled)FromdifferentcompaniesEnterpriseNewsClassification:includingtheclasses“ProductAnnouncement”,“Businessscandal”,“Acquisition”,……Productannouncement:HP'sjust-releasedLaserJetProP1100printerandtheLaserJetProM1130andM1210multifunctionprinters,price…performance
...AnnouncementforLenovoThinkPad
ThinkCentre–price$150offLenovoK300desktopusingcouponcode...LenovoThinkPad
ThinkCentre–price$200offLenovoIdeaPadU450plaptopusing....theirperformanceHPnewsLenovonewsDifferentdistributionFail!Introduction2022/12/30ConceptMotivation(1/3)2022/12/30ConceptLearningforTransferLearning12ExampleAnalysis
Productannouncement:HP'sjust-releasedLaserJetProP1100printerandtheLaserJetProM1130andM1210multifunctionprinters,price…performance
...AnnouncementforLenovoThinkPad
ThinkCentre–price$150offLenovoK300desktopusingcouponcode...LenovoThinkPad
ThinkCentre–price$200offLenovoIdeaPadU450plaptopusing....theirperformanceHPnewsLenovonewsProductwordconceptLaserJet,printer,price,performanceThinkPad,ThinkCentre,price,performanceRelatedProductannouncementdocumentclass:Sharesomecommonwords:announcement,price,performance…indicateMotivation(1/3)2022/12/30ConcMotivation(2/3)2022/12/30ConceptLearningforTransferLearning13ExampleAnalysis:
HPLaserJet,printer,price,performanceetal.LenovoThinkpad,Thinkcentre,price,performanceetal.Thewordsexpressingthesamewordconceptaredomain-dependent
ProductProductannouncementwordconceptindicatesTheassociationbetweenwordconceptsanddocumentclassesisdomain-independent
Motivation(2/3)2022/12/30ConcMotivation(3/3)2022/12/30ConceptLearningforTransferLearning14Furtherobservations:Differentdomainsmayusesamekeywordstoexpressthesameconcept(denotedasidenticalconcept)Differentdomainsmayalsousedifferentkeywordstoexpressthesameconcept(denotedasalikeconcept)Differentdomainsmayalsohavetheirowndistinctconcepts(denotedasdistinctconcept)TheidenticalandalikeconceptsareusedasthesharedconceptsforknowledgetransferWetrytomodelthesethreekindsofconceptssimultaneouslyfortransferlearningtextclassificationMotivation(3/3)2022/12/30ConcPreliminaryKnowledge2022/12/30ConceptLearningforTransferLearning15Basicformulaofmatrixtri-factorization:wheretheinputXistheword-documentco-occurrencematrix
denotesconceptinformation,mayvaryindifferentdomainsFdenotesthedocumentclassificationinformation
indeedistheassociationbetweenwordconceptsanddocumentclasses,mayretainstablecrossdomainsGSPreliminaryKnowledge2022/12/3Previousmethod-MTrickinSDM2010(1/2)2022/12/30ConceptLearningforTransferLearning16SketchmapofMTrick
SourcedomainXs
FsGsFtGtTargetdomainXtSKnowledgeTransferConsideringthealikeconcepts Previousmethod-MTrickinSDMTrick(2/2)OptimizationproblemforMTrick2022/12/30ConceptLearningforTransferLearning17G0isthesupervisioninformationtheassociationSissharedasbridgetotransferknowledgeDualTransferLearning(Longetal.,SDM2012),consideringidenticalandalikeconceptsMTrick(2/2)OptimizationproblTriplexTransferLearning(TriTL)(1/5)2022/12/30ConceptLearningforTransferLearning18Furtherdividethewordconceptsintothreekinds:
F1,identicalconcepts;F2,alikeconcepts;F3,distinctconceptsInput:ssourcedomainXr(1≤r≤s)withlabelinformation,ttargetdomainXr(s+1≤r≤s+t)WeproposeTriplexTransferLearningframeworkbasedonmatrixtri-factorization(TriTLforshort)
TriplexTransferLearning(TriF1,S1andS2
aresharedasthebridgeforknowledgetransferacrossdomainsThesupervisioninformationisintegratedbyGr(1≤r≤s)insourcedomainsTriTL(2/5)OptimizationProblem
2022/12/30ConceptLearningforTransferLearning19F1,S1andS2aresharedasthTriTL(3/5)Wedevelopanalternativelyiterativealgorithmtoderivethesolutionandtheoreticallyanalyzeitsconvergence 2022/12/30ConceptLearningforTransferLearning20TriTL(3/5)WedevelopanalterTriTL(4/5)Classificationontargetdomains When1≤r≤s,Grcontainsthelabelinformation,soweremainitunchangedduringtheiterationswhenxibelongstoclassj,thenGr(i,j)=1,elseGr(i,j)=0Aftertheiteration,weobtaintheoutputGr(s+1≤r≤s+t),thenwecanperformclassificationaccordingtoGr2022/12/30ConceptLearningforTransferLearning21TriTL(4/5)ClassificationontTriTL(5/5)AnalysisofAlgorithmConvergence Accordingtothemethodologyofconvergenceanalysisinthetwoworks[Leeetal.,NIPS’01]and[Dingetal.,KDD’06],thefollowingtheoremholds.Theorem(Convergence):Aftereachroundofcalculatingtheiterativeformulas,theobjectivefunctionintheoptimizationproblemwillconvergemonotonically.2022/12/30ConceptLearningforTransferLearning22TriTL(5/5)AnalysisofAlgorit2022/12/30ConceptLearningforTransferLearning23rec.autosrec.motorcyclesrec.baseballrec.hockeysci.cryptsic.electronicssci.medsci.spacecomp.graphicscomp.sys.ibm.pc.hardwarecomp.sys.mac.hardwarecomp.windows.xtalk.politics.misctalk.politics.gunstalk.politics.mideasttalk.religion.miscrecscicomptalkDataPreparation(1/3)20Newsgroups Fourtopcategories,eachtopcategorycontainsfoursub-categories SentimentClassification,fourdomains:books,dvd,electronics,kitchenRandomlyselecttwodomainsassources,andtherestastargets,then6problemscanbeconstructed
2022/12/30ConceptLearningfor2022/12/30ConceptLearningforTransferLearning24rec.autosrec.motorcyclesrec.baseballrec.hockeysci.cryptsic.electronicssci.medsci.spacerec+sci-baseballcrypySourcedomainautosspaceTargetdomainFortheclassificationproblemwithonesourcedomainandonetargetdomain,wecanconstruct144()
problemsDataPreparation(2/3)Constructclassificationtasks(TraditionalTL)2022/12/30ConceptLearningfor2022/12/30ConceptLearningforTransferLearning25Constructnewtransferlearningproblemsrec.autosrec.motorcyclesrec.baseballrec.hockeysci.cryptsic.electronicssci.medsci.spacerec+sci-baseballcrypyautosspacecomp.graphicscomp.sys.ibm.pc.hardwarecomp.sys.mac.hardwarecomp.windows.xtalk.politics.misctalk.politics.gunstalk.politics.mideasttalk.religion.misccomptalkautosgraphicsMoredistinctconceptsmayexist!DataPreparation(3/3)SourcedomainTargetdomain2022/12/30ConceptLearningfor2022/12/30ConceptLearningforTransferLearning26ComparedAlgorithmsTraditionallearningAlgorithmsSupervisedLearning:LogisticRegression(LR)[Davidetal.,00]SupportVectorMachine(SVM)[Joachims,ICML’99]Semi-supervisedLearning:TSVM[Joachims,ICML’99]TransferlearningMethods:CoCC[Daietal.,KDD’07],DTL[Longetal.,SDM’12]Classificationaccuracyisusedastheevaluationmeasure2022/12/30ConceptLearningfor2022/12/30ConceptLearningforTransferLearning27ExperimentalResults(1/3)SorttheproblemswiththeaccuracyofLRDegreeoftransferdifficultyeasierGenerally,thelowerofaccuracyofLRcanindicatethehardertotransfer,whilethehigheronesindicatetheeasiertotransferharder2022/12/30ConceptLearningfor2022/12/30ConceptLearningforTransferLearning28ExperimentalResults(2/3)ComparisonsamongTriTL,DTL,MTrick,CoCC,TSVM,SVMandLRondatasetrecvs.sci(144problems)TriTLcanperformwelleventheaccuracyofLRislowerthan65%2022/12/30ConceptLearningfor2022/12/30ConceptLearningforTransferLearning29ExperimentalResults(3/3)Resultsonnewtransferlearningproblems,weonlyselecttheproblems,whoseaccuraciesofLRarebetween(50%,55%](Onlyslightlybetterthanrandomclassification,thustheymightbemuchmoredifficult).Weobtain65problemsTriTLalsooutperformsallthebaselines2022/12/30ConceptLearningforConclusions2022/12/30ConceptLearningforTransferLearning30Explicitlydefinethreekindsofwordconcepts,i.e.,identicalconcept,alikeconceptanddistinctconceptProposeageneraltransferlearningframeworkbasedonnonnegativematrixtri-factorization,whichsimultaneouslymodelthethreekindsofconcepts(TriTL)Extensiveexperimentsshowtheeffectivenessoftheproposedapproach,especiallywhenthedistinctconceptsmayexistConclusions2022/12/30ConceptLConceptLearningbasedonProbabilisticLatentSemanticAnalysisforTransferLearning2022/12/30ConceptLearningforTransferLearning31ConceptLearningbasedonProb2022/12/30ConceptLearningforTransferLearning32MotivationProductannouncement:HP'sjust-releasedLaserJetProP1100printerandtheLaserJetProM1130andM1210multifunctionprinters,price…performance
...AnnouncementforLenovoThinkPad
ThinkCentre–price$150offLenovoK300desktopusingcouponcode...LenovoThinkPad
ThinkCentre–price$200offLenovoIdeaPadU450plaptopusing....theirperformanceHPnewsLenovonewsProductwordconceptLaserJet,printer,price,performanceThinkPad,ThinkCentre,price,performanceRelatedProductannouncementdocumentclass:Sharesomecommonwords:announcement,price,performance…indicateRetrospecttheexample
2022/12/30ConceptLearningfor2022/12/30ConceptLearningforTransferLearning33SomenotationsddocumentydocumentclasszwordconceptSomedefinitionse.g.,p(price|Product),p(LaserJet|Product,)wwordrdomaine.g,p(Product|Productannouncement)PreliminaryKnowledge(1/3)2022/12/30ConceptLearningfor2022/12/30ConceptLearningforTransferLearning34PreliminaryKnowledge(2/3)ProductLaserJet,printer,announcement,price,ThinkPad,ThinkCentre,announcement,priceProductannouncementp(w|z,r1)p(w|z,r2)p(z|y)p(w|z,r1)≠p(w|z,r2)E.g.,p(LaserJet|Product,HP)≠p(LaserJet|Product,Lenovo)p(z|y,r1)=p(z|y,r2)E.g.,p(Product|Productannoucement,HP)=p(Product|Productannoucement,Lenovo)Alikeconcept2022/12/30ConceptLearningfor2022/12/30ConceptLearningforTransferLearning35DualPLSA
(D-PLSA)Jointprobabilityoverallvariablesp(w,d)=p(w|z)p(z|y)p(d|y)p(y)GivendatadomainX,theproblemofmaximumloglikelihoodislogp(X;θ)=logΣz
p(Z,X;θ)
θ
includesalltheparametersp(w|z),p(z|y),p(d|y),p(y).Z
denotesallthelatentvariablesPreliminaryKnowledge(3/3)TheproposedtransferlearningalgorithmbasedonD-PLSA,denotedasHIDC2022/12/30ConceptLearningfor2022/12/30ConceptLearningforTransferLearning36Identicalconceptp(w|za)p(za|y)AlikeconceptTheextensionandintensionaredomainindependentp(w|zb,r)p(zb|y)HIDC(1/3)Theextensionisdomaindependent,whiletheintensionisdomainindependent2022/12/30ConceptLearningfor2022/12/30ConceptLearningforTransferLearning37Distinctconceptp(w|zc,r)p(zc|y,r)ThejointprobabilitiesofthesethreegraphicalmodelsHIDC(2/3)Theextensionandintensionarebothdomaindependent2022/12/30ConceptLearningfor2022/12/30ConceptLearningforTransferLearning38Givens+t
datadomainsX={X1,…,Xs,Xs+1,…,Xs+t},withoutlossofgenerality,thefirstsdomainsaresourcedomains,andthelefttdomainsaretargetdomainsConsiderthethreekindsofconcepts:TheLog
likelihoodfunctionislogp(X;θ)=logΣz
p(Z,X;θ)
θ
includesallparametersp(w|za),p(w|zb,r),p(w|zc,r),p(za|y),p(zb|y),p(zc|y,r),p(d|y,r),p(y|r),p(r).HIDC(3/3)2022/12/30ConceptLearningfor2022/12/30ConceptLearningforTransferLearning39UsetheEMalgorithmtoderivethesolutionsEStep:ModelSolution(1/4)2022/12/30ConceptLearningfor2022/12/30ConceptLearningforTransferLearning40M
Step:ModelSolution(2/4)2022/12/30ConceptLearningfor2022/12/30ConceptLearningforTransferLearning41Semi-supervisedEMalgorithm:whenrisfromsourcedomains,thelabeledinformationp(d|y,r)isknownandp(y|r)
canbeinferedp(d|y,r)=1/ny,r,ifdbelongsyindomainr,ny,risthenumberofdocumentsinclassyindomainr,else
p(d|y,c)=0p(y|r)=ny,r/nr
,nr
isthenumberofdocumentsindomainr
whenrisfromsourcedomains,p(d|y,r)andp(y|r)keepunchangedduringtheiterations,whichsupervisetheoptimizingprocessModelSolution(3/4)2022/12/30ConceptLearningfor2022/12/30ConceptLearningforTransferLearning42ClassificationfortargetdomainsAfterweobtainthefinalsolutionsofp(w|za),p(w|zb,r),p(w|zc,r),p(za|y),p(zb|y),p(zc|y,r),p(d|y,r),p(y|r),p(r)Wecancomputetheconditionalprobabilities:
ThenthefinalpredictionisDuringtheiterations,alldomainssharep(w|za),p(za|y),p(zb|y),
whichactasthebridgeforknowledgetransferModelSolution(4/4)2022/12/30ConceptLearningfor2022/12/30ConceptLearningforTransferLearning43BaselinesComparedAlgorithmsSupervisedLearning:LogisticRegression(LG)[Davidetal.,00]SupportVectorMachine(SVM)[Joachims,ICML’99]Semi-supervisedLearning:TSVM[Joachims,ICML’99]TransferLearning:CoCC[Daietal.,KDD’07]CD-PLSA[Zhuangetal.,CIKM’10]DTL[Longetal.,SDM’12]OurMethodsHIDCMeasure:classificationaccuracy2022/12/30ConceptLearningfor2022/12/30ConceptLearningforTransferLearning44Resultsonnewtransferlearningproblems,weselecttheproblems,whoseaccuraciesofLRarehigherthan50%,then334problemsareobtainedExperimentalResults(1/5)2022/12/30ConceptLearningfor2022/12/30ConceptLearningforTransferLearning45Resultsonnewtransferlearningproblems,weselecttheproblems,whoseaccuraciesofLRarehigherthan50%,then334problemsareobtainedExperimentalResults(2/5)2022/12/30ConceptLearningfor2022/12/30ConceptLearningforTransferLearning46ExperimentalResults(3/5)2022/12/30ConceptLearningfor2022/12/30ConceptLearningforTransferLearning47Sourcedomain:S
(rec.autos,
sci.space),Targetdomain:T(rec.sport.hockey,talk.politics.mideast)STSTDistinctconceptSTAlikeconceptExperimentalResults(4/5)2022/12/30ConceptLearningfor2022/12/30ConceptLearningforTransferLearning48ExperimentalResults(5/5)Indeed,theproposedprobabilisticmethodHIDCisalsobetterthanTriTLThismayduetothereasonthatthereismoreclearerprobabilisticexplanationofHIDCp1(z,y)=p2(z,y)orp1(z|y)=p2(z|y)whichisbetter?p(z|y)p(y)2022/12/30ConceptLearningfor2022/12/30ConceptLearningforTransferLearning49[1]FuzhenZhuang,PingLuo,HuiXiong,QingHe,YuhongXiong,ZhongzhiShi:ExploitingAssociationsbetweenWordClustersandDocumentClassesforCross-DomainTextCategorization.SDM2010,pp.13-24.[2]FuzhenZhuang,PingLuo,ZhiyongShen,QingHe,YuhongXiong,ZhongzhiShi,HuiXiong:CollaborativeDual-PLSA:miningdistinctionandcommonalityacrossmultipledomainsfortextclassification.CIKM2010,pp.359-368.[3]FuzhenZhuang,PingLuo,ZhiyongShen,QingHe,YuhongXiong,ZhongzhiShi,HuiXiong:MiningDistinctionandCommonalityacrossMultipleDomainsUsingGenerativeModelforTextClassification.IEEETrans.Knowl.DataEng.24(11):2025-2039(2012).[3]FuzhenZhuang,PingLuo,ChangyingDu,QingHe,ZhongzhiShi:Triplextransferlearning:exploitingbothsharedanddistinctconceptsfortextclassification.WSDM2013,pp.425-434.[4]FuzhenZhuang,PingLuo,PeifengYin,QingHe,ZhongzhiShi.:ConceptLearningforCross-domainTextClassification:aGeneralProbabilisticFramework.IJCAI2013,pp.1960-1966.References2022/12/30ConceptLearningforOutlineConceptLearningforTransferLearningConceptLearningbasedonNon-negativeMatrixTri-factorizationforTransferLearningConceptLearningbasedonProbabilisticLatentSemanticAnalysisforTransferLearningTransferLearningusingAuto-encodersTransferLearningfromMultipleSourceswithAutoencoderRegularizationSupervisedRepresentationLearning:TransferLearningwithDeepAuto-encoders2022/12/3050OutlineConceptLearningforTrTransferLearningfromMultipleSourceswithAutoencoderRegularization2022/12/30TransferLearningUsingAuto-encoders51TransferLearningfromMultipl2022/12/3052Motivation(1/2)TransferlearningbasedonoriginalfeaturespacemayfailtoachievehighperformanceonTargetdomaindataWeconsidertheautoencodertechniquetocollaborativelyfindanewrepresentationofbothsourceandtargetdomaindataElectronicsVideoGames
Compact;easytooperate;verygoodpicture,excited
aboutthequality;lookssharp!Averygood
game!Itisactionpacked
andfullofexcitement.Iamverymuchhooked
onthisgame.52TransferLearningUsingAuto-encoders2022/12/3052Motivation(1/2)TraPreviousmethodsoftentransferfromonesourcedomaintoonetargetdomainWeconsidertheconsensusregularizedframeworkforlearningfrommultiplesourcedomainsDVDBookKitchenElectronicsWeproposeatransferlearningframeworkofconsensusregularizationautoencoderstolearnfrommultiplesourcesMotivation(2/2)2022/12/30TransferLearningUsingAuto-encoders53PreviousmethodsoftentransfeAutoencoderNeuralNetworkMinimizingthereconstructionerrortoderivethesolution:whereh,garenonlinearactivationfunction,e.g.,Sigmoidfunction,forencodinganddecoding2022/12/30TransferLearningUsingAuto-encoders54AutoencoderNeuralNetworkMConsensusMeasure-(1/3)Example:three-classclassificationproblem,threeclassifierspredictinstancesf1f2f3f1f2f3x1111x2333x3222x4231x5313x61232022/12/30TransferLearningUsingAuto-encoders55ConstraintSource1:D1Source2:D2Source3:D3ConsensusMeasure-(1/3)ExamConsensusMeasure-(2/3)Example:three-classclassificationproblem,predictiononinstancex2022/12/30TransferLearningUsingAuto-encoders56Minimalentropy,MaximalConsensusMaximalentropy,MinimalConsensusEntropybasedConsensusMeasure(Luoetal.,CIKM’08)θiistheparametervectorofclassifieri,CistheclasslabelsetConsensusMeasure-(2/3)ExamConsensusMeasure-(3/3)Forsimplicity,theconsensusmeasureforbinaryclassificationcanberewrittenasInthiswork,weimposetheconsensusregularizationtoautoencoders,andtrytoimprovethelearningperformancefrommultiplesourcedomainssincetheireffectsonmakingthepredictionconsensusaresimilar.2022/12/30TransferLearningUsingAuto-encoders57ConsensusMeasure-(3/3)ForSomeNotationsSourcedomainsGivenrsourcedomains:,i.e.,
,.ThefirstcorrespondingdatamatrixisTargetdomainThecorrespondingdatamatrixisThegoalistotrainaclassifier
ftomakeprecisepredictionson.2022/12/30TransferLearningUsingAuto-encoders58SomeNotationsSourcedomaiFrameworkofCRAThedatafromallsourceandtargetdomainssharethesameencodinganddecodingweightsTheclassifierstrainedfromthenewrepresentationareregularizedtopredictthesameresultsontargetdomaindata2022/12/30TransferLearningUsingAuto-encoders59FrameworkofCRAThedatafrOptimizationProblemofCRATheoptimizationproblem:ReconstructionError2022/12/30TransferLearningUsingAuto-encoders60OptimizationProblemofCRAOptimizationProblemofCRATheoptimizationproblem:ConsensusRegularization2022/12/30TransferLearningUsingAuto-encoders61OptimizationProblemofCRAOptimizationProblemofCRATheoptimizationproblem:ThetotallossofsourceclassifiersoverthecorrespondingsourcedomaindatawiththehiddenrepresentationWeighdecayterm2022/12/30TransferLearningUsingAuto-encoders62OptimizationProblemofCRATheSolutionofCRAWeusethegradientdescentmethodtoderivethesolutionofallparametersƞisthelearningrate.ThetimecomplexityisO(rnmk)Theoutput:theencodinganddecodingparameters,andsourceclassifierswithlatentrepresentation.2022/12/30TransferLearningUsingAuto-encoders63TheSolutionofCRAWeusetheTargetClassifierConstructionTwoScheme:Trainthesourceclassifiersbasedonandcombinethemas,whereCombineallthesourcedomaindataasZSandtrainaunifiedclassifierusinganysupervisedlearningalgorithms,e.g.,SVM,LogisticRegression(LR).ThetwoaccuraciesaredenotedasCRAvandCRAu,respectively2022/12/30TransferLearningUsingAuto-encoders64TargetClassifierConstructionDataSets-(1/2)ImageData(fromLuoetal.,CIKM08)(Someexamples)AB
A1A2A3A4B1B2B3B4Threesources:A1B1A2B2A3B3Targetdomain:A4B4Totally,96()3-sourcevs1-targetdomain(3vs1)probleminstancescanbeconstructedfortheexperimentalevaluation2022/12/30TransferLearningUsingAuto-encoders65DataSets-(1/2)ImageData(DataSets-(2/2)SentimentClassification(fromBlitzeretal.,ACL07)Four3-sourcevs1-targetdomainclassificationproblemsareconstructedDVDBookKitchenElectronicsTheaccuracyontargetdomaindataisusedastheevaluationmeasureBothSVMandLRareusedtotrainclassifiersonthenewrepresentation2022/12/30TransferLearningUsingAuto-encoders66DataSets-(2/2)SentimentClassAllComparedAlgorithmsBaselinesSupervisedlearningonoriginalfeatures:SVM[Joachims,ICML’99],LogisticRegression(LR)[Davidetal.,00]Embeddingmethodbasedonautoencoders(EAER)[Yuetal.,ECML’13]MarginalizedStackedDenoisingAutoencoders
(mSDA)[Chenetal.,ICML’12]TransferComponentAnalysis(TCA)[Panetal.,TNN’11]Transferlearningfrommultiplesources(CCR3)(Luoetal.,CIKM’08)Ourmethod:CRAvandCRAuForthemethodswhichcannothandlemultiplesources,wetraintheclassifiersfromeachsourcedomainandmergeddataofallsources(r+1accuracies).Finally,maximal,meanandminimalvaluesarereported.2022/12/30TransferLearningUsingAuto-encoders67AllComparedAlgorithmsBaselinTransferLearningwithMultipleSourcesviaConsensusRegularizationAutoencodersFuzhenZhuang,XiaohuCheng,SinnoJialinPan,WenchaoYu,QingHe,andZhongzhiShi68ExperimentalResults-(1/2)Resultson96imageclassificationproblemsTransferLearningwithMultiplTransferLearningwithMultipleSourcesviaConsensusRegularizationAutoencodersFuzhenZhuang,XiaohuCheng,SinnoJialinPan,WenchaoYu,QingHe,andZhongzhiShi69ExperimentalResults-(2/2)Resultson4sentimentclassificationproblemsTransferLearningwithMultiplConclusionsThewellknownrepresentationlearningtechniqueautoencoderisconsidered,andweformalizetheautoencodersandconsensusregularizationintoaunifiedoptimizationframeworkExtensivecomparisonexperimentsonimageandsentimentdataareconductedtoshowtheeffectivenessoftheproposealgorithm2022/12/30TransferLearningUsingAuto-encoders70ConclusionsThewellknownrSupervisedRepresentationLearning:TransferLearningwithDeepAuto-encoders2022/12/30TransferLearningUsingAuto-encoders71SupervisedRepresentationLearAutoencoderisanunsupervisedfeaturelearningalgorithm,whichcannoteffectivelymakeuseofthelabelinformationLimitationofBasicAutoencoderContributionofThisWorkWeextendAutoencodertomulti-layerstructure,andincorporatethelabelasonelayerMotivation2022/12/30TransferLearnin
温馨提示
- 1. 本站所有资源如无特殊说明,都需要本地电脑安装OFFICE2007和PDF阅读器。图纸软件为CAD,CAXA,PROE,UG,SolidWorks等.压缩文件请下载最新的WinRAR软件解压。
- 2. 本站的文档不包含任何第三方提供的附件图纸等,如果需要附件,请联系上传者。文件的所有权益归上传用户所有。
- 3. 本站RAR压缩包中若带图纸,网页内容里面会有图纸预览,若没有图纸预览就没有图纸。
- 4. 未经权益所有人同意不得将文件中的内容挪作商业或盈利用途。
- 5. 人人文库网仅提供信息存储空间,仅对用户上传内容的表现方式做保护处理,对用户上传分享的文档内容本身不做任何修改或编辑,并不能对任何下载内容负责。
- 6. 下载文件中如有侵权或不适当内容,请与我们联系,我们立即纠正。
- 7. 本站不保证下载资源的准确性、安全性和完整性, 同时也不承担用户因使用这些下载资源对自己和他人造成任何形式的伤害或损失。
最新文档
- 婴幼儿常见营养性疾病-维生素A缺乏症(婴幼儿喂养课件)
- 市场与行业环境分析
- 人教版六年级下册数学期末测试卷加答案(必刷)
- 市场营销学案例分析耐克
- 电子产品品牌设计
- 电子专业毕业设计
- 视觉传达系创业教育实施方案初稿样本
- 家用电器造型设计
- 工业设计转型发展趋势分析
- 幼儿园老师礼仪培训
- 乡村温馨校园建设典型经验申报材料案例
- 【护理管理】提高VTE高危患者预防措施患者执行率
- 钢结构管廊施工组织设计
- 新人教版八年级上册地理 4.2.2 发展农业要因地制宜走科技强农之路 重点习题练习复习课件
- 【台账】有限空间作业场所安全管理台账参考模板范本
- 三下健康成长教案
- 管道沟槽开挖安全安全技术交底
- 投标文件-项目管理实施方案
- 动物营养学水的营养
- 新教材人教版高中物理选择性必修第三册全册教学课件
- 工作总结思维导图
评论
0/150
提交评论