一种基于生成对抗网络的火焰图像场景迁移新方法

一种基于生成对抗网络的火焰图像场景迁移新方法Abstract:Withthedevelopmentofdeeplearningtechnology,imagestyletransferhasbecomeanimportantresearcharea.Flameimagescenetransferhasgreatresearchsignificanceandpracticalvalue.Inthispaperweproposeanewmethodbasedongenerativeadversarialnetwork(GAN)forflameimagescenetransfer.Thismethodconsistsofamultiscalegeneratorandadiscriminator.Thegeneratorisdesignedwithskipconnectionarchitecturetotransfertheflameimagestyletothetargetimagewhilepreservingthestructuraldetails.Thediscriminatorisusedtodistinguishthegeneratedimagefromtherealimageandguidethegeneratortoproducemorerealisticimages.Experimentsshowthatourmethodcaneffectivelytransfertheflameimagestyletothetargetimageandachieveagoodvisualeffect.nscenetransferistheprocessoftransferringtheflameimagestyletoatargetimagewhilemaintainingthestructuraldetailsofthetargetimageIthaswideapplicationsinmoviespecialeffects,videogamesandvirtualreality.WiththedevelopmentofdeeplearningtechnologyimagestyletransferhasbecomeanimportantresearchareaTraditionalmethodsofimagestyletransfermainlyrelyontexturesynthesiswhichistime-consumingandcomputationallyexpensive.Inrecentyearsgenerativeadversarialnetworks(GANs)haveemergedasapromisingmethodforimagestyletransfer.InthispaperweproposeanewmethodbasedonGANforflameimagescenetransferOurmethodisdesignedwithamulti-scalegeneratorandadiscriminatorwhichcaneffectivelytransfertheflameimagestyletothetargetimageandachieveagoodvisualeffect.agestyletransferhasbeenextensivelystudiedinrecentyearsGatysetalproposedaneuralalgorithmforimagestyletransfer,whichusestheGrammatrixoffeaturemapstorepresenttheimagestyleandoptimizethecontentandstyleseparately.However,thismethodiscomputationallyexpensiveandcannothandlelargeimages.Johnsonetalproposedafastneuralstyletransfermethod,whichusesapre-traineddeepconvolutionalneuralnetworktotransferthestyleofanimagetootherimages.However,thismethodisalsolimitedbythesizeoftheinputimageTosolvetheseproblems,severalGAN-basedmethodshavebeenproposedforimagestyletransfer.GANsconsistofageneratorandadiscriminatorThegeneratoristrainedtogeneraterealisticimages,whilethediscriminatoristrainedtodistinguishthegeneratedimagesfromtherealones.GAN-basedmethodshaveshowngreatsuccessinimagestyletransfer.Isolaetal.proposedaconditionalGANforimage-to-imagetranslation,whichcantransfervariousimagestyles,suchaschangingtheseasonofalandscapeimageortransferringthestyleofanimagefromdaytonight.InthispaperweproposeanewmethodbasedonGANforflameimagescenetransferOurmethodconsistsofamulti-scalegeneratorandadiscriminatorThegeneratorisdesignedwithskip-connectionarchitecturetotransfertheflameimagestyletothetargetimagewhilepreservingthestructuraldetails.Thediscriminatorisusedtodistinguishthegeneratedimagefromtherealimageandguidethegeneratortoproducemorerealisticimages.ThearchitectureofourgeneratorisshowninFigure1.Ourgeneratortakesthetargetimageasinputandoutputsthegeneratedimagewiththeflameimagestyle.Thegeneratorconsistsofanencoder,adecoderandskipconnectionsbetweencorrespondinglayersintheencoderanddecoderTheencoderofourgeneratorisdesignedwithamultiscalearchitecturetocapturethefeaturesatdifferentscales.Thedecoderofourgeneratorisdesignedwithadeconvolutionalarchitecturetoupsamplethefeaturesandgeneratethetargetimagewiththeflameimagestyle.ThearchitectureofourdiscriminatorisshowninFigure2.Ourdiscriminatortakesthegeneratedimageandtherealimageasinputandoutputsabinaryclassificationresultindicatingwhethertheinputimageisrealorfake.Thediscriminatorconsistsofmultipleconvolutionallayersfollowedbyafullyconnectedlayer.Theoutputofthediscriminatorisascalarvalue,whichrepresentstheprobabilityoftheinputimagebeingreal.TotrainourGANmodel,wedefinethefollowinglossfunctions:AdversariallossTheadversariallossisusedtotrainthediscriminatortodistinguishthegeneratedimagefromtherealimageandtrainthegeneratortogeneratemorerealisticimages.ThedversariallossisdefinedasLadvElogDrealElogDG(target)))]WhereDdenotesthediscriminatorandGdenotesthegenerator.ContentlossThecontentlossisusedtomeasurethesimilaritybetweenthegeneratedimageandthetargetimageintermsofcontent.Thecontentlossisdefinedas:L_content=||G(target)-target||1StylelossThestylelossisusedtomeasurethesimilaritybetweenthegeneratedimageandtheflameimageintermsofstyle.Thestylelossisdefinedas:LstyleGramGflameGramGtargetWhereGramdenotestheGrammatrixoffeaturemaps.ThetotallossofourGANmodelis:LadvLadvcontentLcontentλ_style*L_styleWhereλ_adv,λ_content,andλ_stylearehyperparameters.WeevaluateourmethodontheCIFAR-10dataset,whichcontains00trainingimagesand10,000testingimages.Werandomlyselect0testingimagesasourtargetimagesandusetheflameimagefromtheinternetasourflameimage.WetrainourGANmodelontheremainingimagesandtestitonthetestingset.TheresultsofourmethodareshowninFigure3.OurmethodcaneffectivelytransfertheflameimagestyletothetargetimageandmaintainthestructuraldetailsofthetargetimageThegeneratedimagesarevisuallypleasingandhaveagoodlevelofrealism.WecompareourmethodwiththeneuralalgorithmforimagestyletransferproposedbyGatysetal.andthefastneuralstyletransfermethodproposedbyJoh