《人工智能与数据挖掘教学课件》l.ppt

2020 2 6 AI DM 1 Chapter8NeuralNetworks PartIII AdvanceDataMiningTechniques 2020 2 6 AI DM 2 What WhyANN 8 1FeedforwardNeuralNetwork HowANNworks workingprinciple 8 2 1SupervisedLearning MostpopularANN BackpropagationNetwork 8 5 1TheBackpropagationAlgorithm Anexample Content 2020 2 6 AI DM 3 What WhyANN ArtificialNeuralNetworks ANN ANNisaninformationprocessingtechnologythatemulatesabiologicalneuralnetwork Neuron 神经元 vsNode Transformation Dendrite 树突 vsInputAxon 轴突 vsOutputSynapse 神经键 vsWeightStartsin1970s becomeverypopularin1990s becauseoftheadvancementofcomputertechnology 2020 2 6 AI DM 4 2020 2 6 AI DM 5 2020 2 6 AI DM 6 WhatisANN Basics TypesofANNNetworkstructure e g Figure17 9 17 10 Turban 2000 version5 p663 NumberofhiddenlayersNumberofhiddennodesFeedforwardandfeedbackward timedependentproblems Linksbetweennodes existorabsentoflinks Theultimateobjectivesoftraining obtainasetofweightsthatmakesalltheinstancesinthetrainingdatapredictedascorrectlyaspossible Back propagationisonetypeofANNwhichcanbeusedforclassificationandestimationmulti layer Inputlayer Hiddenlayer s OutputlayerFullyconnectedFeedforwardErrorback propagation 2020 2 6 AI DM 7 What WhyANN 8 1FeedforwardNeuralNetwork HowANNworks workingprinciple 8 2 1SupervisedLearning MostpopularANN BackpropagationNetwork 8 5 1TheBackpropagationAlgorithm Anexample Content 2020 2 6 AI DM 8 2 HowANN workingprinciple I Step1 CollectdataStep2 SeparatedataintotrainingandtestsetsfornetworktrainingandvalidationrespectivelyStep3 Selectnetworkstructure learningalgorithm andparametersSettheinitialweightseitherbyrulesorrandomlyRateoflearning pacetoadjustweights Selectlearningalgorithm Morethanahundredlearningalgorithmsavailableforvarioussituationsandconfigurations 2020 2 6 AI DM 9 2 ANNworkingprinciple II Step4 TrainthenetworkComputeoutputsCompareoutputswithdesiredtargets ThedifferencebetweentheoutputsandthedesiredtargetsiscalleddeltaAdjusttheweightsandrepeattheprocesstominimizethedelta TheobjectiveoftrainingistoMinimizetheDelta Error Thefinalresultoftrainingisasetofweights Step5 TestthenetworkUsetestset comparingtestresultstohistoricalresults tofindouttheaccuracyofthenetworkStep6 Deploydevelopednetworkapplicationifthetestaccuracyisacceptable 2020 2 6 AI DM 10 2 ANNworkingprinciple III Example Example1 ORoperation seetablebelow Twoinputelements X1andX2InputsCaseX1X2DesiredResults10002011 positive 3101 positive 4111 positive 2020 2 6 AI DM 11 2 ANNworkingprinciple IV Example Networkstructure onelayer seenextpage LearningalgorithmWeightedsum summationfunction Y1 XiWiTransformation transfer function Y1lessthanthreshold Y 0 otherwiseY 1Delta Z YWi final Wi initial Alpha Delta XiInitialParameters Rateoflearning alpha 0 2Threshold 0 5 Initialweight 0 1 0 3Notes WeightsareinitiallyrandomThevalueoflearningrate alpha issetlowfirst 2020 2 6 AI DM 12 ProcessingInformationinanArtificialNeuron x1 w1j x2 Yj w2j Neuronj wijxi Weights Output Inputs Summations Transferfunction 2020 2 6 AI DM 13 What WhyANN 8 1FeedforwardNeuralNetwork HowANNworks workingprinciple 8 2 1SupervisedLearning MostpopularANN BackpropagationNetwork 8 5 1TheBackpropagationAlgorithm Anexample Content 2020 2 6 AI DM 14 3 Back propagationNetwork NetworkTopologymulti layer Inputlayer Hiddenlayer s OutputlayerFullyconnectedFeedforwardErrorback propagationInitializeweightswithrandomvalues 2020 2 6 AI DM 15 Back propagationNetwork Outputnodes Inputnodes Hiddennodes Outputvector Inputvector xi wij 2020 2 6 AI DM 16 3 Back propagationNetwork Foreachnode1 Computethenetinputtotheunitusingsummationfunction2 Computetheoutputvalueusingtheactivationfunction i e sigmoidfunction 3 Computetheerror4 Updatetheweights andthebias basedontheerror5 Terminatingconditions all wijinthepreviousepoch 周期 weresosmallastobebelowsomespecifiedthresholdthepercentageofsamplesmisclassifiedinthepreviousepochisbelowsomethresholdapre specifiednumberofepochhasexpired 2020 2 6 AI DM 17 BackpropagationErrorOutputLayer 2020 2 6 AI DM 18 BackpropagationErrorHiddenLayer 2020 2 6 AI DM 19 TheDeltaRule 2020 2 6 AI DM 20 RootMeanSquaredError 2020 2 6 AI DM 21 3 Back propagation cont IncreasenetworkaccuracyandtrainingspeedNetworktopologynumberofnodesininputlayernumberofhiddenlayers usuallyisone nomorethantwo numberofnodesineachhiddenlayernumberofnodesinoutputlayerChangeinitialweights learningparameter terminatingconditionTrainingprocess FeedthetraininginstancesDeterminetheoutputerrorUpdatetheweightsRepeatuntiltheterminatingconditionismet 2020 2 6 AI DM 22 SupervisedLearningwithFeed ForwardNetworks BackpropagationLearning 2020 2 6 AI DM 23 Summary Decisionsthebuildermustmake NetworkTopology numberofhiddenlayers numberofnodesineachlayer andfeedbackLearningalgorithmsParameters initialweight learningrateSizeoftrainingandtestdata Structureandparametersdeterminethelengthoftrainingtimeandtheaccuracyofthenetwork 2020 2 6 AI DM 24 NeuralNetworkInputFormat Normalization categoricaltonumerical Allinputandoutputmustnumericalandbetween 0 1 CategoricalAttributes e g attributewith4possiblevaluesOrdinal Setto0 0 33 0 66 1Nominal Setto 0 0 0 1 1 0 1 1 NumericalAttributes 2020 2 6 AI DM 25 NeuralNetworkOutputFormat CategoricalAttributes Numericaltocategorical Type0 1Type0 45NumericalAttributes 0 1 toordinaryvalue Min X Max min 2020 2 6 AI DM 26 Homework P264 ComputationalQuestions 2r 0 5 Tk 0 65Adjustallweightsforoneepoch 2020 2 6 AI DM 27 CaseStudy Example BankruptcyPredictionwithNeuralNetworksStructure Three layernetwork back propagationTrainingdata Smallsetofwell knownfinancialratiosDataavailableonbankruptcyoutcomesSupervisednetwork 2020 2 6 AI DM 28 ArchitectureoftheBankruptcyPredictionNeuralNetwork X4 X3 X5 X1 X2 Bankrupt0 Notbankrupt1 2020 2 6 AI DM 29 BankruptcyPrediction Networkarchitecture FiveInputNode