Bayesian Statistics and Inference III贝叶斯统计.ppt

BayesianStatisticsandInferenceIII DrTeng JHarbinInstituteofTechnology Outlines ContrastbetweenMaximumLikelihoodapproachandBayesianapproachExamplesofBayesiandecisionmakingandvalueofinformation Estimationofunknownparametersinstatisticalmodels Bayesianandnon Bayesian Supposewepositaprobabilitydistributiontomodeldata Howdoweestimateitsunknownparameters Example assumedatafollowregressionmodel Wheredotheestimatesoftheregressioncoefficientscomefrom Classicalstatistics maximumlikelihoodestimation Bayesianstatistics Bayesrule EstimatingpercentageofDukieswhoplantogetadvanceddegree SupposewewanttoestimatethepercentageofDukestudentswhoplantogetanadvanceddegree MBA JD MD PhD etc Callthispercentagep Wesample20peopleatrandom and8ofthemsaytheyplantogetanadvanceddegree Whatshouldbeourestimateofp EstimatingtheaverageIQofDukeprofessors Let bethepopulationaverageIQofDukeprofs Supposewerandomlysample25DukeprofsandrecordtheirIQs Whatshouldbeourestimateof Maximumlikelihoodestimation Aprincipledapproachtoestimation Usuallywecanusesubject matterknowledgetospecifyadistributionforthedata But wedon tknowtheparametersofthatdistribution 1 Numberoutof20whowantadvanceddegree binomialdistribution 2 Profs IQs normaldistribution Maximumlikelihoodestimation Weneedtoestimatetheparametersofthedistribution Whydowecare A Sowecanmakeprobabilitystatementsaboutfutureevents B Theparametersthemselvesmaybeimportant Maximumlikelihoodestimation Themaximumlikelihoodestimateoftheunknownparameteristhevalueforwhichthedataweremostlikelytohaveoccurred Let sseehowthisworksintheexamples Advanceddegreeexample LetYbetherandomvariableforthenumberofpeopleoutof20thatplantogetanadvanceddegree Yhasabinomialdistributionwithn 20 andunknownprobabilityp Inthedata Y 8 Ifweknewp thevalueoftheprobabilitydistributionfunctionatY 8wouldbe Maximumlikelihood LabelthefunctionL p L p iscalledthelikelihoodfunctionforp ThemaximumlikelihoodestimateofpisthevalueofpthatmaximizesL p Thisisareasonableestimatebecauseitisthevalueofpforwhichtheobserveddata y 8 hadthegreatestchanceofoccurring FindingtheMLEfordegreeexample Tomaximizethelikelihoodfunction weneedtotakethederivativeofwithrespecttop setitequaltozero andfinallysolveforp Yougetthesamplepercentage EstimatingtheaverageIQofDukeprofessors Let bethepopulationaverageIQofDukeprofs Supposewerandomlysample25DukeprofsandrecordtheirIQs Whatshouldbeourestimateof ModelforProfessors IQs Themathematicalfunctionforanormalcurveforanyprof sIQ whichwelabelY is Allnormalcurveshavethisform withdifferentmeansandSDs Here we llassumethe 15 Wedon tknow whichiswhatwe reafter Modelforall25IQs Weneedthefunctionforall25IQs Assumingeachprof sIQisindependentofotherprofs IQs wehave Modelforall25IQs Withsomealgebraandsimplifications thelikelihoodfunctionis Thefunctionismaximizedwhen isthesampleaverage So weuse132 16asourestimateoftheaverageDukeprof sIQ ThissampleaverageistheMLEfor inanynormalcurve TheBayesianapproachtoestimationofmeans Let sshowhowtocombinedataandpriorinformationtoaddressthefollowingmotivatingquestion WhatisalikelyrangefortheaverageIQofDukeprofessors CombiningthepriorbeliefsandthedatausingBayesRule WecombineourpriorbeliefsandthedatausingBayesrule f data representsourposteriorbeliefsabout Formalizingamodelforpriorinformation Let sassignadistributionfor thatreflectsouraprioribeliefsaboutitslikelyrange Labelthisf Asanexample thecurvedescribingourbeliefsabout isthenormalcurvewithmean 128SD 15 Mathematicalequationfornormalcurve Wecanwritedowntheequationforthisnormalcurve Modelforthedata 25IQs Ifweknew themodelforthedata theprofessors IQs is EstimatingtheaverageIQofDukeprofessors Let bethepopulationaverageIQofDukeprofs Supposewerandomlysample25DukeprofsandrecordtheirIQs Whatshouldbeourestimateof CombiningthepriorbeliefsandthedatausingBayesRule WecombinethemodelforthepriorbeliefsandthemodelforthedatausingBayesrule f data representsourposteriorbeliefsabout Posteriordistribution Usingcalculus onecanshowthatf data isanormalcurvewithmean SD Posteriordistribution Forourdataandpriorbeliefs theposteriorbeliefs f data isanormalcurvewithmean SD Usingtheposteriordistributiontosummarizebeliefsabout Becausef data describesbeliefsabout wecanmakeprobabilitystatementsabout Forexample usinganormalcurvewithmeanequalto132 06andSDequalto2 314 Pr 130 data 813A95 posteriorintervalfor stretchesfrom127 52to136 59 Bayesianstatisticsingeneral Bayesianmethodsexistforanypopulationparameter includingpercentiles ratios etc Themethodisgeneral 1 specifyamathematicalcurvethatreflectspriorbeliefsaboutthepopulationparameter 2 specifyamathematicalcurvethatdescribesthedistributionofthedata givenavalueofthepopulationparameter 3 combinethecurvesfrom1and2mathematicallytogetposteriorbeliefsfortheparameter updatedforthedata DifferencesbetweenfrequentistandBayesian FREQUENTISTParametersarenotrandom Confidenceintervals BAYESIANParametersarerandom Posteriordistributions DecisionMakingUnderRisk Whendoingdecisionmakingunderuncertainty weassumedwehad noidea aboutwhichstateofnaturewouldoccur Indecisionmakingunderrisk weassumewehavesomeidea byexperience gutfeel experiments etc aboutthelikelihoodofeachstateofnatureoccurring TheExpectedValueApproach Givenasetofprobabilitiesforthestatesofnature p1 p2 etc foreachdecisionanexpectedpayoffcanbecalculatedby pi payoffi Ifthisisadecisionthatwillberepeatedoverandoveragain thedecisionwiththehighestexpectedpayoffshouldbetheoneselectedtomaximizetotalexpectedpayoff Butifthisisaone timedecision perhapstheriskoflosingmuchmoneymaybetoogreat thustheexpectedpayoffisjustanotherpieceofinformationtobeconsideredbythedecisionmaker ExpectedValueDecision Supposethebrokerhasofferedhisownprojectionsfortheprobabilitiesofthestatesofnature P S1 2 P S2 3 P S3 3 P S4 1 P S5 1 ExpectedValue 2 100 3 100 3 200 1 300 1 0 2 250 3 200 3 150 1 100 1 150 2 500 3 250 3 100 1 200 1 600 2 60 3 60 3 60 1 60 1 60 100 130 125 60 PerfectInformation Althoughthestatesofnatureareassumedtooccurwiththepreviousprobabilities supposeyouknew eachtimewhichstateofnaturewouldoccur i e youhadperfectinformationThenwhenyouknewS1wasgoingtooccur youwouldmakethebestdecisionforS1 Stock 500 Thiswouldhappenp1 2ofthetime WhenyouknewS2wasgoingtooccur youwouldmakethebestdecisionforS2 Stock 250 Thiswouldhappenp2 3ofthetime Andsoforth ExpectedValueofPerfectInformation EVPI Theexpectedvalueofperfectinformation EVPI isthegaininvaluefromknowingforsurewhichstateofnaturewilloccurwhen versusonlyknowingtheprobabilities Itistheupperboundonthevalueofanyadditionalinformation CalculatingtheEVPI ExpectedReturnWithPerfectInformation ERPI 2 500 3 250 3 200 1 300 1 60 271 ExpectedReturnWithNoAdditionalInformation EV Bond 130 ExpectedValueOfPerfectInformation EVPI ERPI EV Bond 271 130 141 SampleInformation Oneneverreallyhasperfectinformation butcangatheradditionalinformation getexpertadvice etc thatcanindicatewhichstateofnatureislikelytooccureachtime Thestatesofnaturestilloccur inthelongrunwithP S1 2 P S2 3 P S3 3 P S4 1 P S5 1 WeneedastrategyofwhattodogiveneachpossibilityoftheindicatorinformationWewanttoknowthevalueofthissampleinformation EVSI SampleInformationApproach Giventheoutcomeofthesampleinformation werevisetheprobabilitiesofthestatesofnatureoccurring usingBayesiananalysis Thenwerepeattheexpectedvalueapproach usingtheserevisedprobabilities toseewhichdecisionisoptimalgiveneachpossiblevalueofthesampleinformation Example SamuelmanForecast NotedeconomistMiltonSamuelmangivesaneconomicforecastindicatingeitherPositiveorNegativeeconomicgrowthinthecomingyear Usingarelativefrequencyapproachbasedonpastdataithasbeenobserved P Positive largerise 8P Negative largerise 2P Positive smallrise 7P Negative smallrise 3P Positive nochange 5P Negative nochange 5P Positive smallfall 4P Negative smallfall 6P Positive largefall 0P Negative largefall 1 BayesianProbabilitiesGivenaPositiveForecast Prob Positive P PositiveandLargeRise P PositiveandSmallRise P PositiveandNoChange P PositiveandSmallFall P PositiveandLargeFall Prob Positive P Positive LargeRise P LargeRise P Positive SmallRise P SmallRise P Positive NoChange P NoChange P Positive SmallFall P SmallFall P Positive LargeFall P LargeFall 80 20 70 30 30 40 10 0 10 56 P LargeRise Pos P Pos Lg Rise P Lg Rise P Pos P SmallRise Pos P Pos Sm Rise P Sm Rise P Pos P NoChange Pos P Pos NoChg P NoChg P Pos P SmallFall Pos P Pos Sm Fall P Sm Fall P Pos P LargeFall Pos P Pos Lg Fall P Lg Fall P Pos 80 20 56 286 70 30 56 375 50 30 56 268 40 10 56 071 0 10 56 0 50 BestDecisionWithPositiveForecast ExpectedValue 84 180 249 60 WhenSamuelmanpredicts positive ChoosetheStock BayesianProbabilitiesGivenaNegativeForecast Prob Negative P NegativeandLargeRise P NegativeandSmallRise P NegativeandNoChange P NegativeandSmallFall P NegativeandLargeFall Prob Negative P Negative LargeRise P LargeRise P Negative SmallRise P SmallRise P Negative NoChange P NoChange P Negative SmallFall P SmallFall P Negative LargeFall P LargeFall 20 20 30 30 30 60 10 1 10 44 P LargeRise Neg P Neg Lg Rise P Lg Rise P Neg P SmallRise Neg P Neg Sm Rise P Sm Rise P Neg P NoChange Neg P Neg NoChg P NoChg P Neg P SmallFall Neg P Neg Sm Fall P Sm Fall P Neg P LargeFall Neg P Neg Lg Fall P Lg Fall P Neg 20 20 44 091 30 30 44 205 50 30 44 341 60 10 44 136 1 10 44 227 50 BestDecisionWithNegativeForecast ExpectedValue 120 67 33 60 WhenSamuelmanpredicts negative ChooseGold StrategyWithSampleInformation IftheSamuelmanReportisPositive Choosethestock IftheSamuelmanReportisNegative Choosethegold ExpectedValueofSampleInformation EVSI Recall P Positive 56P Negative 44Whenpositive chooseStockwithEV 249Whennegative chooseGoldwithEV 120 ExpectedReturnWithSampleInformation ERSI 56 249 44 120 192 50 ExpectedReturnWithNoAdditionalInformation EV Bond 130 ExpectedValueOfSampleInformation EVSI ERSI EV Bond 192 50 130 62 50 Efficiency Efficiencyisameasureofthevalueofthesampleinformationascomparedtothetheoreticalperfectinformation Itisanumberbetween0and1givenby Efficiency EVSI EVPIFortheJonesInvestmentModel Efficiency 62 50 141 44 BayesianInflationaryBelief Therearetwoplayers thegovernmentandtherepresent