已阅读5页,还剩15页未读, 继续免费阅读
版权说明:本文档由用户提供并上传,收益归属内容提供方,若内容存在侵权,请进行举报或认领
文档简介
1 REINFORCEMENTLEARNING Overview ApplicationstoMusic GautamBhattacharyaMUMT621 2 riseofthemachine letusassumethatweareplayingagainstanimperfectplayer onewhoseplayissometimesincorrectandallowsustowin Forthemoment infact letusconsiderdrawsandlossestobeequallybadforus Howmightweconstructaplayerthatwillfindtheimperfectionsinitsopponent splayandlearntomaximizeitschancesofwinning Sutton R S andA G Barto 1998 ReinforcementLearning AnIntroduction 3 Goals topics WhatisReinforcementLearning History Introduction Individuality ExamplesElementsofaReinforcementLearningSystemTheReinforcementProblem AnExampleApplicationstoMusicQuestions Comments 4 History heterostatictheoryofadaptivesystems developedbyA HarryKlopf butin1979wecametorealizethatperhapsthesimplestoftheideas whichhadlongbeentakenforgranted hadreceivedsurprisinglylittleattentionfromacomputationalperspective Thiswassimplytheideaofalearningsystemthatwantssomething thatadaptsitsbehaviourinordertomaximizeaspecialsignalfromitsenvironment Thiswastheideaofa hedonistic learningsystem or aswewouldsaynow theideaofreinforcementlearning Sutton R S andA G Barto 1998 ReinforcementLearning AnIntroduction 5 Introduction WhatisReinforcementLearning Reinforcementlearningislearningwhattodo howtomapsituationstoactions soastomaximizeanumericalrewardsignal Sutton R S andA G Barto 1998 ReinforcementLearning AnIntroductionThesetwocharacteristics trial and errordelayedrewardThesearethetwomostimportantdistinguishingfeaturesofreinforcementlearning 6 Introduction Theformulationisintendedtoincludejustthesethreeaspects sensation action goal Clearly suchanagentmustbeabletosensethestateoftheenvironmenttosomeextentandmustbeabletotakeactionsthataffectthestate Theagentalsomusthaveagoalorgoalsrelatingtothestateoftheenvironment Sutton R S andA G Barto 1998 ReinforcementLearning AnIntroduction 7 DIFFERENCESWITHRESPECTTOSUPERVISEDLEARNING Learningwithacritic asopposedtolearningwithateacher Reinforcementlearning INTERACTIVElearningIninteractiveproblemsitisoftenimpracticaltoobtainexamplesofdesiredbehaviourthatarebothcorrectandrepresentativeofallthesituationsinwhichtheagenthastoactReinforcementLearninglooksatthebiggerpicture Forexample wehavementionedthatmuchofmachinelearningresearchisconcernedwithsupervisedlearningwithoutexplicitlyspecifyinghowsuchanabilitywouldfinallybeuseful Sutton R S andA G Barto 1998 ReinforcementLearning AnIntroduction 7 8 challenges ChallengesOneofthechallengesthatariseinreinforcementlearningandnotinotherkindsoflearningisthetrade offbetweenexplorationandexploitationThewholeproblemofagoal directedagentinteractingwithanuncertainenvironment Allreinforcementlearningagentshaveexplicitgoals cansenseaspectsoftheirenvironments andcanchooseactionstoinfluencetheirenvironments Moreover itisusuallyassumedfromthebeginningthattheagenthastooperatedespitesignificantuncertaintyabouttheenvironmentitfaces Sutton R S andA G Barto 1998 ReinforcementLearning AnIntroduction 9 Examples Amasterchessplayermakesamove Thechoiceisinformedbothbyplanning anticipatingpossiblerepliesandcounterreplies andbyimmediate intuitivejudgmentsofthedesirabilityofparticularpositionsandmoves Amobilerobotdecideswhetheritshouldenteranewroominsearchofmoretrashtocollectorstarttryingtofinditswaybacktoitsbatteryrechargingstation ItmakesitsdecisionbasedonhowquicklyandeasilyithasbeenabletofindtherechargerinthepastAgazellecalfstrugglestoitsfeetminutesafterbeingborn Halfanhourlateritisrunningat20milesperhour Sutton R S andA G Barto 1998 ReinforcementLearning AnIntroduction 10 Reinforcementlearning Elements POLICY Apolicydefinesthelearningagent swayofbehavingatagiventime Roughly Itisamappingfromperceivedstatesoftheenvironmenttoactionstobetakenwheninthosestates Sutton R S andA G Barto 1998 ReinforcementLearning AnIntroduction 11 Reinforcementlearning Elements REWARDFUNCTION Arewardfunctiondefinesthegoalinareinforcementlearningproblem Itmapseachperceivedstate orstate actionpair oftheenvironmenttoasinglenumber areward indicatingtheintrinsicdesirabilityofthatstate VALUEFUNCTION Specifieswhatisgoodinthelongrun thevalueofastateisthetotalamountofrewardanagentcanexpecttoaccumulateoverthefuture startingfromthatstate CREDITASSIGNMETPROBLEM reinforcementlearningalgorithmslearntogenerateaninternalvaluefortheintermediatestatesastohowgoodtheyareinleadingtothegoal 12 Reinforcementlearning Elements MARKOVDECISIONPROCESS fordiscretetimet 0 1 2 st S wherestisthestateoftheagentattimetandSisthesetofallpossible at A st whereatistheactionattimetandA st isthesetofallpossibleactionsinstatest RewardandNextstatearesampledfromtheirprobabilitydistributionsP rt 1 st at andP st 1 st at InaMarkovSystem thestateandrewardinthenexttimestepdependonlyonthecurrentstate action Canbedeterministici e forcertainstate actiontakenthereisonlyonepossiblerewardandnextstate 13 Reinforcementlearning Elements Thepolicy definestheactiontobetakenatanystatestThevalueofthepolicy V st isthecumulativerewardthatwillbereceivediftheagentfollowsthepolicystartingfromstOptimizevaluesof andV st MODELS Thisissomethingthatmimicsthebehavioroftheenvironment Forexample givenastateandaction themodelmightpredicttheresultantnextstateandnextreward Modelsareusedforplanning E g FiniteHorizon InfiniteHorizonmodels 14 TheReinforcementProblem AnExample http nrm wikipedia org wiki File Jogo da velha tic tac toe png 15 TictacTOE letusassumethatweareplayingagainstanimperfectplayer onewhoseplayissometimesincorrectandallowsustowin Forthemoment infact letusconsiderdrawsandlossestobeequallybadforus Howmightweconstructaplayerthatwillfindtheimperfectionsinitsopponent splayandlearntomaximizeitschancesofwinning Sutton R S andA G Barto 1998 ReinforcementLearning AnIntroduction http nrm wikipedia org wiki File Jogo da velha tic tac toe png 16 Asequenceoftic tac toemoves Thesolidlinesrepresentthemovestakenduringagame thedashedlinesrepresentmovesthatwe ourreinforcementlearningplayer consideredbutdidnotmake Oursecondmovewasanexploratorymove meaningthatitwastakeneventhoughanothersiblingmove theoneleadingto wasrankedhigher Exploratorymovesdonotresultinanylearning buteachofourothermovesdoes causingbackups Sutton R S andA G Barto 1998 ReinforcementLearning AnIntroduction 17 knowledgeupdate letsdenotethestatebeforeagreedymove ands thestateafterthemove thentheupdatetotheestimatedvalueofs givenbyV s V s V s V s V s Where isthestep sizeparameter whichinfluencestherateoflearningThisupdatemethodiscalledTemporalDifferenceLearningMethod 18 applicationstomusic INTERACTIVEMUSICSYSTEMSContemporaryworkinthisfieldincludesinvestigationsintobothmachinelistening realtimeaudioanalysis androbotics AjayKapur sMahaDevi Bot athirteenarmedIndianpercussionistwhichcansynchronisetosensorinputfromahumansitaristHanamakaandcolabpratprsdevelopedasystemthattrainsvirtualplayersfromtheperformanceofatrioofhumanguitarists learningindependentmodelsforreaction withinensembleplayingbehaviour phrase adatabaseofindividualmusicalma terials andgroove timingpreference FrancoisPachetetalldevelopedasystemthatcanlearnth
温馨提示
- 1. 本站所有资源如无特殊说明,都需要本地电脑安装OFFICE2007和PDF阅读器。图纸软件为CAD,CAXA,PROE,UG,SolidWorks等.压缩文件请下载最新的WinRAR软件解压。
- 2. 本站的文档不包含任何第三方提供的附件图纸等,如果需要附件,请联系上传者。文件的所有权益归上传用户所有。
- 3. 本站RAR压缩包中若带图纸,网页内容里面会有图纸预览,若没有图纸预览就没有图纸。
- 4. 未经权益所有人同意不得将文件中的内容挪作商业或盈利用途。
- 5. 人人文库网仅提供信息存储空间,仅对用户上传内容的表现方式做保护处理,对用户上传分享的文档内容本身不做任何修改或编辑,并不能对任何下载内容负责。
- 6. 下载文件中如有侵权或不适当内容,请与我们联系,我们立即纠正。
- 7. 本站不保证下载资源的准确性、安全性和完整性, 同时也不承担用户因使用这些下载资源对自己和他人造成任何形式的伤害或损失。
最新文档
- 青岛市2027年高三第二次联考高考物理试卷(含答案解析)
- 2026-2027学年四川省绵阳市高三适应性调研考试高考物理试题(含答案解析)
- 小镇青年返乡创业:智慧养老平台下沉市场的机遇与挑战
- 十五五规划前瞻:智能家居安防的国家战略与安全底座
- 深度复盘智能网联汽车网络安全:2026年渗透率与融资趋势
- 十五五银发经济机遇:智能备餐进度屏适老化交互设计研究
- 双碳约束下智能按摩锤:绿色材料应用能否成为家电巨头第二曲线
- 2026年商务礼品市场调研报告:行业礼品定制化创新与市场前景
- 2026年环保产业小微门店绿色技术发展与政策解读报告
- 2026年人工智能行业应用报告及市场潜力分析
- 2027年天津农商银行校园招聘笔试备考试题及答案解析
- 康复医学科常见功能障碍康复指南(2026版)
- 2026-2030训练健身器材市场盈利模式及经营管理风险分析研究报告
- 2026年湖北省荆州市公安招聘辅警考试题库含答案
- 安徽建工集团管理制度
- 图形的位似课时1(课件)2026-2027学年北师大版九年级数学上册
- 消防工程从入门到精通
- YDT 5102-2024 通信线路工程技术规范
- 银行员工禁止性行为课件
- 第3章-民族医药基础知识
- 除痛病历管理制度
评论
0/150
提交评论