2024大模型时代自身智能机器人

RESEARCHCENTERFORSOCIALCOMPUTINGANDINFORMATIONRESEARCHCENTERFORSOCIALCOMPUTINGANDINFORMATIONHARBININSTITUTEOF HARBININSTITUTEOF研制出一种由机械蒸汽驱动的鸟状飞行器，并被命名为“鸽子”。““鸽子”HARBININSTITUTEOF莱昂纳多·达·1495年左右绘制了人形机器人的草图。现在被称为莱昂纳多的机器

1973年,KUKA公司推出的世界第一台拥有六HARBININSTITUTEHARBININSTITUTEOFHARBININSTITUTEOF HARBINHARBININSTITUTEOF①自主能力机器 智能机器人 人②泛化能力（通用能力） 世界第一台全尺寸人形机器 人形运动能力重大进 人形机器人成功商业落 人形动作能力迈入新纪

Aldebaran公司研发的

HARBININSTITUTEOFHARBININSTITUTEOFHARBINHARBININSTITUTEOF①自主能力②泛化能力（通用能力）机器 智能机器人 人 医疗微创机器 物流运输机器 展厅服务机器 家庭清洁机器HARBINHARBININSTITUTEOF

RESEARCHCENTERFORSOCIALCOMPUTINGANDINFORMATIONHARBINHARBININSTITUTEOF 像人类一样工作的机器人 各方面强于人类的机器人 有意识和情感的机器人HARBINHARBININSTITUTEOF 一定的自主能一定的泛化能力（通用能力HARBINHARBININSTITUTEOFRESEARCHCENTERFORSOCIALCOMPUTINGANDINFORMATION

并综合分析当前所有状态（具身感知和规划（具身推理向下位机下发送运动指令（具身执行HARBININSTITUTEOF 向下位机下发送运动指令（具身执行HARBININSTITUTEOF

RESEARCHCENTERFORSOCIALCOMPUTINGANDINFORMATION

HARBINHARBININSTITUTEOFHARBINHARBININSTITUTEOF并综合分析当前所有状态（具身感知向下位机下发送运动指令（具身执行HARBINHARBININSTITUTEOF收集所有传感器采集的环境信息和自身状态。并综合分析当前所有状态（具身感知这是什么？如何打开它 人的一些手势是什么意思 HARBINHARBININSTITUTEOF根据当前状态，对自身下一步的运动做出决策和规划（具身推理HARBINHARBININSTITUTEOF根据当前状态，对自身下一步的运动做出决策和规划（具身推理HARBINHARBININSTITUTEOF向下位机下发送运动指令（具身执行）（形式包括代码、技能库API、关节旋转角度等多模态大模 扩散小模较低位置泛化场景泛化泛物品泛化化位置泛化力对于生成技能库API或代码API形式的运动指令：现实世界场景过于复杂，构建完整的技能库几Ⓖ因此，当前人工智能还不足以让机器人更智能，需要RESEARCHCENTERFORSOCIALCOMPUTINGANDINFORMATION

HARBININSTITUTEOFHARBININSTITUTEOFRichardHeld,AlanHein.Movement-producedstimulationinthedevelopmentofvisuallyguidedbehavior.1963JouranalofComparativeandPhysiologicalPsychologyHARBINHARBININSTITUTEOF抽象的智能（围棋、文本处理、图像识别抽象的智能（围棋、文本处理、图像识别

物 场感 感行 表感 感

推 具体分

基于规 基于大模HARBINHARBININSTITUTEOFCONTENT

RESEARCHCENTERFORSOCIALCOMPUTINGANDINFORMATIONRESEARCHCENTERFORSOCIALCOMPUTINGANDINFORMATIONHARBININSTITUTEHARBININSTITUTEOF表HARBINHARBININSTITUTEOF&HARBININSTITUTEOFHARBININSTITUTEOFRESEARCHCENTERFORSOCIALCOMPUTINGANDINFORMATIONHARBINHARBININSTITUTEOFXuetal.UMPNet:UniversalManipulationPolicyNetworkforArticulatedObjects.2022RA-Dongetal.Tactile-RLforInsertion:GeneralizationtoObjectsofUnknownHARBINHARBININSTITUTEOFOccupancyNetworkHARBINHARBININSTITUTEOFQietal.Pointnet:Deeplearningonpointsetsfor3dclassificationandsegmentation.2017Qietal.Pointnet++:Deephierarchicalfeaturelearningonpointsetsinametricspace.2017HARBINHARBININSTITUTEOFFengetal.Meshnet:Meshneuralnetworkfor3dshaperepresentation.2019HARBINHARBININSTITUTEOFZhouetal.VoxelNet:End-to-EndLearningforPointCloudBased3DObjectDetection.2018DeepSDF（SignedDeepSDF（SignedDistanceHARBININSTITUTEOFParketal.DeepSDF:LearningContinuousSignedDistanceFunctionsforShapeRepresentation.2019OccupancyNetworksOccupancyNetworksHARBININSTITUTEOFMeschederetal.OccupancyNetworks:Learning3DReconstructioninFunctionSpace.2019

HARBINHARBININSTITUTEOFHARBINHARBININSTITUTEOFWangetal.NormalizedObjectCoordinateSpaceforCategory-Level6DObjectPoseandSizeEstimation.2019HARBINHARBININSTITUTEOFHARBINHARBININSTITUTEOFHARBINHARBININSTITUTEOFFangetal.AnyGrasp:RobustandEfficientGraspPerceptioninSpatialandTemporalDomains.2022T-HARBINHARBININSTITUTEOF[1]Huangetal.CoPa:GeneralRoboticManipulationthroughSpatialConstraintsofPartswithFoundationModels.2024HARBINHARBININSTITUTEOFXuetal.UMPNet:UniversalManipulationPolicyNetworkforArticulatedObjects.2022RA-Dongetal.Tactile-RLforInsertion:GeneralizationtoObjectsofUnknownHARBINHARBININSTITUTEOFLiuetal.TowardReal-WorldCategory-LevelArticulationPoseEstimation.2022HARBINHARBININSTITUTEOFLiuetal.AKB-48:AReal-WorldArticulatedObjectKnowledgeBase.2022Cageetal.CAGE:ControllableArticulationGEneration.2024HARBINHARBININSTITUTEOFHARBINHARBININSTITUTEOFLietal.Category-LevelArticulatedObjectPoseEstimation.2020HARBINHARBININSTITUTEOFWeietal.Self-supervisedNeuralArticulatedShapeandAppearanceModels.2022HARBINHARBININSTITUTEOFHARBINHARBININSTITUTEOF首先以原始物体点云作为输入，基于物体组件级分割，得到物体初始URDFLvetal.SAGCI-System:TowardsSample-Efficient,Generalizable,CompositionalandIncrementalRobotLearning.2022HARBINHARBININSTITUTEOF[1]Moetal.Where2Act:FromPixelstoActionsforArticulated3DObjects.2024HARBINHARBININSTITUTEOFXuetal.UMPNet:UniversalManipulationPolicyNetworkforArticulatedObjects.2022RA-Dongetal.Tactile-RLforInsertion:GeneralizationtoObjectsofUnknownHARBINHARBININSTITUTEOFGirdharetal.Imagebind:Oneembeddingspacetobindthemall.2023Zhuetal.Languagebind:Extendingvideo-languagepretrainington-modalitybylanguage-basedsemanticalignment.2024Dongetal.Tactile-rlforinsertion:Generalizationtoobjectsofunknowngeometry.2024HARBINHARBININSTITUTEOF[1]Yuetal.PreciseRoboticNeedle-ThreadingwithTactilePerceptionandReinforcementLearning.2023RESEARCHCENTERFORSOCIALCOMPUTINGANDINFORMATIONHARBINHARBININSTITUTEOFHARBININSTITUTEHARBININSTITUTEOF HARBINHARBININSTITUTEOFHARBINHARBININSTITUTEOF激光传感器工作原 雷达传感器工作原Sun,etal.Aqualityimprovementmethodfor3Dlaserslampointcloudsbasedongeometricprimitivesofthescanscene.2021Kong,etal.Multi-modaldata-efficient3dsceneunderstandingforautonomousdriving.2024Zheng,etal.Scene-awarelearningnetworkforradarobjectdetection.2021Yang,etal.Anego-motionestimationmethodusingmillimeter-waveradarin3Dscenereconstruction.2022

HARBINHARBININSTITUTEOFSingh,etal.SporadicAudio-VisualEmbodiedAssistiveRobotNavigationForHumanTracking.2023Gan,etal.Look,listen,andact:Towardsaudio-visualembodiednavigation.2020Roberge,etal.StereoTac:Anovelvisuotactilesensorthatcombinestactilesensingwith3Dvision.2023Padmanabha,etal.Omnitact:Amulti-directionalhigh-resolutiontouchsensor.2020Armada,etal.Co-operativesmell-basednavigationformobilerobots.2004Ciui,etal.Chemicalsensingattherobotfingertips:Towardautomatedtastediscriminationinfoodsamples.2018ACSSinai,etal.Scenerecognitionwithinfra-red,low-light,andsensorfusedimagery.1999Kim,etal.Firefightingrobotstereoinfraredvisionandradarsensorfusionforimagingthroughsmoke.2015FireShimoyama,etal.SeeingNearby3DScenesusingUltrasonicSensors.2022Mulindwa,etal.Indoor3Dreconstructionusingcamera,IMUandultrasonicsensors.2020HARBININSTITUTEHARBININSTITUTEOF HARBINHARBININSTITUTEOFDurrantetal.Simultaneouslocalizationandmap:partI.2006Taketomieal.VisualSLAMalgorithms:Asurveyfrom2010to2016.2017HARBINHARBININSTITUTEOFDeTone,etal.Towardgeometricdeepslam.2017Tateno,etal.Cnn-slam:Real-timedensemonocularslamwithlearneddepthprediction.2017Li,etal.Undeepvo:Monocularvisualodometrythroughunsuperviseddeeplearning.2018

HARBINHARBININSTITUTEOFHARBINHARBININSTITUTEOF7HARBINHARBININSTITUTEOF主动映射任务，即下一个最佳视图（NestBestView）任务，旨在找到更好的观测视点或基于拓扑的信息增益度量确定下一个最佳视 Collander,etal.Learningthenextbestviewfor3dpointcloudsviatopologicalfeatures.2021Gazani,etal.Bagofviews:Anappearance-basedapproachtonext-best-viewplanningfor3dreconstruction.2023 NBV（NBO）

HARBINHARBININSTITUTEOFLiu,etal.Object-awareguidanceforautonomousscenereconstruction.2018Dong,etal.Multi-robotcollaborativedensescenereconstruction.2019HARBINHARBININSTITUTEOFANL（Activeneurallocalization）通过端到端强化学习（包括感知模块和策略模块）最大化移动Chaplot,etal.Activeneurallocalization.2018HARBINHARBININSTITUTEOFHARBININSTITUTEHARBININSTITUTEOF HARBINHARBININSTITUTEOF物体识 空间关系推 时序变化检HARBINHARBININSTITUTEOFMASKRedmon,etal.Youonlylookonce:Unified,real-timeobjectdetection.2016He,etal.Maskr-cnn.2017He,etal.Deepresiduallearningforimagerecognition.2016HARBINHARBININSTITUTEOFPathaketalPathak,etal.Learninginstancesegmentationbyinteraction.2018HARBINHARBININSTITUTEOFSeeingbyMoving模仿人类“通过绕着同一物体走动来获取多个观察视角”的策略，使机器Fang,etal.Movetoseebetter:Self-improvingembodiedobjectdetection.2020HARBINHARBININSTITUTEOFopen openGoyal,etal.Rel3d:Aminimallycontrastivebenchmarkforgroundingspatialrelationsin3d.2020Yang,etal.Spatialsense:Anadversariallycrowdsourcedbenchmarkforspatialrelationrecognition.2019Kuznetsova,etal.Theopenimagesdatasetv4:Unifiedimageclassification,objectdetection,andvisualrelationshipdetectionatscale.2020HARBINHARBININSTITUTEOFroboticvisionsceneunderstandingHall,etal.Theroboticvisionsceneunderstandingchallenge.2020Park,etal.Changesim:Towardsend-to-endonlinescenechangedetectioninindustrialindoorenvironments.2021Prabhakar,etal.Cdnet++:Improvedchangedetectionwithdeepneuralnetworkfeaturecorrelation.2020Sakurada,etal.Weaklysupervisedsilhouette-basedsemanticscenechangedetection.2020HARBINHARBININSTITUTEOF Gadre,etal.Continuousscenerepresentationsforembodiedai.2022Zhang,etal.Object-levelchangedetectionwithadualcorrelationattention-guideddetector.2021HARBINHARBININSTITUTEOFYang,etal.4dpanopticscenegraphgeneration.2024RESEARCHCENTERFORSOCIALCOMPUTINGANDINFORMATIONHARBINHARBININSTITUTEOFHARBINHARBININSTITUTEOFHARBINHARBININSTITUTEOF&&HARBININSTITUTEOF[1]Narayananetal.ProxEmo:Gait-basedEmotionLearningandMulti-viewProxemicFusionforSocially-AwareRobotNavigation.2020HARBINHARBININSTITUTEOF[1]Zhangetal.Recurrentneuralnetworkformotiontrajectorypredictioninhuman-robotcollaborativeassembly.2020HARBINHARBININSTITUTEOFHARBINHARBININSTITUTEOF [1]Jiangetal.Motiongpt:Humanmotionasaforeignlanguage.2024HARBINHARBININSTITUTEOF [1]ChenLHetal.MotionLLM:UnderstandingHumanBehaviorsfromHumanMotionsandVideos.2024arXivpreprintRESEARCHCENTERFORSOCIALCOMPUTINGANDINFORMATIONHARBINHARBININSTITUTEOFHARBINHARBININSTITUTEOF陪伴机器 智慧家 工业机器[1]MaFetal.Facialexpressionrecognitionwithvisualtransformersand[1]MaFetal.Facialexpressionrecognitionwithvisualtransformersandattentionalselectivefusion.2021IEEETransactionsonAffectiveHARBININSTITUTEOF如几何特征（关键点坐标）、纹理特征（局部二值模式，LBP）和动作单元（ActionUnits,HARBINHARBININSTITUTEOFMaFetal.Facialexpressionrecognitionwithvisualtransformersandattentionalselectivefusion.2021IEEETransactionsonAffectiveWangKetal.Regionattentionnetworksforposeandocclusionrobustfacialexpressionrecognition.2020IEEETransactionsonImageWuYetal.Edge-AI-drivenframeworkwithefficientmobilenetworkdesignforfacialexpressionrecognition.2023ACMTransactionsonEmbeddedComputingSystemsHARBINHARBININSTITUTEOFFigure01机器人（基于OpenAI大模型）听到人类说“我饿了”之后，

HARBINHARBININSTITUTEOFHARBINHARBININSTITUTEOFHARBINHARBININSTITUTEOF动态图注意力网络模块化注意力网络指代表达理解的动态图注意网络(DGA)的整体架 YuLetal.Mattnet:Modularattentionnetworkforreferringexpressioncomprehension.2018ProceedingsoftheIEEEconferenceoncomputervisionandpatternrecognitionYangSetal.Dynamicgraphattentionforreferringexpressioncomprehension.2019ProceedingsoftheIEEE/CVFInternationalConferenceonComputerVisionRESEARCHCENTERFORSOCIALCOMPUTINGANDINFORMATIONHARBININSTITUTEOFHARBININSTITUTEOFHARBINHARBININSTITUTEOFANDRESEARCHCENTERFORSOCIALCOMPUTINGANDINFORMATION

物 场感 感行 表感 感

推 具体分

基于规 基于大模HARBINHARBININSTITUTEOFRESEARCHCENTERFORSOCIALCOMPUTINGANDINFORMATIONHARBININSTITUTEHARBININSTITUTEOF，HARBINHARBININSTITUTEOF与轨迹规划的区别（Motion任务规划：将人类指令分解为给定技能集的HARBINHARBININSTITUTEOF Fikesetal.STRIPS:ANewApproachtotheApplicationof.TheoremProvingtoProblemSolving.1971Carbonelletal.PRODIGY:anintegratedarchitectureforplanningandlearning.1991SIGARTAuetal.SHOP2:AnHTNPlanningSystem.2003HARBINHARBININSTITUTEOF Howeetal.PDDL-theplanningdomaindefinitionlanguage.1998Lifschitzetal.WhatIsAnswerSetProgramming?.2008.HARBINHARBININSTITUTEOFXuetal.RegressionPlanningNetworks.2019.HARBINHARBININSTITUTEOFLiuetal.LLM+P:EmpoweringLargeLanguageModelswithOptimalPlanningProficiency.2023HARBINHARBININSTITUTEOFHuangetal.LanguageModelsasZero-ShotPlanners:ExtractingActionableKnowledgeforEmbodiedAgents.2022.HARBINHARBININSTITUTEOF[1]Singhetal.ProgPrompt:GeneratingSituatedRobotTaskPlansusingLargeLanguageModels.2023AutonomousHARBINHARBININSTITUTEOF[1]Driessetal.PaLM-E:AnEmbodiedMultimodalLanguageModel.2023.HARBINHARBININSTITUTEOFAhnetal.DoAsICan,NotAsISay:GroundingLanguageinRoboticAffordances.2022.HARBINHARBININSTITUTEOFYietal.EgoPlan-Bench:BenchmarkingMultimodalLargeLanguageModelsforHuman-LevelPlanning.2023.Srivastavaetal.BEHAVIOR:BenchmarkforEverydayHouseholdActivitiesinVirtual,Interactive,andEcologicalEnvironments.2021.Puigetal.Watch-and-help:Achallengeforsocialperceptionandhuman-aicollaboration.2020.Shridharetal.ALFRED:ABenchmarkforInterpretingGroundedInstructionsforEverydayTasks.2020.HARBINHARBININSTITUTEOF

文本大模型（只拆解任务文本大模型（只拆解任务 HARBININSTITUTEHARBININSTITUTEOF

动作（一般就是轨迹

动作（一般就是轨迹RESEARCHCENTERFORSOCIALCOMPUTINGANDINFORMATIONHARBINHARBININSTITUTEOFHARBINHARBININSTITUTEOF 路径规划算 避障算HARBINHARBININSTITUTEOF视觉导航（VisualHARBINHARBININSTITUTEOF非端到端的方 端到端的方HARBINHARBININSTITUTEOFHARBINHARBININSTITUTEOFVTNetTransformer(VT[1]Duetal.VTNet:VisualTransformerNetworkforObjectGoalNavigation.ICLRHARBINHARBININSTITUTEOF空间布局建模：使用地图构建和路径规划的联合架构，如Guptaetal知识图谱与深度强化学习结合贝叶斯关系记忆Guptaetal.Cognitivemappingandplanningforvisualnavigation.2020Chenetal.Abehavioralapproachtovisualnavigationwithgraphlocalizationnetworks.2019Ramakrishnanetal.Occupancyanticipationforefficientexplorationandnavigation.2020Yangetal.Visualsemanticnavigationusingscenepriors.2018YiWu.2019.Bayesianrelationalmemoryforsemanticvisualnavigation.2019HARBINHARBININSTITUTEOF三种辅助任务（动作条件对比预测编码、逆动力学学和时间距离估计），Dosovitskiyetal.Learningtoactbypredictingthefuture.2017Mishkinetal.Benchmarkingclassicandlearnednavigationincomplex3denvironments.2019Wijmansetal.DD-PPO:LearningNear-PerfectPointGoalNavigatorsfrom2.5BillionFrames.2019Yeetal.Auxiliarytasksspeeduplearningpointgoalnavigation.2021Wortsmanetal.Learningtolearnhowtolearn:Self-adaptivevisualnavigationusingmeta-learning.2019HARBINHARBININSTITUTEOF主要通过RNNLSTM,TransformerHARBINHARBININSTITUTEOFZhuetalVision-LanguageNavigationwithSelf-SupervisedAuxiliaryReasoningTasksHARBINHARBININSTITUTEOFHARBINHARBININSTITUTEOFCLIP将短语与当前输入图片计算相似度，选择最恰当的[1]Dorbalaetal.CLIP-Nav:UsingCLIPforZero-ShotVision-and-LanguageNavigation.2022HARBINHARBININSTITUTEOF[1]YuetalL3MVN:LeveragingLargeLanguageModelsforVisualTargetNavigationHARBINHARBININSTITUTEOFNavGPT将视觉观察的文本描述、导航历史和未来可探索方向作为输入来推理代理的当前状态，并做出接[1]ZhouetalNavGPT:ExplicitReasoninginVision-and-LanguageNavigationwithLargeLanguageModels.2023RESEARCHCENTERFORSOCIALCOMPUTINGANDINFORMATIONHARBINHARBININSTITUTEOF具身问答任务最早由Dasetal.提出，该任务下机器人需要主动探索环境，定位目标物体或位置，[1]Dasetal.EmbodiedQuestionAnswering.2018HARBINHARBININSTITUTEOFLuoetal针对嘈杂环境设计鲁棒性更强Lietal提出Model-basedRL的EQA算法，通过“想象”下一个子目标环境图片，提高探Chaplotetal多任务联合学习，将Dasetal.EmbodiedQuestionAnswering.2018Luoetal.Robust-EQA:RobustLearningforEmbodiedQuestionAnsweringWithNoisyLabels.2023Lietal.WalkingwithMIND:MentalImageryeNhanceDEmbodiedQA.2019ACMChaplotetal.EmbodiedMultimodalMultitaskLearning2019HARBINHARBININSTITUTEOFYuetal.Multi-TargetEmbodiedQuestionAnswering.2019Tanetal.Multi-agentEmbodiedQuestionAnsweringinInteractiveEnvironments.2020Tanetal.Knowledge-BasedEmbodiedQuestionAnswering.2021HARBINHARBININSTITUTEOFBlindBenchmark：在EQAv1数据集上，不基于环境观察回答问题，2018年提出时成为当时RESEARCHCENTERFORSOCIALCOMPUTINGANDINFORMATIONHARBINHARBININSTITUTEOFRESEARCHCENTERFORSOCIALCOMPUTINGANDINFORMATION

物 场感 感行 表感 感

推 具体分

基于规 基于大模HARBINHARBININSTITUTEOFHARBINHARBININSTITUTEOFHARBINHARBININSTITUTEOFRESEARCHCENTERFORSOCIALCOMPUTINGANDINFORMATIONHARBINHARBININSTITUTEOFHARBINHARBININSTITUTEOF模仿学习可以分为两部分：对图像的编码， HARBININSTITUTEOFHARBININSTITUTEOFJangetal.BC-Z:Zero-ShotTaskGeneralizationwithRoboticImitationLearning.2021Brohanetal.RT-1:RoboticsTransformerforReal-WorldControlatScale.2022

上绕过，可以向下绕过，

此方法将动作进行聚类此方法将动作进行聚类[1]Shafiullahetal.BehaviorTransformers:Cloningkmodeswithonestone.NIPS2022HARBININSTITUTEOFHARBININSTITUTEOFHARBININSTITUTEOF[1]Florenceetal.ImplicitBehavioralCloning.CoRL

HARBINHARBININSTITUTEOFHARBINHARBININSTITUTEOF[1]Chietal.DiffusionPolicy:VisuomotorPolicyLearningviaActionDiffusion.RSSHARBINHARBININSTITUTEOF[1]Parietal.TheSurprisingEffectivenessofRepresentationLearningforVisualImitation.RSSRESEARCHCENTERFORSOCIALCOMPUTINGANDINFORMATIONHARBINHARBININSTITUTEOF强化学习（Reinforcementlearning无模型强化学习（Model-Free基于模型的强化学习（Model-BasedTD-MPC2Model-TD-MPC2Model-HARBININSTITUTEOFDynamic𝑟=𝑅(𝑧𝑎,𝑞=𝑄(𝑧𝑎,状态预测模型𝑧′=𝑑(𝑧𝑎=𝑝(𝑧𝑧=ℎ(𝑠 [1]HansenetalTD-MPC2:Scalable,RobustWorldModelsforContinuousControl.2024（TD-MPC的）dynamic更新策略得到

dynamic更新策略pre得到EUCLIDModel-EUCLIDModel-HARBININSTITUTEOF[1]YuanetalEUCLID:TowardsEfficientUnsupervisedReinforcementLearningwithMulti-choiceDynamicsModel.2023Grasp-Q-NetworkModel-Grasp-Q-NetworkModel-HARBININSTITUTEOF[1]JoshietalRoboticGraspingusingDeepReinforcementLearning.2020RESEARCHCENTERFORSOCIALCOMPUTINGANDINFORMATION

文本大模型（只拆解任务文本大模型（只拆解任务 HARBININSTITUTEHARBININSTITUTEOF

动作（一般就是轨迹

动作（一般就是轨迹HARBINHARBININSTITUTEOF当前，像Octo、RT-X和OpenVLA等模型已经在高质量数据集OpenXEmbodiedemnt的基础上HARBINHARBININSTITUTEOFHARBINHARBININSTITUTEOFL1L2HARBINHARBININSTITUTEOFHARBININSTITUTEOF数据规模减少数据多样性降低HARBINHARBININSTITUTEOF

HARBINHARBININSTITUTEOFHARBININSTITUTEOF

HARBINHARBININSTITUTEOF研究表明，使用OpenXEmbodiment数据集训练的模型如RT-X、Octo和OpenVLA，与先前模型Padalkaretal.Openx-embodiment:Roboticlearningdatasetsandrt-xmodels.2023Teametal.Octo:Anopen-sourcegeneralistrobotpolicy.2024Kimetal.OpenVLA:AnOpen-SourceVision-Language-ActionModel.2024HARBINHARBININSTITUTEOFDaietal.Graspnerf:Multiview-based6-dofgraspdetectionfortransparentandspecularobjectsusinggeneralizablenerf.2023IEEEInternationalConferenceonRoboticsandAutomationHARBINHARBININSTITUTEOFWangetal.Robogen:Towardsunleashinginfinitedataforautomatedrobotlearningviagenerativesimulation.2023HARBINHARBININSTITUTEOFShawetal.Videodex:Learningd