浅谈人工智能下一个十年

1、人工智能的第三次浪潮2人工智能历史3人工智能领域发展趋势Powered byClaude ShannonShannon, Claude E. XXII. Programming a computer for playing chess. Philosophical magazine 41.314 (1950): 256-275.1950计算机象棋博弈1954图灵测试Alan TuringTuring, Alan M. Solvable and unsolvable problems. Science News-ens. fr 39 (1954).4人工智能领域发展趋势Powered by195

2、6达特茅斯会议McCarthy, J., et al. Dartmouth Conference. Dartmouth Summer Research Conference on Artificial Intelligence. 19561959一般问题解决器JohnMarvin McCarthy MinskyNathan RochesterClaudeShannonHerbertJ.C. ShawAllen NewellSimonNewell, A.; Shaw, J.C.; Simon, H.A. (1959). Report on a general problem-solving pr

3、ogram. Proceedings of the International Conference on Information Processing. pp. 256264.5人工智能领域发展趋势Powered byDaniel BobrowBobrow, Daniel G.Natural language input for a computer problem solving system. (1964)1964 理解自然语言输入1966 ELIZA人机对话Joseph Weizenbaum Weizenbaum, Joseph. ELIZAa computerprogram for

4、the study of natural language communication between man and machine. Communications of the ACM 9.1 (1966): 36-45.6人工智能领域发展趋势owered by1968 世界首个专家系P 统 DENDRALEdward FeigenbaumBuchanan, Bruce, Georgia Sutherland, and Edward A. Feigenbaum. Heuristic DENDRAL: a program for generating explanatory hypothes

5、es in organic chemistry. Defense Technical Information Center, 1968.7人工智能领域发展趋势Powered byRandall DavisDrew McDermott, Jon DoyleMcDermott D, Doyle J. Non-monotonic logic IJ. Artificial intelligence, 1980, 13(1): 41-72.1976 大规模知识库构建与维护Applications of meta level knowledge to the construction, maintenan

6、ce and use of large knowledge basesM. Stanford University, Computer Science Department, AI Laboratory, 1976.1980 非单调逻辑8人工智能领域发展趋势Powered by1980 计算机战胜双陆棋世界冠军Hans BerlinerBerliner H J. Backgammon computer program beats world championJ. Artificial Intelligence, 1980, 14(2): 205-220.9人工智能领域发展趋势owered by

7、1987 基于行为的机器P人学Rodney BrooksBrooks R. A robust layered control system for a mobile robotJ.Robotics and Automation, IEEEJournal of, 1986, 2(1): 14-2310人工智能领域发展趋势owered by1987 自我学习双陆棋P程序Gerry TesauroTesauro G. TD-Gammon, a self-teaching backgammon program, achieves master-level playJ. Neural computati

8、on, 1994, 6(2): 215-219.11人工智能领域发展趋势Powered byTim Berners-LeeBerners-Lee, Tim. Semantic web road map. (1998).McGuinness, Deborah L., and Frank Van Harmelen. OWL web ontology language overview. W3C recommendation 10.2004-03 (2004): 10.1998 语义互联网路线图2004 OWL语言12人工智能领域发展趋势Powered byGeoffrey HintonLe, Qu

9、oc V., et al. Building high- level features using large scale unsupervised learning. arXiv preprint arXiv:1112.6209 (2011).2006 深度学习Hinton, Geoffrey E., Simon Osindero, and Yee-Whye Teh. A fast learning algorithm fordeep belief nets. Neural computation 18.7(2006): 1527-1554.2011 高层抽象特征构建13人工智能领域发展趋势

10、Powered by2009 谷歌自动驾驶汽车Sebastian ThrunMarkoff, John. Google cars drive themselves, in traffic. The New York Times 10 (2010): A1.14人工智能领域发展趋势Powered by2011 沃森获得Jeopardy冠军IBMs WatsonMarkoff, John. Computer program to take on Jeopardy!. The New York Times (2009).2011 自然语言问答Apples Siri Sadun, Erica, and

11、 Steve Sande. Talking to Siri: Learning the Language of Apples Intelligent Assistant. Que Publishing, 2013.15人工智能近10 年17AI趋势：从感知到认知From perceptron to cognitionComputingPerceptionCognitionStorage & ComputingRecognize text, images, objects, voicesOrganize and generate knowledge, reasoning18Artificial

12、IntelligenceAlphaGoImage recognitionSelf-driving1920DDPG(2015)A3C(2016)Perceptron(1958)Frank Rosenblatt Cornell University psychologistBPNN/MLP(1986)Hopfield Network(1982) recurrent & feedbackGeoffery Hinton University of Toronto deep learningNeocognitron(1980) convolution & poolingLeNet/CNN(1998)Ya

13、nn LecunNew York University deep learningAlexNet(2012)Relu, dropout & biggerVGG(2014)GoogLeNet(2015)ResNet(2016)Kaiming He MSRA = FAIRcomputer visionDenseNet(2017)RBM(1986/2006)stackDeep Belief Nets(2006)VAE(2013)AutoEncoder(1989/2006) Denosing Autoencoder(2008)Variational InferenceMax WellingUniver

14、sity of Amsterdam statistical learningGAN(2014)DCGAN(2014) WGAN(2017) PGGAN(2017)Ian Goodfellow Google Braindeep adversarial learningRNN/LSTM(1997)Jrgen SchmidhuberIDSIAUniversal AISeq2Seq(2014)RNN in Speech Recognition(2013)Yoshua Bengio University of Montreal Deep learningNeural Probabilistic Lang

15、uage Model(2003)word2Vec(2013)SeqGAN(2017) LeakGAN(2018)Character CNN(2015) self-attention(2017)Deep Q- learning(2013)AlphaGo(2016) Double DQN(2015)Dueling Net(2016)David Silver DeepMindReinforcement learningAlpha Zero(2017) Capsule Nets(2017)算法BERTPre-trainFine tuneBeat all state-of-the-arts on 11

16、NLP tasks in 2018/pdf/1810.04805.pdf21XLNetAutoregressive ModelBeat BERT in 2019/pdf/1810.04805.pdf22ALBERTA Lite BERTParameter-reduction techniquesBeat XLNet and all the others/pdf/1810.04805.pdf23Video-to-Video SynthesisThe best video synthesis performance/abs/1808.0660124graph_netBy DeepMind/abs/

17、1808.0660125MoCoUnsupervised visual representation learningMomentum contrastive learningOutperform its supervised pre-training counterparts/abs/1911.057222627SimCLRSimplified contrastive learning frameworkOutperform previous self-supervised and semi- supervised methods on ImageNet/abs/2002.05709人工智能

18、未来28第三代人工智能的理论体系早在2015年，张钹老师就提出第三代人工智能体系的雏形； 2017年DARPA发起XAI项目，从可解释的机器学习系统、人机 交互技术以及可解释的心理学理论三个方面，全面开展可解释性 AI系统的研究2018年底，正式公开提出第三代人工智能的理论框架体系建立可解释、鲁棒性的人工智能理论和方法发展安全、可靠、可信及可扩展的人工智能技术推动人工智能创新应用具体实施路线图与脑科学融合，发展脑启发的人工智能理论数据与知识融合的人工智能理论与方法第三代人工智能的理念在国内外 获得广泛影响力29认知图谱 (Cognitive Graph)知识图谱, 认知推理, 逻辑表达30知识

19、图谱“Knowledge graph”由Google于2012年提出知识工程，专家系统CYC: 世界上历史最长的AI项目 (1985)Tim Berners Lee Father of WWW Turing AwardEdward Feigenbaum Father of KB Turing Award31认知图谱：算法与认知的结合Quality CafThe Quality Cafe is a now-defunct diner in Los Angeles, California. The restaurant has appeared as a location featured in

20、a number of Hollywood films, including Old School, Gone in 60 Seconds, .Los AngelesLos Angeles is the most populous city in California, the second most populous city in the United States, after New York City, and the third most populous city in North America.Old SchoolOld School is a 2003 American c

21、omedy film released by DreamWorks Pictures andThe Montecito Picture Company and directed by Todd Phillips.Todd PhillipsTodd Phillips is an American director, producer, screenwriter, and actor. He is best known for writing and directing films, including Road Trip (2000), Old School (2003), Starsky &

22、Hutch (2004), and The Hangover Trilogy.Alessandro Moschitti is a professor of the CS Department of the University of Trento, Italy. He is currently a Principal Research Scientist of the Qatar Computing Research Institute (QCRI)Alessandro MoschittiTsinghua UniversityTsinghua University is a major res

23、earch university in Beijing and dedicated to academic excellence and global development.Tsinghua is perenniallyranked as one of the topacademic institutions inChina, Asia, and worldwide.32算法: BIDAF, BERT, XLNet目标：理解整个文档，而不仅仅是局部片段但仍然缺乏在知识层面上的推理能力BiDAFBERTXLNet33挑战 : 可解释性34大部分阅读理解方法都只能看做黑盒:输入: 问题和文档输出

24、: 答案文本块 (在文档中的起止位置)如何让用户可以验证答案的对错:推理路径或者子图每个推理节点上的支撑事实用于对比的其他可能答案和推理路径36和认知科学的结合Dual Process Theory (Cognitive Science)System 1 IntuitiveSystem 2 Analytic371. From Bengios NIPS2019 KeynoteReasoning w/ Cognitive GraphSystem 1:Knowledge expansion by association in text when readingSystem 2:Decision ma

25、king w/ all the informationSystem 1IntuitiveSystem 2Analytic38CogQA: Cognitive Graph for QAAn iterative framework corresponding to dual process theorySystem 1extract entities to build the cognitive graphgenerate semantic vectors for each nodeSystem 2Do reasoning based on semantic vectors and graphFe

26、ed clues to System 1 to extract next-hop entitiesQuestionQuality cafTodd PhillipsGone in 60 secondsOld schoolLos AngelesDominic SenaSystem 2Cognitive Graphlocation featured in a number ofSystem 1inputHollywood films, icncluluedsing Old School,Gone in 60 Secondspredict39Cognitive Graph: DL + Dual Pro

27、cess Theory?System 1: implicit knowledge expansion1. M. Ding, C. Zhou, Q. Chen, H. Yang, and J. Tang. Cognitive Graph for Multi-Hop Reading Comprehension at Scale. ACL19.40System 2: explicit decisionCognitive Graph: DL + Dual Process TheorySystem 1: implicit knowledge expansion1. M. Ding, C. Zhou, Q

28、. Chen, H. Yang, and J. Tang. Cognitive Graph for Multi-Hop Reading Comprehension at Scale. ACL19.41System 2: explicit decision42Cognitive Graph: Representation,Reasoning, and Decision?43认知与推理Trillion-scale common-sense knowledge graphTim Berners Lee Turing Award WinnerEdward Feigenbaum Turing Award

29、 Winner* AI = Knowledge + Intelligence1. J. Tang, J. Zhang, L. Yao, J. Li, L. Zhang, and Z. Su. ArnetMiner: Extraction and Mining of Academic Social Networks. KDD08. pp.990-998.Big DataKnowledgeIntelligenceRelated Publications44Ming Ding, Chang Zhou, Qibin Chen, Hongxia Yang, and Jie Tang. Cognitive

30、 Graph for Multi-Hop Reading Comprehensionat Scale. ACL19.Jie Zhang, Yuxiao Dong, Yan Wang, Jie Tang, and Ming Ding. ProNE: Fast and Scalable Network RepresentationLearning. IJCAI19.Yukuo Cen, Xu Zou, Jianwei Zhang, Hongxia Yang, Jingren Zhou and Jie Tang. Representation Learning for AttributedMulti

31、plex Heterogeneous Network. KDD19.Fanjin Zhang, Xiao Liu, Jie Tang, Yuxiao Dong, Peiran Yao, Jie Zhang, Xiaotao Gu, Yan Wang, Bin Shao, Rui Li, and Kuansan Wang. OAG: Toward Linking Large-scale Heterogeneous Entity Graphs. KDD19.Qibin Chen, Junyang Lin, Yichang Zhang, Hongxia Yang, Jingren Zhou and Jie Tang. Towards Knowledge-BasedPersonalized Product Description Generation in E-commerce. KDD19.Yifeng Zhao, Xiangwei Wang,