硕士学位论文-基于多模态语料库的手势隐喻研究.doc

分类号 密级硕士学位论文题目：基于多模态语料库的手势隐喻研究英文并列题目：A Self-compiled Multimodal Corpus-basedStudy on Gestural Metaphors研究生：专业：英语语言文学研究方向：语言学导师：指导小组成员：学位授予日期：江 南 大 学地址：无锡市蠡湖大道1800号二一九二一九年十一月A Self-compiled Multimodal Corpus-based Study on Gestural MetaphorsZhongYuanSubmitted in Partial Fulfillment of the RequirementsFor the degree of Master of ArtsSupervised by Yan MinfenSchool of Foreign StudiesJiangnan UniversityWuxi, Jiangsu Month 6独 创 性 声 明本人声明所呈交的学位论文是本人在导师指导下进行的研究工作及取得的研究成果。尽我所知，除了文中特别加以标注和致谢的地方外，论文中不包含其他人已经发表或撰写过的研究成果，也不包含本人为获得江南大学或其它教育机构的学位或证书而使用过的材料。与我一同工作的同志对本研究所做的任何贡献均已在论文中作了明确的说明并表示谢意。签名：日期：关于论文使用授权的说明本学位论文作者完全了解江南大学有关保留、使用学位论文的规定：江南大学有权保留并向国家有关部门或机构送交论文的复印件和磁盘，允许论文被查阅和借阅，可以将学位论文的全部或部分内容编入有关数据库进行检索，可以采用影印、缩印或扫描等复制手段保存、汇编学位论文，并且本人电子文档的内容和纸质论文的内容相一致。保密的学位论文在解密后也遵守此规定。签名：导师签名：日期：ACKNOWLEDGEMENTSMy deepest gratitude goes first and foremost to Professor Yan Minfen, my supervisor, for her constant encouragement and guidance. She has walked me through all the stages of the writing of this thesis. Without her consistent and illuminating instruction, this thesis could not have reached its present form. Second, I would like to express my heartfelt gratitude to Professor Li Jianxue, as well as other professors and teachers in School of foreign languages who have offered the enlightening academic courses, warm encouragement and valued suggestions for me during my course of my study for the M.A. Degree. Last, my thanks would go to my beloved family for their loving considerations and great confidence in me all through these years. I also owe my sincere gratitude to my friends and my fellow classmates who gave me their help and time in listening to me and helping me work out my problems during the difficult course of the thesis.摘 要自从Lakoff & Johnson(1980)的划时代巨著我们赖以生存的隐喻出版以来，隐喻就不再单纯的认为只与语言有关，而是一种思维与行为方式，即思维隐喻的概念隐喻理论（CMT）。语言符号系统，除了有声语言和文字语言之外，还有其它非语言符号系统，如图像、声音、手势等。这些非语言符号系统也可以用来构建和表达概念，因而也具有隐喻性。图像、声音、手势是语言的视觉、听觉模态表征，其隐喻性涉及多种感官模态的调用。Forceville（2006）等学者将纯语言领域的概念隐喻研究拓展到跨学科的多模态研究平台，提出了反映图像、声音、手势隐喻的多模态隐喻理论。手势是指人类用语言中枢建立起来的一套用手掌和手指位置、形状的特定语言系统。政治演说是公众演说的一种重要形式，它是针对国家的内政事务和外交关系表明立场，阐明观点，宣传主张的一种演说，在其政治生活中扮演着极其重要的角色。政治演讲中手势语具有隐喻性。随着计算机技术的发展，多模态语料库技术日趋成熟，这也为多模态隐喻研究提供了契机。为了验证上述假设，本文自建了多模态隐喻语料库。语料库来源选取了奥巴马两次就职演说及四篇国会演讲，根据McNeill (1992)关于手势的分类为框架，以多模态视频分析软件ELAN为数据收集和分析工具，重点回答以下三个问题：1)奥巴马演说中手势语的类别及其频率分布是什么? 2)奥巴马演说中隐喻性手势的类别有哪些？其分类有力的论证了什么? 3)演讲-手势语类中多模态意义构建的机制及其功能特点是什么?本文采用多媒体语料标注工具ELAN作定量统计，并进行定性分析。研究发现：1) 奥巴马演说中，手势语的类别分别是：节拍性手势(beats)(69%），隐喻性手势(metaphorics)(12%)，指示性手势(deitics)(11%)，提示性手势(delicates)(7%)，图示性手势(iconics)(1%)。隐喻性手势在五类手势中(节拍性手势 69%, 隐喻性手势 12%, 指示性手势11%, 提示性手势 7%, 图示性手势1% )所占的比例不大, 但其它类型的手势或强调目标词(节拍性和提示性手势), 或指代具体概念(图示性手势), 指示性手势虽兼具指代具体与抽象概念的功能, 但是却不包含跨域映射。与其他类型的手势相比，隐喻性手势具有呈现抽象概念，实现跨域映射的功能, 因此在演讲-手势语类中具备不可或缺的作用, 值得进一步研究。2）奥巴马演说中隐喻性手势的使用类型根据其相伴随的话语是否具有隐喻性可以分为“隐喻手势+隐喻性言语”(84%)、“隐喻手势+非隐喻性言语”(16%)两种方式，前者手势的隐喻又可分为本体性手势隐喻(62%)、方位性手势隐喻(34%)、转喻性手势隐喻(4%)。后者则可分为时间手势隐喻(82%)，本体性手势隐喻(10%)和转喻性手势隐喻(8%), 。前一种方式证明了言语隐喻和手势隐喻在人类的认知系统中具有同一性, 而后一种方式则证明了隐喻思维可以通过手势进行独立表达,有力的回应了反对派对概念隐喻理论囿于语言层面循环论证的质疑。3）演说中多模态意义构建主要是通过隐喻与转喻的互动实现的，多模态意义的构建方式主要有以下三种：转喻为隐喻提供基础(73%)；转喻占主导地位的构建方式(4%)和多模态隐喻与转喻互动的动态性和层次性(23%)。三种不同的互动形式下目标词的使用呈现出一定的特点: 前两种作用方式中手势可呈现出静态和动态的双重特征，其中的动态手势目的在于完成源域到目标域的指称功能或是构建两者之间的相似关系，而第三种作用方式中的动态手势则为实现从转喻到隐喻的层层推进；第一种方式作用下的隐喻性手势所涵盖的目标词词性(N, V, VP, Adj, Prep, Adv)较第二种方式和第三种方式所涵盖的目标词性广, 说明该模式为多模态意义构建的主要模式，这三种方式作用下的隐喻性手势同动词、动词词组和名词的搭配最多(76%),显示了手势隐喻在展示说话者即时注意力焦点变化以及“在整个概念化过程激活对应侧面，为语言加工中焦点的转移提供线索”(Langacker, 2008: 249)的重要作用。隐喻性手势在演说中的功能可以从演讲者和受众两个角度去分析：对于演讲者(addresser/gesturer)来说，可以通过外化抽象概念使意义表达更加清楚明了，实现演说的劝导功能，同观众形成共鸣。对于受众(addressee/hearer)来说，视觉-听觉两种模态的双重呈现有助于调动听众以往的经验，从而促进从源域手势到目标域演说词的跨域映射，最终达成对演说内容多模态意义的理解。本文的研究成果将从理论上进一步印证并发展概念隐喻、尤其是多模态隐喻理论。从实践方面来说，本文的研究成果将对中国未来的语言教学提供相关启示：教师不仅要注重学生言语隐喻能力的提高，而且要将学生的多模态隐喻能力作为评判学生隐喻能力高低的指标之一，这不仅有利于学生“多模态识读能力”(Hu Zhuanglin, 2007: 7)的提高，而且也有助于实现更加高效的交流与沟通。 关键词：手势语；政治演讲；多模态隐喻；类型；特点；功能iiiABSTRACTSince the publication of the epoch-making book “Metaphors we live by” written by Lakoff and Johnson(1980), metaphor is not primarily thought as a matter of language, but structures thought and action, i.e., the nature of human thinking is metaphorical. Apparently, metaphorists considering themselves adherents of the Conceptual Metaphor Theory(CMT), if the conceptual nature claim of metaphor is correct, evidences should go beyond other non-verbal modes, such as pictures, sounds, gestures and so on, thus led to the upsurge of multimodal metaphor pioneered by Charles Forceville(2006).With the development of computer technology, the technology of multimodal corpus is becoming increasingly mature, in this thesis, we adopt the method of self-compiled corpus analysis, we select president Obamas two inaugural addresses in 2008 and 2013 respectively, as well as four national addresses from 2010 to 2013. This paper mainly studies the following research questions in detail: 1) What are the ratios and characteristics of gestures used in the speech-gesture synchronizations?2) What are the types of metaphorical gestures according to the target sentences they accompany? And what are the implications according to the classifications?3) What are the inner mechanisms and characteristics of multimodal meaning constructions in the speeches? And what is the function of metaphorics in speech-gesture synchronization?The thesis met these research aims by an extensive study of relevant literature and the implementation of quantitative study with the help of multimedia annotation software ELAN and qualitative research afterwards. Through detailed and scientific research, the research findings are presented as following: 1) The proportion of metaphorics (12%) in the speeches doesnt rank as the highest in the total five types of gestures (beats, 69%; metaphorics, 12%; deitics, 11%; indicates, 7%; iconics, 1%), but the functions of these four types of gestures are either stress target words (beats and indicates), or describe concrete entities (iconics), detics can present both concrete and abstract entities, but they do not include cross-domain mappings. So metaphorics overweigh others in that it can present abstract entities and also it involves cross-domain mappings, so it needs further research. 2) Metaphorics in the speeches can be divided into “metaphorics + metaphorical utterances” (84%) and metaphorics + non-metaphorcal utterances (16%) according to the metaphoricity of the target sentences. Metaphorics in the former type can be further divided into “ontological metaphorics” (62%), “orientational metaphor” (34%) and metonymics (4%) which proves that verbal metaphor and gestural metaphor are identical in mechanism in humans cognitive system, in the latter type, metaphorics can be divided into “temporal metaphorics” (82%), “ontological metaphorics”(10%) and metonymics (8%), it proves that humans metaphorial thought can be presented by gestural modes alone.3) According to the qualitative analysis of the annotations, we find out there are three interactive patterns in multimodal meaning making patterns: 1) “Metonymy within metaphor” multimodal meaning making process (73%), 2)“Metonymy-prominent” multimodal meaning making process (4%), 3) “A-ING IS B-ING” multimodal meaning making process (23%). Through the classifications and analysis of the three patterns of multimodal meaning construction, we also find out: the metaphorical gestures of the first two interactive can be both static and dynamic, but different from the dynamic nature of the third interactive patterns, in which the metonymic mapping process is motivated level upon level by metaphorical mapping process, dynamic metaphorics in the first two interactive patterns only aim to fulfill its referential function or form a similarity function between the source domain and target domain; the lexical categories of the first interactive pattern (N, V, VP, Adj, Prep, Adv) is much more comprehensive than that of the other two patterns, which shows it is the most frequently occur interactive pattern in multimodal meaning construction; such lexical categories as verbs, verbal phrases and nouns are most frequently occur words that accompany metaphorics, which shows the function of metaphorics in highlighting the focus of linguistic processing.(Langacker, 2008: 249)The functions of metaphorics can be viewed from both the gesturer/addresser and the addressee/hearer. Through the perspective of the gesturer, they can raise consonance among the audiences and make their meanings fully expressed through the exbodiment of concept ideas. From the perspectives of the latter, they can get a better understanding of the multimodal meanings in speech-gesture synchronization through the duplication of both the visual and verbal modes.As to the above mentioned research findings, in the first place theoretically, they will prove and further develop the present conceptual metaphor as well as multimodal metaphor, and also practically, they will provide implications for language teachings in China, that is more attention should be paid to the improvement of students multimodal metaphor abilities, which is helpful for the enhancement in “multiliteracy” (Hu Zhuanglin, 2007: 7) as well as the effectiveness in communication.Key words: political speech; multimodal metaphor; metaphorics; functionTABLE OF CONTENTSACKNOWLEDGEMENTSii摘 要iABSTRACTivTABLE OF CONTENTSviiLIST OF TABLESxLIST OF FIGURESxiChapter One INTRODUCTION11.1 Background of the Research11.2 Research Focus and Objectives3 1.2.1 Focus of the Study错误！未定义书签。 1.2.2 Significance of the Study错误！未定义书签。1.3 Layout of the Thesis5Chapter Two LITERATURE REVIEW。72.1 Survey of Multimodal Metaphor7 2.1.1 Conceptual Metaphor7 2.1.2 Visual Metaphor9 2.1.3 An Overall Review of the Study of Multimodal Metaphor112.2 Gestural Study12 2.2.1 Overview of Gesture Studies12 2.2.2 Gesture and Verbal Discourse16 2.2.3 Functional Roles of Gestures17 2.2.4 Metaphor and Gesture182.3 Summary21Chapter Three RESERACH METHODOLOGY233.1 Research Questions233.2 The Identification of Metaphoric Gestures233.3 Data and Data Collection26 3.3.1 The Genre of Political Speeches26 3.3.2 Self-complied Corpus273.4 Instruments and Research procedure31 3.4.1 Instruments31 3.4.2 The Research Procedure32Chapter Four MULTIMODAL ANALYSIS OF POLITICAL SPEECHES364.1 The Distribution and Features of Gestures in Obamas Speeches36 4.1.1 The Distribution of Gestures36 4.1.2 The General Features of Gestural Types394.2 Multimodal Analysis of Verbal Speeches and Metaphorical Gestures45 4.2.1 Metaphorical Gestures Plus Metaphorical Utterances45 4.2.2 Metaphorical Gestures plus Non-metaphorical Utterances52 4.2.3 Summary544.3 Multimodal Meaning Construction in Speech-Gesture Synchronization55 4.3.1 The Interactive Patterns of Metaphor and Metonymy in Co-Speech Metaphorical Gestures56 4.3.2 Characteristics of the Interactive Patterns in Metaphorical Gestures59 4.3.3 Summary664.4 Functions of Metaphorical Gestures in Speech-Gesture Synchronization67 4.4.1 The Speakers/Gesturers Point of View67 4.4.2 The Hearer/Addressers Point of View674.5 Summary68Chapter Five CONCLUSION725.1 Major Findings of the Research725.2 Implications of the Study745.3 The Present Studys Limitations and the Suggestions for Further Research76REFERENCES78附录： 作者在攻读硕士学位期间发表的论文99LIST OF TABLESTable 2.1 Correspondence Across Two Domains.8Table 2.2 Typology of pictorial metaphors.10Table 2.3 Gestures Relationship to Speech.12Table 4.1 Distributions of Different Types of Gestures.40Table 4.2 Frequencies of Different Types of Metaphorical Gestures.50Table 4.3 The Characteristics of the Two Interactive Patterns.61Table 4.4 Classification of Metaphorical Gestures.65Table 4.5 The Characteristics of Interactive Pattern I.66.Table 4.6 The Characteristics of Interactive Pattern.67 xiiLIST OF FIGURES Figure 3.1 The White House Website Page.24 Figure 3.2 A list of Media Files in the Selected Directory.26 Figure 3.3 The Selection of the Corpus to be Analyzed.26 Figure 3.4 The Initializing Elan Window.27 Figure 3.5 Annotation Process of ELAN 4.6.1 .28 Figure 3.6 Add New Types and Tiers in ELAN 4.6.1.29 Figure 3.7 Annotation Viewer.30 Figure 3.8 Annotation StatisticsViewer. .31 Figure 4.1 The Distribution of Gestures used in Obamas Speeches. .34 Figure 4.2 Beats of Words. .36 Figure 4.3 Beats of Parallelism.36 Figure 4.4 Frequency of Deitics Types and Their Referents.37 Figure 4.5 Frequency of Different Types of Indicates.38 Figure 4.6 CUTS IS OBJECT. .43 Figure 4.7 .We will extend a hand if you are willing to clench your fist.44 Figure 4.8 HIGH STATUS IS UP.45. Figure 4.9 “Future is in Right and Past is in Left”.48 Figure 4.10 Multimodal Mapping Process in the “Path” Metaphor.51 Figure 4.11 Cross-Domain Mapping Process of the “Pointing finger” Metaphor.52 Figure 4.12 General Cross-domain Mapping Process in Metaphorical Gestures.54 Figure 4.13 Frequency of the Interactive Patterns in Metaphorical gestures.55 Figure 4.14 Frequencies of Different Properties of Referents in Interactive Pattern.59 Figure 4.15 Frequencies of Different Properties of Referents in Interactive Pattern .61 Chapter One INTRODUCTIONChapter One INTRODUCTION1.1 Background of the ResearchMetaphor is pervasive in everyday life; it has long been viewed as a device of poetic imagination and the rhetoric flourish, ie, characteristics of languages alone. Since the publication of Lakoff & Johns influential book Metaphors We Live By(1980) which claims that “our ordinary conceptual system, in terms of which we both think and act, is fundamentally metaphorical in nature,”(Lakoff & Johnson, 1980: 4) thus conceptual metaphor theory is brought into being, studies on metaphor concerning on its nature, working mechanism, and its relations to culture, experience, and human thought have brought a metaphormania in 1970s in the western academic.More than two decades later, Forceville(2006b: 379)claims that:“if this tenet of CMT is correct, metaphor should manifest itself not just in language but also via other modes of communication, such as pictures, music, sounds and gestures”, thus spring up the study of multimodal metaphor, the representative book in this field is Forceville & Urios- Aparisis Multimodal metaphor(2009). Which is a collection of thesis concerning the roles of different modes play in the identification and interpretation of metaphors studied, such as: advertisements, political cartoons, comics, animation, music compositions, oral conversations, lectures, and feature films, etc. The study of multimodal metaphor abroad mainly focuses on four dimensions(Zhao xiufeng 2011: 4) : 1) the characteristics of genres, including advertisements (Forceville 1996; Caballero 2009) and cartoons (Yus 2009; Teng & Sun 2002) ; 2) the study of distinctive featur