（外国语言学及应用语言学专业论文）完形填空用于词汇测试的探索性研究.pdf

shanghai jiao tong university m.a. thesis 3 acknowledgements my sincere gratitude first goes to professor wu jiang, my supervisor, whose patient and warm guidance has always been invaluable in the writing of this thesis paper. from planning the thesis to its fulfillment, i benefited enormously from his suggestions, comments and corrections. his inspiration and encouragement also proved equally valuable when i came across difficulties in the process. without his painstaking efforts in revising and polishing my draft, the completion of the present paper would not have been possible. my sincere thanks also go to all professors who have taught me during the ma courses in the school of foreign languages of shanghai jiao tong university. finally, id like to extend my deep gratitude to my beloved parents and dear classmates for the love, encouragement, and understanding they have always provided me during the past two and a half years post graduate study. shanghai jiao tong university m.a. thesis 4 an exploratory study on the effectiveness of cloze procedure for measuring vocabulary in esl proficiency test abstract cloze procedure, a test format familiar to chinese english learners, has been widely used in a variety of language proficiency tests, such as the cet test. it originates from the theories of gestalt psychology that humans have the tendency to view phenomena as whole and thus to complete or close incomplete patterns. for a long time there have been discussions among the language testers on what cloze procedure measures. some researches show a high correlation between cloze test and vocabulary test. therefore, this study aims at exploring the feasibility of cloze procedure used as a vocabulary measure. the thesis consists of five chapters. chapter 1 gives a brief introduction to the study, including the rationale and aims of the research as well as the outline of the thesis. chapter 2 reviews the literature of theoretical and empirical research on language testing, vocabulary assessment and cloze procedure. chapter 3 and chapter 4 are the main body of the thesis, justifying the validity of cloze procedure for measuring vocabulary. a total of 131 second-year college students completed a vocabulary test with three different formats including mc cloze, mc sentence completion, and word translation. both quantitative and qualitative studies were made for the research. for quantitative study, descriptive analysis was conducted, including shanghai jiao tong university m.a. thesis 5 item analysis and correlation analysis; for qualitative study, a questionnaire was made to collect responses from the subjects on the test paper. the final chapter draws conclusions and makes recommendations for further study. the results of the experiment show that compared with the traditional vocabulary test, the cloze procedure not only measures test-takers knowledge of vocabulary, but also their communicative lexical ability. because of its fairly satisfactory validity and practicality, the cloze procedure may possibly become a good choice of test format for vocabulary tests in the administration of a large-scale english proficiency test. the responses from the subjects in the qualitative study indicate that the cloze procedure used for vocabulary test will bring some beneficial backwash effects on vocabulary learning and teaching. key words: cloze procedure, vocabulary assessment, gestalt psychology, validity, communicative lexical ability shanghai jiao tong university m.a. thesis 6 完形填空用于词汇测试的探索性研究 摘要 完形填空用于词汇测试的探索性研究 摘要 完形填空是中国英语学习者所熟悉的一种题型被广泛应用于各种大型考 试如大学英语四级考试完形的概念来自于格式塔心理学此学派认为人们 都有一种自然的心理倾向来填充某一图式中的空缺对于完形填空究竟测试了 学习者何种语言能力一直是语言测试领域引起争论的问题研究表明完形 填空考试成绩和词汇考试成绩有很高的相关性为此作者提出了用完形填空 来测试词汇的设想作者认为与传统的词汇测试相比完形填空通过提供篇 章这一大的语境不但测试出被试者对词汇知识的掌握更测试出被试者在语 境中实际运用词汇的能力 全文由五章组成第一章是绪论从整体上对本文要研究的问题进行了介 绍其中包括研究目的和对本文结构的解释第二章是对语言测试词汇测试 和完形填空测试理论的一个综述并提出了用完形填空来测试词汇的理论依据 第三章第四章是全文的主体作者通过实验研究探讨了完形填空用于词汇 测试的效度一共有 131 名大二的学生参加了由作者设计的词汇测试该试卷 包括多项选择完形填空单句词汇测试和单词翻译三种题型作者对该词汇测 试的成绩进行了定量研究包括描述分析题项分析和相关分析并且采用问 卷调查的形式对该词汇测试进行了定性研究实验结果证明完形填空作为一种 综合性题型在测试词汇上具有较高的效度可以测试出被试者交际词汇能力 即在语境中运用词汇的能力第五章总结了研究发现, 并且提出了对本实验研 究改进的建议 在研究的基础上作者认为如果完形填空用于词汇测试将对词汇学习和 教学产生良好的后效对学习者而言他们会通过阅读在语境中习得词汇教 师在教授词汇时也会注重词汇在不同语境中的用法 关键词关键词完形填空词汇测试格式塔心理学效度交际词汇能力 shanghai jiao tong university m.a. thesis 9 list of figures and tables figure 2-1 three dimensions of vocabulary assessment.11 figure 4-1 histogram of the mc cloze test.29 figure 4-2 histogram of the mc sentence completion test.29 figure 4-3 histogram of the word translation test.30 figure 4-4 scattergram of scores of mc cloze and sentence completion test.34 table 4-1 descriptive statistics of the scores of each test.28 table 4-2 facility value of each item in three tests.31 table 4-3 correlations between the mc cloze, mc sentence completion and word translation test.33 table 4-4 correlations between the mc cloze test and components of cet-6.36 shanghai jiao tong university m.a. thesis 10 chapter 1 introduction 1.1 rationale of the study cloze procedure has been used in language testing with a long history, and is familiar to chinese english learners. as one of the most widely used testing devices, cloze procedure is adopted in a variety of language proficiency tests, including several large-scale standardized language proficiency tests in china, such as cet (college english test), tem (test for english major) and pets (public english test system). the cloze procedure, developed by wilson taylor in 1953, is based on one of the theories of gestalt psychology: namely, our tendency to view phenomena as whole and thus to complete or close incomplete patterns. according to taylor (1953), the essence of the cloze test is its reliance on the human tendency to complete a familiar but not-quite-finished patternto “see” a broken circle as a whole one, for example, by mentally closing up the gaps. he further pointed out that “a cloze unit may be defined as any single occurrence of a successful attempt to reproduce accurately a part deleted from a massage (any language product), by deciding from the context that remains what the missing part should be”. therefore, cloze procedure has been valued for testing and teaching purposes in that it is believed to call into action semantic, syntactic and pragmatic knowledge of a language in an integrative way that approximates to natural language processes (oller, 1973). however, can cloze procedure be used exclusively to measure vocabulary? it seems that cloze procedure has no direct relationship with vocabulary assessment. in a cloze procedure, every nth word is deleted from a selected reading passage. students are asked to read the passage, inserting a word into each blank to create a meaningful reading. therefore, cloze procedure originated as a diagnostic reading assessment technique. as a typical integrative format, cloze procedure is now claimed to test the overall linguistic proficiency in a second or foreign language. it is true that cloze procedure has never been seen primarily as a vocabulary measure, but presumably test-takers need to draw strongly on their vocabulary knowledge in completing the cloze task. research studies also show that score of cloze tests correlates high with shanghai jiao tong university m.a. thesis 11 that of vocabulary tests. these research findings provide the empirical basis of the present study. as we know, one of the most commonly used formats for vocabulary tests is designed in the way that the test item is based on context of the sentence, in which one word or phrase is deleted as the question area, followed by four options among which, one option is the correct answer and the other three are the distracters. in the present thesis, the author defines such a test format as multiple-choice sentence completion (mc sentence completion). however, under the influence of communicative approaches to language teaching and testing, there is a tendency in esl proficiency test that vocabulary tests are constructed to assess the learners ability to comprehend vocabularies which occur in the whole text. accordingly, in language teaching, learners should be taught to pay more attention to the meanings of the vocabulary in a broad context instead of in an isolated sentence or in isolation. therefore, the sentence-based test items are naturally not an ideal device to assess the learners real vocabulary knowledge, as vocabulary knowledge should be assessed through test-takers performance of integrative tasks that show how well they can draw on all their language knowledge to use the language for various communicative purposes. in line with this new tendency in language testing, the author tries to examine whether the cloze procedure is an effective test method for measuring vocabulary. cloze procedure is a fill-in-the-blanks activity. the learner uses clues from the context to supply words that have been deliberately removed from the text. so in contrast with the traditional vocabulary test, cloze procedure is an integrative type of language tests. to complete the cloze task, learners have to depend on both the knowledge of individual words and the contextual clues to identify which particular word fits the blank. in terms of practicality, cloze procedure seems relatively easy to administer and score, and possibly becomes a good choice of test format for the standardized language proficiency test. hopefully, this thesis can help language learners and teachers have a profound understanding of cloze procedure and evoke the attention and discussion on the effectiveness of cloze procedure for measuring shanghai jiao tong university m.a. thesis 12 vocabulary in esl proficiency test. 1.2 aims of the study assuming that vocabulary knowledge is a fundamental component of language proficiency, we claim that one of the primary purposes of language testing is to assess whether learners have acquired adequate knowledge of vocabulary that enables them to communicate successfully in the second language. the critical issue lies in how to measure vocabulary in a more communicative context. can cloze procedure be used as a good measure of vocabulary? in the thesis, the author attempts to look into the following three research questions: question 1:what language ability is tested in cloze procedure? question 2:is cloze procedure valid for measuring learners knowledge of vocabulary? question 3:what are the merits of context-dependent test of vocabulary in language testing? in general, the aims of the present study are two-fold: 1) to review the theories and practices in cloze procedure and vocabulary assessment so as to construct the theoretical framework which can serve as a basis for the validation of cloze procedure used for measuring vocabulary; 2) to examine the validity of the cloze procedure modified by the author for the research. 1.2 outline of the thesis the thesis consists of five chapters. chapter 1, a preliminary introduction to the study, includes the rationale and aims of the study as well as the outline of the thesis. chapter 2 reviews the literature of theoretical and empirical research on language testing, vocabulary assessment and cloze procedure. the justification of using cloze procedure as a vocabulary measure is also included in this part. chapter 3 describes methodological concerns such as the research design and data collection for the study. chapter 4 presents the interpretation of data analysis, based on both quantitative and shanghai jiao tong university m.a. thesis 13 qualitative studies. the final chapter draws conclusions and makes recommendations for further study, in the hope that eventually a more effective vocabulary test can be developed. shanghai jiao tong university m.a. thesis 14 chapter two literature review 2.1 basic considerations in language testing 2.11 types of language testing not all language tests are of the same kind. they differ with respect to how they are designed, and what they are for: in other words, in respect to test method and test purpose. in terms of test method, distinctions should be made between direct and indirect testing, between discrete point and integrative testing, between norm-referenced and criterion-referenced testing. discrete point testing refers to the testing of one element at a time, item by item. the appearance of discrete point testing is under the influence of psychometric-structuralist approach, which emphasizes the knowledge of language as consisting of mastery of the features of the language as a system. this might involve, for example, a series of items each testing a particular grammatical structure. integrative testing, by contrast, does not seek to separate language skills into neat divisions in order to improve test reliability; instead, they are often designed to assess the learners ability to use two or more skills simultaneously. this might involve, for instance, writing a composition, making notes while listening to a lecture, taking a dictation, or completing a cloze passage (hughes, 1989). language tests also differ according to their purposes. in fact, the same form of test may be used for differing purposes, although in other cases the purpose may also affect the form (mcnamara, 2002). there are four types of language tests: proficiency tests, achievement tests, diagnostic tests and placement tests. the most familiar distinction in terms of test purposes is that between achievement and proficiency tests. generally speaking, achievement tests are associated with the process of instruction. they accumulate evidence during, or at the end of a course of study in order to see whether and where progress has been made in terms of the goals of learning. whereas the achievement test looks back on what should have been learnt, the proficiency test looks forward, defining a students language proficiency with reference to a particular task which he or she will be required to perform. the content of a proficiency test, shanghai jiao tong university m.a. thesis 15 therefore, is not based on the content or objectives of language courses which students taking the test may have followed. rather, it is based on a specification of what students have to be able to do in the language in order to be considered proficient (hughes, 1989). 2.12 qualities of language testing for testers, developing valid and reliable tests, which have beneficial backwash effects on teaching and learning, and which are also practical is what they pursue most. in other words, validity, reliability, practicality and beneficial backwash effects are four desirable qualities that determine the overall usefulness of a given test. to construct a good language test is by no means easy. there are various issues that testers need to consider, beginning from the content of the test specifications to the post-test report. but among all these issues, the most important one in language testing is: does the test test what it is supposed to measure? in other words, what is the validity of the test like? henning (1987) in his book a guide to language testing defines validity as the appropriateness of a given test or any of its component parts as a measure of what it is purported to measure. in general, there are five kinds of validity: face validity, content validity, construct validity, predictive validity and concurrent validity. the purpose of validation in language testing is to ensure the defensibility and fairness of the interpretation on test performance. reliability is clearly another essential quality of a language test. by reliability is meant the stability of test scores; for unless test scores are relatively consistent, they cannot provide testers with any information at all about the ability that testers want to measure. a reliable test score will be consistent across different characteristics of the testing situation (bachman therefore, only global integrative tests such as cloze and dictation go beyond the measurement of a limited part of language competence by tapping the single indivisible faculty of overall proficiency. however, most linguists still agree that language proficiency should be broken down to delineate various grammar domains and language skills. in other words, one-factor representation of l2 proficiency is implausible and a general factor can be divided into relatively more analytical components. in accordance with this componential model of language proficiency, it is still necessary to identify and measure the learners mastery of the separate elements of the target language in language proficiency tests, including vocabulary. to test vocabulary has its significance. as we know, words are an essential building block of language, the units of meaning from which larger structures such as sentences, paragraphs and whole texts are formed. vocabulary thus refers to the total number of words that make up a language. it is commonly believed that no matter how well the student learns grammar, no matter how successful the sounds of l2 are mastered, without words to express a wide range of meanings, communication in an l2 just cannot happen in any meaningful ways. therefore, expanding ones vocabulary knowledge is one of the fundamental goals for language learners. in fact, language learners generally agree that many of their difficulties in both receptive and productive use of language arise from their inadequate vocabulary knowledge. accordingly, we need vocabulary tests to monitor learners progress in vocabulary learning and to assess how adequate their vocabulary knowledge is to meet their shanghai jiao tong university m.a. thesis 19 communication needs. since vocabulary is so important to language learning, it is necessary to explore