How to Design an English Test Paper.doc

How to Design an English Test PaperTian ShiguoShandong Education InstituteAbstractIt is quite necessary for teachers of English to know the basic knowledge and techniques about how to design good English test papers. To meet this necessity, this paper first tells of the basic principles and requirements of designing English test papers, then the procedures, and, at last, the characters of some test questions and the techniques when designing them.English tests are a very important part in the teaching of English language. The two aspects concerned with English tests are: how to prepare for the tests for the students or other examinees and how to design English test papers for teachers of English or other designers of English test papers. Generally, in order to tell their students how to pass various English examinations, teachers of English pay more attention to the former and less attention to the latter. This paper, with the intention of mainly favoring those inexperienced designers of English test papers, will tell of the principles, the general requirements, the usual procedures, and some techniques of designing good English test papers. 。The basic principles and requirements for designingTo design good English test papers, the first thing for the designers to know is the basic principles and requirements for designing. The following principles and requirements should be abided by:1. A good English test paper ought to show the characteristics and the requirements of the test type the test paper belongs to.Different types of English tests have different characteristics, and therefore have different requirements. For example, a grade test mainly measures the students pass rate in accordance with the English curriculum. Just because of this, a grade test paper should stick closely to the goals and the teaching contents prescribed in the curriculum, and the test questions ought to cover as much the language knowledge as contained in the textbook. Otherwise, what the students have really acquired cannot be tested. In addition, the distribution of the grades of the students in the test should be moderate, with the number of the higher and the lower grades being small and the number of the middle being large, for this pattern of grades reflects the students normal conditions of their studies. In a proficiency test, the candidates are tested whether they can arrive at a certain level, so the most important thing for the designer of this type of test is to consider the standard of the test, without thinking a lot about the distribution of the grades of the candidates. It is quite natural that the number of the candidates who have passed the test is sometimes small, while it may be large at some other times.2. The test questions should have proper indices of difficulty, reliability and item discrimination.(1) The item difficulty index requirements: item difficulty index is an important index to decide whether a test is good or bad, and it has great influence on the reliability, the validity and the item discrimination. In a norm-referenced test, the proper item difficulty index means two things: the average item difficulty index of all the test questions is about 0.5; the item difficulty index of 2/3 of the test questions should be between 0.3 and 0.70.(2) The validity index requirements: the standard validity index should be between 0.40 and 0.70.(3) The reliability index requirements: it is generally accepted that the reliability index of the grade test should be over 0.90 (4) The item discrimination requirements: the item discrimination index is the main index to show the quality of a test question. In a large-scale screening test, the item discrimination index had better be over 0.4. 3. Paying enough attention to the tests function.Any test papers must fulfill the function of the test well. A tests function usually involves three things: (1)To guide the direction of teaching; (2) To assess the students achievements; (3) To evaluate the results of teaching.4. Requirements for concrete operations(1).The designing should be in accordance with the goal of the test. (2) The language knowledge, the verbal skills and the communicative competence of the examinees should be tested in an overall and objective way. (3) The scope of both the language knowledge and the skills (including the input skill, the output skill and the comprehensive skill) to be tested should be very clear. (4) The words to guide the examinees how to work at the questions should be definite. (5) There are no errors in the paper. (6) The forms of the questions adopted should be proper. (7) Key to the questions should be definite and reasonable. (8) The points for the questions should be distributed properly.(9) The standards for grading should be specific, definite and easy to be put into practice. (10) The working time the test questions take should be in accordance with the time set for the test. 。The general procedures for designingThe following six steps are usually followed while working:1. Keep in mind the goal, the principles, the requirements of the test and the scope of the language knowledge for the test2. Draw up the bi-directional detailed listAccording to the goal, the principles, the requirements and the scope of the language knowledge of the test, the test designer should draw up the bi-directional detailed list (that is, two-split chart) for the whole paper. On top of the list, is the goal of the test, and on the left are the contents. The main body of the list is the related weight. See Table l, please.Table l The bi-directional detailed list for an English proficiency Test(designed by Guangzhou Institute of Foreign Languages)GoalWeightContentsKnowledgeComprehensionApplicationAnalysisComprehensive useThe number of itemsPercentageTimeGrammar10102012.520 min.Vocabulary202012.520 min.Reading40402540 min.Cloze test202012.520 min.Listening353521.930 min.Writing252515.630 min.Total3075102025160100160 min.3. Decide the forms of questionsAs we all know, reading and writing abilities are tested by written examinations and hearing and speaking abilities are tested by oral and listening tests. Since written examinations are widely used in our daily life, here we only discuss how to decide the forms of questions in written examinations.It is important that the forms of questions serve the contents to be tested and the goal of the test. Multiple choice is a convenient form in testing all kinds of language knowledge and verbal skills, and so it often covers the most part of a paper. In NMET(National Matriculation English Test)1999, there are 95 points of objective questions (all in the form of multiple choice) and 55 points of subjective questions. As is shown in many excellent English test papers, nowadays emphasis is laid on testing the students comprehensive abilities. It is just because of this, the form of test questions should be effective in serving this purpose. Therefore, English test paper designers should be careful enough when choosing the forms of test questions.If there are no off-the-peg forms of questions for a designer to use, he can invent a new form. For example, “error-correcting in the passage”, which is used in NMET, is a new form of question. Like “cloze test”, it can test the students abilities of using the language in discourse. In a “cloze test”, the students only need to choose the right one among the four alternatives, but in “error-correcting in the passage”, the students have to decide what is correct out of the void in the face of blanks. That is why “error-correcting in the passage” is more difficult than a “cloze test”. If an examinee doesnt have good sense of language, well-knit knowledge foundation and profound ability in using the target language, he will be at a loss in front of such a form of question. The following is the “error-correcting in the passage” in NMET2001:V. Error-correcting in the passage (there are 10 sub-questions in total, each of which is 1 point. The full marks are 10 points.)Like most of my classmates, I have nei- 76. ther brother nor sisters - in any other words, 77. I am an only child. My parents love medearly of course and will do all they canmake sure that I get a good education. 78. They did not want me to do any work at 79. family; they want me to devote all my 80. time to my studies so that Ill get goodmarks in all my subject. We may be one 81. family and live under a same roof, but we 82. do not seem to get much time to talkabout together. It looks as if my parents 83. treat me as a visitor and a guest. Do they 84. really understand their own daughter? What 85. things are in other homes, I wonder.(key: 76.; 77. leaving out “any”; 78. maketo make; 79. diddo; 80. familyhome; 81. subjectsubjects; 82.athe; 83. leaving out “about”; 84. andor; 85. WhatHow)When deciding the forms of questions, the designer had better draw a “two-split table of the contents and the forms (types of questions)”. See Table 2, please. Table 2 A two-split table of contents and formsTypes of questionsContentsObjective questionsSubjective questionsTrue or falseClozeMatch the words or phrasesMultiple choiceQuestions and answers in situationsGuided writingTranslation (from Chinese to English)Translation (from English to Chinese)VocabularyStructureReading ComprehensionWritingComprehensive use4.Write out the test questionsWhile writing out the test questions, the designer should work in accordance with the prepared two-split table. First, the collected test questions should be 23 times as many as those required in order to be sufficiently enough for selecting. In the course of writing out the questions, key to the questions and the methods for grading should also be taken into consideration.The designers ought to write out all the questions personally. But sometimes, very good questions might be found. At this time, the designer should remember that not any ready questions are to be copied. If a question is so good that you are eager to use it, be sure to make some changes in them. For example, if a multiple-choice question is used again, the order of the options can be changed. One of the confusing options may be changed into the key, and visa versa. Do remember that the source of the questions should be wide. Search for as many available materials as you can!In addition, it should be pointed out that, to design an English test paper for a large-scale examination, it is quite necessary to set up a test-question storeroom, because it can save a lot of time, energy and money as well as can keep up a high reliability of the test paper. 5. Decide the keyAs is mentioned above, key to the questions and the methods for grading has already been taken into account while wiling writing out the test questions, though it might be sketchy. Key to every question and the methods for grading should be given cautiously. The designer should pay enough attention to two things: (1) Key to an objective question must be unique. Ambiguous or two or three keys are forbidden. (2) Key to a subjective question is usually called “key for reference”. “Reference keys” to some questions should be offered as many as possible.Here is the reference key to the guided writing in NMET1998. It is simple, clear, and fit for the requirements of the question well. It is requested in the question that the Chinese version of the question doesnt need to be translated, and the key offered here is done in this way. For example, “你的老师认为 (Your teacher thinks )”, which appears in the question, is not translated in the key.The test question and its reference key are as follows:UII Writing (One question in total, 15%)English magazine BBC ENGLLSH has a special column. “Can I help you?” which is set up to answer English questions asked by readers. The editor of the column is Jill welfare (female). The address of this magazine is: BBC English Magazine, PO Box 76, Bush House, London wc2B 4PH, U.K.Suppose your name is Li Hua, a student in grade 3 in No. I Middle School of Beijing. Please write a letter to this magazine to ask two questions.First, your teacher thinks, “He is taller than I.” But you have heard “He is taller than me.” You want to know which one is right. Why?Second, you want to know the difference between “further” and “farther” in meaning and in usage.Attention: You are only required to provide clearly the questions you want to ask. Dont translate the Chinese version of the questions. The letter should be written in 80120 words. Dont use your real name and real address in the letter.The key:One possible version: No. 1 Middle School Senior 3 Beijing Peoples Republic of China _ July 1988Ms/Miss Jill Welfare“Can I help you?”BBC ENGLISH MagazinePo Box 76, Bush HouseLondon WC2B 4PHU.K.Dear Ms/Miss Welfare,I have two questions to ask you.1. I have heard some people say “He is taller than me” instead of “He is taller than I”. Which one is correct? Can you please explain why?2. Do “further” and “farther” mean the same thing? If not, whats the difference? When do we use the one and when do we use the other?Will you please answer those questions for me?Thank you very much. Yours truly, Li Hua (85 words)6. Decide the methods for gradingThe most important thing in deciding the methods for grading is: the grading must be fair and reasonable.(1) For objective questionsIt is easy to grade the objective questions. But when deciding the methods for grading, it must be noticed that the points each question is worth in different types of questions should be different according to the difficulty of the questions. If 0.5 point is given when a correct choice is made in a single item, than more points (2 points in general) should be given when a correct choice is made in reading comprehension.In order to avoid the factor of guess in the objective test, the points should be revised. The formula for revising is:The number of wrong answersThe revised number= The number of correct answers The number of choices 1(2) For subjective questionsIts never to be too detailed to give the methods for grading subjective questions. “The grading method of dividing the papers into nine piles” can be used in order to grade more accurately. The steps are as follows:First, read through all the papers, underline the wrong words and the wrong sentences, and assess the proportion of the wrong words and sentences in all the papers. Second, divide them into three piles in terms of the better, the middle and the worse concerning the contents. Then, again, divide the papers of every pile into three piles in this way. So, there are nine piles altogether. Read through the papers again, and, if there are some, try to find out the excellent ones or the worst ones. The excellent ones should be upgraded, while the worst ones should be reduced to a lower rank. When feeling sure of the relatively reasonable classification of all the papers, the designer can now grade the papers. Generally, the first class is 95 points. 5 points will be subtracted for each class. The ninth class, i.e. the last class, fails to pass the exam. Although this method takes more time, it meets the requirements of reliability because it is relatively fair and reasonable.The following is the grading table for the guided writing in Matriculation English Test 1988 (MET88A). See table 3, please. Table 3 The grading table for MET88UIIPaper number The number of the words is 30 less than words requiredLanguage (7 points)Contents (7 points)Score (The full marks are 15 points.)For any of the following cases, deduct points like this:If irrelevant to the subject, deduct 715 points. For the rest conditions, the deduction is as follows:All are right, proper, clear and coherent.There are several mistakes, but not influence the expression.There are enough mistakes to influence the expression.There are many mistakes. Only a few sentences can make sense.Errors are all over, and almost not anything can make sense.Wrong form in the letter (Time, appellation, signature, address)If the following important points are neglected or not expressed, the deduction is like this:prologueQuestion 1Question 2Concluding remarks; thankfulnessThe Two expressions. Which one is right? Why?X X and X X. Whats the similarity and the difference between them in meaning and usage?-1-0-12-34-56-7-1-0.5-3-2-0.5。The characters of some types of questions and the techniques for designing themThe designers should pay more attention to some types of questions, for there might be some difficulties when designing them.1. Making choice in a single itemThis type of questions has two strong points: one is its flexibility in testing, being very convenie