数据分析2.doc

Table 4.6 below is the results of paired samples t-test (Naming-Matching). The two variables in the analysis are score of naming test and score of matching test. The figures in the table show that the two tailed significance is 0.764; and the lower level is -4.548 and the upper level is 3.382. Zero is included in the span from lower level to upper level. All these well prove that there is no significant difference between the result of naming test and that of matching test.Table 4.6 Paired Samples Test (NAMING-MATCHING)Paired Samples TestPaired DifferencestdfSig. (2-tailed)MeanStd. DeviationStd. Error Mean95% Confidence Interval of the DifferenceLowerUpperPair 1NAMING - MATCHING-.5839.3901.917-4.5483.382-.30423.764Table 4.7 Paired Samples Test (CHECKLIST-NAMING)Paired Samples TestPaired DifferencestdfSig. (2-tailed)MeanStd. DeviationStd. Error Mean95% Confidence Interval of the DifferenceLowerUpperPair 1CHECKLIST - NAMING4.1676.8251.3931.2857.0492.99123.007 Then we come to the paired samples t-test (Naming-Checklist). The two variables in the analysis are score of naming test and score of checklist test. In table 4.7 above, we can see that the two tailed significance is 0.007, which is statistically significant; and the lower level is 1.285 and the upper level is 7.049. Zero is not included in the span from lower level to upper level, and thus, conclusion comes that the difference between the two variables is significant. That is, the result of naming test is significantly different from that of checklist test.The Paired-item Significant Difference AnalysisThe analysis in the previous section shows that the total scores of naming test and checklist test are significantly different, but the difference between the two tests and matching test is not significant. Further study about the difference between the corresponding items in two tests is conducted by pairing the concerned corresponding items of two tests. Accordingly, there are altogether 60 pairs of items and the only difference for each pair is the test format. Here the author not only pair the items of naming test with checklist test, but also pair naming test items and checklist test items with items of matching test respectively. Paired samples t-tests were conducted to explore the results. Therefore, there are three paired-item analysis altogether. As for the paired item significant difference analysis, the case is a bit different from that of the total score analysis. First, in the paired-item analysis of naming test and checklist test, no significant difference is found in 56.7 percent of the items and 43.3 percent of the items show significant difference according to the 95% confidence interval of difference.Then, in the paired-item analysis of naming test and matching test, no significant difference can be found in 83.3 percent of the 60 items; only scores of 16.7 percent of the items are significantly different from each other.And in the paired-item analysis of checklist test and matching test, the results seem to be more complicated. Although no significant difference is found between the total scores of these two tests, the paired-item analysis indicates that there are significant differences in 33.3 percent of the 60 items between these two tests. Details of these items will be analyzed in the discussion part.The Item DifficultyItem difficulty is an important index of a test. Item difficulty expresses the proportion or percentage of participants who answered the item correctly. The item difficulty index (P) is calculated by P=R/T, in which R is the number of correct answers and T is the total number of participants. Item difficulty can range from 0.0 (none of the participants answered the item correctly) to 1.0 (all of the participants answered the item correctly). Average difficulty index is between 0.4 and 0.6.In the descriptive statistics, we have seen that the mean of checklist test is 38.92; then we can calculate the difficulty of the test by dividing the total score, that is, 60 in the this study, with the mean of the test. So the difficulty of the checklist test is 0.65. And in the same way, we can get the difficulty of matching test and naming test; which is 0.59 and 0.58 respectively. That is to say, difficulties of the three tests are all acceptable.Besides, this research has undergone item difficult study on each test to make it more concrete. The results are shown in table 4.6 below:Table 4.8 The Item Difficult StudyP0.20.2P0.40.4P0.60.6P0.8P0.8CHECKLIST0%6.7%20%23.3%50%MATCHING0%16.7%36.7%30%16.7%NAMING3.3%20%36.7%16.7%23.3%P value in the table means the percentage of the participants who have got the correct answer to the number of the total participants. The table shows that the checklist test is on the easy side for the participants. Half of the items are easy for them and none of the items seem to be very difficult for them. The difficulty of the matching test and naming test is quite acceptable with most of the item difficulty falls between 0.4 and 0.6. Words tested are the same, but item difficulty is quite different. Items in matching test and naming test are comparatively much more difficult than in checklist test. Tentative explanation for this situation is that the participants may have over-estimated their vocabulary knowledge.Besides, by analyzing the paired items of checklist test and naming test, it is found that 50 pairs of items increase in the difficulty from the one in checklist test to the one in naming test in the whole 60 pairs.The Item DiscriminationAn index of item discrimination is the point-biserial correlation, that is to say, how well the item serves to discriminate between students with higher and lower levels of knowledge. The point-biserial correlation reflects the degree of relationship between scores on the item and total test scores. The item discrimination index (D) measures the difference between the percentage of participants who answered the item correctly in the upper group (PU), i.e., the top 27 percent scorers, and the percentage of those in the lower group (PL) who got the correct answer, i.e., the bottom 27 percent scorers; thus D=PU-PL. The index will be positive if better students answered the item correctly more frequently than poorer students did, and negative if the opposite occurred. A negative index is denoted by a minus sign in front of the value.The value of a positive point-biserial discrimination index can range between 0 and1; the closer the value is to 1, the better the discrimination. The value of a negative point-biserial discrimination index can range between -1 and 0, but positive values are desirable. Item discrimination is greatly influenced by item difficulty. Items with a difficulty of either 0 or 1 will always have a discrimination index of 0, and item discrimination is maximized when item difficulty is close to 0.5. As a general rule, discrimination index of 0.2 and above are considered to be desirable.Table 4.7 The discrimination index of the three testsTest FormatDiscrimination IndexCHECKLIST0.47MATCHING0.65NAMING0.43The figures in table 4.7 indicate that the three tests have well discriminated participants of different levels for their discrimination index are higher than 0.2. The analysis of discrimination was made to each item as well. For the checklist test, discrimination index of 28 of the 60 words tested is above 0.2. And discrimination index of 46 of the 60 words tested in naming test is above 0.2. Furthermore, when it comes to compare each concrete paired-item to see if there is any difference in discrimination within each pair, it finds out that 40 of the 60 pairs of items in naming test has better discrimination than the corresponding one in checklist test. When comparing the discrimination index of checklist test and naming test to matching test, the overall discrimination of matching test is the best of the three. And in the paired-item comparison, 44 items of the matching test has a higher discrimination index than the corresponding one in the checklist test; and 32 items of the matching test possess a higher discrimination index than the corresponding one in the naming test.The only difference between these three tests is the testing format; therefore, all these can well prove that the matching format enjoys better discrimination than the checklist format and naming format though the discrimination index of the two tests are acceptable.The criterion-related analysisAs stated in chapter two, criterion-related validity is a predominantly quantitative and a posteriori concept, concerned with the extent to which test scores correlate with a suitable external criterion of performance with established properties. Criterion-related validity divides into two types: concurrent and predictive. Concurrent validity looks for a criterion which we believe is also an indicator of the ability being tested (Bachman 1990:248). Test scores could be correlated with another measure of performance, usually an older, longer, established test, taken at the same time for teachers rankings students, or even student self-assessment.In the present research, a matching test was designed to be the external criterion. Matching test was chosen because it has been proved by many researchers to be a valid test of vocabulary size and has been widely used. The author has verified validity and reliability of the matching test used in this study before the experiment was carried out.Correlation analysis in this study is conducted mainly between the matching test scores and that of the other two tests, using the Pearson Correlations. Pearsons correlation reflects the degree of linear relationship between two variables. On general sense, r is used to present Pearsons correlation. Pearsons r can range from -1 to +1. An r of -1 indicates a perfect negative linear relationship between variables, an r of 0 indicated no linear relationship between variables, and an r of 1 indicates a perfect positive relationship between variables.Table 4.9 shows that the Sig. (2-tailed) is 0.000 for the correlation concerned which means the result is statistically meaningful. The Pearsons r in table 4.9 is 0.810, which means the scores of checklist test are highly correlated with that of matching test.Table 4.9 The correlation between Checklist and MatchingCorrelationsCHECKLISTMATCHINGCHECKLISTPearson Correlation1.810*Sig. (2-tailed).000N2424MATCHINGPearson Correlation.810*1Sig. (2-tailed).000N2424*. Correlation is significant at the 0.01 level (2-tailed). Table 4.10 shows that the Sig (2-tailed) is 0.000 for the correlation concerned. The Pearsons r in table 4.10 is 0.814, which is a little higher than the correlation between Checklist and Matching. That is, naming test are also highly correlated with matching test.Table 4.10 The correlation between Naming and MatchingCorrelationsMATCHINGNAMINGMATCHINGPearson Correlation1.814*Sig. (2-tailed).000N2424NAMINGPea