Nature guideline 中山大学 循证医学.pptx

Statistical Guideline of Nature Ji-Qian Fang School of Public Health Sun Yat-Sen University 2008.10,An editorial of Nature Medicine (2005) : “Some of the articles published in Nature and Nature Medicine were criticized due to the deficiency in statistical issues”.,Challenge to Nature Medicine,What happened?,Emili Garca-Berthou and Carles Alcaraz (Girona Univ., Spain) published an article in BMC Medical Research Methodology (May 2004). They reviewed 181 research papers of Nature (2001) , found that 38% of them have at least one mistake in statistics. Since then, a series of critical articles have been published, of which one written by Robert Matthews (The Financial Times) analyzed the statistical methodology of the articles in Nature Medicine (2000).They found that 31% of the authors had misunderstood the meaning of P-value, even some one reported the P-value with unnecessary precision ( 0.002387).,Independent statistical “audit”,Nature Medicine invited two experts from the University of Columbia to work out “statistical audit” , especially to evaluate 21 articles published in 2003 with a list of consolidated criteria on statistics. They found that some papers almost did not have any quantitative analysis, and some contained very complicated statistical and mathematical issues. While most of them have just used a litter statistical testing, but with incomplete descriptions such that one could hardly assess whether they were appropriate or not.,Checklist of statistical adequacy,Reported n at start of study and for each analysis Provided sample size calculation or justification Examples We believed that . . . the incidence of symptomatic deep venous thrombosis or pulmonary embolism or death would be 4% in the placebo group and 1.5% in the ardeparin sodium group. Based on 0.9 power to detect a significant difference (P=0.05, two-sided), 976 patients were required for each study group. To compensate for non-evaluable patients, we planned to enroll 1000 patients per group,To have an 85% chance of detecting as significant (at the two sided 5% level) a five point difference between the two groups in the mean SF- 36 general health perception scores, with an assumed standard deviation of 20 and a loss to follow up of 20%, 360 women (720 in total) in each group were required.,3. Identified all statistical methods unambiguously 4. If statistical methods were described adequately, were any of them clearly inappropriate? Example All data analysis was carried out according to a preestablished analysis plan. Proportions were compared by tests with continuity correction or Fishers exact test when appropriate. Mean serum retinol concentrations were compared by t test. . . Two sided significance tests were used throughout.,Multivariate analyses were conducted with logistic regression. The durations of episodes and signs of disease were compared by using proportional hazards regression.,Methods for additional analyses, such as subgroup analyses and adjusted analyses: Example Proportions of patients responding were compared between treatment groups with the Mantel-Haenszel chisquared test, adjusted for the stratification variable, methotrexate use. . . . it was planned to assess the relative benefit of CHART in an exploratory manner in subgroups: age, sex, performance status, stage, site, and histology. To test for differences in the effect of CHART, a chisquar test for interaction was performed, or when appropriate a chi-squared test for trend (131).,5. Provided alpha for all statistical tests 6. Specified whether tests were one-sided or two-sided 7. Stated whether the data met the assumptions of the test 8. Reported actual P values for primary analyses,Example The data of two samples were adequately normally distributed（Shapiro-Wilk test：P1=0.466；P2 0.482） and the two population variances were equal at the significant level 0.10（F1.345；P=0.261）, so two independent samples t test was used（t=4.137； df=18；P=0.001）. The results indicated a statistically significant difference between effects of two drugs at two-tailed significant level 0.05 and the average increase of concentration of Hb was higher in patients taking the new drug, which could also be observed from the 95% confidence interval of the difference of two population means (3.829, 11.731).,9. Were the statistical measures (mean, standard error, standard deviation, etc.) reported, and were they clearly labeled? Example The results show that the mean SD of IL-2 for the experimental group (n=31) was 16.00IU/ml 7.50 IU/ml and for the control group (n=30) was 20.00IU/ml8.00 IU/ml; the difference between the two group means was 4.00IU/ml, and the 95% CI of the difference was（0.0304, 7.9696)（IU/mL),10. Was the unit of analysis clearly stated in all comparisons? 11. Are mean and standard deviation used to describe data sets that may be non-normally distributed or when the sample size is very small?,表4-5 试验组和对照组治疗前血气分析结果,What are the problems?,12. Explanation of unusual or complex statistical Methods Example In order to compare the effects of common feed, feed with plasma protein and feed with bioprotein on weight growing to weaning young pigs，30 weaning young pigs were matched to 10 blocks by gender, days of age and baseline weight. Then 3 individuals in each block were randomly assigned to 1 of 3 treatment groups. After 10 days, the changes in weights from baseline were measured. - Random block design,The mean change of weight SD was 3.33kg0.48kg for the group of common feed, 3.83kg 0.61kg for that of plasma protein, and 4.10kg 0.68kg for that of bioprotein. Results of two-way ANOVA under the significance level of 0.05 indicated statistically significant differences among 3 treatment groups (F=6.8112,P=0.0063). Similar results were found among 10 blocks (F=2.7407, P=0.0328). - Results of ANOVA,13. Explanation of data exclusions, if any Example The primary analysis was intention-to-treat and involved all patients who were randomly assigned One patient in the alendronate group was lost to follow up; thus data from 31 patients were available for the intention-to-treat analysis. Five patients were considered protocol violators . . . Consequently, 26 patients remained for the per- protocol analyses,Protocol deviations Authors should report all departures from the protocol, including unplanned changes to interventions, examinations, data collection, and methods of analysis. The nature of the protocol deviation and the exact reason for excluding participants after randomization should always be reported.,14. Explained reasons for any discrepancy between initial n and n for each analysis Example Initially, the 60 rats were randomly divided into 3 groups, 15 for each, to receive 3 levels of doses respectively. However, at the end of the first week, 2 rats in the group of low dose escaped; on the 40-th day, 1 rat in the group of high dose and 1 in the control group escaped ,15. Explained method of treatment assignment (randomization, if any) Example Determination of whether a patient would be treated by streptomycin and bed-rest (S case) or by bed-rest alone (C case) was made by reference to a statistical series based on random sampling numbers drawn up for each sex at each centre by Prof. Bradford Hill; the details of the series were unknown to any of the investigators or to the coordinator and were contained in a set of sealed envelopes, each bearing on the outside only the name of the hospital and a number. After acceptance of a patient by the panel, the envelope was opened at the central office; the card inside told the medical officer of the centre if the patient was to be an S or a C case.,16. Explained any data transformation,Example 18 patients with acute encephalitis B in a clinic were randomly allocated into 3 groups. Each group accepted different kind of treatments, say treatment A, B and C; and the fevering days were measured as the effects of treatments. Please make an inference from the differences of means of fevering days among the three groups whether the treatments had different effects.,Consider the two assumptions of one-way ANOVA. The fevering days are positively skew from the normal distribution; and the ratio of is closed to 10, the assumption of homogeneity of variances is also abandoned. Therefore, a square root transformation of the scale for the fevering days is applied The new scales have been used in computation of one-way ANOVA. It resulted in that there is no significant difference on the average fevering days (scales of square roots) among the three kinds of treatments.,17. Discussed adjustments for multiple testing Example Multiple comparison with Bonferroni adjustment (alpha level of 0.0167) revealed that the effects of the two treatments with protein were significantly higher than that of common feed, while the difference between the Two treatments with protein was not statistically significant. -Multiple comparison,For graphs 18. Were effect sizes distorted? (by truncation of