研究设计实验研究的基本问题.ppt

School of Public Administration & Policy Dr. Kaifeng Yang,研究设计:实验研究的基本问题,问题,研究设计的核心要诣 实验设计的基本过程和环节 实验设计的影响因素,Given the research question, what type of evidence is needed to answer the question (or test the theory) in a convincing way?,Why? Making decisions to ensure validity,Planning: how to collect, analyze, and interpret data in order to test hypotheses Who Where What How When Can results be generalized (external validity)? Are the relationships true (internal validity)? Time order if necessary,School type,Academic Achiv.,Childs ability,School type,Academic Achiv.,Parent resource,Home facility,School type,Academic Achiv.,Parent education value,Child valuation,School type,Academic Achiv.,Key: Dealing with rival hypotheses (another example),When you observe X and Y co-vary, the reason may be:,x,y,z,1,x,y,z,2,x,y,z,3,x,y,z,4,x,y,z,5,x,y,z,6,Coping with Rival Hypotheses,Operation Fright - Juvenile Delinquency Visit prisonmonths .70% not arrested Even without the program, no more than 30% would have been arrested Family background: Volunteered are different Committed crimes but not been caught Only temporary effects Participation labels them as potential criminals,The New Design,Two groups: similar characteristics relevant to delinquency; one attend the program, another not. Interview both groups about delinquent activity and family background before the program Interview all families One year after the visit, interview both groups about delinquent activity and changes in family During step 4, also check arrest and conviction records for both groups Two years after the visit, repeat steps 4 and 5 Three years after, repeat 4 and 5.,问题,研究设计的核心要诣 实验设计的基本过程和环节 实验设计的影响因素,Pretest,Stimulus,Posttest,Randomization Or matching,Comparability,Compare,Compare,Exp.,Control,Staging Experiments,Pretesting,Subject recruitment,Intro to experiment,Manipulation of indep variable,Measurement of dep variable,Debriefing,Acquisition of informed consent,Random assignment,Manipulation check,Key Features of Experimental Approach,1. Manipulation 2. Control To test hypotheses, the experimenter deliberately introduces changes into the environments of subjects and observes or measures the effects of the changes.,Random Assignment vs. Random Sampling,Random sampling is a method of drawing a sample of cases, such as the pool of subjects in an experiment. Random assignment is a method of assigning subjects from the pool to experimental conditions.,Experimental Design,Experimental Design (Randomized Controlled) Group Random. Obser.1 Treat Obser.2 Comparison Exper. Re Oe1 X Oe2 Oe2-Oe1 Control Rc Oc1 Oc2 Oc2-Oc1 Two groups Randomization Treatment manipulation Pre and post-tests on the dependent variable Control of the environment of the experiment,Pretest,Stimulus,Posttest,Randomization Or matching,Comparability,Compare,Compare,Exp.,Control,Pre-experimental vs. True Experimental Designs,Pre-experimental designs are designs that contain one or more features of true experimental design features, such a comparison group or random assignment. True experimental designs are those designs that meet the basic requirements for an experiment.,Pre-experimental Designs,Design 1: The One-Shot Case Study X O Design 2: The One-Group Pretest-Posttest Design O1 X O2 Design 3: The Static-Group Comparison X O1 O2,True Experimental Designs,Design 4: The Pretest-Posttest Control Group Design O1 X O2 R O3 O4 Design 5: The Posttest Only Control Group Design X O2 R O4,True Experimental Designs (2),Design 6: The Solomon Four-Group Design O1 X O2 O3 O4 R X O5 O6 Designs 4-6 are called between-subjects designs because different groups of subjects are compared with one another. Within subjects design: X1 O1 X2 O2 Each subject acts as his/her own control that is, O1 is compared with O2 for each subject.,Quasi-experimental Designs,Quasi-experimental designs lack some features of a true experimental design, “but permits stronger inferences about cause and effect than pre-experimental designs, by means of special design features and supplementary data.” The quasi-experimental design falls into four subdivided categories, including 1) the separate-sample pretest posttest design, 2) Non-equivalent control group designs, 3) interrupted time-series design, and 4) multiple time-series design.,Quasi-experimental Designs,1) Separate-sample pretest-posttest design O1 X R X O4 2) Nonequivalent control group designs O1 X O2 R O3 O4,Quasi-experimental Designs (3),3) Interrupted time-series design O1 O2 O3 O4 X O5 O6 O7 O8 4) Multiple time-series design O1 O2 O3 O4 X O5 O6 O7 O8 O1 O2 O3 O4 O5 O6 O7 O8,Experimentation Outside the Laboratory,Field experiments: studies that meet all the requirements of a “true” experiment but are conducted in a natural setting. Experimental designs in survey research Unit of analysis other than individuals Evidence of the versatility of experimentation comes from the incorporation of experimental design methodology into sample surveys and from the use of units of analysis other than individuals. Because of its greater degree of control, experimentation is the preferred approach for testing causal hypotheses.,问题,研究设计的核心要诣 实验设计的基本过程和环节 实验设计的影响因素,Experiment as a Social Occasion,The particular cues in an experimental situation that communicate to subjects what is expected and what the experimenter hopes to find are called demand characteristics. Subjects in experiments often experience anxiety about being evaluated. Subjects come to experiments expecting the possibility that they will be evaluated. evaluation apprehension. Experimenters expectations about how the experiment will turn out. experimenters may unintentionally communicate to subjects their expectations about how the subjects “should” respond so as to confirm their hypotheses.,Minimizing Bias Due to the Social Nature of Experimentation,Several strategies for handling the problems raised by demand characteristics and experimenter bias To ask subjects about their perception of the experimental situation In a cover story, providing the subject with a false hypothesis about the purpose of the experiment. To measure the dependent variable in a setting different from the one in which the independent variable is manipulated. Double-blind technique: the experimenter does not know which condition a subject is in. Reduce the amount of contact between the experimenter and subjects (e.g., using taped recordings),Research Design Validity,Internal Validity Whether, in a given study or research project, the independent variable did indeed cause or lead to changes in the dependent variable External Validity: About Generalizing whether and to what extent the conclusions in your study would hold for other persons (or other units of analyses) in other places and at other times.,Internal Validity,IV,DV,Example: Imagine that we are stud