Marketing Research Causal Designs.pdf

Causal Designs 6 Questions to Guide Your Learning Q1 What is causality Q2 What are the types of evidence necessary to infer causal relationships Q3 What is an experiment and what s the difference between lab and field experiments Q4 What are the extraneous factors that can affect the interpretation of research results Q5 What are standard test markets electronic test markets and simulated test marketing M arketing managers frequently wish to answer the question Does X cause Y For example Will a 5 increase in price hurt customer demand Will a cereal package that has been redesigned for kids improve families attitudes toward the product These research questions are best answered via experimentation and causal analysis Concept of Causality The scientific concept of causality is different from the everyday notion The everyday interpretation of the statement X causes Y suggests that X is the only cause whereas the scientific state ment holds that X is only one of a number of determining con ditions In addition whereas the everyday interpretation implies a completely deterministic relationship i e X must always lead to Y the scientific understanding of causality implies a prob abilistic relationship i e the occurrence of X makes Y more likely Finally the scientific concept implies that we can never prove that X is a cause of Y We infer that a relationship exists an inference based on data perhaps acquired in a controlled ex perimental setting but the scientific approach recognizes the fallibility of any procedure Thus the question becomes What kinds of evidence can be used to support scientific inferences There are three basic kinds of evidence concomitant variation time order of the variables and elimination of other possible causal factors Concomitant Variation Consider the statement X is a cause of Y Evidence of con comitant variation is the extent to which X and Y occur together or vary together in the way predicted by the hypothesis There are two cases the qualitative and the quantitative Consider the qualitative case first Suppose we were interested in testing the statement The quality of our dealers X drives Copyright 2009 Cengage Learning Inc All Rights Reserved May not be copied scanned or duplicated in whole or in part our company s market share Y We believe that where we have good dealers we have good market penetration and where we have poor dealers we have less of the market If X is to be considered a cause of Y we should expect to find the following In those territories with good dealers we would expect to have sa tisfactory market shares and in those territories where poor dealers are located we would expect to have unsatisfactory market shares However if we found that the proportion of territories with unsatisfactory market shares was higher where the good dealers were located we would conclude that the hypothesis was untenable InTable6 1 the100dealersineachofthecompany ssalesterritorieshavebeen classified as good orpoor and the firm s market penetration in eachsales territory has been categorized by management as either satisfactory or unsatisfactory Thistableprovidesevidenceofconcomitantvariation Wherewefindthepresence of X a good dealer we also find the presence of Y good market share In fact 67 of the good dealers are found in territories where our market share is satisfactory A perfect relationship would yield data in which all good dealers were located in territories with satisfactory market shares and all poor dealers were located in territories with unsatisfactory market shares The pure case will rarely be found in practice because other causal factors will produce some deviation from this one to one correspondence between X and Y So we search for the proportion of cases having X that also possess Y With continuous variables the approach is similar e g a firm s advertising dollar expenditure is logically considered a cause X and sales the effect Y If we believe that the higher the level of advertising expenditure the greater the sales then we would expect to find a positive relationship between the variables If we analyze the relationship between X and Y and find supporting evidence of concomitant variation we can say that the association makes the hypothesis X causes Y more tenable but it does not prove it 1Similarly the absence of an association between X and Y cannot be taken in and of itself as evidence that there is no causal relationship between X and Y because we are always inferring rather than proving that a causal relationship exists Consider first the case in which there is positive evidence of concomitant var iation Table 6 2 suggests that candy consumption is affected by marital status 2 Single people are more likely than married people to eat candy regularly 75 of the single people in the sample ate candy regularly whereas only 63 of the married people were regular consumers The evidence is strong based on a rather large sample of 3 009 cases Thus can we safely conclude that marriage causes a decrease in candy consumption or are there other possible explanations What about the effects of age Married people are usually older than single people and perhaps older people eat less candy Table 6 3 shows the Evidence of Concomitant Variation Qualitative CaseTABLE 6 1 Market Share Y Dealer Quality XSatisfactoryUnsatisfactoryTotal Good40 67 20 33 60 100 Poor10 25 30 75 40 100 1 We ll discuss in subsequent chapters how to test for concomitant variation Note that association or correlation between X and Y does not mean there is causality between X and Y 2 Adapted from Hans Zeisel Say It with Figures 5thed Harper and each group was exposed to a different experimental treatment at the first point in time and at the same time and then the responses of the two groups were subsequently measured again simultaneously Extraneous Variables We need to design experiments carefully because we want to be able to con clude that the observed response was due to our experimental manipulations In particular we need to rule out extraneous factors as possible causes Extraneous factors fall into several categories History refers to the specific events external to the experiment but occurring at the same time that may affect the dependent variable Suppose a major ap pliance manufacturer was interested in consumers price sensitivity for re frigerators Suppose they proposed the following experiment to take place in Detroit Sales at regular prices would be monitored for a 4 week period then prices would be cut 10 and sales monitored for 4 more weeks We d compare weekly sales at the lower price with those at the higher price The experiment is a pre test intervention post test design O1XO2 What if soon after the price reduction the union contract with the auto industry expired and there was a strike The purchase of refrigerators and other major appliances would likely be postponed so we might see fewer sales at the lower price than at the higher one Would we therefore conclude that higher prices induce the sales of more units No because we know there were ex tenuating circumstances that caused the aberration Unfortunately the effects of history on a research conclusion are rarely so obvious There are always many variables that can affect what we observe and whose effect is subtle or hidden We need to isolate the effect given that we re rarely in a position to physically control it Seasonality is a related phenomenon Literal seasons are obvious e g sales of snow shovels in the north will be greater in the late fall and winter But many products have purchasing cycles so whether you re testing price changes in store ads product placement etc seasonality can confound your results The best solution is to be sure to run the experiment particularly in a field vs lab study for a sufficient duration so as to give the season a chance to cycle through Extraneous variables are alternative explanations 108Part 2Determine Research Design Copyright 2009 Cengage Learning Inc All Rights Reserved May not be copied scanned or duplicated in whole or in part Maturation specifically refers to changes occurring within the test units e g consumers that are not due to the effect of the experimental variable but result from the passage of time For example people get older become tired or grow hungry and distracted Measured changes in attitude toward a product may occur simply because people have become bored and disinterested while using the product and not because of the product or ad Similarly individuals in a consumer panel may change brands over time not because of any changes in marketing strategies but perhaps their marital or family status changed Maturation effects can occur in test units other than people Organizations also change Dealers grow become more successful diversify etc Stores change traffic increases its composition changes the store s physical makeup decays and then is renovated etc Of course the type of maturation depends on the duration of the experi ment We wouldn t worry about maturation of people in an experiment that lasted only a week On the other hand if they were participating in an interview and the questioning lasted a couple of hours they could very well grow tired or feel pressed for time Thus responses to later questions may be affected because their personal situation has changed and for no other reason Testing is concerned with the fact that the experiment itself may affect the responses There are two kinds the main testing effect and the interactive testing effect The main testing effect is the effect of a prior observation Oton a later ob servation Ot 1 The first administration in and of itself is responsible for the change In many situations the main testing effect is a result of respondents desire to be consistent Thus in successive administrations of an attitude ques tionnaire respondents reply in a consistent manner even though there has been some change in their attitudes Even in a single survey administration responses to the latter part of the questionnaire are somewhat conditioned by responses to the early questions The main testing effect may also be reactive there are very few things in social science that can be measured so that the process of measurement itself doesn t change what s being measured For example people asked about their attitude toward Honda at time t may begin to think about Honda somewhat and their attitude might even change slightly by time t 1 There is also an interactive testing effect which means that a prior measurement affects perceptions of the experimental variable For example people who are asked to indicate their attitudes toward Chevrolet may become more aware of the Chevrolet ads than those who are not queried Yet if we are interested in the attitude impact of the ads we are interested in their effect on the population as a whole and not simply on those individuals in our sample The results of the two testing effects are different In the O1X O2 study the main testing effect has an effect from O1to O2 The interactive testing effect on the other hand has an effect from O1to X Instrument variation includes any changes in measuring instruments that might account for differences The change may occur in the instrument itself or it may result from variations in its administration When many interviewers are used it is difficult to ensure that they will all ask the same questions with the same voice inflections use the same probes develop the same rapport and so on As a result the recorded differences between say the awareness level of two respondents may not reflect a true difference but rather a difference that arose because each interviewer handled the interview slightly differently The same thing can occur with interviews conducted by the same interviewer it s unlikely Extraneous factors History Maturation Testing main interactive Instrument variation Statistical regression Selection bias Experimental mortality Chapter 6Causal Designs109 Copyright 2009 Cengage Learning Inc All Rights Reserved May not be copied scanned or duplicated in whole or in part that each situation will be handled in exactly the same way Interviewers may become more adept at eliciting the desired responses or they may become bored with the project and tired of interviewing We might modify the measuring instrument during the course of the re search project If the changes are big e g a new set of attitude statements the responses would probably be analyzed separately Sometimes though a minor modification is needed such as a slight change in wording to make a specific question more understandable Although slight this kind of change should be checked for variations in the reported answers Statistical regression is the tendency of extreme cases to move closer to the average during the course of an experiment For example if people are chosen for the study because they are frequent drinkers then over the next couple of surveys the reported alcohol intake for those people would appear closer to the average Alternatively a study investigating the use of a brand of orange juice might reveal several families who consumed 10 cartons in a week If they had had house guests it wouldn t be surprising if subsequent consumption would be less Selection bias arises from the way in which test units are selected and assigned in an experiment It is said to be present when there is no way of certifying that groups of test units were equivalent before being tested Consider the standard news reaction to a presidential address Many say the president has convinced them that the X pick your topic crisis is real In a CNN poll last week 60 of those who heard the president s speech agreed that there is a worldwide crisis while 40 of those who didn t hear the speech be lieve the crisis is real The results 60 40 make it appear that the president is persuasive but the fallacy is that there is no way of knowing their attitudes be fore the speech Watching the speech is voluntary and thus the exposed and unexposed groups inevitably possess a systematic difference on the factors de termining the choice Republicans listen to the speeches of Republican candi dates Democrats listen to Democrats people with a favorable attitude toward a product pay more attention to the product s ads and so on If we are to con clude that exposure to the experimental stimulus TV special speech ad whatever was responsible for the observed effect we must ensure that the groups were equal prior to exposure EthicalDilemma6 2 The regional sales manager for a large chain of men s clothing stores asks you to establish whether increasing his salespeople s commission will result in better sales performance Specifically he wants to know whether increasing the commission on limited lines of clothing will result in better sales on those lines along with the penalty of fewer sales on the remaining lines or even whether raising the commission on all lines will produce greater sales on all lines Suppose that you think that the best way to investigate the issue is through a field experiment in which some salespeople receive increased commission on a single line others receive increased commission across the board and still others make up a control group whose members receive no increase in commission Are there ethical problems inherent in such a design Is the control group being deprived any benefits 110Part 2Determine Research Design Copyright 2009 Cengage Learning Inc All Rights Reserved May not be copied scanned or duplicated in whole or in part The equality of groups is established in two ways matching or randomization Suppose we have 20 stores 10 of which will comprise an experimental group and the other 10 a control Sales of anything in the stores are likely to be cor related with the store s traffic so to make the groups as similar as possible before introducing an intervention into the experimental group we would match the stores according to some external criterion such as annual sales or square feet of floor space Then we d assign one store from each matched pair to the ex perimental group and one to the control Matching isn t perfect It s hard to match test units on any but a few char acteristics so the test units may be equal in terms of the variables chosen but unequal in terms of others In addition if the matched characteristic turns out not to be an important determinant of the response the researcher has wasted time and money in matching the test units Randomization doesn t have these problems and is generally the preferred procedure We would assign the 20 stores at random to each of the groups using a table of random digits Randomization produces groups that are equal on the average when the sample is large enough to allow the positive and negative deviations about the average to balance Experimental mortality is the loss of test units during the course of an experi ment The problem is that there is no way of knowing if the people who have dropped out would have responded to the experiment in the same way as those that were retained In the example of testing a special display in a grocery store suppose that in the middle of the experiment the managers of two of the stores in the experimental group got nervous they thought the display wasn t work ing so they used the display for another product This would reduce the num ber of experimental stores to eight Our interest is in the average store sales in the experimental group vs those in the control group and we d have no way of knowing if the experimental group average would have been higher or lower if the two dropout stores had continued participating We can t simply assume that the sales would have been like those in the other experimental stores They might have been but maybe not We need to design studies so that this issue is eliminated Specific Designs Threetypesofexperimentaldesignsarecommonlydistinguished pre experimental designs true experi