6SIGMA工具 Intro to Survey Design.ppt

IntroductiontoTransactionalSurveys,天马行空官方博客：,Objectives,IntroduceconceptsinSurveyPlanningIntroduceconceptsinQuestionnaireDesignIntroduceconceptsSampleDesignandDataCollectionDemonstrateapplicationsofCustomerSatisfactionSurveys,天马行空官方博客：,TransactionalSurveys,SectionISurveyPlanning,WhatisanIdealSurvey?,FromtheSurveyorsperspectiveMaximizesthequalityofresponse(validityofresponse)Maximizesthequantityofresponse(responserate)FromtheRespondentsperspectiveMaximizestherewardstotherespondentMinimizesthecoststotherespondent,SurveyDesign,PreliminariesStatementofobjectivesTargetpopulationInstrumentdesignQuestionnairedesignThepretestSampledesignThesamplingframeSamplingmethod,DatacollectionMethodofmeasurementSelectionandtrainingoffield-workersOrganizationoffieldworkDataanalysisDatamanagementAnalysisReporting,SurveyObjective,“Define”themetricstobeusedtotracktheprocess“Measure”processperformanceusingreliableandvalidmeasurementinstruments“Analyze”thedatatoidentifyopportunitiesVerifiesresultsfromactionstakento“Improve”theprocess“Control”theprocessthroughon-goingtrackingofmetrics,UsesofSurveys,Processesprocessimprovement/screeningprocesscapabilityprocesscontrol/monitoringProgramsneedsassessmentprogramevaluationmonitorandcontrol,Peoplehiring/selectionperformanceappraisalformemployeeattitudesurveyqualityofworklifesurveyProductsand/orServicesmarketresearchsurveybusinesscasesurveycustomersatisfactionsurvey,ProductorProcessSurveys,Design,orredesign,aproductorprocess(usedforpreliminaryallocationofresources)In-depthinterviewswithdistinctcustomersegmentsEmphasisonqualitativequestionsMeasuresofabsoluteimportanceProcessorProductQualityMonitoring(usedformarginalreallocationofresources)InvolvesrepresentativesamplesofcustomersEmphasisonquantitativequestionsMeasuresofrelativeimportance,TimeDomainofTransactionSurveys,RelationshipIntegratedexperienceovertimeMayinvolveseveraltransactionsTransactionFocusedonanspecifictransactionexperienceConductedassoonaspossibleaftertheexperience,ValidationofInternalMetrics,CustomermetricsEaseoforderplacementProductavailabilityOn-timedeliverOrdercompletenessOrderaccuracyDamagefreegoodsDelivertocorrectlocationProblemresolutionInvoiceaccuracyTimelinessofcredits,Internalmetrics%Abandonedcalls%Backorder%Carrieron-timedelivery%Perfectorderfulfillment%Returnsfordamage%Correctdeliveries#Daysfromordertodelivery%Unansweredcalls#Daysinproblemresolution%Billinginquires%Billingerrors#Daystoissuecredit,TransactionalSurveys,SectionIIQuestionnaireDesign,IssueswithQuestionnaireDesign,QuestionorderingContextOpen-versus-closedquestionsResponseoptionsTypeofinformationQuestionwordingmistakesmade,QuestionOrdering,Firstquestion-relatedtostatedpurposeMostfamiliartoleastSensitiveanddifficultquestionsinthemiddleLogicalflowofquestionswithtransitionsObjectivebeforesubjectivequestionsDemographicsattheendofsurvey,Context,AffectsresponsestoadjacentquestionsSaliencyaffectsfollowingquestionsTendencytoanswerconsistentlyMaybeusedtocontextualizegeneralquestionsMaybeusedtoprimethememoryofaspecificevent,OpenVersusClosed,Open-endedquestionsWhenresponsecategoriescannotbeenumeratedoranticipatedWhenlistingcategoriesofresponsewillbiastherespondentClosed-endedquestionsIncludesorderedandunorderedresponsescalesMayormay-notincludeaneutralcategory,Closed-EndedResponseOptions,UnorderedresponsesMultiplechoice(independentalternatives)Ranking,preference,fixedsumOrderedresponsesGradationonasingledimensionBalanced,maycontainaneutralcategoryPartiallyclosedUnorderedresponseswith“other”asachoice,TypesofInformation,AttitudesBeliefs(knowledge)BehaviorAttributes(demographics),Whataresomeexamplesofeach?,QuestionWordingMistakesMade,WordsnotuniformlyunderstoodbyallrespondentsAbbreviations,toocrypticDoublemeaningofthequestionAllrespondentshaveprerequisiteinformationtoanswerallquestionsQuestionassumestoomuchabouttherespondentsbehaviororexperienceQuestionisreallytwoquestionsQuestionistoodifficultorobjectionableQuestionbiasesrespondentsanswer,OtherConsiderations,LengthofsurveyTypeofsurveyBurdenonrespondentRelevancyoftopictorespondentSpacingAdequatespaceforresponses(open-ended)Include“whitespace”DirectionsSimple,clear,easytofollow(mustpilot),CommonPitfalls,TopicnotwellboundedorspecifiedIll-definedpopulationRaisingexpectationsyoucantfulfillSelf-incrimination/socialdesirability“Mono-numerosis”,TransactionalSurveys,SectionIIISampleDesignandDataCollection,IssueswithSurveyDataCollection,CollectionmethodsSampledesignSamplesizeandfrequency,CollectionMethods,PersonalinterviewsTelephoneinterviewsSelf-administeredquestionnaires(mail,electronic-includingadaptivequestionnaires)DirectobservationOther:Panelinterviews(1interviewers,1respondent)Groupinterviews(1interviewer,1respondents)On-siteformsurvey(1administrator,1respondents),SampleDesign,Itisusuallynoteconomicallyfeasibletoadministerasurveytoallmembersofatargetpopulation(acensussurvey)Primaryconsideration:thesampleneedstoberepresentativeofthelargerpopulation,leadingtoanunbiasedsurveysampleMajorcharacteristicsofsampleschosenforstatisticalinference:RandomselectionisusedSamplesizecanbestatisticallydeterminedStatisticalinferencescanbemadeaboutimportantparameters,SamplingPlans,Threemaintypesofsamplingapproaches:SimplerandomsamplingStratifiedrandomsamplingClustersampling,SomeHelpfulDefinitions,Elements-objectonwhichmeasurementistakenPopulation-collectionofelementsaboutwhichaninferenceistobemadeSamplingunitsnon-overlappingcollectionsofelementsFrame-listofsamplingunitsSample-collectionofsamplingunitsdrawnfromaframeorframes,SimpleRandomSampling,SimpleRandomSamplinginsuresthateverysamplefromthepopulationhasequalprobabilityofbeingselectedForexample,if100peopleoutof1000inacompanyaretobeselectedforasurvey,eachcouldbenumbered1to1000,saybyalphabeticalorder,with100chosenusingarandomnumbergenerator,StratifiedSampling,StratifiedrandomsamplingconsistsofseparatingtheelementsofthepopulationintomutuallyexclusivegroupscalledstrataRandomsamplesarethentakenfromeachstrataTheprimarypurposeofsuchaselectionprocessistoinsurethateachstratumisproperlyrepresented,ExampleofStratifiedSampling,Forexample,hospitalmanagementisinterestedinasurveywhichtrackspatientsatisfaction.Itisdesiredthatthesurveypopulation,size1200,mimictheethnicbreakdowninthecommunity.ThisisaProportionalStratifiedPlan:RacialPopulationSampleGroupSize(000s)%RepresentationWhite6,00060720Black1,50015180Hispanic1,50015180Asian1,00010120Theadvantagesofastratifiedrandomsampleinclude:Withinstratumhomogeneityofvarianceleadingtogreateroverallprecision.EstimatesofCustomerSatisfactioncanbeobtainedforeachstratum.,ClusterSampling,Clustersamplingbeginssampleselectionbyrandomlychoosinggroupsandthensamplesfromthegroups.Suchgroupingsmightincludecounties,productlines,physiciangroups,etc.Forexample,samplingforasurveyofemergencyroomphysicianswouldstartwitharandomselectionofhospitals(clusters),andthenarandomselectionofthephysicianswithinthehospitalsselected.Thisisanexampleofatwo-stageclustersampling.Otherlevelsarepossible:Forexample,anotherlevelinthepreviousexamplemightbethecorporationsthatownthehospitals.Corporationswouldberandomlyselected,thenaselectionofthehospitalswithinthecorporationswouldbeselected,andfinallytheselectionofthephysicianswithinthehospitalswouldbeselected.Thisisanexampleofathree-stageclustersampling.Theadvantageofclustersamplingisthatitcanvastlysimplify,andreducethecostsassociatedwith,thesamplingprocess.Thedisadvantageisthepossibilityofsamplingerroroccurringateachofthestages.,IssueswithSampleSize,Statisticallyspeaking,samplesizedependson:ThesamplingplanemployedThestatistic(s)ofinterestEstimatesofinherentvariabilityRequiredprecisionThesizeofthepopulationMorepracticallyspeaking,samplesizealsodependson:CostperelementofdatacollectionFrequencywithwhichdataneedstobereportedAnnualbudgetfortheongoingtrackingDecisionsthatwill,orwillnot,bemadebasedonresults,SimpleRandomSamplingWithContinuousData,Forasimplerandomsampleandcontinuousdata,wehavethefollowingrelationship:Where:Eisthemarginoferror,Z/2istheZvaluefora1-confidenceinteval,andS2isthesamplevariance,forwhichthereisapre-testestimate,PopulationSizeN100,000,N100,000,N100,000,SampleSizeExercise,Amortgagelendingcompanyhasdonebusinesswith2500customersincalendaryear1999.Thelastandmostimportantquestionofitscustomersatisfactionsurveyfortheyearaskstherespondent,“Wouldyouconsiderdoingbusinesswithusinthefuture?Yes,No”Asamplesurveyistaken,asopposedtoacompletecensussurvey,withamaximumerrorofplusorminus3%anda95%confidenceintervalandnoknowledgeaboutp?Whatwouldbeaprudentchoiceofn?,Forasimplerandomsampleandbinomialdata,wehavethefollowingrelationship:WhereEisthemarginoferror,Z/2istheZvaluefora1-confidenceinterval,andPisthesampleproportion,forwhichthereisapre-testestimate,SolutionbyHand,ExcelSpreadsheet,SampleSizeforProportions.xls,ExcelTable,SampleSizeforProportions.xls,QuestionsforSampleFrequency,Dowewanttomonitortheongoingprocesscontinuously?Whatsortofsamplinginterval?Howoftenisthesamplerefreshed?Howlongshouldaparticularsamplebecalledon?Whatsortofanalysisinterval?Shouldwesummarizeweekly,monthly,quarterly,annually?Whatisthedatareallygoingtobeusedfor?ProcesscontrolValidationofinternalmetricsManagementincentivesystems,TransactionalSurveys,SectionIVCustomerSatisfactionSurveys,CustomerSatisfactionModel,SupplierPerformance,CustomerExpectations,CustomerSatisfaction,WhatDoWeAsktheCustomers?,SupplierPerformance,CustomerExpectations,CustomerSatisfaction,Doweaskcustomersabouttheirperceptionofourperformance(oneachattribute)?,Doweaskcustomerswhatperformancetheyexpectfromus(oneachattribute)?,Doweaskcustomerswhethertheyaresatisfied?,CustomerSatisfactionModel,PerformanceOnAvailability,ExpectationsonAvailability,SatisfactionWithAvailability,OverallSatisfaction,PerformanceOnAccuracy,ExpectationsonAccuracy,SatisfactionWithAccuracy,HierarchyofCustomerSatisfaction,SatisfactionwithProductInformation,SatisfactionwithTimetoConnect,SatisfactionWithOrderPlacement,OverallSatisfaction,SatisfactionwithConditionofPackaging,SatisfactionwithTimetoDelivery,SatisfactionWithOrderDelivery,Primary,Secondary,Tertiary,IssueswithInstrumentDesign,MacroviewOverallcompositionFlowofthequestionsMicroviewSpecificwordingofquestionsScalingoftheresponses,SatisfactiononAllThreeLevels,IntroductionandrespondentqualificationOverallsatisfactionSatisfactionwithprocessoneSatisfactionwithattribute1aSatisfactionwithattribute1bSatisfactionwithattribute1cSatisfactionwithprocesstwoSatisfactionwithattribute2aSatisfactionwithprocessthreeDemographics(ifneeded)Thank-youandreleaseofinformation,PerformanceAsTertiaryQuestions,IntroductionandrespondentqualificationOverallSatisfactionSatisfactionwithProcessOnePerformanceonAttribute1aPerformanceonAttribute1bPerformanceonAttribute1cSatisfactionwithProcessTwoPerformanceonAttribute2aSatisfactionwithProcessThreeDemographics(ifneeded)Thank-youandReleaseofInformation,PerformanceandImportanceAsTertiaryQuestions,IntroductionandrespondentqualificationOverallsatisfactionSatisfactionwithprocessoneImportanceofattribute1aImportanceofattribute1bImportanceofattribute1cPerformanceonattribute1aPerformanceonattribute1bPerformanceonattribute1cSatisfactionwithprocesstwoImportanceofattribute2aPerformanceonattribute2aSatisfactionwithprocessthreeDemographics(ifneeded)Thank-youandreleaseofinformation,Exercise,Situation:wanttosurveycustomersatisfactionofanordertoinvoiceprocessprocessincludes:1)customerplacinganorderoverthephone,2)supplierpackagingtheorder,3)3rdpartyshippinganddelivery,and4)supplierinvoicesthecustomerAssignment:FlowcharttheprocessIdentifypotentialcustomersatisfiersanddissatisfiersChoosetheoutlineofthesurveyDescribethequalificationsofarespondentChoosethebasiclayoutofquestionsPreparepresentationforclass,MicroView,ExamplesofQuestions(wordingandscaling)StatedoverallsatisfactionStatedattributesatisfactionStatedattributeimportanceStatedattributeperformanceDerivedattributeimportance,StatedOverallSatisfaction,Example:Onascaleof1to7where1represents“VeryDissatisfied”and7represents“VerySatisfied”,howwouldyourateyouroverallsatisfactionwithUnitedAirlinesduringthistrip?,StatedAttributeSatisfaction,Example:Onascaleof1to7where1represents“VeryDissatisfied”and7represents“VerySatisfied”,howwouldyourateyoursatisfactionwiththefollowingaspectsofUnitedAirlinesduringthistrip?ontimedepartureontimearrivalphysicalcomfortworkenvironmentqualityoffoodpricesafety,StatedAttributeImportance,Example:Onascaleof1to7where1represents“Notatallimportant”and7represents“ExtremelyImportant”,howwouldyouratetheimportanceofthefollowingaspectsofairlineperformanceduringthistrip?ontimedepartureontimearrivalphysicalcomfortworkenvironmentqualityoffoodpricesafety,StatedAttributePerformance,Example:Onasc