多元回归模型和建模

1、2019/11/19,Applied Stat for MBA05D1,1,多元回归模型与建模,2019,年,5,月,2019/11/19,Applied Stat for MBA05D1,2,一、多元线性回归问题,1,一元回归问题的困惑巴特勒,Butler,运输公司的例子,p661,行驶距离,英里,运送货物次数,行驶时间,小时,100,4 9.3,50,3,4.8,100 4,8.9,100,2,6.5,50,2,4.2,80,2,6.2,75,3 7.4,65 4,6,90 3,7.6,90,2,6.1,2019/11/19,Applied Stat for MBA05D1,3,2,做行驶

2、时间,行驶距离的一元回归,Coefficients t Stat,P-value,Intercept 1.273913,0.909454 0.389687,行驶距离,英里,0.067826 3.976755,0.00408,回归方程为,可以看出方程整体检验和自变量检验的,P,值为,0.0041,一元回归能够,显著成立。但是判定系数,偏小,说明有些因变量的解释,因素（例如运货次数）没有引入,0.6221,0.6641,2,2,R,R,0.664,2,R,x,y,0678,0,2739,1,2019/11/19,Applied Stat for MBA05D1,4,p,p,p,p,p,p,x,x,

3、x,y,E,x,x,x,N,y,N,e,e,x,x,x,y,2,2,1,1,0,2,2,2,1,1,0,2,2,2,1,1,0,0,3,那么,假定误差项,更多的自变量,多元回归理论模型,2019/11/19,Applied Stat for MBA05D1,5,4,来自,p,元回归模型的容量为,n,的样本,注意,的第,1,个脚码,k,表示变量编号,k=1,p,第,2,个脚码,i= 1,n,表示样本编号,2,1,2,2,22,12,1,1,21,11,n,pn,n,n,p,p,y,x,x,x,y,x,x,x,y,x,x,x,ki,x,2019/11/19,Applied Stat for MBA

4、05D1,6,5,多元回归总体模型和古典假定,总体模型表示式为,古典假定,1,E(e,i,0;,E(y,i,x,1i,p,x,pi,2,对于所有的,i,Var,e,i,3,e,i,是服从正态分布,N,0,的,4,对于不同的,e,i,e,j,i,j,是相互独立的,0,1,1,0,1,1,0,i,i,p,p,i,i,i,i,p,p,i,i,y,x,x,e,i,1,2,n,E,y,x,x,E,e,2019/11/19,Applied Stat for MBA05D1,7,6,多元线性回归方程的估计,2,i,0,1,1,2,2,0,1,0,1,2,0,1,1,2,2,0,1,min,i,i,i,p,p

5、i,p,p,p,p,k,k,i,p,y,x,x,x,b,b,b,b,y,b,b,x,b,x,b,x,b,使用最小二乘方法估计,得出,的估计值,估计的回归方程是,的含义是甚麽,2019/11/19,Applied Stat for MBA05D1,8,巴特勒公司二元线性回归模型的估计,自变量,x,1,行驶距离,x,2,运货次数,回归方程,2,1,923,0,0611,0,869,0,x,x,y,2019/11/19,Applied Stat for MBA05D1,9,7,多元回归方程变差分解和判定系数,R,2,总变差的分解,SST=SSR+SSE,多元判定系数,R,2,SSR/SST,多重相关

6、系数,r,调整（修正）的判定系数,巴特勒公司二元线性回归模型的判定系数,2,2,1,1,2,2,1,1,R,R,R,R,p,n,n,一般的有,876,0,904,0,2,2,R,R,2,2,R,R,adj,2019/11/19,Applied Stat for MBA05D1,10,8,对回归方程的检验,F,Test for Overall,Significance,问题：因变量和所有自变量之间是否存在显著,的关系,判定系数,R,2,可以,做方程的整体检验，但是遇,到分布的困难,检验假设,拒绝域,F,和,R,2,的关系,R,2,pF,n-p-1+pF,0,H,2,1,0,p,1,1,p,n,p

7、,F,F,p,n,SSE,p,SSR,MSE,MSR,2019/11/19,Applied Stat for MBA05D1,11,9,对回归系数的检验,t,Test for,Individual Significance,检验假设,检验统计量,拒绝域,0,0,1,0,k,k,H,H,的标准误差,是,其中,k,b,s,b,k,b,s,t,k,k,b,k,1,1,2,2,p,n,t,t,p,n,t,t,k,k,或者,2019/11/19,Applied Stat for MBA05D1,12,10,巴特勒公司线性回归模型的,Excel,输出,回归统计,R,0.951,R,2,0.904 adj,

8、R,2,0.876 s=0.573,n,10,方差分析,df,SS MS,F,Significance F,回归,2,21.601 10.800 32.878,0.00027624,残差,7,2.299 0.328,总计,9,23.9,Here: SSR=21.601, SSE=2.299, SST=23.9,系数估计和检验,Coefficients,标准误差,t,Stat,p,value,Intercept,0.8687 0.9515,0.9129,0.39163,行驶距离,英里,0.0611 0.0099,6.1824,0.00045,运送货物次数,0.9234,0.2211,4.1763

9、,0.00416,2019/11/19,Applied Stat for MBA05D1,13,11,多重共线性,Multicollinearity,1,巴特勒运输公司例题的修改,行驶距离,运送货物次数,修改数,行驶时间,100,4 (4) 9.3,50,3,2,4.8,100 4 (4,8.9,100,2,4,6.5,50,2 (2,4.2,80,2,3,6.2,75,3 (3) 7.4,65 4,3,6,90 3,4,7.6,90,2,4,6.1,2019/11/19,Applied Stat for MBA05D1,14,2,巴特勒运输公司例题的回归结果,一元回归方程,二元回归方程,运输

10、次数修改后的二元回归方程,F,检验,p,值,0.021,括弧内表示系数的,p,值,0.004,0.390,622,0,664,0,067,0,27,1,2,2,1,R,R,x,y,0.877,0.0042,0.0005,0.3916,904,0,9234,0,0611,0,8687,0,2,2,2,1,R,R,x,x,y,0.573,0.78,0.25,0.41,668,0,4671,0,0868,0,2989,1,2,2,2,1,R,R,x,x,y,2019/11/19,Applied Stat for MBA05D1,15,3,多重共线性问题讨论,巴特勒运输回归结果说明,增加解释变量不会降

11、低,R,2,的值,但是,adj R,2,的值却会降低,前两个回归方程的系数,p,值都很低,说明甚麽,后一个修改运输,次数的二元回归的两个系数,p,值都很高,以至通不过检验,但是后一个,方程总体检验的,F,值的,p,值却为,0.021(0.05,水平下方程成立,原因是修改运输次数数据,使得,x,1,x,2,的相关系数由,0.16,升至,0.97,发生了共线性,自变量发生多重共线性,会出现一些,甚至全部,变量通,不过检验,但是方程总体检验却能通过,此时的解释变量系数估计值,很不可靠,经验表明：解释变量数据彼此的相关系数绝对值大于,0.7,回归,结果就不可信,处理办法就是剔除,p,值高的变量,对,2

12、,个以上解释变量,自变,量的相关矩阵和方差膨胀因子,Variance Inflation Factors,简记,作,VIF,是识别多重共线性的有效方法,有专门软件加以精确检验,2019/11/19,Applied Stat for MBA05D1,16,12,利用模型进行预测,使用计算机软件产生回归模型,通过检验判断你的模型,直接利用模型可以预测自变量,x,01,x,02,x,0p,对应的因变量期望值,E,y,0,的估计,预测,E,y,0,和,y,0,的置信区域需要某些专门软,件,0,y,0,y,2019/11/19,Applied Stat for MBA05D1,17,13,多元回归的残差

13、分析,多元回归的残差分析作用方法和一元基,本相同,主要的差异在于,多自变量的观测值的杠,杆率,h,i,的计算比较复杂,需要使用专门软,件,回归分析建模应用中可以看到残差分析,的应用,2019/11/19,Applied Stat for MBA05D1,18,二,定性自变量,Qualitative Independent,Variable,1,虚拟变量,Dummy variable,方差分析中定性变量的解决方案：引入因子，处理,回归分析的解决方案：引入虚拟变量,如何定义虚拟变量,例,x,2,0,女性,x,2,1,男性,如何解释回归模型,期望值模型为,女性,男性,截距变化，斜率相同,2,2,1,

14、1,0,x,x,y,E,1,1,0,x,y,E,1,1,2,0,x,y,E,2019/11/19,Applied Stat for MBA05D1,19,2.Johnson,过滤水股份公司例子,Johnson,公司对遍布南弗罗里达州的,水过滤系统提供维修服务。为了估计服,务时间和成本，公司希望能够对顾客的,每一次维修请求预测必要的维修时间,他们收集的数据中包含就近一次维修至,今的时间（月数）、故障的类型（电子,和机械）以及相应的维修时间（小时,你能够建立起一个预测方程吗,2019/11/19,Applied Stat for MBA05D1,20,1)Johnson,公司数据,维修时间,小时,

15、最后维修至本次维修请求时间,月,故障类型,2.9 2,电子,1,3.0,6,机械,0,4.8,8,电子,1,1.8,3,机械,0,2.9 2,电子,1,4.9 7,电子,1,4.2,9,机械,0,4.8,8,机械,0,4.4,4,电子,1,4.5,6,电子,1,2019/11/19,Applied Stat for MBA05D1,21,散点图,维修时间-维修间隔散点图,0,1,2,3,4,5,6,0,2,4,6,8,10,维修间隔,维,修,时,间,有正相关的关系，可做一元回归。但是似乎可以看,出有两条接近平行的直线拟合这些散点,2019/11/19,Applied Stat for MBA0

16、5D1,22,2,建立维修时间,上次维修间隔,故障性质的回归方程,第一个回归方程,第二个回归方程,解释你得到的回归方程！讨论,x,2,的作用,括弧内表示系数的,p,值,二元比一元的判定系数增大许多,0.016,0.008,534,0,304,0,15,2,2,1,R,x,y,0.005,0.000,0.087,859,0,2627,1,3876,0,9305,0,2,2,1,R,x,x,y,2019/11/19,Applied Stat for MBA05D1,23,3,更复杂的定性变量,如果有,3,种定性状态，如何设虚拟变量,例,复印机销售地区是,A,B,C,三个地区,已知不同,地区销售量不

17、但与价格有关而且与地区也有关系,利用,回归分析建立销售量模型。设,x,1,是价格，还需要,2,个虚,拟变量,回归方程期望值表示为,地区,A,方程,地区,B,方程,地区,C,方程,注意,k,种状态,需要引入,k,1,个虚拟变量,其它,销售地区是,B,0,1,2,x,其它,销售地区是,C,0,1,3,x,3,3,2,2,1,1,0,x,x,x,y,E,1,1,2,0,x,y,E,1,1,3,0,x,y,E,1,1,0,x,y,E,2019/11/19,Applied Stat for MBA05D1,24,三、广义线性模型,有些复杂的曲线关系也可以用多元回归方法拟合,1,模拟高阶曲线关系,Curv

18、ilinear Relationships,1,Reynolds,公司是一家生产工业天平和实验室,设备的企业。公司管理人员想要对公司销售人员,的工作年限和天平的销售数量之间的关系进行研,究。他们随机抽取了,15,名销售人员，得到相应的,数据,2019/11/19,Applied Stat for MBA05D1,25,Reynolds,公司天平销售量与人员雇用月数,天,平,销售人员,天,平,销售人员,销售量,雇用月数,销售量,雇用月数,275,41 89,40,296,106 235,51,317,76 83,9,376,104 112 12,162,22 67,6,150,12 325,56

19、,367,85 189,19,308,111,2019/11/19,Applied Stat for MBA05D1,26,2,散点图和一元回归结果,0,50,100,150,200,250,300,350,400,0,20,40,60,80,100,120,销售人员雇用月数,天,平,销,售,量,2019/11/19,Applied Stat for MBA05D1,27,R,2,0.781174,Coefficients,标准误差,t,Stat,p,value,Intercept,111.22792,21.628002,5.1427738,0.000189,销售人员雇用月数,2.376774

20、9,0.3488932,6.8123279,1.239E-05,可以看出销售量和人员雇用月数的回归方程为,Sale = 111.23+2.38Months,0.00012,方程的显著性也很高。但是从散点图看出似乎有非线性,趋势，而且判定系数也不算大。从下页残差表和残差图,看出有明显非线性特征，考虑加入二次项,x,2,做为第二个,解释变量，做二阶回归,2019/11/19,Applied Stat for MBA05D1,28,Reynolds,公司案例,残差表,预测天平销售量,残差,标准残差,208.6756926 66.32430742,1.390020675,363.166061,67.1

21、6606097,1.407662093,291.862814,25.13718598,0.526823567,358.4125112,17.58748883,0.368597488,163.5169695 -1.516969516,0.031792552,139.7492205 10.25077947,0.214835193,313.2537881,53.7462119,1.126409738,375.0499355,67.04993546,1.405228342,206.2989177 -17.29891768,0.362549632,232.4434416,2.556558435,0.05

22、3580191,132.6188958 -49.61889584,1.039909707,139.7492205,27.74922053,0.581566423,125.4885711,58.48857114,1.225799805,244.3273161 80.67268394,1.69073305,156.3866448 32.61335518,0.683508652,2019/11/19,Applied Stat for MBA05D1,29,Reynolds,公司案例残差图,Reynolds公司标准残差,2,1.5,1,0.5,0,0.5,1,1.5,2,100,150,200,250

23、,300,350,400,预测值,标,准,残,差,2019/11/19,Applied Stat for MBA05D1,30,3,二阶回归结果,R,2,0.90,Coefficients,标准误差,t,Stat,p,value,Intercept,45.34758,22.77465,1.99114 0.0697,雇用月数,6.344807,1.057851,5.99782 6.24E-05,月数平方,0.03449,0.008948 -3.85388 0.0023,回归方程为,Sale = 45.35+6,34(Months)-0,35,Months,2,0.000) (0.002,整个方程

24、,F,检验的,p,值为,0.000,无论系数和方程高度显著通过检验,下页给出二阶回归的标准化残差，相当规范,2019/11/19,Applied Stat for MBA05D1,31,Reynolds,二阶回归残差,2,1.5,1,0.5,0,0.5,1,1.5,2,0,50,100,150,200,250,300,350,400,预测值,标,准,残,差,2019/11/19,Applied Stat for MBA05D1,32,2,因变量对数模型,1,汽车耗油问题,2,散点图,有负线性相关趋势,英里,加仑,28.7,29.2,34.2,27.9,33.3,26.4,23.9,30.5,1

25、8.1,19.5,14.3,20.9,车重,磅,2289,2113,2180,2448,2026,2702,2657,2106,3226,3213,3607,2888,例,12,辆汽车重量与油耗,英里/加仑问题散点图,0,10,20,30,40,2000,2500,3000,3500,4000,汽车重量(磅,英,里,加,仑,2019/11/19,Applied Stat for MBA05D1,33,3,一元回归,判定系数和变量系数都很显著，方程应该可以被接受,回归统计,Multiple R,0.9671524,R Square,0.9353838,Adj.R Squ,0.9289222,标准

26、误差,1.6705265,观测值,12,方差分析,df,SS,MS,F,Sign. F,回归,1,403.97591,403.975913,144.76005,2.85E-07,残差,10,27.906587,2.79065874,总计,11,431.8825,Coef,标准误差,t Stat,P-value,Intercept,56.09568,2.5821354,21.7245306,9.547E-10,Weight,0.011644,0.0009677,12.031627,2.85E-07,2019/11/19,Applied Stat for MBA05D1,34,4,一元回归残差分析

27、,残差呈楔形，有随汽车重量增加而增大的异方差趋势,英里,加仑油,标准残差,2,1.5,1,0.5,0,0.5,1,1.5,2,2.5,10,15,20,25,30,35,英里,加仑预测值,标,准,残,差,2019/11/19,Applied Stat for MBA05D1,35,5,因变量对数一元回归分析,E(lnY),0,1,x,系数显著性有提高,回归统计,Multiple R,0.9735,R Square,0.94771,Adj.R Squ,0.94248,标准误差,0.06425,观测值,12,方差分析,df,SS,MS,F,Sign. F,回归,1,0.7482155,0.7482

28、15,181.2244,9.84E-08,残差,10,0.0412867,0.004129,总计,11,0.7895021,Coef,标准误差,t Stat,P-value,Intercept,4.52423,0.0993186,45.5527,6.26E-13,车重(磅,0.0005,3.722E-05,13.462,9.84E-08,2019/11/19,Applied Stat for MBA05D1,36,6,因变量对数一元回归分析残差分析,标准残差分布比较均匀，方程可以更好的被接受,对数回归标准残差,2,1.5,1,0.5,0,0.5,1,1.5,2,2,2.5,3,3.5,4,ln

29、英里/加仑预测值,标,准,残,差,2019/11/19,Applied Stat for MBA05D1,37,3,其他常用的非线性变换为线性的公式,0,1,0,1,1,2,2,l,n,l,n,l,n,E,Y,x,E,Y,x,x,1,2,1,2,l,n,l,n,l,n,l,n,l,n,Y,A,xx,Y,A,x,x,1,0,E,Y,x,2019/11/19,Applied Stat for MBA05D1,38,四,变量选取方法,上面一些例子说明选取合适的解释变量至关重要,对,于一组备选的解释变量进行挑选,逐步回归,Stepwise,是,十分有效的方法。逐步回归建立在向前选择和向后消元的,基础之

30、上,逐步回归的基本思想是,备选的解释变量依照对因变,量的相关程度和在回归方程中的地位,按照一定的规则逐,步吸纳和剔除,直到不能吸纳和剔除为止,不少统计软件都具有逐步回归功能,例如,SAS,SPSS,Minitab,StaPro,等,2019/11/19,Applied Stat for MBA05D1,39,1,增加或删除变量的,F,检验,F,检验用来检验已含,x,1,x,k,的模型再增加自变量,x,k+1,或者从已含,x,1,x,k,x,k+1,删除,x,k+1,若,F,F,1,n,k,1)-1,则应该增加,或不删除,x,k+1,否则不,应增加,或删除,x,k+1,k,1,则有,S,S,E,

31、S,S,E,1,1,M,S,E,F,F,n,k,简,化,完,全,增,加,变,量,数,增,加,变,量,数,完,全,3,1,1,2,SSE,1,SSE,SSE,2,1,2,1,1,n,F,n,x,x,x,x,x,F,2019/11/19,Applied Stat for MBA05D1,40,增加或删除变量的,F,检验的巴特勒例题,巴特勒例题的一元回归和二元回归方程分别为,0.0041,括号内为变量系数的,p,值,0.0004) (0.0042,F,检验中的分子分母分别为,F,统计量的,p,值,0.0042,x,2,应该增加,或不应删除,可以看出,F,统计量的,p,值就是二元中,x,2,系数的,p

32、,值,1,0678,0,2739,1,x,y,2,1,9234,0,0611,0,869,0,x,x,y,3284,0,1,2,10,730,5,299,2,029,8,1,2,1,2,1,1,x,x,SSE,x,x,SSE,x,SSE,59,5,7,1,45,17,3284,0,730,5,0,5,0,F,F,2019/11/19,Applied Stat for MBA05D1,41,2,逐步回归的基本步骤,1,给定显著性水平,2,选择与被解释变量相关系数最高的变量做一元回归；如果,该变量,p,值不显著，则回归失败结束；否则一元回归方程成立,进入,3,3,在一元回归基础上利用,F,检验筛选

33、其余变量，选择其中显,著性水平,p,值）小于,且,F,值最大的一个变量做二元回归,如果不,存在这种变量,只能得出一元回归方程,回归结束；否则二元回,归成立,进入,4,4,在二元回归基础上利用,F,检验筛选其余变量，选择其中显,著性水平小于,且,F,值最大的一个变量做,3,元回归；如果不存在这,种变量,只能得出二元回归方程,回归结束；否则在引入,3,元基础上,进入第,5,步,2019/11/19,Applied Stat for MBA05D1,42,逐步回归的基本步骤（续,5,已有,k,个变量被引入基础上利用,F,检验对已被引入的变量,做检验，删除其中显著性水平,p,值）大于,且,F,值最小的

34、一个变,量，做,k,1,元回归，然后继续做删除检验（每次删除,1,个变量,直到没有符合被删除条件的变量为止,进入第,6,步,6,在,m,个变量被引入基础上利用,F,检验筛选未被引入的变量,选择其中显著性水平小于,且,F,值最大的一个变量做,m,1,元归,然后回到第,5,步；否则如果不存在这种变量,只能得出,m,元回归,方程,回归结束,1 2 3 4 5 6,结束,2019/11/19,Applied Stat for MBA05D1,43,3,逐步回归的几个问题,1,对于给定的显著性水平,逐步回归一定会结束,其结果唯一；不同的,回归结果不同,2,前三步只引进变量，不剔除变量,3,可以分别设定不

35、同的,进,和,出,但是要求,进,出,否则可能形成死循环不能结束回归,2019/11/19,Applied Stat for MBA05D1,44,4,大型问题分析,逐步回归的应用,教材,740,页提供,9,个变量的,Cravens,数据，讨论,8,个自变量对因,变量,SALES,的多元回归问题。相关系数阵为,利用,StaPro,软件做逐步回归，结果在以下各片,SALES,TIME,POTEN,ADV,SHARE,CHANGE,ACCTS,WORK,TIME,0.622881,POTEN,0.597812,0.453978,ADV,0.596177,0.24917,0.1741,SHARE,0.

36、483511,0.106105,0.21067,0.264458,CHANGE,0.489181,0.251469,0.268287,0.376516,0.08547,ACCTS,0.753986,0.757789,0.478643,0.200037,0.403011,0.327415,WORK,0.11722,0.17941,0.25884,0.27223,0.349345,0.28766,0.19885,RATING,0.401878,0.101172,0.358703,0.411462,0.02356,0.549333,0.228613,0.27691,2019/11/19,Applie

37、d Stat for MBA05D1,45,逐步回归的应用,第一步,Results of stepwise regression for SALES,Step 1 - Entering variable: ACCTS,Summary measures,Multiple R,0.7540,R-Square,0.5685,Adj R-Square,0.5497,StErr of Est,881.0930,ANOVA Table,Source,df,SS,MS,F,p-value,Explained,1,23524074.9269,23524074.9269,30.3018,0.0000,Unexp

38、lained,23,17855474.0000,776324.9565,Regression coefficients,Coefficient,Std Err,t-value,p-value,Lower limit,Upper limit,Constant,709.3239,515.2460,1.3767,0.1819,356.5423,1775.1900,ACCTS,21.7218,3.9460,5.5047,0.0000,13.5588,29.8847,2019/11/19,Applied Stat for MBA05D1,46,逐步回归的应用,第二步,Step 2 - Entering

39、variable: ADV,Summary measures,Change, Change,Multi. R,0.880398,0.126412,0.167658,R-Square,0.775101,0.206606,0.363426,Adj R-Sq,0.754656,0.204922,0.372766,StErr,650.3916,230.701,0.26184,ANOVA Table,Source,df,SS,MS,F,p-value,Explained,2,32073346,16036673,37.91093,7.44E-08,Unexplained,22,9306203,423009

40、.2,Regression coefficients,Coeffi,Std Err,t-value,p-value,Lower limit,Upper limit,Constant,50.29406,407.6095,0.123388,0.90292,795.037,895.6253,ACCTS,19.04827,2.972908,6.407283,1.9E-06,12.88283,25.21371,ADV,0.226527,0.050388,4.49562,0.00018,0.122028,0.331026,2019/11/19,Applied Stat for MBA05D1,47,逐步回

41、归的应用,第三步,Step 3 - Entering variable: POTEN,Summary measures,Change, Change,Multi. R,0.909793,0.029394,0.033388,R-Square,0.827723,0.052621,0.06789,Adj R-Sq,0.803112,0.048456,0.064209,StErr,582.6357,67.7559,0.10418,ANOVA Table,Source,df,SS,MS,F,p-value,Explained,3,34250798,11416933,33.6322,3.32E-08,Un

42、explained,21,7128751,339464.3,Regression coefficients,Coeffi,Std Err,t-value,p-value,Lower limit,Upper limit,Constant,327.235,394.4004,0.8297,0.416039,1147.44,492.9654,ACCTS,15.55396,2.999367,5.185747,3.87E-05,9.316432,21.79148,ADV,0.216071,0.045327,4.766888,0.000104,0.121807,0.310335,POTEN,0.021922,0.008656,2.532663,0.019359,0.003921,0.039922,2019/11/19,Applied Stat for MBA05D1,48,逐步回归的应用,第四步,Step 4 - Entering variable: SHARE,Summary measures,Change, Change,Multi. R,0.94892,0.039127,0.043007,R-Square,