支持向量机matlab工具箱

1、2 内容该工具箱包括了二种分类,二种回归,以及一种一类支持向量机算法(1) Main_SVC_C.m - C_SVC二类分类算法(2) Main_SVC_Nu.m - Nu_SVC二类分类算法(3) Main_SVM_One_Class.m - One-Class支持向量机(4) Main_SVR_Epsilon.m - Epsilon_SVR回归算法(5) Main_SVR_Nu.m - Nu_SVR回归算法%-%3 使用(1) 目录下以Main_开头的文件即是主程序文件,直接按快捷键F5运行即可(2) 工具箱中所有程序均在Matlab6.5环境中调试通过，不能保证在Matlab其它版本正确

2、运行%-% % Support Vector Machine Matlab Toolbox 1.0 - C Support Vector Classification% Platform : Matlab6.5 / Matlab7.0% Copyright : LU Zhen-bo, Navy Engineering University, WuHan, HuBei, P.R.China, % E-mail : % Homepage : % Reference : Chih-Chung Chang,

3、 Chih-Jen Lin. LIBSVM: a Library for Support Vector Machines% Solve the quadratic programming problem - quadprog.mclcclearclose all% -% 定义核函数及相关参数C = 200; % 拉格朗日乘子上界ker = struct(type,linear);%ker = struct(type,ploy,degree,3,offset,1);%ker = struct(type,gauss,width,1);%ker = struct(type,tanh,gamma,1,

4、offset,0);% ker - 核参数(结构体变量)% the following fields:% type - linear : k(x,y) = x*y% poly : k(x,y) = (x*y+c)d% gauss : k(x,y) = exp(-0.5*(norm(x-y)/s)2)% tanh : k(x,y) = tanh(g*x*y+c)% degree - Degree d of polynomial kernel (positive scalar).% offset - Offset c of polynomial and tanh kernel (scalar, n

5、egative for tanh).% width - Width s of Gauss kernel (positive scalar).% gamma - Slope g of the tanh kernel (positive scalar).% -% 构造两类训练样本n = 50;randn(state,3);x1 = randn(n,2);y1 = ones(n,1);x2 = 5+randn(n,2);y2 = -ones(n,1);figure(1);plot(x1(:,1),x1(:,2),bx,x2(:,1),x2(:,2),k.);hold on;X = x1;x2; %

6、训练样本,nd的矩阵,n为样本个数,d为样本维数Y = y1;y2; % 训练目标,n1的矩阵,n为样本个数,值为+1或-1% -% 训练支持向量机ticsvm = C_SVC_Train(X,Y,C,ker);t_train = toc% svm 支持向量机(结构体变量)% the following fields:% ker - 核参数% x - 训练样本% y - 训练目标;% a - 拉格朗日乘子% -% 寻找支持向量a = svm.a;epsilon = 1e-8; % 如果小于此值则认为是0i_sv = find(aepsilon); % 支持向量下标plot(X(i_sv,1),

7、X(i_sv,2),ro);% -% 测试输出x1,x2 = meshgrid(-2:0.05:7,-2:0.05:7);rows,cols = size(x1);nt = rows*cols; % 测试样本数Xt = reshape(x1,nt,1),reshape(x2,nt,1);ticYd = C_SVC_Sim(svm,Xt); % 测试输出t_sim = tocYd = reshape(Yd,rows,cols);contour(x1,x2,Yd,0 0,m); % 分类面hold off;function K = CalcKernel(ker,x,y)% Calculate ke

8、rnel function. % x: 输入样本,n1d的矩阵,n1为样本个数,d为样本维数% y: 输入样本,n2d的矩阵,n2为样本个数,d为样本维数% ker 核参数(结构体变量)% the following fields:% type - linear : k(x,y) = x*y% poly : k(x,y) = (x*y+c)d% gauss : k(x,y) = exp(-0.5*(norm(x-y)/s)2)% tanh : k(x,y) = tanh(g*x*y+c)% degree - Degree d of polynomial kernel (positive sca

9、lar).% offset - Offset c of polynomial and tanh kernel (scalar, negative for tanh).% width - Width s of Gauss kernel (positive scalar).% gamma - Slope g of the tanh kernel (positive scalar).% ker = struct(type,linear);% ker = struct(type,ploy,degree,d,offset,c);% ker = struct(type,gauss,width,s);% k

10、er = struct(type,tanh,gamma,g,offset,c);% K: 输出核参数,n1n2的矩阵%-% 转成列向量x = x;y = y;%-%switch ker.typecase linearK = x*y;case ployd = ker.degree;c = ker.offset;K = (x*y+c).d;case gausss = ker.width;rows = size(x,2);cols = size(y,2); tmp = zeros(rows,cols);for i = 1:rowsfor j = 1:colstmp(i,j) = norm(x(:,i

11、)-y(:,j);endend K = exp(-0.5*(tmp/s).2);case tanhg = ker.gamma;c = ker.offset;K = tanh(g*x*y+c);otherwiseK = 0;endfunction svm = C_SVC_Train(X,Y,C,ker)% 输入参数:% X 训练样本,nd的矩阵,n为样本个数,d为样本维数% Y 训练目标,n1的矩阵,n为样本个数,值为+1或-1% C 拉格朗日乘子上界% ker 核参数(结构体变量)% the following fields:% type - linear : k(x,y) = x*y% po

12、ly : k(x,y) = (x*y+c)d% gauss : k(x,y) = exp(-0.5*(norm(x-y)/s)2)% tanh : k(x,y) = tanh(g*x*y+c)% degree - Degree d of polynomial kernel (positive scalar).% offset - Offset c of polynomial and tanh kernel (scalar, negative for tanh).% width - Width s of Gauss kernel (positive scalar).% gamma - Slope

13、 g of the tanh kernel (positive scalar).% 输出参数:% svm 支持向量机(结构体变量)% the following fields:% ker - 核参数% x - 训练样本% y - 训练目标;% a - 拉格朗日乘子% -% 解二次优化n = length(Y);H = (Y*Y).*Calckernel(ker,X,X);f = -ones(n,1);A = ;b = ;Aeq = Y;beq = 0;lb = zeros(n,1);ub = C*ones(n,1);a0 = zeros(n,1);options = optimset;opti

14、ons.LargeScale = off;options.Display = off;a,fval,eXitflag,output,lambda = quadprog(H,f,A,b,Aeq,beq,lb,ub,a0,options);eXitflag% -% 输出 svmsvm.ker = ker;svm.x = X;svm.y = Y;svm.a = a;function Yd = C_SVC_Sim(svm,Xt)% 输入参数:% svm 支持向量机(结构体变量)% the following fields:% ker - 核参数% type - linear : k(x,y) = x*

15、y% poly : k(x,y) = (x*y+c)d% gauss : k(x,y) = exp(-0.5*(norm(x-y)/s)2)% tanh : k(x,y) = tanh(g*x*y+c)% degree - Degree d of polynomial kernel (positive scalar).% offset - Offset c of polynomial and tanh kernel (scalar, negative for tanh).% width - Width s of Gauss kernel (positive scalar).% gamma -

16、Slope g of the tanh kernel (positive scalar).% x - 训练样本% y - 训练目标;% a - 拉格朗日乘子% Xt 测试样本,nd的矩阵,n为样本个数,d为样本维数% 输出参数:% Yd 测试输出,n1的矩阵,n为样本个数,值为+1或-1% -%ker = svm.ker;X = svm.x;Y = svm.y;a = svm.a;% -% 求 bepsilon = 1e-8; % 如果小于此值则认为是0i_sv = find(aepsilon); % 支持向量下标tmp = (Y.*a)*Calckernel(ker,X,X(i_sv,);

17、% 行向量b = 1./Y(i_sv)-tmp;b = mean(b);% -% 测试输出nt = size(Xt,1); % 测试样本数tmp = (Y.*a)*Calckernel(ker,X,Xt);Yd = sign(tmp+b); % Support Vector Machine Matlab Toolbox 1.0 - Nu Support Vector Classification% Platform : Matlab6.5 / Matlab7.0% Copyright : LU Zhen-bo, Navy Engineering University, WuHan, HuBei

18、, P.R.China, % E-mail : % Homepage : % Reference : Chih-Chung Chang, Chih-Jen Lin. LIBSVM: a Library for Support Vector Machines% Solve the quadratic programming problem - quadprog.mclcclearclose all% -% 定义核函数及相关参数nu = 0.2; % nu - (0,1 在支持向量数与错分样本数之间进行

19、折衷ker = struct(type,linear);%ker = struct(type,ploy,degree,3,offset,1);%ker = struct(type,gauss,width,1);%ker = struct(type,tanh,gamma,1,offset,0);% ker - 核参数(结构体变量)% the following fields:% type - linear : k(x,y) = x*y% poly : k(x,y) = (x*y+c)d% gauss : k(x,y) = exp(-0.5*(norm(x-y)/s)2)% tanh : k(x,

20、y) = tanh(g*x*y+c)% degree - Degree d of polynomial kernel (positive scalar).% offset - Offset c of polynomial and tanh kernel (scalar, negative for tanh).% width - Width s of Gauss kernel (positive scalar).% gamma - Slope g of the tanh kernel (positive scalar).% -% 构造两类训练样本n = 50;randn(state,3);x1

21、= randn(n,2);y1 = ones(n,1);x2 = 5+randn(n,2);y2 = -ones(n,1);figure(2);plot(x1(:,1),x1(:,2),bx,x2(:,1),x2(:,2),k.);hold on;X = x1;x2; % 训练样本,nd的矩阵,n为样本个数,d为样本维数Y = y1;y2; % 训练目标,n1的矩阵,n为样本个数,值为+1或-1% -% 训练支持向量机ticsvm = Nu_SVC_Train(X,Y,nu,ker);t_train = toc% svm 支持向量机(结构体变量)% the following fields:%

22、 ker - 核参数% x - 训练样本% y - 训练目标;% a - 拉格朗日乘子% -% 寻找支持向量a = svm.a;epsilon = 1e-8; % 如果小于此值则认为是0i_sv = find(aepsilon); % 支持向量下标plot(X(i_sv,1),X(i_sv,2),ro);% -% 测试输出x1,x2 = meshgrid(-2:0.05:7,-2:0.05:7);rows,cols = size(x1);nt = rows*cols; % 测试样本数Xt = reshape(x1,nt,1),reshape(x2,nt,1);ticYd = Nu_SVC_Si

23、m(svm,Xt); % 测试输出t_sim = tocYd = reshape(Yd,rows,cols);contour(x1,x2,Yd,0 0,m); % 分类面hold off;function K = CalcKernel(ker,x,y)% Calculate kernel function. % x: 输入样本,n1d的矩阵,n1为样本个数,d为样本维数% y: 输入样本,n2d的矩阵,n2为样本个数,d为样本维数% ker 核参数(结构体变量)% the following fields:% type - linear : k(x,y) = x*y% poly : k(x,y

24、) = (x*y+c)d% gauss : k(x,y) = exp(-0.5*(norm(x-y)/s)2)% tanh : k(x,y) = tanh(g*x*y+c)% degree - Degree d of polynomial kernel (positive scalar).% offset - Offset c of polynomial and tanh kernel (scalar, negative for tanh).% width - Width s of Gauss kernel (positive scalar).% gamma - Slope g of the

25、tanh kernel (positive scalar).% ker = struct(type,linear);% ker = struct(type,ploy,degree,d,offset,c);% ker = struct(type,gauss,width,s);% ker = struct(type,tanh,gamma,g,offset,c);% K: 输出核参数,n1n2的矩阵%-% 转成列向量x = x;y = y;%-%switch ker.typecase linearK = x*y;case ployd = ker.degree;c = ker.offset;K = (

26、x*y+c).d;case gausss = ker.width;rows = size(x,2);cols = size(y,2); tmp = zeros(rows,cols);for i = 1:rowsfor j = 1:colstmp(i,j) = norm(x(:,i)-y(:,j);endend K = exp(-0.5*(tmp/s).2);case tanhg = ker.gamma;c = ker.offset;K = tanh(g*x*y+c);otherwiseK = 0;endfunction svm = Nu_SVC_Train(X,Y,nu,ker)% 输入参数:

27、% X 训练样本,nd的矩阵,n为样本个数,d为样本维数% Y 训练目标,n1的矩阵,n为样本个数,值为+1或-1% nu 控制参数% ker 核参数(结构体变量)% the following fields:% type - linear : k(x,y) = x*y% poly : k(x,y) = (x*y+c)d% gauss : k(x,y) = exp(-0.5*(norm(x-y)/s)2)% tanh : k(x,y) = tanh(g*x*y+c)% degree - Degree d of polynomial kernel (positive scalar).% offs

28、et - Offset c of polynomial and tanh kernel (scalar, negative for tanh).% width - Width s of Gauss kernel (positive scalar).% gamma - Slope g of the tanh kernel (positive scalar).% 输出参数:% svm 支持向量机(结构体变量)% the following fields:% ker - 核参数% x - 训练样本% y - 训练目标;% a - 拉格朗日乘子% -% 解二次优化n = length(Y);H = (

29、Y*Y).*Calckernel(ker,X,X);f = zeros(n,1);A = -ones(1,n);b = -nu;Aeq = Y;beq = 0;lb = zeros(n,1);ub = ones(n,1)/n;a0 = zeros(n,1);options = optimset;options.LargeScale = off;options.Display = off;a,fval,eXitflag,output,lambda = quadprog(H,f,A,b,Aeq,beq,lb,ub,a0,options);eXitflag% -% 输出 svmsvm.ker = k

30、er;svm.x = X;svm.y = Y;svm.a = a;function Yd = Nu_SVC_Sim(svm,Xt)% 输入参数:% svm 支持向量机(结构体变量)% the following fields:% ker - 核参数% type - linear : k(x,y) = x*y% poly : k(x,y) = (x*y+c)d% gauss : k(x,y) = exp(-0.5*(norm(x-y)/s)2)% tanh : k(x,y) = tanh(g*x*y+c)% degree - Degree d of polynomial kernel (posi

31、tive scalar).% offset - Offset c of polynomial and tanh kernel (scalar, negative for tanh).% width - Width s of Gauss kernel (positive scalar).% gamma - Slope g of the tanh kernel (positive scalar).% x - 训练样本% y - 训练目标;% a - 拉格朗日乘子% Xt 测试样本,nd的矩阵,n为样本个数,d为样本维数% 输出参数:% Yd 测试输出,n1的矩阵,n为样本个数,值为+1或-1% -

32、%ker = svm.ker;X = svm.x;Y = svm.y;a = svm.a;% -% 求 bepsilon = 1e-8; % 如果小于此值则认为是0i_sv = find(aepsilon); % 支持向量下标tmp = (Y.*a)*Calckernel(ker,X,X(i_sv,); % 行向量b = 1./Y(i_sv)-tmp;b = mean(b);% -% 测试输出nt = size(Xt,1); % 测试样本数tmp = (Y.*a)*Calckernel(ker,X,Xt);Yd = sign(tmp+b); % Support Vector Machine M

33、atlab Toolbox 1.0 - One-Class Support Vector Machine% Platform : Matlab6.5 / Matlab7.0% Copyright : LU Zhen-bo, Navy Engineering University, WuHan, HuBei, P.R.China, % E-mail : % Homepage : % Reference : Chih-Chung Chang, Chih-Jen Lin. LIBSVM: a Librar

34、y for Support Vector Machines% Solve the quadratic programming problem - quadprog.mclcclearclose all% -% 定义核函数及相关参数nu = 0.15; % nu - 0,1 在支持向量数与错分样本数之间进行折衷% 支持向量机的 nu 参数(取值越小，异常点就越少)ker = struct(type,linear);%ker = struct(type,ploy,degree,3,offset,1);%ker = struct(type,gauss,width,200);%ker = struct

35、(type,tanh,gamma,1,offset,0);% ker - 核参数(结构体变量)% the following fields:% type - linear : k(x,y) = x*y% poly : k(x,y) = (x*y+c)d% gauss : k(x,y) = exp(-0.5*(norm(x-y)/s)2)% tanh : k(x,y) = tanh(g*x*y+c)% degree - Degree d of polynomial kernel (positive scalar).% offset - Offset c of polynomial and tan

36、h kernel (scalar, negative for tanh).% width - Width s of Gauss kernel (positive scalar).% gamma - Slope g of the tanh kernel (positive scalar).% -% 构造一类训练样本n = 100randn(state,1);x1 = randn(floor(n*0.95),2);x2 = 4+randn(ceil(n*0.05),2);X = x1;x2; % 训练样本,nd的矩阵,n为样本个数,d为样本维数figure(3);plot(x1(:,1),x1(:

37、,2),bx,x2(:,1),x2(:,2),k.);axis(-5 8 -5 8); hold on;% -% 训练支持向量机ticsvm = One_Class_SVM_Train(X,nu,ker);t_train = toc% svm 支持向量机(结构体变量)% the following fields:% ker - 核参数% x - 训练样本% y - 训练目标;% a - 拉格朗日乘子% -% 寻找支持向量a = svm.a;epsilon = 1e-10; % 如果小于此值则认为是0i_sv = find(aepsilon); % 支持向量下标plot(X(i_sv,1),X(

38、i_sv,2),ro);% -% 测试输出x1,x2 = meshgrid(-4:0.05:6,-4:0.05:6);rows,cols = size(x1);nt = rows*cols; % 测试样本数Xt = reshape(x1,nt,1),reshape(x2,nt,1);ticYd = One_Class_SVM_Sim(svm,Xt); % 测试输出t_sim = tocYd = reshape(Yd,rows,cols);contour(x1,x2,Yd,0 0,m); % 分类面hold off;function K = CalcKernel(ker,x,y)% Calcul

39、ate kernel function. % x: 输入样本,n1d的矩阵,n1为样本个数,d为样本维数% y: 输入样本,n2d的矩阵,n2为样本个数,d为样本维数% ker 核参数(结构体变量)% the following fields:% type - linear : k(x,y) = x*y% poly : k(x,y) = (x*y+c)d% gauss : k(x,y) = exp(-0.5*(norm(x-y)/s)2)% tanh : k(x,y) = tanh(g*x*y+c)% degree - Degree d of polynomial kernel (positi

40、ve scalar).% offset - Offset c of polynomial and tanh kernel (scalar, negative for tanh).% width - Width s of Gauss kernel (positive scalar).% gamma - Slope g of the tanh kernel (positive scalar).% ker = struct(type,linear);% ker = struct(type,ploy,degree,d,offset,c);% ker = struct(type,gauss,width,

41、s);% ker = struct(type,tanh,gamma,g,offset,c);% K: 输出核参数,n1n2的矩阵%-% 转成列向量x = x;y = y;%-%switch ker.typecase linearK = x*y;case ployd = ker.degree;c = ker.offset;K = (x*y+c).d;case gausss = ker.width;rows = size(x,2);cols = size(y,2); tmp = zeros(rows,cols);for i = 1:rowsfor j = 1:colstmp(i,j) = norm

42、(x(:,i)-y(:,j);endend K = exp(-0.5*(tmp/s).2);case tanhg = ker.gamma;c = ker.offset;K = tanh(g*x*y+c);otherwiseK = 0;endfunction svm = One_Class_SVM_Train(X,tmp,nu,ker)% 输入参数:% X 训练样本,nd的矩阵,n为样本个数,d为样本维数% nu 控制参数% ker 核参数(结构体变量)% the following fields:% type - linear : k(x,y) = x*y% poly : k(x,y) = (

43、x*y+c)d% gauss : k(x,y) = exp(-0.5*(norm(x-y)/s)2)% tanh : k(x,y) = tanh(g*x*y+c)% degree - Degree d of polynomial kernel (positive scalar).% offset - Offset c of polynomial and tanh kernel (scalar, negative for tanh).% width - Width s of Gauss kernel (positive scalar).% gamma - Slope g of the tanh

44、kernel (positive scalar).% 输出参数:% svm 支持向量机(结构体变量)% the following fields:% ker - 核参数% x - 训练样本% y - 训练目标;% a - 拉格朗日乘子% -% 解二次优化n = size(X,1);H = Calckernel(ker,X,X);f = zeros(n,1);for i = 1:nf(i, = -Calckernel(ker,X(i,X(i,);endA = ;b = ;Aeq = ones(1,n);beq = 1;lb = zeros(n,1);ub = ones(n,1)/(nu*n);a

45、0 = zeros(n,1);options = optimset;options.LargeScale = off;options.Display = off;a,fval,eXitflag,output,lambda = quadprog(H,f,A,b,Aeq,beq,lb,ub,a0,options);eXitflag% -% 输出 svmsvm.ker = ker;svm.x = X;svm.y = ;svm.a = a;function Yd = One_Class_SVM_Sim(svm,Xt)% 输入参数:% svm 支持向量机(结构体变量)% the following fi

46、elds:% ker - 核参数% type - linear : k(x,y) = x*y% poly : k(x,y) = (x*y+c)d% gauss : k(x,y) = exp(-0.5*(norm(x-y)/s)2)% tanh : k(x,y) = tanh(g*x*y+c)% degree - Degree d of polynomial kernel (positive scalar).% offset - Offset c of polynomial and tanh kernel (scalar, negative for tanh).% width - Width s of Gauss kernel (positive scalar).% gamma - Slope g of the tanh kernel (positive scalar).% x - 训练样本% y - 训练目标;% a - 拉格朗日乘子% Xt 测试样本,nd的矩阵,n为样本个数,d为样本维数% 输出参数:% Yd 测试输出,n1的矩阵,n为样本个数,值为