logistic回归

1、logistic 回归1 .算法思想根据给定的数据集确定分类的边界.这个分类的边界就是我们所要求的回归函数.所谓的回归其实就是最正确拟合,回归函数就是确定最正确回归参数,然后对不同的特征赋予不同的权重2 .算法根底(1)所采用的映射函数是 sigmoid 函数,sigmoid函数比0-1函数(正方形波)更好的原 因是sigmoid函数在局部上看是平滑的,而在全局上看是接近跳跃的.而 0-1函数它本身是 跳跃的,不够平滑,误差比拟大.(2)根据回归函数计算出了一个结果,然后代入sigmoid 函数,就可以得到一个位于 0与1之间的函数值,然后根据这个函数值得大小就可以判断类别；如果是二类分类问题

2、值大于0.5属于1类,否那么属于0类(3)最正确回归系数确定的方法是梯度上升法：a.梯度上升法是用来求函数的最大值的,常说的梯度下降法是用来求函数的最小值的b.所谓的梯度其实就是数学中的导数,也就是数据变化最大的方向.一般用倒三角符号来表示梯度.c.公式为w= w+ a.tidu(f(w), 其中a是步长,该公式会一直被迭代直到次数到达某一个 值,或者到达某个误差允许的范围.3 .算法的优缺点优点：计算比拟简单,易于理解说明缺点：有可能会欠拟合适用的数据：标称数据和数值数据4 .算法的python 实现 (1)创造简单的数据from numpy import *from math import

3、 *import matplotlib.pyplot as plt# create the datadef createdata (filename):fr = open(filename, r) lines = fr.readlines() dataset = labelset = for each in lines:current_data = each.strip().split()dataset.append(1.0, float(current_data0), float(current_data1) labelset.append(int(current_data2)return

4、dataset, labelset(2)定义sigmoid 函数# define the sigmoid fuctiondef sigmoid (x): return 1.0/(1+ exp(-x)(3)定义梯度上升算法# define the gradascentdef gradascent (dataset, lableset): datamatrix = mat(dataset)y = mat(lableset).transpose() m, n = shape(datamatrix)a = 0.001maxloop = 500w = ones(n, 1) for i in range(

5、maxloop):l = datamatrix*wh = ones(m, 1)j =0 for each in l: hj = sigmoid(each) j += 1error = y - hw += a * datamatrix.transpose()*error return w(4)定义随机梯度下降算法,这是一个改良的算法,之所以它是一个改良的算法是由于他 节省了计算资源# improve the graddef gradimprove(dataset, datalable, times = 150):datamatrix = array(dataset)m,n = shape(dat

6、amatrix)weights = ones(n)for i in range(times): dataindex = range(m)for j in range(m): a = 4/(i + j + 10)+0.01 randindex = int(random.uniform(0, len(dataindex) t = sum(datamatrixrandindex*weights) h = sigmoid(t)error = datalablerandindex - hweights += a*datamatrixrandindex*error del (dataindexrandin

7、dex)return weights(5)绘制logstic函数# plot the regression functiondef plotregression (weights):datamat, datalable =createdata(F:data/machinelearninginaction/Ch05/testSet.txt) datastr = array(datamat) n = shape(datastr)0 x1 = 口 y1 = 口 x2 = 口 y2 = 口for i in range(n):if datalablei = 1:x1.append(datastri, 1

8、) y1.append(datastri, 2) else :x2.append(datastri, 1) y2.append(datastri, 2) fig = plt.figure() ax = fig.add_subplot(111) ax.scatter(x1, y1, s=30, c=red, marker=s) ax.scatter(x2, y2, s=30, c=green) x = arange(-3.0, 3.0, 0.1)y = (-weights0-weights1*x)/weights2ax.plot(x, y)plt.show()(6)对测试向量进行分类# clas

9、sify the vectordef classify (testdata, weights):testsum = sum(testdata*weights)classnum = sigmoid(testsum)if classnum 0.5: return 0else : return 1(7)进行十折交叉验证,这里针对的是判定马是否得病的案例# the multi testdef multitest (times):errorall = 0.0for i in range(times): error = horse() errorall += errorerrorrate = errora

10、ll/float(times)print the %d errorrate is %f % (times, errorrate) return errorrate(8)5.具体应用：判断一匹马是不是得病了# create the horse functiondef horse ():fr1 = open(F:data/machinelearninginaction/Ch05/horseColicTraining.txt)fr2 = open(F:data/machinelearninginaction/Ch05/horseColicTest.txt)lines = fr1.readlines(

11、)dataset = labelset = for each in lines:current_data = each.strip().split(t)vector = for i in range(21):vector.append(float(current_datai)dataset.append(vector)labelset.append(float(current_data21)weights = gradimprove(dataset, labelset, 500)test_lines = fr2.readlines()testdata = testlable = for eac

12、h in test_lines:current_data = each.strip().split(t)vector =for i in range(21):vector.append(float(current_datai)testdata.append(vector)testlable.append(float(current_data21)error = 0.0for i in range(len(testdata):lable = classify(testdatai, weights)if lable != testlablei:error += 1.0errorrate = error/float(len(testdata) p