神经网络研究报告.docVIP

实验室开放项目报告题目：神经网络研究与实现学生姓名：学 号：日 期： 2012年12月24日目录1 引言31.1 课题背景31.2 神经网络的理论基础32 实验分析33 实验训练部分设计43.1神经网络原理示意图43.2神经网络训练过程44. 部分功能实现截图75代码示例86. 总结与展望参考文献：25神经网络1 引言1.1 课题背景神经网络特有的非线性适应性信息处理能力，克服了传统人工智能方法对于直觉的缺陷，使之在神经专家系统、模式识别、智能控制、组合优化、预测等领域得到成功应用。神经网络与其它传统方法相结，将推动人工智能和信息处理技术不断发展。近年来，神经网络在模拟人类的认知的道路上更加深入发展，并与模糊系统、遗传算法、进化机制等结合，形成计算智能，成为人工智能的一个重要方向。在此背景下，本文论述了神经网络算法的理论与实现。初步了解这种新的算法，加深对信息处理的能力。1.2 神经网络的理论基础学习功能是神经网络最主要的特征之一，各种学习算法的研究，在神经网络理论和实践发展中起着重要作用。基于这种智能系统，研究者先后提出了感知器、线性、BP等算法。而各种算法的基础都是对权重和阈的一个修改和调整。而单层和多层的复杂程度区别，决定了人工神经网络的优越性和智能性。2 需求分析神经网络的形成，是个学习和训练的过程，要达到预想的智能性，必须经过大量的数据训练,大概归结为下面三点：1神经网络初始化和设置样本值2训练和支持停止训练3计算全局误差 本论文主要探讨training。3 实验训练部分设计3.1 神经网络原理示意图3.2 神经网络训练过程神经网络的实现是个判断，调整，记忆的过程。神经网络的训练过程包括三个方面：一是计算隐含层的输入和输出这个过程集中处理了PropagateLayer中，这个函数有3个参数，分别为net，upper(高层)Lower(底层)，这个函数主要是通过循环的调用这个函数，来求出高层神经元的输入和输出。神经元的输入是通过于前一层的每一个神经元之间的权值乘以那个神经元的输出的和，在函数中通过变量sum来求输入，然后我们使用sigmoid function来求出对应神经元的输出，然后保存到每一层的输出数组中去，而这个函数的功能也就完成了这些功能。void NET:PropagateLayer(NET* Net, LAYER* Lower, LAYER* Upper)/传播 INT i,j; REAL Sum; for (i=1; iUnits; i+) Sum = 0; for (j=0; jUnits; j+) Sum += Upper-Weightij * Lower-Outputj; Upper-Outputi = 1 / (1 + exp(-Net-Gain * Sum); 可以看出程序和算法中所说的一样，在计算权值的过程。从红色那个语句我们可以看出，它使用的是sigmoid function，这里的Net-Gain是一个系数，这个为1。sigmoid function=1 / (1 + exp(-x);二是调整网络的权值这个过程主要在两个函数中完成，这两个函数其实也就是整个BP网络的精髓所在，也是BP网络能学习的要点之处，这两个函数分别为ComputeOutputError和BackpropagateLayer，前者是用来求出上面输出层的输出值与我们的样本值(期望值)之间的差距，而后者则是通过ComputeOutputError求出的误差来调整一个变量，这个变量为每一个神经元的Error的值，这个值书上公式中的d，这个值与稍后的调成权值的有着直接的关系。ComputeOutputError ：这个函数处理的是输出层上的东西，通过对每一神经元的循环，然后比较每一个神经元的输出值与期望值(Target),然后通过公式(d(m,i)=(y(m,i)-y(s,i)*(y(m,i)*(1-y(m,i),m为输出层)来求出输出层的每一个神经元的误差，然后通过求和再求出net的总误差。void ComputeOutputError(NET* Net, REAL* Target) /计算输出与实际之间的误差 INT i; REAL Out, Err; Net-Error = 0; for (i=1; iOutputLayer-Units; i+) Out = Net-OutputLayer-Outputi; Err = Targeti-1-Out; Net-OutputLayer-Errori = Net-Gain * Out * (1-Out) * Err; Net-Error += 0.5 * sqr(Err);/这个就是标准的J函数，也就是整个net的误差 d(m,i)=(y(m,i)-y(s,i)*(y(m,i)*(1-y(m,i),m为输出层 Out * (1-Out) 对应于y(m,i)*(1-y(m,i) Err = Targeti-1-Out对应于y(m,i)-y(s,i)所以可以看出，在计算d(m,i)是完全与算法对应的。这里我们可以找到每一层的Error数组用来存储的是每一个神经元的d;这里我们已经计算出输出层的d了，我们就可以根据上面提到的公式去退出前几层的d；BackpropagateLayer：这个函数也有三个参数，分别为net,lower(底层),upper(高层)。这个函数是通过输出层的误差也反向的去改变前面几层的与权值直接相关的d这个值。通过循环的使用这个函数来有限次的改变这个值，这个值的计算的公式为d(k,i)=y(k,i)*(1-y(k,i)*w(k+1,l,i)*d(k+1,l)来计算的，程序中通过使用循环来求的上层权值与error的乘积之和，然后在求出这层的d值。 void BackpropagateLayer(NET* Net, LAYER* Upper, LAYER* Lower) INT i,j; REAL Out, Err; for (i=1; iUnits; i+) /i=1排除了输入层 Out = Lower-Outputi; Err = 0; for (j=1; jUnits; j+) Err += Upper-Weightji * Upper-Errorj; Lower-Errori = Net-Gain * Out * (1-Out) * Err; 公式为d(k,i)=y(k,i)*(1-y(k,i)*w(k+1,l,i)*d(k+1,l)，这里的k不是输出层循环中的Err += Upper-Weightji * Upper-Errorj;就是对应于w(k+1,l,i)*d(k+1,l)而Out * (1-Out)就是对应的y(k,i)*(1-y(k,i)，通过上面的调整，我们基本已经完成了对d进行了调整 三是修改权值公式：W(k,i,j)= - * d(k,i) * y(k-1,j)这个过程主要在AdjustWeights完成，这个函数主要是根据上BackpropagateLayer 求出来的error 值来调整每一层每一个神经元之间的权值。这个调整和一开始的赋值差不多，也是通过3重循环来改变，每一个神经元之间的权值。使用的公式就是我们上面提到的求W(k,i,j)的公式。不过这个的是Eta,而d就是每一个神经元的error的值，y就是上一层的某个神经元的输出值。 void AdjustWeights(NET* Net)/调整权重 INT l,i,j; REAL Out, Err, dWeight; for (l=1; lNUM_LAYERS; l+) for (i=1; iLayerl-Units; i+) for (j=0; jLayerl-1-Units; j+) Out = Net-Layerl-1-Outputj; Err = Net-Layerl-Errori; dWeight = Net-Layerl-dWeightij; Net-Layerl-Weightij += Net-Eta * Err * Out + Net-Alpha * dWeight; Net-Layerl-dWeightij = Net-Eta * Err * Out; Out = Net-Layerl-1-Outputj;为上一层的输出值，也就是公式中的Y(k-1,j) Net-Layerl-Weightij += Net-Eta * Err * Out + Net-Alpha * dWeight; 也就是对权值进行修改，其中 Net-Eta * Err * Out 是前面公式的W，也就是权值的变化量 4.部分功能实现截图 5神经网络的实现过程代码实现：#include #include #include typedef int BOOL;typedef int INT;typedef double REAL;#define FALSE 0#define TRUE 1#define NOT !#define AND &#define OR |#define MIN_REAL -HUGE_VAL#define MAX_REAL +HUGE_VAL#define MIN(x,y) (x)(y) ? (x) : (y)#define LO 0.1#define HI 0.9#define BIAS 1#define sqr(x) (x)*(x)typedef struct /* A LAYER OF A NET: */ INT Units; /* - number of units in this layer */ REAL* Output; /* - output of ith unit */ REAL* Error; /* - error term of ith unit */ REAL* Weight; /* - connection weights to ith unit */ REAL* WeightSave; /* - saved weights for stopped training */ REAL* dWeight; /* - last weight deltas for momentum */LAYER;class NET /* A NET: */private:LAYER* Layer; /* - layers of this net */ LAYER* InputLayer; /* - input layer */ LAYER* OutputLayer; /* - output layer */ REAL Alpha; /* - momentum factor */ REAL Eta; /* - learning rate */ REAL Gain; /* - gain of sigmoid function */ REAL Error; /* - total net error */public:void NormalizeSunspots();/使太阳黑子正常化void InitializeApplication(NET* Net);/初始化应用程序void FinalizeApplication(NET* Net);/完成应用程序void GenerateNetwork(NET* Net);/形成网络void RandomWeights(NET* Net);/随机重量void SetInput(NET* Net, REAL* Input);void GetOutput(NET* Net, REAL* Output);void SaveWeights(NET* Net);/保存void RestoreWeights(NET* Net);/恢复void PropagateLayer(NET* Net, LAYER* Lower, LAYER* Upper);/传播层void PropagateNet(NET* Net);/传播网络void ComputeOutputError(NET* Net, REAL* Target);/计算void BackpropagateLayer(NET* Net, LAYER* Upper, LAYER* Lower);void BackpropagateNet(NET* Net);void AdjustWeights(NET* Net);void SimulateNet(NET* Net, REAL* Input, REAL* Output, REAL* Target, BOOL Training);void TrainNet(NET* Net, INT Epochs);/训练网络void TestNet(NET* Net);/测试void EvaluateNet(NET* Net);/估价; #define NUM_LAYERS 3#define N 30#define M 1INT UnitsNUM_LAYERS = N, 10, M;#define FIRST_YEAR 1700#define NUM_YEARS 280#define TRAIN_LWB (N)#define TRAIN_UPB (179)#define TRAIN_YEARS (TRAIN_UPB - TRAIN_LWB + 1)#define TEST_LWB (180)#define TEST_UPB (259)#define TEST_YEARS (TEST_UPB - TEST_LWB + 1)#define EVAL_LWB (260)#define EVAL_UPB (NUM_YEARS - 1)#define EVAL_YEARS (EVAL_UPB - EVAL_LWB + 1)REAL Sunspots_NUM_YEARS;REAL Sunspots NUM_YEARS = 0.0262, 0.0575, 0.0837, 0.1203, 0.1883, 0.3033, 0.1517, 0.1046, 0.0523, 0.0418, 0.0157, 0.0000, 0.0000, 0.0105, 0.0575, 0.1412, 0.2458, 0.3295, 0.3138, 0.2040, 0.1464, 0.1360, 0.1151, 0.0575, 0.1098, 0.2092, 0.4079, 0.6381, 0.5387, 0.3818, 0.2458, 0.1831, 0.0575, 0.0262, 0.0837, 0.1778, 0.3661, 0.4236, 0.5805, 0.5282, 0.3818, 0.2092, 0.1046, 0.0837, 0.0262, 0.0575, 0.1151, 0.2092, 0.3138, 0.4231, 0.4362, 0.2495, 0.2500, 0.1606, 0.0638, 0.0502, 0.0534, 0.1700, 0.2489, 0.2824, 0.3290, 0.4493, 0.3201, 0.2359, 0.1904, 0.1093, 0.0596, 0.1977, 0.3651, 0.5549, 0.5272, 0.4268, 0.3478, 0.1820, 0.1600, 0.0366, 0.1036, 0.4838, 0.8075, 0.6585, 0.4435, 0.3562, 0.2014, 0.1192, 0.0534, 0.1260, 0.4336, 0.6904, 0.6846, 0.6177, 0.4702, 0.3483, 0.3138, 0.2453, 0.2144, 0.1114, 0.0837, 0.0335, 0.0214, 0.0356, 0.0758, 0.1778, 0.2354, 0.2254, 0.2484, 0.2207, 0.1470, 0.0528, 0.0424, 0.0131, 0.0000, 0.0073, 0.0262, 0.0638, 0.0727, 0.1851, 0.2395, 0.2150, 0.1574, 0.1250, 0.0816, 0.0345, 0.0209, 0.0094, 0.0445, 0.0868, 0.1898, 0.2594, 0.3358, 0.3504, 0.3708, 0.2500, 0.1438, 0.0445, 0.0690, 0.2976, 0.6354, 0.7233, 0.5397, 0.4482, 0.3379, 0.1919, 0.1266, 0.0560, 0.0785, 0.2097, 0.3216, 0.5152, 0.6522, 0.5036, 0.3483, 0.3373, 0.2829, 0.2040, 0.1077, 0.0350, 0.0225, 0.1187, 0.2866, 0.4906, 0.5010, 0.4038, 0.3091, 0.2301, 0.2458, 0.1595, 0.0853, 0.0382, 0.1966, 0.3870, 0.7270, 0.5816, 0.5314, 0.3462, 0.2338, 0.0889, 0.0591, 0.0649, 0.0178, 0.0314, 0.1689, 0.2840, 0.3122, 0.3332, 0.3321, 0.2730, 0.1328, 0.0685, 0.0356, 0.0330, 0.0371, 0.1862, 0.3818, 0.4451, 0.4079, 0.3347, 0.2186, 0.1370, 0.1396, 0.0633, 0.0497, 0.0141, 0.0262, 0.1276, 0.2197, 0.3321, 0.2814, 0.3243, 0.2537, 0.2296, 0.0973, 0.0298, 0.0188, 0.0073, 0.0502, 0.2479, 0.2986, 0.5434, 0.4215, 0.3326, 0.1966, 0.1365, 0.0743, 0.0303, 0.0873, 0.2317, 0.3342, 0.3609, 0.4069, 0.3394, 0.1867, 0.1109, 0.0581, 0.0298, 0.0455, 0.1888, 0.4168, 0.5983, 0.5732, 0.4644, 0.3546, 0.2484, 0.1600, 0.0853, 0.0502, 0.1736, 0.4843, 0.7929, 0.7128, 0.7045, 0.4388, 0.3630, 0.1647, 0.0727, 0.0230, 0.1987, 0.7411, 0.9947, 0.9665, 0.8316, 0.5873, 0.2819, 0.1961, 0.1459, 0.0534, 0.0790, 0.2458, 0.4906, 0.5539, 0.5518, 0.5465, 0.3483, 0.3603, 0.1987, 0.1804, 0.0811, 0.0659, 0.1428, 0.4838, 0.8127 ;REAL Mean;REAL TrainError;REAL TrainErrorPredictingMean;REAL TestError;REAL TestErrorPredictingMean;FILE* f;void NET:NormalizeSunspots() INT Year; REAL Min, Max; Min = MAX_REAL; Max = MIN_REAL; for (Year=0; YearNUM_YEARS; Year+) Min = MIN(Min, SunspotsYear); Max = MAX(Max, SunspotsYear); Mean = 0; for (Year=0; YearAlpha = 0.5; Net-Eta = 0.05; Net-Gain = 1; NormalizeSunspots(); TrainErrorPredictingMean = 0; for (Year=TRAIN_LWB; Year=TRAIN_UPB; Year+) for (i=0; iM; i+) Out = SunspotsYear+i; Err = Mean - Out; TrainErrorPredictingMean += 0.5 * sqr(Err); TestErrorPredictingMean = 0; for (Year=TEST_LWB; Year=TEST_UPB; Year+) for (i=0; iLayer = (LAYER*) calloc(NUM_LAYERS, sizeof(LAYER*); for (l=0; lLayerl = (LAYER*) malloc(sizeof(LAYER); Net-Layerl-Units = Unitsl; Net-Layerl-Output = (REAL*) calloc(Unitsl+1, sizeof(REAL); Net-Layerl-Error = (REAL*) calloc(Unitsl+1, sizeof(REAL); Net-Layerl-Weight = (REAL*) calloc(Unitsl+1, sizeof(REAL*); Net-Layerl-WeightSave = (REAL*) calloc(Unitsl+1, sizeof(REAL*); Net-Layerl-dWeight = (REAL*) calloc(Unitsl+1, sizeof(REAL*); Net-Layerl-Output0 = BIAS; if (l != 0) for (i=1; iLayerl-Weighti = (REAL*) calloc(Unitsl-1+1, sizeof(REAL); Net-Layerl-WeightSavei = (REAL*) calloc(Unitsl-1+1, sizeof(REAL); Net-Layerl-dWeighti = (REAL*) calloc(Unitsl-1+1, sizeof(REAL); Net-InputLayer = Net-Layer0; Net-OutputLayer = Net-LayerNUM_LAYERS - 1; Net-Alpha = 0.9; Net-Eta = 0.25; Net-Gain = 1;void NET:RandomWeights(NET* Net) INT l,i,j; for (l=1; lNUM_LAYERS; l+) for (i=1; iLayerl-Units; i+) for (j=0; jLayerl-1-Units; j+) Net-Layerl-Weightij = RandomEqualREAL(-0.5, 0.5); void NET:SetInput(NET* Net, REAL* Input) INT i; for (i=1; iInputLayer-Units; i+) Net-InputLayer-Outputi = Inputi-1; void NET:GetOutput(NET* Net, REAL* Output) INT i; for (i=1; iOutputLayer-Units; i+) Outputi-1 = Net-OutputLayer-Outputi; /* 支持停止训练 */void NET:SaveWeights(NET* Net) INT l,i,j; for (l=1; lNUM_LAYERS; l+) for (i=1; iLayerl-Units; i+) for (j=0; jLayerl-1-Units; j+) Net-Layerl-WeightSaveij = Net-Layerl-Weightij; void NET:RestoreWeights(NET* Net) INT l,i,j; for (l=1; lNUM_LAYERS; l+) for (i=1; iLayerl-Units; i+) for (j=0; jLayerl-1-Units; j+) Net-Layerl-Weightij = Net-Layerl-WeightSaveij; /* 信号的传播 */void NET:PropagateLayer(NET* Net, LAYER* Lower, LAYER* Upper) INT i,j; REAL Sum; for (i=1; iUnits; i+) Sum = 0; for (j=0; jUnits; j+) Sum += Upper-Weightij * Lower-Outputj; Upper-Outputi = 1 / (1 + exp(-Net-Gain * Sum); void NET:PropagateNet(NET* Net) INT l; for (l=0; lLayerl, Net-Layerl+1); /* backpropagating错误 */void NET:ComputeOutputError(NET* Net, REAL* Target) INT i; REAL Out, Err; Net-Error = 0; for (i=1; iOutputLayer-Units; i+) Out = Net-OutputLayer-Outputi; Err = Targeti-1-Out; Net-OutputLayer-Errori = Net-Gain * Out * (1-Out) * Err; Net-Error += 0.5 * sqr(Err); void NET:BackpropagateLayer(NET* Net, LAYER* Upper, LAYER* Lower) INT i,j; REAL Out, Err; for (i=1; iUnits; i+) Out = Lower-Outputi; Err = 0; for (j=1; jUnits; j+) Err += Upper-Weightji * Upper-Errorj; Lower-Errori = Net-Gain * Out * (1-Out) * Err; void NET:BackpropagateNet(NET* Net) INT l; for (l=NUM_LAYERS-1; l1; l-) BackpropagateLayer(Net, Net-Layerl, Net-Layerl-1); void NET:AdjustWeights(NET* Net) INT l,i,j; REAL Out, Err, dWeight; for (l=1; lNUM_LAYERS; l+) for (i=1; iLayerl-Units; i+) for (j=0; jLayerl-1-Units; j+) Out = Net-Layerl-1-Outputj; Err = Net-Layerl-Errori; dWeight = Net-Layerl-dWeightij; Net-Layerl-Weightij += Net-Eta * Err * Out + Net-Alpha * dWe