FAULT DIAGNOSIS APPROACH BASED ON HIDDEN MARKOV MODEL AND SUPPORT VECTOR MACHINE (2).doc

chinese journal of mechanical engineering92- vol.20, no. 5, 2007liu guanjunliu xinminqlujinghu niaoqingcollege of mechatronics engineeringand automation,national university of defensetechnology,changsha 410073, chinafault diagnosis approach based on hidden markov model and support vector machine*abstract: aiming at solving the problems of machine-learning in fault diagnosis, a diagnosis approach is proposed based on hidden markov model (hmm) and support vector machine (svm). hmm usually describes intra-class measure well and is good at dealing with continuous dynamic signals. svm expresses inter-class difference effectively and has perfect classify ability. this approach is built on the merit of hmm and svm. then, the experiment is mad; in the transmission system of a helicopter. with the features extracted from vibration signals in gea.box, this hmm-svm based diagnostic approach is trained and used to monitor and diagnose th; ge-vaox s faults. the result shows that this method is better than !imm-based and svm-basod duvgnosiig methods in higher diagnostic accuracy with small training samples key words: hidden markov mode. support vector imchijie fzl: diagnosis0 introductiongearboxes are very imporiaul to the transmission system of a helicopter and they affect trie helicopters reliability and safety directly. it is significant to diagnose the faults of the gearboxes rapidly and efficiently. but the present machine-learning approaches (neural network, for example) that be used widely in condition monitoring and fault diagnosing have some shortcomings, such as: diagnosis is the matching result of the present information and the templates. the relation between former information and latter information is ignored. experiential risk minimization (erm) principle is adopted. so lots of samples are essential to train the practicable model. but these samples are very difficult to be acquired.hidden markov model (hmm) is a statistical model extended from markov model. hmm is capable of characterizing a doubly embedded stochastic process with an underlying stochastic process that, also unobservable (hidden), can be observed through another set of stochastic processes. hmm is a parametric model characterized by the state transition probabilities, the instantaneous probabilities of test outcomes given the system state and the initial state distribution. these parameters can be adaptively estimated by the well-known baum-welch algorithm. hmm, which is the statistical model of continuous dynamic series, has the precise data structures and reliable computation. now it becomes the dominating approach for speech recognitioni many researchers have applied hmm to condition monitoring and fault diagnosing2 3, and the favorable results that is better than that of the neural network is obtained4.support vector machine (svm) is a novel powerful machine-learning method with small-samples based on vc dimension theory and structural risk minimization (srm) principle. svm implements well trade-off between the quality of the approximation of the given data and the complexity of the approximating function by srm, and owns high generalization performance. the svm has more excellent feature than ann. and these features are as follows: the svm could take the optimal solution in the condition of a small number of samples. the optimal solution for the svm is transformed to solve a quadratic programming problem. in this method, the global optimum solution could be taken, but only the local optimum solution could be gained for the ann algorithm. (d algorithm for the svm transforms the sample space (ss) into the high dimensional feature space (hdfs) by the nonlinear transformation. in the hdfs,it structures linear classification function to achieve the nonlinear classification in the ss. it indicates that the machine learning has good generalization performance, and solves the dimensionality problem51. the svm has been successfully applied to fault diagnose because of their excellent classification ability6 71.hmm is good at dealing with sequential inputs, while svm shows superior performance in classification. furthermore, hmm usually provides an intra-class measure while svm proposes inter-class difference. since these two classifiers use different criteria, they can be combined to yield an ideal one. so a hybrid hmm and svm fault diagnostic approach is presented to solve the unstable fault diagnosing problems.1 hmm-svm based diagnostic model1.1 hidden markov modelshmm are extensions of markov models to include the case where the observations are probabilistic functions of the states rather than the states themselves. an hmm is characterized by several parameters. the first parameter is the transition probability distribution a=ag, where ay is the probability of being in state sj at time m-l provided that the state at time / is sh i.e.(1)kq.sqs,) i,jnwhere q, denotes the state at time / and n is the number of states. the second parameter of an hmm is the observation probability distribution, b= bjk)(2)b)(k) = p(okq,=sj) kj(=5,) 1/jvwhich is the probability of 5, being the initial state.a compact notation x=(aji,k) is used to define an hmm. the probability of a given observation sequence, 0=o, o2,-,ot, can be calculated as(4)p(0a) = z xs0fls,s,ajosj* this project is supported by national natural science foundation of china(no. 50375153). received january 9, 2007; received in revised form may 24, 2007; accepted june 14,2007the maximum likelihood (ml) method can be used to reestimate the model parameters, x=(aj, ic), as followschinese journal of mechanical engineering93 (7)(8)emj ok)(5)bl(k) =(,)where ntj is the expected number of transitions from s, to sj, nt is the expected number of transitions from sj, rrtj is the expected number of times in sj.training of an hmm for a given observation sequence can be realized by the so-called baum-welch algorithm. starting with initial or pre-estimated hmm parameters, the algorithm updates the parameters, by calculating the ml estimates, step by step increasing the probability of the observation sequence in each step. the training procedure along with the other features of hidden markov models is explained in detail in ref. 1.1.2 svm algorithmsthis section briefly introduces the theory of svm. a more detailed description of svm can be founr! in ref. 5.statistical learning theory(slt), which is a small-sample sia tistics introduced by va jn1 ic, si al in 1970s, provides us an uniform framework for i-iiral learning problem. and a novel powerful learning meihod ca led svm is developed based on it. the svm, which can solve small-sample learning problem, has been successfully applied in pattern recognition and function approximation.based on the structural risk minimization principle from the computational learning theory, svm seeks a decision surface to separate the training data points into two classes and makes decisions based on the support vectors that are selected as the only effective element from the training set.as for the binary classification, assume that the training set is(xl,ymxi,yi),-,(x.,y.) fle(l,-l) = l,2,-, (6)a separating hyper-plane divides it into two sides, each side containing points with the same class label only. the goal of the svm learning is to find the optimal separating hyper-plane (osh) that has the maximal margin to both sides. this can be formula-rized asj min (w) = -wf = -(ww) s.t. yi(w.x,) + b-qthe dual problem isi nitmin q(a) = -alajylyj(xlxj)-yiats.t. a, 0 i=-,2,n x.y,a, =0 the decision function ismin q(a) = -aiajylyjk(xixj)-1al(10)2 ij-i*-ins.t. 0z, c ( = l,2,-,n xjyz, =0rv,the weights can be calculated by minimizing the mean square value of the residual error over an analysis window31.the observational data of vibration signal of mechanical system behaves just like the linear auto-regressive model in that the next sample of the signal is relate to the p previous samples. so in this paper, the reflection coefficients of the polynomial transfer function of the linear auto-regressive model was chosen as condition features. the p was suitably chosen by the preexistent knowledge and system experiments.if there is periodicity in the fault signal, the preprocessing of getting rid of the periodicity must be taken. then, the linear auto-regressive model can be built.after preprocessing, every vibration signal is divided into t windows of equal length. the windows overlap each other with some points. here every window is 128 points long and overlaps 64 points each other. a set of features was extracted from each window. the features for a single window were selected to be the reflection coefficients of the polynomial transfer function of the linear auto-regressive model for that window. the order of the model here is 6. these features were taken as the state observation of the helicopters gearbox.2.2 training of the hmm-svm diagnostic model2.2.1 training of hmmfig. 2 initial markov chainbecause the global optimum result depends on the initial values of x, segmental *-means segmentation with clustering isbecause continuous gaussian mixed hmm is better than discrete hmm in smaller distortion and clearer classification, continuous gaussian mixed hmm with rapid left-to-right markov chain was adopted here. the number of states and mixed gaussian distribution should be chosen properly to arrive practicable hmm according to actual condition, the smaller the samples is, the fewer the number of states and mixed gaussian distribution are. in this paper, 4 states markov chain (fig. 2) was used and its initial distribution was x=(l, 0, 0, 0), the initial values of transition probability matrix a were uniform. two gaussian distributions were used in each state.here to obtain the good initial value. in order to increase the adaptability of hmm, multiple observation sequences baum-welch algorithm was used. that isx,=!(o#,(o/(0, i/i) i i nlli,(,0h(o)o)/p(o i x)1 7v-1(16)1/ jniwoow0!*)i t :j)0;j)ipx)t _ m 1-1 q,*vti=i 1=1where a is the forward variable, fl is the backward variable, l is the number of observation sequences11.a set of values of initial model is estimated through a-means segmentation with clustering algorithm, then normal hmm, dots-erosion hmm, crack hmm, roller fault hmm and outer race fault hmm were trained with 10 samples of each different condition. 2.2.2 training of svmthe hmm probability of the training samples were calculated by trained hmm. these results were normalized for svm to train.so far we have only discussed using svm to solve two-class problems. however, if we are interested in conducting fault modes classification experiments, we will need to choose multiple classes classifiers. the best method of extending the two-class classifiers to multi-class problems is not clear. previous work has generally constructed a one vs. all classifier for each class, or constructed a one vs. one classifier for each pair of classes. the one vs. all approach works by constructing for each class a classifier, which separates that class from the remainder of the data. a given test example is then classified as belonging to the class whose boundary maximizes. so the number of classifiers is equal to the number of classes. the one vs. one approach simple constructs for each pair of classes a classifier which separates those classes. a test example is then classified by all of the classifiers, and is said to belong to the class with the largest number of positive outputs from these sub-classifiers6 71. so the number of classifiers is equal to n(n+)/2, n is the number of classes. the one vs. one approach has the merit of being trained and expanded easily. so we choose one vs. one approach.commonly used kernels include: linear kernel, polynomial kernels, rbf kernels, sigmoid kernels. here we choose rbf-4fi(17)k(x,xi) = exp2r2the optimal values of the cand c parameters have been determined after a series of experiments and assumed a=1.62 and c=. the decomposition method is adopted to train the svm classifiers151. in this experiment, there are 5 different conditions, so c = 10, svm classifiers are trained.23 diagnostic results2.3.1 results of hmm-svm based diagnostic modelfirst the unknown samples were divided into 15 windows. 6-order coefficient of the ar model is extracted from each window. these form the observation series (7m5). then the probability p(0|a,)(i=l,2,-, 5) of every hmm is calculated by trained hmm through viterbi algorithm. these probabilities were normalized for svm. last svm classifiers made decision with voting manners.all 500 samples are discriminated by the hmm-svm diagnostic model. table 1 presents the diagnostic results. the resultschinese journal of mechanical engineering.95.show that all the samples are i