FAULT DIAGNOSIS APPROACH BASED ON HIDDEN MARKOV MODEL AND SUPPORT VECTOR MACHINE.doc

fault diagnosis approach based on hidden markov model and support vector machine*abstract: aiming at solving the problems of machine-learning in fault diagnosis, a diagnosis approach is proposed based on hidden markov model (hmm) and support vector machine (svm). hmm usually describes intra-class measure well and is good at dealing with continuous dynamic signals. svm expresses inter-class difference effectively and has perfect classify ability. this approach is built on the merit of hmm and svm. then, the experiment is mad; in the transmission system of a helicopter. with the features extracted from vibration signals in gea.box, this hmm-svm based diagnostic approach is trained and used to monitor and diagnose th; ge-vaox s faults. the result shows that this method is better than !imm-based and svm-basod duvgnosiig methods in higher diagnostic accuracy with small training samples key words: hidden markov mode. support vector imchijie fzl: diagnosis0 introductiongearboxes are very imporiaul to the transmission system of a helicopter and they affect trie helicopters reliability and safety directly. it is significant to diagnose the faults of the gearboxes rapidly and efficiently. but the present machine-learning approaches (neural network, for example) that be used widely in condition monitoring and fault diagnosing have some shortcomings, such as: diagnosis is the matching result of the present information and the templates. the relation between former information and latter information is ignored. experiential risk minimization (erm) principle is adopted. so lots of samples are essential to train the practicable model. but these samples are very difficult to be acquired.hidden markov model (hmm) is a statistical model extended from markov model. hmm is capable of characterizing a doubly embedded stochastic process with an underlying stochastic process that, also unobservable (hidden), can be observed through another set of stochastic processes. hmm is a parametric model characterized by the state transition probabilities, the instantaneous probabilities of test outcomes given the system state and the initial state distribution. these parameters can be adaptively estimated by the well-known baum-welch algorithm. hmm, which is the statistical model of continuous dynamic series, has the precise data structures and reliable computation. now it becomes the dominating approach for speech recognitioni many researchers have applied hmm to condition monitoring and fault diagnosing2 3, and the favorable results that is better than that of the neural network is obtained4.support vector machine (svm) is a novel powerful machine-learning method with small-samples based on vc dimension theory and structural risk minimization (srm) principle. svm implements well trade-off between the quality of the approximation of the given data and the complexity of the approximating function by srm, and owns high generalization performance. the svm has more excellent feature than ann. and these features are as follows: the svm could take the optimal solution in the condition of a small number of samples. the optimal solution for the svm is transformed to solve a quadratic programming problem. in this method, the global optimum solution could be taken, but only the local optimum solution could be gained for the ann algorithm. (d algorithm for the svm transforms the sample space (ss) into the high dimensional feature space (hdfs) by the nonlinear transformation. in the hdfs,it structures linear classification function to achieve the nonlinear classification in the ss. it indicates that the machine learning has good generalization performance, and solves the dimensionality problem51. the svm has been successfully applied to fault diagnose because of their excellent classification ability6 71.hmm is good at dealing with sequential inputs, while svm shows superior performance in classification. furthermore, hmm usually provides an intra-class measure while svm proposes inter-class difference. since these two classifiers use different criteria, they can be combined to yield an ideal one. so a hybrid hmm and svm fault diagnostic approach is presented to solve the unstable fault diagnosing problems.1 hmm-svm based diagnostic model1.1 hidden markov modelshmm are extensions of markov models to include the case where the observations are probabilistic functions of the states rather than the states themselves. an hmm is characterized by several parameters. the first parameter is the transition probability distribution a=ag, where ay is the probability of being in state sj at time m-l provided that the state at time / is sh i.e.where q, denotes the state at time / and n is the number of states. the second parameter of an hmm is the observation probability distribution, b= bjk)where ok is the atb observation and m is the number of distinct observations. if the observations are continuous, a continuous probability density function, generally a weighted sum of several gaussian distributions, is assigned to each state.the last parameter is the initial state distribution, which is the probability of 5, being the initial state.a compact notation x=(aji,k) is used to define an hmm. the probability of a given observation sequence, 0=o, o2,-,ot, can be calculated asthe maximum likelihood (ml) method can be used to reestimate the model parameters, x=(aj, ic), as followswhere ntj is the expected number of transitions from s, to sj, nt is the expected number of transitions from sj, rrtj is the expected number of times in sj.training of an hmm for a given observation sequence can be realized by the so-called baum-welch algorithm. starting with initial or pre-estimated hmm parameters, the algorithm updates the parameters, by calculating the ml estimates, step by step increasing the probability of the observation sequence in each step. the training procedure along with the other features of hidden markov models is explained in detail in ref. 1.1.2 svm algorithmsthis section briefly introduces the theory of svm. a more detailed description of svm can be founr! in ref. 5.statistical learning theory(slt), which is a small-sample sia tistics introduced by va jn1 ic, si al in 1970s, provides us an uniform framework for i-iiral learning problem. and a novel powerful learning meihod ca led svm is developed based on it. the svm, which can solve small-sample learning problem, has been successfully applied in pattern recognition and function approximation.based on the structural risk minimization principle from the computational learning theory, svm seeks a decision surface to separate the training data points into two classes and makes decisions based on the support vectors that are selected as the only effective element from the training set.as for the binary classification, assume that the training set isa separating hyper-plane divides it into two sides, each side containing points with the same class label only. the goal of the svm learning is to find the optimal separating hyper-plane (osh) that has the maximal margin to both sides. this can be formula-rized asthe nonlinear support vector machine maps the input variable into a high dimensional feature space, and applies the linear support vector machine in the feature space. it turns out that the computation of this linear svm in the feature space can be carried out in the original space through a kernel trick. therefore we do not really need to know the feature space and the transformation to the feature space. commonly used kernels include: linear kernel, polynomial kernels, rbf kernels, sigmoid kernels.1.3 hmm-svm based fault diagnosishw.m based fault diagnosis is determined by the largest probability cutpu! of all hmms. an unknown signal that is dis-iwbsd by noise nury make uiany hmms to output almost the same probabilities. lii this condition, a potential error may occur if dscison rtas made only by largest probability. svm based approach diagnoses fault according to the present signal feature and ignores the relationship between former and latter signals. an impossible result may win in this competition, then a mistake occurs.hmm uses sequential inputs to recognize the patterns and provides an intra-class measure, while svm shows superior performance in classification and proposes inter-class difference. since these two classifiers use different criteria, they can be combined to yield an ideal one that get rid of the shortcomings of hmm or svm. precise classification of svm can remedy the shortcoming of hmm that makes decision only by the largest probability. and according to the characteristic of hmm that is good at dealing with sequential inputs, hmm can be used to compute the matching degree between the unknown signal and all hmms. the probability output is used as feature for svm to decide. the error of svm based approach that judges only by the feature of single moment can be reduced. thereby a hybrid hmm and svm fault diagnostic approach is presented.the principle of hybrid hmm-svm based fault diagnostic model is shown in fig. 1. at first, features are extracted from the training samples after they are preprocessed. then these features are used to train hmm through baum-welch algorithm, and the unknown signals probability outputs of different hmms are calculated by viterbi algorithm. these features are used to train svm, too. for the diagnosis of the fault, feature is extracted from the unknown signal after it is preprocessed. then matching degrees between the unknown signal and all hmms are calculated. at last, the features and probability outputs are sent to svm to discriminate, and the diagnostic result is achievedthus the problem can be converted like that with kernel k2 hmm-svm based diagnostic of gearboxthe experimental object is the transmission system of a helicopter that worked at the rotate speed of 2 800 r/min and the output twist moment of 600 n m. the diagnostic experiment is done through replacing normal components with some faulty ones. the vibration signals are detected by the acceleration sensors. then they are amplified and sampled with 10 khz sample rate. data was gathered on five different conditions: normal;dots-erosion of gear; crack of gear; roller fault; outer race fault. for each condition 100 vibration samples are collected. therefore, experimental data consisted of 500 vibration samples and each sample has 1 024 points data. ten samples of each condition are randomly drawn as training samples; all 500 signals are used to test the performance of the hmm-svm based diagnostic approach.the diagnostic process consists of data preprocess and feature extraction and model training and fault diagnosing.2.1 feature extractiona linear auto-regressive model can be used to predict the value of the next sample of a signal as a linear combination of the previous samples. the next sample of the signal, j, is predictedas the weighted sum of the p previous samples, .vi, j_2 ,s*-p and can be expressed asthe transfer function of the model is given bythe residual error, e is defined as the difference between the actual and the predicted values of the next sample and can be expressed asthe weights can be calculated by minimizing the mean square value of the residual error over an analysis window31.the observational data of vibration signal of mechanical system behaves just like the linear auto-regressive model in that the next sample of the signal is relate to the p previous samples. so in this paper, the reflection coefficients of the polynomial transfer function of the linear auto-regressive model was chosen as condition features. the p was suitably chosen by the preexistent knowledge and system experiments.if there is periodicity in the fault signal, the preprocessing of getting rid of the periodicity must be taken. then, the linear auto-regressive model can be built.after preprocessing, every vibration signal is divided into t windows of equal length. the windows overlap each other with some points. here every window is 128 points long and overlaps 64 points each other. a set of features was extracted from each window. the features for a single window were selected to be the reflection coefficients of the polynomial transfer function of the linear auto-regressive model for that window. the order of the model here is 6. these features were taken as the state observation of the helicopters gearbox.2.2 training of the hmm-svm diagnostic model2.2.1 training of hmmbecause continuous gaussian mixed hmm is better than discrete hmm in smaller distortion and clearer classification, continuous gaussian mixed hmm with rapid left-to-right markov chain was adopted here. the number of states and mixed gaussian distribution should be chosen properly to arrive practicable hmm according to actual condition, the smaller the samples is, the fewer the number of states and mixed gaussian distribution are. in this paper, 4 states markov chain (fig. 2) was used and its initial distribution was x=(l, 0, 0, 0), the initial values of transition probability matrix a were uniform. two gaussian distributions were used in each state because the global optimum result depends on the initial values of x, segmental *-means segmentation with clustering is here to obtain the good initial value. in order to increase the adaptability of hmm, multiple observation sequences baum-welch algorithm was used. that iswhere a is the forward variable, fl is the backward variable, l is the number of observation sequences11.a set of values of initial model is estimated through a-means segmentation with clustering algorithm, then normal hmm, dots-erosion hmm, crack hmm, roller fault hmm and outer race fault hmm were trained with 10 samples of each different condition. 2.2.2 training of svmthe hmm probability of the training samples were calculated by trained hmm. these results were normalized for svm to train.so far we have only discussed using svm to solve two-class problems. however, if we are interested in conducting fault modes classification experiments, we will need to choose multiple classes classifiers. the best method of extending the two-class classifiers to multi-class problems is not clear. previous work has generally constructed a one vs. all classifier for each class, or constructed a one vs. one classifier for each pair of classes. the one vs. all approach works by constructing for each class a classifier, which separates that class from the remainder of the data. a given test example is then classified as belonging to the class whose boundary maximizes. so the number of classifiers is equal to the number of classes. the one vs. one approach simple constructs for each pair of classes a classifier which separates those classes. a test example is then classified by all of the classifiers, and is said to belong to the class with the largest number of positive outputs from these sub-classifiers6 71. so the number of classifiers is equal to n(n+)/2, n is the number of classes. the one vs. one approach has the merit of being trained and expanded easily. so we choose one vs. one approach.commonly used kernels include: linear kernel, polynomial kernels, rbf kernels, sigmoid kernels. here we choose rbthe optimal values of the cand c parameters have been determined after a series of experiments and assumed a=1.62 and c=. the decomposition method is adopted to train the svm classifiers151. in this experiment, there are 5 different conditions, so c = 10, svm classifiers are trained.23 diagnostic results2.3.1 results of hmm-svm based diagnostic modelfirst the unknown samples were divided into 15 windows. 6-order coefficient of the ar model is extracted from each window. these form the observation series (7m5). then the probability p(0|a,)(i=l,2,-, 5) of every hmm is calculated by trained hmm through viterbi algorithm. these probabilities were normalized for svm. last svm classifiers made decision withvoting manners.all 500 samples are discriminated by the hmm-svm diagnostic model. table 1 presents the diagnostic results. the resultsshow that all the samples are identified correctly.in table 1, al means normal, a2 mea