关 键 词：

风力 发电 并网 系统 控制 分析

资源描述：

风力发电并网系统控制分析,风力,发电,并网,系统,控制,分析

内容简介：

Seediscussions,stats,andauthorprofilesforthispublicationat:/publication/268966235OnHeterogeneousMachineLearningEnsemblesforWindPowerPredictionCONFERENCEPAPERJANUARY2015DOI:10.1016/j.renene.2015.11.073CITATION1READS302AUTHORS:JustinHeinermannCarlvonOssietzkyUniversittOldenburg10PUBLICATIONS13CITATIONSSEEPROFILEOliverKramerCarlvonOssietzkyUniversittOldenburg159PUBLICATIONS535CITATIONSSEEPROFILEAllin-textreferencesunderlinedinbluearelinkedtopublicationsonResearchGate,lettingyouaccessandreadthemimmediately.Availablefrom:JustinHeinermannRetrievedon:14March2016On Heterogeneous Machine Learning Ensembles for Wind Power PredictionJustin Heinermann and Oliver KramerDepartment of Computing Science, University of Oldenburg, 26111 Oldenburg, Germanyjustin.philipp.heinermann, oliver.krameruni-oldenburg.deAbstractFor a sustainable integration of wind power into theelectricity grid, a precise prediction method is required.In this work, we investigate the use of heterogeneousmachine learning ensembles for wind power prediction.We first analyze homogeneous ensemble regressors thatmake use of a single base algorithm and compare deci-sion trees to k-nearest neighbors and support vector re-gression. As next step, we construct heterogeneous en-sembles that make use of multiple base algorithms andbenefit from a gain of diversity of the weak predictors.In the experimental evaluation, we show that a combi-nation of decision trees and support vector regressionoutperforms state-of-the-art predictors (improvementsof up to 37% compared to support vector regression)as well as homogeneous ensembles while requiring ashorter runtime (speed-ups from 1.60 to 8.78). Theexperiments are based on large wind time series datafrom simulations and real measurements.1IntroductionPrecise predictions of the wind power production are crucialfor the integration of wind turbines into the power grid. Be-sides numerical weather predictions, machine learning algo-rithms yield good forecasting results (Kramer, Treiber, andGieseke 2013; Salcedo-Sanz et al. 2014). E.g., support vec-tor regression (SVR) or k-nearest neighbors (k-NN) regres-sion can provide suitable results for short-term forecast hori-zons. However, there are two main drawbacks: First, in orderto reach the best prediction accuracy possible with these al-gorithms, the computation time for both training and testinggrows infeasibly large. Second, the prediction performanceneeds to be improved further to cope with the actual energymarkets needs.It has been shown that a good alternative to the well-known machine learning algorithms is to combine severalbasic models to machine learning ensembles: By employinga number of so-called weak predictors, which use standardmachine learning algorithms, and combining their outputs,the accuracy of classification and regression can be amelio-rated while reducing the computation time, see DietterichCopyright c ? 2015, Association for the Advancement of ArtificialIntelligence (). All rights reserved.(2000). Especially for real-world problems, machine learn-ingensemblesarepromisingapproaches.Incontrasttostate-of-the-art machine learning algorithms, ensemble methodsrequire less tuning and expert domain knowledge. Neverthe-less, in order to find an optimal ensemble predictor, usuallya trade-off between multiple objectives has to be made: Forexample, a lower prediction error is often achieved by in-vesting more computation time, see Rokach (2010).In this work, we discuss the practical use of bagging en-sembles for the task of wind power forecasting, which canbe formulated as regression problem. We investigate whichensemble setting can offer the best results in a reasonableruntime. In the first step, we compare homogeneous ensem-ble predictors, where every weak predictor makes use of thesame base algorithm. In this paper, we use decision trees,k-NN regression, and support vector regression as base al-gorithms.Going further, we propose the use of heterogeneous en-semble predictors for wind power prediction. We investigatethe question, if heterogeneous ensembles that consist of dif-ferent types of base predictors, perform better than homo-geneous ensembles. A reason for the success is the gain ofdiversity of the weak predictors. Our comprehensive exper-imental results show that a combination of Decision Trees(DT) and SVR yields better results than the analyzed homo-geneous predictors while offering a decent runtime behavior.1.1Related WorkKusiak, Zhang and Song (2009) successfully apply differ-ent methods to short-term wind power prediction, one ofwhich is the bagging trees algorithm. Fugon et al. (2008)compare various algorithms for wind power forecasting andshow that random forests with and without random inputselection yield a prediction performance similar to SVR,but recommend to prefer a linear model when the compu-tation time grows too large. Heinermann and Kramer (2014)achieve good wind power prediction results using supportvector regression ensembles.In the field of numerical weather forecasts, it is quite com-mon to use ensemble postprocessing: Gneiting et al. (2005)found ensembles to reduce the prediction error by apply-ing the ensemble model output statistics (EMOS) methodto diverse weather forecasts. Similarly, Thorarinsdottir andGneiting (2010) are using a so-called “heteroscedastic cen-54Computational Sustainability: Papers from the 2015 AAAI Workshopsored regression” for maximum wind speed prediction overthe American Pacific Northwest.A similar domain to wind power prediction is time se-ries prediction for solar power output: Chakraborty et al.(2012) built up an ensemble of a weather forecast-drivenNa ve Bayes Predictor as well as a kNN-based and a Motif-based machine learning predictor. The results of the threepredictors are combined with a Bayesian Model Averaging.Chakraborty et al. show that the prediction error can be re-duced by inducing ensemble methods to forecasting poweroutput.2BackgroundIn this section, we briefly describe our machine learning ap-proach to wind power prediction and provide the relevantbackground to machine learning ensembles.2.1Machine Learning Approach to Wind PowerPredictionShort term wind power prediction can be treated as a regres-sion problem. In contrast to numerical weather predictions,statistical learning methods usually make only use of thetime series data itself. The objective is to predict the mea-surement after a forecast horizon . The input patterns con-sist of a past of time steps, which we call the feature win-dow. In this work, we use a spatio-temporal model based onthe one proposed by Kramer, Treiber, and Gieseke (2013),which showed the benefit of involving neighboring turbinesto the input vector. Let pi(t) be the measurement of a turbineiatatimet,and1 i mtheindicesofthemneighboringturbines. Then, for a target turbine with index j we define apattern-label-pair (x,y) for a given time t0as p1(t0 ).p1(t0).pm(t0 ).pm(t0)! pj(t0+ )(1)In our experiments, we use the NREL western wind re-sources dataset1. It consists of simulated wind power out-put for 32,043 wind turbines in the US, given in 10-minutetime resolution for the years 2004 - 2006. For every turbine,there are 157,680 wind speed and power output measure-ments available.2.2Machine Learning Ensembles for SupervisedLearningThe idea of ensemble methods can be described as as build-ing “a predictive model by integrating multiple models”(Rokach 2010). One of the advantages is the possible im-provement of prediction performance. Another reason forutilizing ensemble methods is the reduction of computa-tional complexity, which can be helpful on very large datasets. There are countless variants of different ensemble al-gorithms. A comprehensive overview and empirical analy-sis for ensemble classification is given by Bauer and Kohavi(1999). Another, more up-to-date review paper is given byRokach (2010).1/An important and famous approach is bagging, whichstands for bootstrap aggregating, and was introduced byBreiman (1996). The main idea is to build independent pre-dictors using samples of the training set and average (orvote)the outputofthesepredictors. Breimanshowsthat bag-ging ensembles of decision trees as well as regression treeswork very well in comparison with single trees. Further-more, he gives arguments for the question “why baggingworks” (Breiman 1996). A popular variant of bagging ap-proaches is the random forest algorithm (Breiman 2001) that“uses a large number of individual, unpruned decision trees”(Rokach 2010). Every decision tree is built with a subsetsample from the training set, but only uses N of the avail-able features of the patterns.There exist more sophisticated ensemble approaches likeAdaBoost (Freund, Schapire, and others 1996) or StackedGeneralization (Wolpert 1992), but, as we want to give aproof of concept, we limit ourselves here to bagging.One key ingredient to successful building of ensem-bles is the concept of diversity: All the weak predictorsshould behave different if not uncorrelated (Rokach 2010;Brown et al. 2005). Then, the ensemble prediction improves.There are many ways to generate such diversity, like manip-ulating the used training sample, the used features, and theweak predictors parameters. Another possibility is the hy-bridization. An example is given by Kramer et al. (2012):support vector machine and k-NN classification are com-bined and complement each others behavior.3Regression Ensembles for Wind PowerPredictionOur objective is to find out, if heterogeneous machine learn-ing ensembles are a superior alternative to state-of-the-artmachine learning predictors. We decided to implement arelatively simple bagging approach with weighting, whichhas some advantages. While the implementation is straight-forward and offers a moderate computational cost, we con-sider the approach sufficient for a proof of concept, whichis also shown in the experimental evaluation. Another mo-tivation for this kind of ensemble algorithm is that the fa-mous Random Forest method yields very good results, too,and is relatively fast compared to boosting algorithms. Thelatter ones could outperform bagging algorithms for someapplications, but also tend to overfitting, and can hardly beparallelized. A comparison to more sophisticated ensembleapproaches like AdaBoost as well as stacked generalizationwill be subject to future work.3.1Ensemble AlgorithmOur algorithm is outlined in Algorithm 1. As usual in su-pervised learning, a training set Xtrainwith known labelsis given. The most important meta-parameters of the algo-rithm are the number n of weak predictors, the number s ofsamples, and the types of base algorithms used for each pre-dictor. Both n and s have to be chosen large enough in orderto provide satisfying results. However, the best choice for nand s depends on the base algorithm used, which also influ-ences the runtime significantly. The main work of the algo-55Algorithm 1 Training of Ensemble Predictor1: Inputs:Xtrain= (x1,y1).,(xm,ym) Rd RNumber of predictors: nNumber of samples: sTypes of Algorithm to use: A=ai|i 1.n2: Returns:Predictors: P = pi|i = 1,.,nWeights: W = wi R|i = 1,.,n3: for i = 1 to n do4:Xsample sample(Xtrain,s)5:Xval Xtrain Xsample6:pi trainPredictor(ai,Xsample)7:wi1MSE(pi,Xval)8: end forrithm takes place in the for-loop beginning in line 5. Eachpass trains one weak predictor piand its assigned weightwi. For each weak predictor, a subset of Xtrainwith sizes is sampled and used as training set Tifor the particularpredictor pi. In order to calculate weight wi, the remainingtraining patterns are used as a validation set Tval. The valuewiis then obtained by testing pion Tvaland taking the in-verse of the mean squared error (MSE).When the training algorithm computed the predictors andweights, for an unknown distance x the predicted label iscomputed by:b y =Pki=1wi pi(x)Pki=1wi(2)Each predictors output pi(x) is computed and then weightedby wiin the resulting weighted average. In a realistic sce-nario, one would perform all calls of piin parallel usingmulti- or manycore processors. In our experiments, we onlyemployed one CPU core for the runtime measurements.As pointed out by Ho (1998), random forests and bag-ging classifiers in general my benefit from sampling fromthe feature space, i.e., taking only a random subset of theavailable features into account. Besides the number of fea-tures used, the choice can be done with or without replace-ment. Although replacement is sometimes considered asuseful (Rokach 2010; Dietterich 2000), there is no explicitrule when to use it. In a preliminary experiment, we foundno evidence that random feature subspaces help to increasethe accuracy, but of course can help to decrease runtime.Because we did not find a significant difference betweensampling with replacement and sampling without replace-ment, we employ sampling without replacement. For the ba-sic comparison of the weak predictors, all features are used.For the comparison of heterogeneous ensembles with state-of-the-art predictors, the number of features sampled will beconsidered again because of the possible trade-off betweenruntime and accuracy. Concerning the sampling from thetraining set, we also found no evidence for the supremacyof replacement, but due to the recommendations in litera-ture (Breiman 1996; Rokach 2010), we decided to employsampling with replacement in the following experiments.3.2Choice of the Base Algorithms and Trainingof Weak PredictorsSince the number of possible settings is huge, one has tomake some assumptions to limit the number of combina-tions. First, we want to give an overview over the choice ofbase algorithms and parameters.The decision tree algorithm is a powerful yet simple toolfor supervised learning problems (Hastie, Tibshirani, andFriedman 2009). The main idea is to build a binary tree,which partitions the feature space into a set of rectangles.The assignment of the unknown pattern to one of these rect-angles is used for computing the sought label. While thereare different algorithms for building up decision trees, welimit ourselves to the famous CART algorithm (Breiman etal. 1984). Besides moderate computational costs, the mainadvantage of decision trees is their interpretability.The SVR algorithm often provides very good predictionresults and is regarded as state-of-the-art regression tech-nique. In general, the support vector machine (SVM) algo-rithm maximizes a geometric margin between the instancesof the classes to be separated. Similar to SVM classifica-tion, the SVR algorithm aims at finding a prediction func-tionf : Rd R R that computes an accurate predictionvalue for an unseen pattern x Rd. For more detailed in-formation about the algorithm, we refer to, e.g., (Steinwartand Christmann 2008). We utilized a RBF kernel and chooseregularization parameter C = 10,000 and kernel bandwidth = 1e 5.A famous yet relatively simple approach for classificationand regression is the k-nearest neighbors (k-NN) model, see(Hastie, Tibshirani, and Friedman 2009). The prediction av-erages the label information of the k nearest neighbors, i.e.,via f(x) =1kPiNk(x)yi, where Nkdenotes the set of in-dices for the k nearest neighbors in T w.r.t. a distance metriclike Euclidean distance. While a na ve implementation takesO(|S| |T|) time for a training set T and a test set S, moreefficient implementations with spatial data structures, e.g.k-d trees, are available (Hastie, Tibshirani, and Friedman2009). Logarithmic runtime is offered for small dimension-alities d 15. For parameter k, we make a random choicein the interval 1;25.In a preliminary experiment, we compare different regres-sion algorithms composed to ensembles: Table 1 shows acomparison of decision trees, SVR, and k-NN used as weakpredictors. A general observation is that increasing n ands decreases the prediction error. With the given n and s, noclear decision between decision trees and SVR can be made,but we stick to these two basic algorithms in the further ex-periments rather than k-NN.3.3Heterogeneous EnsemblesOf course, when dealing with forecasting tasks, the firstgoal is to reach the lowest error possible. While it is pos-sible to decrease the prediction error by using ensembles,a feasible runtime is equally important. If it takes hours totrain a regressor for one turbine, a model would be unus-able for a large number of turbines to be forecasted thetime needed for parameter-tuning and cross-validation not56Table 1: Comparison (MSE) of ensemble predictors consisting of different base algorithms used as weak predictor. For everyturbine, the best result is printed in bold. Each experiment has been repeated 10 times.Base AlgorithmDecision TreeSVRk-NNn323225625632322562563232256256s5001,0005001,0005001,0005001,0005001,0005001,000Casper11.1710.9310.8910.6210.9910.8410.9510.8713.1012.4413.0212.44Las Vegas10.8410.6110.5110.2710.2610.2610.2710.2712.8412.3612.8112.30Hesperia7.987.827.767.597.627.617.607.599.418.969.368.98Reno14.7614.5314.4714.1914.1114.1014.0013.9818.9218.0319.1418.14Vantage7.316.977.006.836.616.586.576.578.447.938.438.07mentioned. Therefore, our goal is to reach a good predictionperformance as well as a short runtime for both training andtesting.As seen in Table 1, there is no clear answer which algo-rithm to prefer. We propose to use heterogeneous ensemblesbuiltuponSVRanddecisiontreeregressors.Asshowninthefollowing experiments, the result is a robust prediction algo-rithm that offers a reasonable runtime behavior. The exper-iments in the following section address the question, if het-erogeneous ensembles offer a better performance than ho-mogeneous ensembles. The second question is: Can hetero-geneous ensembles help to decrease the computation timeneeded?4Experimental ResultsIn our experiments, we analyze heterogeneous ensemblesbuilt upon SVR and decision tree regressors. As data source,we use the NREL western wind resources dataset. Eachgrid point has a maximum power output of 30MW and 10-minute datais given forthe years 2004- 2006. Therefore, forevery station there are 157,680 wind speed and power out-put measurements available. In our experiments, we use thepower output data of ten wind parks2that consist of the tar-get wind turbine and the 10 neighbored turbines. Therefore,As training data set, the data from 01/2004 until 06/2005 isused and the data from 7/2005 until 12/2006 serves as testdata set. The experiments were run on an Intel Core i5 at3.10GHz with 8GB of RAM. The algorithms were imple-mented in Python utilizing the kNN, decision tree, and SVRimplementations of Scikit-learn (Pedregosa et al. 2011). Asmetric for the prediction performance we use the MSE.4.1Mixed Ensemble with Coefficient First, we analyze heterogeneous ensembles that employ bothSVR predictors and decision trees. We define a coefficient that specifies the amount of weak predictors trained with thedecision tree algorithm. Hence, 1 determines the num-ber of SVR predictors in the ensemble. Figure 1 shows theexperimental results for two wind turbines. For both (a) and(b), three ensemble settings were analyzed showing MSEand training time: s is set to 1,000 and n is set to 32, 64, or128. The higher the number of predictors is, the lower the2The corresponding IDs of the turbines in the NREL datasetare: Casper = 23167, Hesperia = 2028, Las Vegas = 6272, Reno =11637, Vantage = 28981prediction error becomes. But also the runtime increase isnotably.3With = 0, we observe an ensemble with onlySVR predictors, with = 1, only decision trees are cho-sen. The interesting point is that, given a sufficient numbern, the prediction quality becomes maximal with an in themiddle range. This points to an advantageous behavior ofheterogeneous ensembles: With an of approximately 0.5,the training and testing time of the ensemble predictor de-creases dramatically compared to = 0.0. Therefore, theparameter can be seen as explicit tuning parameter for thetrade-off between prediction performance and runtime. 6.5 6.6 6.7 6.8 6.9 7 7.1 0 0.2 0.4 0.6 0.8 1MSE32x100064x1000128x1000 0 100 200 300 400 500 0 0.2 0.4 0.6 0.8 1Training Time32x100064x1000128x1000(a) Mixture for a wind tur-bine near Vantage 10 10.1 10.2 10.3 10.4 10.5 10.6 0 0.2 0.4 0.6 0.8 1MSE32x100064x1000128x1000 0 100 200 300 400 500 0 0.2 0.4 0.6 0.8 1Training Time32x100064x1000128x1000(b) Mixture for a wind tur-bine near Las VegasFigure 1: Mixing of SVR and DT. With a balanced mixture,a better MSE is reached within a shorter training time.4.2Meta-Ensemble Combining SVR Ensemblesand Decision Tree EnsemblesWhile the results of the former experiment are promising,we have to point out that the parameter was varied forfixednands.Onehastoconsiderthatdifferentweakpredic-tors show different behavior and could benefit from differentcombinations of n and s. I.e., we will see that a large amountof predictors is a good possibility to ameliorate decision treeensembles while the runtime does not suffer as much as inSVR ensembles. Instead of combining some SVR predictorsand some decision trees, one could possibly better combinea huge number of decision trees and maybe also increasesample number s.Therefore, we have to give a fair comparison, which con-siders both prediction performance and runtime. First, we3The testing time was similar to the training time.57Table 2: Behavior of different setups of ensembles, whichare then combined.(a) SVR ensemblessetupnsttrainttestMSE145007.587.1110.78241,00014.6913.0910.43342,00029.9425.4610.5843250059.7355.9110.305321,000117.86105.6110.266322,000245.85206.6810.35764500119.40111.9110.208641,000236.26211.8310.269642,000489.92410.8110.26(b) Decision Tree EnsemblessetupnsttrainttestMSE13220004.913.6510.442324,0007.743.8510.2733240,00081.963.8210.0642562,00042.6728.8610.1352564,00063.1729.1710.05625640,000733.1530.359.8671,0242,000161.60113.9710.1581,0244,000258.03115.5510.0391,02440,0002,859.18136.719.82(c) Combinations of one DT ensemble and on SVR ensemble toone heterogeneous ensemble. For every combination, the MSEis shown. The row denotes which SVR ensemble is employed,whereas the column shows which DT ensemble is used.D1D2D3D4D5D6D7D8D9S110.1310.049.8710.059.999.8210.069.999.81S210.039.969.759.979.919.709.979.909.69S310.059.999.819.999.949.7710.009.949.76S410.029.959.769.959.909.729.979.899.71S59.969.899.729.909.859.679.919.849.66S69.989.929.759.929.879.709.939.869.70S79.989.919.729.919.869.689.929.869.67S89.999.929.749.929.879.699.939.869.68S99.979.909.739.909.859.689.919.849.67analyze the behavior of the homogeneous ensembles basedon SVR or decision trees. We try to find good combinations,which are computable in a feasible time. The result can beseen in Table 2: Like assumed, one can train more decisiontrees with a larger sample in the same time than SVR pre-dictors. The central point of the experiment is the equally-weighted combination of one SVR ensemble and one DTensemble at a time to one heterogeneous ensemble. The re-sults of these combinations are depicted in Table 2(c), whichhas the form of a matrix. In every cell of the matrix, the usedSVR ensemble is given by the row and the used decision treeensemble is given by the column. In the table, only the MSEis given for clear arrangement. The training and test timesfor one predictor is approximately the sum of the respec-tive times of the two combined ensembles. Besides the verypromising results, which outperform the homogeneous en-sembles, we also can see that the combination of two weakerensembles takes less time to deliver the same prediction er-ror. We visualize this behavior for two wind turbines in Fig-ure 2.4.3Comparison with State-of-the-Art PredictorsAt last, we give a comparison of our heterogeneous ensem-ble method to SVR, which is considered as state-of-the-artregressor. Because SVR outperformed k-NN in all cases,we do not visualize k-NN results. The parameters k andaccordingly C and were optimized with a 10-fold cross- 9.6 9.8 10 10.2 10.4 10.6 10.8 0 500 1000 1500 2000 2500 3000 3500MSETraining Timedecsvrdec+svr(a) Las Vegas: MSE dependingon training time 9.6 9.8 10 10.2 10.4 10.6 10.8 0 100 200 300 400 500 600MSETest Timedecsvrdec+svr(b) Las Vegas: MSE dependingon test time 6.3 6.4 6.5 6.6 6.7 6.8 6.9 7 0 500 1000 1500 2000 2500 3000 3500MSETraining Timedecsvrdec+svr(c) Vantage: MSE dependingon training time 6.3 6.4 6.5 6.6 6.7 6.8 6.9 7 0 100 200 300 400 500 600MSETest Timedecsvrdec+svr(d) Vantage: MSE dependingon test timeFigure 2: Behavior of runtime and prediction performancefor homogeneous and heterogeneous ensembles for twowind turbines near Las Vegas and Vantage. The heteroge-neous combinations can outperform the homogeneous en-sembles. In particular, the solutions in each bottom left cor-ner show ensembles with a very short computation time aswell as a very low error.validation. The training times are measured using the opti-mal parameters, so the huge amount for parameter tuningis not included. The testing times showed similar behavior.In Figure 3, we visualized three setups: First, standard SVRwith RBF kernel and tuned C and . As comparison, weshow two heterogeneous ensembles in the style of Section4.2: Both consist of one SVR ensemble with n = 32 ans = 1,000 and one decision tree ensemble with n = 256and s = 10,000. The first one uses all 33 features available,whereas the second only makes use of 15 randomly chosenfeatures without replacement. The comparison shows thatin most cases the ensemble predictor outperforms classicalSVR. Further, the training time is much shorter. If one mustmake a trade-off and decrease training or testing time, hemight want to use a feature-reduced variant.5ConclusionsWind power can only be integrated into the power grid, if areliable forecast can be given in a reasonable time. After an-alyzing different types of ensemble predictors, we presenteda heterogeneous ensemble approach utilizing both DT andSVR. In our comprehensive experimental evaluation, weshowed that our approach yields better results within a muchshorter computation time. The trade-off between predictionperformance and computation time can easily be managedby adapting the parameters like number of predictors, num-ber of samples, and number of features used. In the future,we plan to implement automatic ensemble optimization andthe integration of other regression algorithms.58Figure 3: Comparison of MSE and training time for fivewind turbines. Our ensemble using all features yields thebest MSE in four cases, but only takes a small amount of thetime taken by standard SVR. If training time is consideredmore important than MSE, on can reduce the number of fea-tures used without loosing much of prediction performance.AcknowledgementsWe thank the ministry of science and culture of Lower Sax-ony for supporting us with the PhD Programm System In-tegration of Renewable Energies. Furthermore, we thankthe US National Renewable Energy Laboratory (NREL) forproviding the wind data set.ReferencesBauer, E., and Kohavi, R. 1999. An empirical comparisonof voting classification algorithms: Bagging, boosting, andvariants. Machine learning 36(1-2):105139.Breiman, L.; Friedman, J.; Stone, C.; and Olshen, R. 1984.Classification and Regression Trees. The Wadsworth andBrooks-Cole statistics-probability series. Taylor & Francis.Breiman, L. 1996. Bagging predictors. Machine learning24(2):123140.Breiman, L.2001.Random forest