文稿图像理解93robust real time face detection

1、Face DetectionXuejin Chen1Face Detection is Challenging Pose (Out-of-Plane Rotation)Presence or absence of structural componentsFacial expressionOcclusionOrientation (In-Plane Rotation)Imaging conditionslighting (spectra, source distribution and intensity) and camera characteristics (sensor response

2、, gain control, characteristics (sensor response, gain control, lenses), resolutionRobust Real-Time Face Detection2Scan classifier over locs. & scalesRobust Real-Time Face Detection3Detecting Faces in Images: A SurveyMing-Hsuan Yang, Member, IEEE, David J. Kriegman, Senior Member, IEEE, Narendra

3、 Ahuja, Fellow, IEEEIEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 24, NO. 1, JANUARY 2002Samples of Sung and Poggio 98Some images are scanned from newspapers and, thus, have low resolution. Though most faces in the images are upright and frontal. Some faces in the images appea

4、r in different poseDatabase by Rowley et al. 130 images with a total of 507 frontal faces.Also includes 23 images of the second data set used by Sung and Poggio, 1998. Most images contain more than one face on a cluttered backgroundA good test set to assess algorithms which detect upright frontal fa

5、ces. Database by Rowley et al. Some images contain hand-drawn cartoon faces. Most images contain more than one face and the face size varies significantly. Another Database by Rowley et al. For detecting 2D faces with frontal pose and rotation in image50 images with a total of 223 faces, of which 21

6、0 are at angles 10 degrees. Profile Views Database208 images Each image contains faces with facial expressions and in profile viewsSchneiderman and Kanade, 00Kodak Face DatabaseA common test bed for direct benchmarking of face detection and recognition algorithms300 digital photos Captured in a vari

7、ety of resolutionsFace size ranges from as small as 13x13 pixels to as large as 300 x300 pixels. Test Sets for Face DetectionRobust Real-Time Face Detection11Methods to Detect/Locate Faces: Before 2002Knowledge-based methodsEncode human knowledge of what constitutes a typical face (usually, the rela

8、tionships between facial features)Feature invariant approaches Aim to find structural features of a face that exist even when the pose, viewpoint, or lighting conditions varyTemplate matching methodsSeveral standard patterns stored to describe the face as a whole or the facial features separately Ap

9、pearance-based methodsThe models (or templates) are learned from a set of training images which capture the representative variability of facial appearance Knowledge-Based MethodsMulti-resolution focus-of-attention approachLevel 1 (lowest resolution)apply the rule “the center part of the face has 4

10、cells with a basically uniform intensity” to search for candidatesLevel 2local histogram equalization followed by edge detectionLevel 3search for eye and mouth features for validationRobust Real-Time Face Detection13Yang and Huang 94Knowledge-Based MethodsHorizontal/vertical projection to search for

11、 candidatesSearch eyebrow/eyes, nostrils/nose for validationDifficult to detect multiple people or in complex backgroundRobust Real-Time Face Detection14Kotropoulos & Pitas 94Feature-Based methods Bottom-up approachDetect facial features (eyes, nose, mouth, etc) firstFacial featuresedge, intensi

12、ty, shape, texture, color, etcAim to detect invariant featuresGroup features into candidates and verify themRobust Real-Time Face Detection15Leung et al. 95Feature-Based Methods Pros: Features are invariant to pose and orientation changeCons:Difficult to locate facial features due to several corrupt

13、ion (illumination, noise, occlusion)Difficult to detect features in complex backgroundRobust Real-Time Face Detection16Template Matching MethodsStore a templatePredefined: based on edges or regionsDeformable: based on facial contours (e.g., Snakes)Templates are hand-coded (not learned)Use correlatio

14、n to locate facesCons:Templates needs to be initialized near the face images Difficult to enumerate templates for different poses (similar to knowledge-based methods)Robust Real-Time Face Detection17Ration Template Sinha 94Lanitis et al. 95Appearance-Based Methods Train a classifier using positive (

15、and usually negative) examples of faces RepresentationPre processingTrain a classifierSearch strategyPost processingView-basedRobust Real-Time Face Detection18ClassifiersNeural network: Multilayer Perceptrons PCA, Factor AnalysisSVMMixture of PCA, FADistribution-based methodsNave BayesHidden Markov

16、modelSparse network of winnowsInductive learning AdaboostRobust real-time face detectionPaul A. Viola and Michael J. JonesIntl. J. Computer Vision57(2), 137154, 2004(originally in CVPR2001)(slides adapted from Bill Freeman, MIT 6.869, April 2005)Robust Real-Time Face Detection19“Learn” classifier fr

17、om dataRobust Real-Time Face Detection20Training Data5000 faces (frontal)108 non facesFaces are normalizedScale, translationMany variationsAcross individualsIlluminationPose (rotation both in plane and out)Characteristics of AlgorithmRobust Real-Time Face Detection21Feature set (is huge about 16M fe

18、atures)Efficient feature selection using AdaBoostNew image representation: Integral Image Cascaded Classifier for rapid detectionFastest known frontal face detector for gray scale imagesIntegral ImageRobust Real-Time Face Detection22Allows for fast feature evaluationDo not work directly on image int

19、ensitiesCompute integral image using a few operations per pixel (similar with Haar Basis functions)Characteristics of AlgorithmRobust Real-Time Face Detection23Feature set (is huge about 16M features)Efficient feature selection using AdaBoostNew image representation: Integral Image Cascaded Classifi

20、er for rapid detectionCascaded ClassifierRobust Real-Time Face Detection24Combining successively more complex classifiers in a cascade structure Dramatically increases the speed of the detector by Focusing attention on promising regions of the image. Focus of attention approachesIt is often possible

21、 to rapidly determine where in an image a face might occur (Tsotsos et al., 1995; Itti et al., 1998; Amit and Geman, 1999; Fleuret and Geman, 2001). More complex processing is reserved only for these promising regions. The key measure of such an approach is the “false negative” rate of the attention

22、al process. Cascaded ClassifierRobust Real-Time Face Detection25Training processAn extremely simple and efficient classifier Used as a “supervised” focus of attention operator. A face detection attentional operator Filter out over 50% of the image Preserving 99% of the faces over a large datasetThis

23、 filter is exceedingly efficientit can be evaluated in 20 simple operations per location/scale OverviewRobust Real-Time Face Detection26Features: form and computingCombing features to form a classifier: AdaBoostConstructing cascade of classifiersExperimental resultsDiscussions FeaturesRobust Real-Ti

24、me Face Detection27Using features rather than image pixelsFeatures act to encode ad-hoc domain knowledge that is difficult to learn using a finite quantity of training dataMuch faster than a pixel-based systemImage featuresRobust Real-Time Face Detection28“Rectangle filters” Papageorgiou et al. 1998

25、Similar to Haar wavelets Differences between sums of pixels inadjacent rectanglesAbout 160000 rectangle features for a 200 x200 image Integral ImageRobust Real-Time Face Detection29Huge library of filtersRobust Real-Time Face Detection30Feature DiscussionRobust Real-Time Face Detection31Primitive wh

26、en compared with steerable filters, etcExcellent for the detailed analysis of boundaries, image compression, and texture analysis.Sensitive to the presence of edges, bars, and other simple image structureQuite coarse: only three orientations (|, X, -)Overcomplete: 400 times, aspect ratio, locationCo

27、mputational AdvantageRobust Real-Time Face Detection32Face detector scans the input at many scalesstarting at the base scale: detect face at a size of 24 24 pixels, Then at 12 scales, 1.25 larger than the last384 288 pixel image is scanned at the top scaleThe conventional approach:Compute a pyramid

28、of 12 images (smaller and smaller image)A fixed scale detector is scanned at each image. Computation of the pyramid directly requires significant time. It takes around .05 seconds to compute a 12 level pyramid of this size (on an Intel PIII 700 MHz processor)Implemented efficiently on conventional h

29、ardware (using bilinear interpolation to scale each level of the pyramid)Computational AdvantageRobust Real-Time Face Detection33Define a meaningful set of rectangle featuresA single feature can be evaluated at any scale and location in a few operations. Effective detectors is constructed with two r

30、ectangle features. Computational efficiency of featuresFace detection process can be completed for an entire image at every scale at 15 frames per secondAbout the same time required to evaluate the 12 level image pyramid alone. Learning Classification FunctionsRobust Real-Time Face Detection34Any ma

31、chine learning methodsGiven the feature set and training setMixture of Gaussian model (Sung and Poggio, 1998)Simple image feature and neural network (Rowley et al. 1998)Support Vector Machine (Osuna et al. 1997b)Winnow learning procedure (Roth et al. 2000)160000 featuresEven though each feature can

32、be computed very efficiently, computing the complete set is prohibitively expensiveSimple and Efficient ClassifierRobust Real-Time Face Detection35Select a small number of important features from a huge library of potential features using AdaBoost Freund and Schapire,1995AdaBoost, Adaptive BoostingR

33、obust Real-Time Face Detection36Formulated by Yoav Freund and Robert Schapire.1 It is a meta-algorithm, can be used in conjunction with many other learning algorithms to improve their performance. AdaBoost is adaptive Boosting: aimed at reducing bias by combining sequential learnersAdaptive: for sub

34、sequent classifiers, adjust the weights of the samples misclassified by previous classifiers. Calls a weak classifier repeatedly in a series of rounds from T classifiers. For each calla distribution of weights Dt is updated that indicates the importance of examples in the data setOn each round, the

35、weights of each incorrectly classified example are increased Or alternatively, the weights of each correctly classified example are decreased, The new classifier focuses more on those examplesAdaBoostRobust Real-Time Face Detection37Given ,InitializeFor For each classifier that minimizes the error w

36、ith respect to the distribution is the weighted error rate of classifierIf , then stop Choose , typically Update where is a normalized factor (choose so that Dt+1 will sum_x=1)11( ,),.,(,)mmx yxy, 1, 1iixX yY 11( ),1,.,D iimm1,.,tT: 1, 1thX tDargminttthHh( )( )ttitiD iyh x0.5ttR11ln2ttttth1( )exp( )

37、( )ttitittD iy h xDiZtZAdaBoostRobust Real-Time Face Detection38Output the final classifierThe equation to update the distribution Dt is constructed so thatAfter selecting an optimal classifier for the distributionExamples that the classifier identified correctly are weighted less Examples that is i

38、dentified incorrectly are weighted more. When the algorithm is testing the classifiers on the distributionit will select a classifier that better identifies those examples that the previous classifier missed. 1( )( )TtttH xsigna h x0,( )( )( )0,( )( )titititiy ih xa y h xy ih xAdaBoostRobust Real-Ti

39、me Face Detection39Weak classifier 1AdaBoostRobust Real-Time Face Detection40Weak classifier 1Weights increasedAdaBoostRobust Real-Time Face Detection41Weak classifier 1Weights increasedWeak classifier 2AdaBoostRobust Real-Time Face Detection42Weak classifier 1Weights increasedWeak classifier 2Weigh

40、ts increasedAdaBoostRobust Real-Time Face Detection43Weak classifier 1Weights increasedWeak classifier 2Weights increasedWeak classifier 3AdaBoostRobust Real-Time Face Detection44Weak classifier 1Weights increasedWeak classifier 2Weights increasedWeak classifier 3Final classifier Good Reference on B

41、oostingRobust Real-Time Face Detection45Friedman, J., Hastie, T. and Tibshirani, R. Additive Logistic Regression: a Statistical View of Boosting “We show that boosting fits an additive logistic regression model by stagewise optimization of a criterion very similar to the log-likelihood, and present

42、likelihood based alternatives. We also propose a multi-logit boosting procedure which appears to have advantages over other methods proposed so far.”AdaBoostRobust Real-Time Face Detection46A very small number of features can be combined to form an effective classifierBoost the classification perfor

43、mance Combining a collection of weak classification functions to form a stronger classifierWeak learnerDo not expect even the best classification function to classify the training data wellThe first round of learningExamples are re-weighted in order to emphasize those which were incorrectly classifi

44、ed by the previous weak classifier. The final strong classifier takes the form of a perceptron, a weighted combination of weak classifiers followed by a threshold.6Training error of the strong classifier approaches zero exponentially in the number of roundsAdaBoostRobust Real-Time Face Detection47Se

45、lecting a small set of good classification functions have significant varietySelect effective features which have significant varietyRestrict the weak learner to classification functions Each function depends on a single feature Select the single rectangle feature which best separates the positive a

46、nd negative examples1if ( )( , , )0pf xph x f potherwisethreshold24x24 subwindow featurePolarity indicating the direction of inequalityConstructing the classifierRobust Real-Time Face Detection48Perceptron yields a sufficiently powerful classifierUse AdaBoost to efficiently choose best featuresadd a

47、 new hi(x) at each roundeach hi(xk) is a “decision stump”b=Ew(y x q)a=Ew(y x F_target i i + 1 ni = 0; Fi = Fi1 while Fi f Fi1 ni ni + 1 Use P and N to train a classifier with ni features using AdaBoost Evaluate current cascaded classifier on validation set to determine Fi and Di . Decrease threshold

48、 for the ith classifier until the current cascaded classifier has a detection rate of at least d Di1 (this also affects Fi ) N If Fi Ftarget Evaluate the current cascaded detector on the set of non-face images put any false detections into the set NSimple ExperimentRobust Real-Time Face Detection68A

49、 monolithic 200-feature classifier and A cascade of ten 20-feature classifiersTraining using 5000 faces + 10000 nonface sub-windowsRobust Real-Time Face Detection69Simple ExperimentRobust Real-Time Face Detection70A monolithic 200-feature classifier and A cascade of ten 20-feature classifiersTrainin

50、g using 5000 faces + 10000 nonface sub-windowsLittle difference between them in terms of accuracyBut cascaded classifier is nearly 10 times fastersince its first stage throws out most non-faces so that they are never evaluated by subsequent stages.Detector Cascade DiscussionRobust Real-Time Face Det

51、ection71Similar to Rowley et al. (1998) (fast)Trained two neural networksOne was moderately complexfocused on a small region of the image, detected faces with a low false positive rate. Second neural network much fasterfocused on a larger regions of the image, and detected faces with a higher false

52、positive rateThis methodtwo stage cascade include 38 stagesTraining DatasetRobust Real-Time Face Detection724916 hand labeled faces scaled and aligned to a base resolution of 24 by 24 pixels.Structure of the Detector CascadeRobust Real-Time Face Detection7338 layer cascade of classifiers included a

53、total of 6060 featuresFirst classifier constructed using two featuresrejects about 50% of non-faces while correctly detecting close to 100% of faces. The next classifier has ten features rejects 80% of nonfaces whiledetecting almost 100% of faces. The next two layers are 25-feature classifiers Then

54、three 50-feature classifiers Then classifiers with variety of different numbers of features chosen accordingSpeed of Face Detector Robust Real-Time Face Detection74Speed is proportional to the average number of features computed per sub-window.On the MIT+CMU test set, an average of 9 features (/ 606

55、1) are computed per sub-window.On a 700 Mhz Pentium III, a 384x288 pixel image takes about 0.067 seconds to process (15 fps).Roughly 15 times faster than Rowley-Baluja-Kanade and 600 times faster than Schneiderman-Kanade.Scanning The DetectorRobust Real-Time Face Detection75Multiple scalesScaling is

56、 achieved by scaling the detector itself, rather than scaling the imageThe features can be evaluated at any scale with the same costLocationsSubsequent locations are obtained by shifting the window some number of pixels Dchoice of D affects both speed and accuracya step size 1 pixel tends to decreas

57、e the detection rate slightly while also decreasing the number of false positivesRobust Real-Time Face Detection76Integration of Multiple DetectionsRobust Real-Time Face Detection77Postprocess: combine overlapping detections into a single detectionThe set of detections are first partitioned into dis

58、joint subsetsTwo detections are in the same subset if their bounding regions overlap. Each partition yields a single final detection. The corners of the final bounding region are the average of the corners of all detections in the set.Decreases the number of false positives.Integration of Multiple D

59、etectionsRobust Real-Time Face Detection78A simple Voting Scheme further improves resultsThree detections performed similarly on the final task, but in some cases errors were different. Retaining only those detections where at least 2 out of 3 detectors agree. This improves the final detection rate

60、as well as eliminating more false positives. Since detector errors are not uncorrelated, the combination results in a measurable, but modest, improvement over the best single detector.Sample resultsRobust Real-Time Face Detection79MIT + CMU test setFailure CasesRobust Real-Time Face Detection80Trained on fron