机器人资料-论文-040-Object Tracking in Shape Parameter Space.pdf

abstractin this paper, we develop a robust visual servo system for object tracking applications of a nonholonomic mobile robot. the system mainly consists of an adaptive shape tracking algorithm and a robust visual servo controller. the adaptive shape tracking algorithm is designed to automatically detect the shape contours of moving objects, extract the shape parameters, and continuously track the object in shape parameter space. based on direct measurements of the shape parameters, the visual servo controller is designed by using the sliding mode control technique and is robust to uncertainties of the objects motion. through a lyapunov-based stability analysis, a sufficient condition on selection of control gains to achieve the tracking goal in finite time is provided. keywords: visual servo, mobile robot, visual tracking, sliding mode control, randomized hough transform, shape detection. i. introduction recently, the use of vision systems in feedback control, i.e. visual servoing, has gained increasing attention both in robotics and control community. it has been addressed by researchers for a wide range of applications including object tracking, surveillance, inspection, monitoring, and automatic mobile robot navigation 1. the build-up of a visual servo system involves the design of a visual tracking algorithm, which continuously senses the positions of object features from the vision measurements, and the design of a visual servo controller based on the sensed object positions. many research efforts have contributed to the development of visual tracking algorithms 2-17, which can be mainly classified into three categories, namely optic-flow based, feature based, and active contour based. the optic-flow based tracking algorithms 2-7 involve the computation of the optical flow field, from which one can extract the motion information of the object. the feature-based tracking algorithms estimate the object position based on variations of the objects features in an image sequence 8-12. the active contour method models the outline of rigid or nonrigid (deformable) objects as parameterized curves 13-17. during tracking the parameterized curves are moved in an energy minimization process under the influence of image forces. regarding the design of visual servo controllers, a fair amount of research works have addressed the issue for robot systems in recent years 1. particularly for mobile robot systems, several control algorithms 18-21 have been developed to realize vision-based feedback control. for example, visual servo techniques using image features of certain geometric primitives are proposed in 18 and are extended to control a mobile hand-eye system 19. stabilizing control laws based on direct measurement from the image are proposed in 20 to deal with vision guided navigation tasks for a nonholonomic mobile robot tracking an arbitrarily shaped ground curve. an image-based adaptive tracking control law is proposed in 21 to cope with model uncertainties of mobile robots. in this paper, we develop a robust visual servo system for object tracking applications of a nonholonomic mobile robot, focusing on the design of the visual tracking algorithm and the visual servo controller, such that the system can automatically detect the shape contours of moving objects, extract the shape parameters, continuously update the shape parameters, and then control the mobile robot to track one moving object of interest. we adopt the adaptive shape tracking algorithm proposed in our recent work 22 for visual tracking. as compared with those feature-based algorithms mentioned above, the adaptive shape tracking algorithm does not assume a priori knowledge about the object shapes. instead it detects the shape contours of moving objects and extracts their shape parameters based on the randomized hough transform method and has the advantage of computational efficiency and robustness to noise and object occlusion 22. according to direct measurements of the shape features from the adaptive shape tracking algorithm, we design the visual servo controller by using the sliding mode control technique to cope with the uncertainty of model parameters and the random nature of object movements. furthermore, through a lyapunov-based stability analysis, we derive a sufficient condition on selection of control gains to guarantee achievement of the tracking goal in finite time. the remainder of the paper is organized as follows. the robust visual servo control of a mobile robot for object tracking in shape parameter space jong-hann jean and tien-pao wu department of electrical engineering, st. johns and st. marys institute of technology, tamsui, taipei, taiwan, r. o. c. e-mail: .tw 43rd ieee conference on decision and control december 14-17, 2004 atlantis, paradise island, bahamas 0-7803-8682-5/04/$20.00 2004 ieee thc11.1 4016 adaptive shape tracking algorithm is described in section 2, where a demonstrative algorithm for detection of ellipses is provided. in section 3 the visual servo controller based on the sliding mode control technique is presented together with the stability analysis of the visual servo system. in section 4 several simulations are provided to verify the validness of the system. finally, conclusions are given in section 5. ii. adaptive shape tracking most visual servo applications use point features 1, for example, the coordinates of points in the image, the distance between two points and the orientation of the line connecting those two points, and etc., to identify the objects positions. obviously point features are sensitive to partial occlusion, and consequently we adopt shape features in this paper for their robustness to noise and object occlusion. in this section, we briefly present the adaptive shape tracking algorithm proposed in our recent work 22, which can senses the positions of the objects shape features based on the randomized hough transform (rht) method. we assume the shape contours of moving objects to be tracked can be partially approximated as parameterized curves such as circles, ellipses, triangles, or polygons but with unknown parameters. for example, the human head can be approximated as an ellipse which can be considered as the feature of a human being for visual tracking. the adaptive shape tracking algorithm, as shown in fig. 1, mainly consists of a motion detector, a shape detection algorithm based on the rht method, and an adaptive object tracking algorithm. a. motion detection the motion detection is implemented with the concept of “moving edges”. the implementation can be found in 22. the moving edges highlight the outlines of moving objects as shown in fig. 2, and provide useful information to the shape detection. the computational complexity of shape detection is much simplified when using moving edges than using the edge image directly. b. shape detection using rht method the hough transform (ht) is a standard technique for detecting parametric shapes. however the ht method suffers the problem in performance. to overcome the performance issue, the rht method was first introduced in 1990 by xu et. al. 23 and was successfully extended by mclaughlin 24 to detect shapes with nonlinear parametric equations. based on different parametric equations, the rht method can be used to detect various analytic shapes such as circles, ellipses, triangles, and polygons. therefore the adaptive shape visual tracking algorithm does not assume a priori knowledge about the object shapes. in the following we demonstrate the rht-based shape detection algorithm using the example of ellipse detection. it is easy to extend the algorithm to track circles, triangles, polygons and other parameterized shapes 22. c. rht based ellipse detection we implement the rht-based ellipse detection algorithm with reference to mclaughlins work 24 and schuler work 25 and adopt an efficient least-squared fitting method 26 to compute the ellipse parameters. algorithm 2-1: ellipse detection based on rht step 1: select five pixels randomly from the edge pixels of the image supposing these pixels are located on an ellipse. step 2: use the least-squares fitting technique 26 to obtain the coefficients, a to f, of the following ellipse equation, . 04 , 0 222 =+bacfeydxcybxyax step 3: compute the 5-tuple parameter (xe, ye, ra, rb,?) which consists of the ellipse centre, the radii of the major and minor axes, and the angle of the major axis, defined by the following equation. 1 )sin)(cos)()sin)(cos)( 2 2 2 2 = + + b ee a ee r xxyy r yyxx step 4: after a specified amount of parameters is found, select parameters which repeat over a predefined number of times for further check. step 5: perform the shape matching as follows to further check if the found parameters are valid. for each parameter, determine an ellipse. grow the ellipse to a constant width and form an elliptical ring. find and count the edge pixels which lie within the elliptical ring. if the (a) i(k) (b) i(k-1) (c) (d) (e) fig. 2. (a), (b) the frames at time k and k-1, (c) binarized result of image subtraction between i(k) and i(k-1), (d) edge image of i(k-1), (e) moving edges (logical and between (c) and (d). motion detection shape detection object tracking in translational subspace object tracking in rotation/ scaling subspace parameter update fig. 1. flowchart of the adaptive shape tracking algorithm 4017 counting result is greater than a predefined threshold, indicate that the ellipse is actually existent. d. adaptive object tracking after identifying the shapes of objects, the system proceeds to continuously track locations of these moving objects in the shape parameter space. the tracking algorithm use the process of shape matching as described below to update the shape parameters corresponding to the translational, rotational, and scaling motion of the object. given a shape parameter ?, we first compute the parameterized curve, grow the curve to a certain constant of width, and denote the resultant contour as ?(?). by summing the number of 1-pixels in the edge image along the shape contour ?(?), we define a score function as follows, ? = )(),( ,),( 1 )( yx d yxi n (2) where n? is the total number of pixels on the shape contour ?(?), id is the matrix representation of the edge image and contains only binary values, that is, pixels on an edge are 1 (=white) and all other pixels are 0 (=black). consider a subset s of the parameter space. we define the process of shape matching over the set s as maximizing the score function ?(?) over the set s. by going through the subset s in the parameter space, we can get the best matched parameter ?* corresponding to the largest score, that is, .)(maxarg * = s (3) the best matched shape contour ?( ?*) corresponds to the distribution of edge pixels which most resembles the objects shape. in this sense the best matched parameter ?*denotes the matched position of the object in the set s. because the higher dimension of the parameter space may result in more space and time requirements to update parameters, the tracking algorithm uses a decomposition of the parameter space into lower dimensional subspaces to enhance the computational efficiency. based on the observation that the rotational/scaling motion is relatively slower than the translational motion, it is natural to decompose the parameter space into the translational subspace and the rotational/scaling subspace. in the following we present the adaptive object tracking algorithm. for the sake of brevity we describe the algorithm in case of tracking one object. it is easy to extend the algorithm for tracking multiple objects. algorithm 2-2: adaptive object tracking step 1: as the output of the motion detector and the shape detection algorithm in section 2, we obtain the shape parameter of the object, ?0,indicating the initial position of the object. step 2: capture a new image frame and compute the edge image of the new image frame as id. step 3: perform shape matching over the translational subspace st(or a finite subset of st). denote the best matched parameter as ?kt, )(maxarg = + tk s t k , (4) where the sub-index k denotes the number of iterations. step 4: perform shape matching over the rotational/scaling subspace sr(or a finite subset of sr). denote the best matched parameter as ?kr, )(maxarg = + r k t s r k . (5) step 5: update the objects shape parameter as follows. r kk = +1 (6) return to step 2 and continue the object tracking. in practice we perform the shape matching only over a finite subset of parameters which defines a search range and its size will depend on the moving speed of the object. since the translational movement is usually larger than the rotational/scaling movement, the algorithm searches a larger range in st than in sr. this can be treated as the tracking algorithm roughly finds the 2d position of the object shape in the image, then fine-tunes its pose and scale around that position. iii. visual servo controller a. problem formulation consider a mobile robot of the unicycle type carrying a fixed camera and performing the task of tracking a moving object on a 2d plane. as depicted in fig. 3, we assume the camera is mounted along the heading direction of the mobile robot and a coordinate frame rc is attached to the optical center of the camera, pc, with the y-axis being in the direction pointing out of the paper and the z-axis being along the optical axis of the camera. let (xc, yc, zc) denote the coordinate of the point pc with respect to an inertial frame r0 and ? be the angle between the heading direction of the mobile robot and the z-axis of the frame r0. based on the assumption that the mobile robot moves on a 2d plane without slips, the kinematic equation of a mobile robot can be written as ? ? ? ? ? ? ? ? ? ? = ? ? ? ? ? ? ? ? ? ? sin cos c c c c v v z x ? ? ? , (7) where vc and ? are the linear velocity and the angular velocity of the mobile robot respectively, and yc is a constant due to planar motion. the equation (7) is one of the well-known nonholonomic systems. using the pin-hole model for the camera as shown in fig. 4, we obtain the following relationships, f zhx t c tt t c = , (8) 4018 where (cxt, h, czt) is the position vector of the objects shape feature pt with respect to the frame rc, (?t, ?t) is the image coordinate of pt projected on the image plane, and f is the focal length of the camera. applying the homogeneous transformation of the frame r0 with respect to rc to equation (7), we have the equation of relative motion between the object and the mobile robot as ? ? ? ? ? ? ? ? ? ? + = ? ? ? ? ? ? ? ? ? ? t t c t t c t t c t c xvv zv z x cos sin ? ? ? (9) where vt is the linear velocity of the object, is the angle between moving directions of the object and the mobile robot, and t is the angular velocity of the object. for the goal of object tracking, we have to drive the mobile robot to keep the moving object always in sight of the camera. this goal can be achieved if we keep the position of the shape feature, pt, on the heading direction and within a short distance of the mobile robot, that is, dt c t c zzx=and0, (10) where zd is a positive constant. by using (8), the control goal (10) can be rewritten in terms of the image coordinates of the shape feature as dtt =and0, (11) where d d z hf = . (12) b. visual servo control design in this subsection, we present the design of the visual servo controller to achieve the control goal described as (10) or, equivalently, as (11). based on the relative motion equation (9), we can design the control law for v and by using the sliding mode control technique as follows. () () dtvtz t t c tx kvv k z v = += sgn sgn , (13) where tx v and tz v are estimates for the velocity components, sin ttx vv = and cos ttz vv =, respectively, the constants v k and kare positive control gains to be determined, and sgn() denotes the sign function. it is noteworthy that the control law (13) is implemented based on measurements of the shape features, t and t , obtained from the adaptive shape tracking algorithms. for convenience of presentation, we tentatively make the following assumption. assumption 1: during the period of tracking, the moving object is always in front of the mobile robot, that is, 0 t cz . fig. 3. a mobile robot mounted with a fixed camera tracks a moving object on a 2d plane. fig. 4. projection of the objects feature position onto the image plane it is obvious that assumption 1 is a basic requirement to keep the moving object always in sight of the camera. define the tracking errors, ex and ez as dt c z t c x zze xe = = , (14) then the system (9) can be rewritten in terms of the tracking errors as () xtzz dztxx evve zeve += += ? ? . (15) similarly, based on assumption 1 and equation (8), the control law (13) can be rewritten as () ( ) x t c tx t c t c tx ek z v xk z v sgn sgn += += . (16) () ( ) zvtz dt c vtz ekv zzkvv sgn sgn += += . (17) applying equations (16) and (17) to the system (9), we can get the following error equations. ( )( ) ( ) zvxtztzz xdxztxx ekekvve ezkeekve sgn sgnsgn += = ? ? (18) where tztztz txtxtx vvv vvv = = . object h (cxt, cz t) ? ? z f () tt , pc pt cz t image plane v (xc , zc) (xt , zt) vt object z r0p0 rc z pc pt mobile robot 4019 c. stability analysis lemma 1: consider the nonholonomic mobile robot system described by equation (9). suppose the object velocity is estimated by tx v and tz v with bounded errors, that is, ztztzxtxtx cvvcvvand .(20) proof: define the following positive function v as a candidate of the lyapunov function. )( 2 1 22 zx eev+= (21) the time derivative of v along the error equations (18