IROS2019国际学术会议论文集1520

Air To Ground Collaboration For Energy-effi cient Path Planning For Ground Robots Minghan Wei and Volkan Isler1 AbstractWe study a fundamental motion planning problem of navigating a ground robot to a goal position with minimum energy consumption. Most existing solutions for this problem require an energy consumption model as a function of the environment and the robot motion. Obtaining such models is diffi cult which prevents the practical applicability of path planning algorithms for energy optimization. To address this issue, we present a new approach based on the assumption that the energy consumption for the ground robot is correlated with ground appearance. The fi rst main contribution of this paper is the validation of the ground appearance assumption by experiments using actual energy consumption data obtained by ground robots. We then show how aerial images collected by an unmanned aerial vehicle can be used to generate the energy cost map of a given environment, which can further be used for planning energy-effi cient paths for ground robots. I. INTRODUCTION Mobile robots are becoming increasingly useful to perform tasks such as environmental monitoring, mowing, and search- ing. Many robots such as small aerial and ground vehicles are powered by batteries and subject to energy constraints. Therefore, it is important for fi eld robots to operate in an energy-effi cient fashion for long-term autonomy. It has been shown that actuation is one of the main consumers of the energy on a robot 17 12. Thus planning energy- effi cient paths can elongate the operation time of the robots performing fi eld tasks. Path planning is a fundamental robotics research topic and thus has been widely studied, including variations focusing on energy effi ciency. Many algorithms have been developed to output optimal energy-effi cient paths. These approaches require an energy consumption map as input. Consider the case shown in Fig 1. If we know how much energy it takes to move a unit distance on the road and grass respectively, we are able to plan an energy-effi cient path (the green one). Unfortunately, the diffi culty of obtaining such models undermines practical applicability of these algorithms. For example, in the existing literature 16 10 17, the energy cost of a path is calculated using the friction coeffi cient and ground inclination angle. However, measuring the friction coeffi cient and inclination angle for every position in the environment is not easy. In this paper, we revisit this important path planning problem with an eye toward obtaining realistic inputs for energy-effi cient planners. To address the challenge of es- timating the energy consumption, we present an approach 1 Wei and Isler are with the Department of Computer Science and Engineering, College of Science and Engineering, University of Minnesota, United States.weixx526, A B Fig. 1.To navigate from A to B, the solid green path are preferred since it is mostly on fl at road surface and consumes less energy. If we know how much energy it takes the ground robot to move a unit distance on the road and grass, we can plan an energy effi cient path without explicitly measuring the ground friction coeffi cient everywhere. based on the following assumption: in a given environment, the energy cost is correlated with the ground appearance. The main contributions of our work are: (1) We validate this assumption (Sec. V-A) using actual energy consumption data collected by a ground robot. (2) We are able to effi ciently build an energy consumption map of a given environment using aerial images collected by unmanned aerial vehicles (UAVs). The energy consumption map is then used to plan energy-effi cient paths for ground robots. II. RELATEDWORK Path planning is a fundamental robotics research topic. The well-known Dijkstras algorithm is able to fi nd the shortest path between two nodes in a graph. In the simplest case where the energy cost is only related to distance, we can discretize an area into grids and represent it using a graph. An optimal path can then be found by applying Dijkstras algorithm or other heuristic methods such as A. However, for practical robots, minimizing the distance does not always mean that the energy cost is also minimized. Considering the example in Fig. 1. The shortest path from A to B is a straight segment shown by the dashed path. But the green path consumes less energy for ground robots since the road is fl at. To fi nd an energy-effi cient path for a practical robot, we need an energy cost model of robot motion. In the existing literature 16, 10 17, the energy cost of the motion is mostly calculated using variants of Equation 1 based on physics laws. E = mg(ucos + sin)l,(1) where mg are the weight of the robot, u is the friction coeffi cient, is the ground inclination angle, and l is the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Macau, China, November 4-8, 2019 978-1-7281-4003-2/19/$31.00 2019 IEEE1949 traveling distance. This model mainly considers the traction resistance due to friction and gravity , as shown in 16. Since large inclination angles may make some traversal headings at certain places impermissible, constraints can be added for energy-effi cient as well as safe path planning 6. At the same time of consuming energy, the ground robots can also harvest energy with a photovoltaic solar panel. Plonski et al. studied how to construct a solar-power map and use the map to plan energy-effi cient paths for ground robots 13 14. Kaplan et al. presented a method based on particle swarm optimization to plan paths so that the ambient energy is harvested and the navigation time is minimized 8. Note that the model in Equation 1 is for constant-velocity motion. To improve the energy effi ciency while moving, it is also necessary to consider the cost of the acceleration and turning on a path. Finding the optimal velocity profi le along a path has also received a lot of attention in the literature. Mei et al. related the motor speed with the energy consumption and demonstrated up to 51% energy saving in simulation 11. Given a trajectory, Tokekar and Isler presented a closed form solution to fi nd the velocity profi le to minimize the energy consumption 18. Based on this solution, they also presented an algorithm to fi nd the minimum energy trajectory along with the velocity profi le. When fi nding the optimal trajectory and velocity profi le, these methods assume that the energy cost is uniform in the environment. The energy cost model of Equation 1, which calculates the energy cost using friction and gravity force, performs well on a fl at surface. However, its application to large fi elds may be restricted due to two causes. First, it is not clear how to obtain the ground friction and inclination angle of every position in the given fi eld. Second, the rough ground surfaces often cause the robot to shake during motion even if the ground inclination angle is zero. Neither the friction nor gravity force could model the energy cost of motion on rough terrains easily. Thus we propose to use a data-driven approach to obtain the energy consumption model from visual inputs and energy-consumption measurements. Recent studies have also made use of visual inputs for path planning. Unmanned aerial vehicles (UAV) are used to collect the image data for ground vehicle path planning. Chavez-Garcia et al. proposed to use the convolutional neural network to predict from images whether a path is traversable or not 2. In an unknown environment, Delmerico et al. used observations from a fl ying robot to plan a path which minimizes the total exploration and navigation time to a target 4. These papers do not consider the energy cost of the planned paths. In this paper, we show that from visual inputs we are able to plan paths for fi eld robots to minimize the energy consumption. More specifi cally, we use a UAV to fl y over the area of interest to collect the visual data. Then we sample a few positions in the area and let the ground robot run on them to collect the energy cost data. By visually segmenting the whole area into multiple parts according to the measured samples, we generate the energy cost of every position in the environment. Therefore, we are able to plan a path to minimize the energy cost given a starting and goal position. III. PROBLEMFORMULATION Before formally stating our problem, we fi rst introduce our energy consumption model based on the actual measure- ments of ground robots. We measure the energy cost of the ground robot directly from the motor voltage V and current I. The energy cost is given by: E = Z T 0 VtItdt.(2) When the robot moves at constant speed, dividing Equation 2 by the traveled distance d, yields the energy cost for moving a unit distance. p = E l .(3) We call this value the unit-distance energy consumption, denoted as p. The energy consumption of a given path L can then be calculated by: EL= Z pldl,(4) where dl is the differential of the path, and plis the corresponding unit-distance energy consumption. Given the energy consumption model, the problem of this paper is summarized as follows. We are given a robot that is able to measure its energy consumption during its motion. The terrain types in the environment are known. Given a starting s and target g position, the goal is to plan a path sg so that the total energy consumption is minimized. In this problem, we make two assumptions. (1) For the same terrain class in the given environment, the unit-distance energy consumption p is a constant plus Gaussian noise when the ground inclination angle is zero. This assumption is validated in Sec. V-A. Note that due to the ground inclination angle, different traveling directions at the same location may have different p values. Our method considers the inclination angle in Sec. IV-A. (2) We assume that the robot operates at constant speed. With these two assumptions, we estimate p from the ground appearance. IV. ENERGY-EFFICIENTPATHPLANNING In this section, we present our methodology to plan energy-effi cient paths. We fi rst introduce how to build the energy cost map (the unit-distance energy consumption p of each position in the given environment). Then we present details of using this map to generate paths. A summary of our method is demonstrated in Fig.2. A. Energy-cost Map Building The unit-distance energy consumption p at a location can be directly measured by the ground robot using Equation 2 and Equation 3. A naive way to build the energy cost map of the given environment could be to cover the whole area and record p for every position. However, this process is tedious and thus not feasible in practice, especially when the fi eld is large. Instead, we build the map in two steps: collecting aerial 1950 Take energy measurements Collect aerial images Terrain segmentation Mosaic and 3D reconstruction Build Energy cost map Path planning Image patch, ground inclination Fig. 2.Our method starts from collecting aerial images of the given envi- ronment. By collecting the unit-distance energy consumption information in each terrain class, we can effi ciently build an energy-cost map for energy- effi cient path planning. (a)(b) Fig. 3. Covering the fi eld with a UAV to collect image data. (a). The fl ight trajectory of a UAV to cover a corn fi eld with back-and-forth motion. (b). A single view from the UAV camera. images and then taking sample measurements to generate the energy cost map for the whole area. By our assumption, p is correlated with the ground appear- ance. Thus the fi rst step for building the energy cost map is to obtain ground appearance information. We use a UAV to collect aerial images of the environment. The UAV we use is equipped with a camera facing downward. The advantage of using a UAV to collect the images is that it can quickly cover a large area and visit areas that may not be traversable for ground robots. To plan a path to cover a given environment, many algorithms have been proposed in the literature 3 5 1. We use a simple back-and-forth motion to cover the fi eld. Fig. 3(a) shows an example of a UAV path to cover a corn fi eld. To plan an energy-effi cient path, we need complete ground information, which a single aerial image may not be able to provide. Using image mosaic 9, we can merge all the collected views to analyze the terrain. In our experiment, we use a commercially available software, Agisoft, for this purpose. Fig. 4(a) shows a view of the entire area using a set of single view images shown in Fig. 3(b). The second step is to generate the energy cost map by the aerial images. The whole view from image mosaic can give us the terrain class of each location. Our assumption is that p is a constant plus noise in the same terrain class when the inclination angle is zero. Therefore, to obtain p for the whole fi eld, we sample a few regions in each terrain class to measure p with the ground robot. Note that the size of the sampled regions should be small compared to the whole fi eld. The rectangles in Fig. 4(a) represent the areas we sample. In each region except the corn areas, the robot drives in constant speed. The position, traveling time, the motor voltage and current are recorded along the way. (a)(b) Fig. 4.Connecting the ground appearance with the unit distance energy consumption. (a). A large view of the fi eld can be built using image mosaic. The rectangles correspond to the area we measure the energy consumption data in the second step. (b). The segmentation result of the fi eld. Road area is marked by red, grass area is marked by green, and corn area is marked blue. The position provides the traveling distance d. The motor voltage, current, and the traveling time together provide the total energy consumption E by Equation 2. The unit-distance energy cost p is calculated by Equation 3. For the corn area, we assign a large p value so that it will be avoided at the path planning stage. Now we discuss how to process the ground inclination angle in our model. The ground inclination angle will make the unit-distance energy consumption p non-consistent for different moving directions. We need to specify three ques- tions: (1) how to measure p when the ground inclination angle is not zero, (2) how to use p for the whole fi eld with different inclination angles, (3) and how to obtain the ground inclination angle effi ciently over the fi eld. Firstly, when measuring p for the ground with a slope, we make p independent of the inclination angle in this way. Let ab be a straight segment for measuring p. The robot fi rst follows this path from a to b to calculate p1. The second time we let the robot follow the path from b to a to get p2. On one direction the robot saves energy due to the gravity and the other spends more. The average of these two values is regarded as the unit-distance energy consumption of the corresponding position. Secondly, to include the ground inclination angle in our model, we process it in a similar way to the energy cost model in 16 10 17: for a direction with the inclination angle , the unit-distance energy cost is (p+mg sin). Thirdly, the ground inclination angle is easily accessible by performing 3D reconstruction with the aerial images collected in our method. For the implementation of 3D reconstruction, we also use the software Agisoft. We obtain the energy cost map by applying the measured p values to the corresponding terrain classes. Now we introduce how we segment the terrain classes. Recall that the number of terrain types in the scene is given. We use a similar approach to the work in 4 which divides the input image into small patches and classify the patches. To classify the pathes, we build a network that uses the pretrained features from ResNet (on theImageNet dataset15). We add two fully connected layers on top of it. To train the network, we 1951 (a) 020406080100120140 time(s) 100 150 200 250 300 350 400 carpet indoor ground concrete road dirt road grass p(J/m) (b) Fig. 5.(a) The ground system we use for experiments. (b) The unit- distance energy consumption when moving in constant speed on fi ve different grounds. label a few regions in the image to obtain the ground truth for the corresponding image patches. Figure 4(b) shows a segmentation result with three classes (road, grass and corn). The mosaic image is of size 10254 by 8437. The patches are of size 32 by 32. The total number of patches from the image is 84485 and we use 1000 of them to train the network. B. Energy-Effi cient Path Planning After segmenting the whole area and taking measurements, we have the unit-distance energy cost for the whole area. We plan paths based on the image so that we can scale the image to balance the space complexity and the resolution of the path. By comparing the pixel distance with the actual size of the area, we can obtain the actual distance between two adjacent pixels. Together with the unit-distance energy consumption data, we can calculate the energy cost from one pixel to its neighbor. We use a graph to represent the environment where each pixel is a node. Let d be a user-chosen parameter. Two nodes have an edge between them if the corresponding pixel distance is within d. In the graph, the robot can move along the edges and the energy costs between the two pixels are used as the weights. With this graph, existing planning algorithms such as Dijkstra algorithm (which we use) can be applied to obtain an optimal path. V. EXPERIMENTS In this section, we test our methods with fi eld data and experiments for two purposes. The fi rst is to validate the assumption that in a given environment the terrain visual appearance is correlated with energy consumption for a ground robot. We then show the path planning results with our method. For both purposes, we use a four-wheel ground rob