本课题研究实时嵌入式盲点安全辅助系统

1、本课题研究 实时嵌入式盲点安全辅助系统 Bing-Fei Wu, Chih-Chung Kao, Ying-Feng Li, andMin-Yu Tsai电气与控制工程学院本文提出了一种有效的电子车辆在白天和夜间的盲点区摩托车检测系统场景。该方法确定的车和摩托车的检测在白天的阴影和边缘特征，汽车和摩托车可以通过夜间车灯检测定位。首先，进行定位车辆的位置。然后，垂直和水平边缘被用来验证是否存在的车辆。之后，追踪程序操作在连续的帧跟踪相同的车辆。最后，驱动behavior is judged by the trajectory. Second, the lamps in the nighttime

2、 are extracted based on automatic histogram thresholding,行为是由轨迹判断。第二，基于直方图自动阈值提取and are veried by spatial and temporal features to against the reection of the pavement. The proposed real-time vision-based,是受空间和时间的功能的限制，对路面的重新检验。所提出的实时的基于视觉Blind Spot Safety-Assistance Systemhas implemented and evalua

3、ted on a TI DM6437 platformto performthe vehicle detection on盲点安全辅助系统来实现一个TI DM6437平台上进行车辆检测评价。real highway, expressways, and urban roadways, and works well on sunny, cloudy, and rainy conditions in daytime and night time.真正的公路，高速公路，城市道路，工作在晴天，阴天，多雨的条件下，在白天和夜间的时间进行。Experimental results demonstrate t

4、hat the proposed vehicle detection approach is eective and feasible in various environments.实验结果表明，所提出的车辆检测方法是在各种环境中有效的，可行的。1。简介In recent years, the driving safety has become the most近年来，驾驶的安全性已成为重要问题。汽车事故伤亡人数逐年增加。根据事故，来自国道高速公路局的数据，主要发生事故的原因是人为的疏忽。因此，碰撞预警技术得到极大的重视。驾驶安全的几种辅助产品ucts are promoted, inclu

5、ding lane departure warning systems产品的推广也越来越多，包括车道偏离警告系统(LDWSs), blind spot information systems (BLISs), and so（ldwss），盲点信息系统（幸福）等。forth. These products could providemore information about这些产品可以提供the vehicle surroundings with the driver, so that the driver车辆周围环境和司机的更多的信息，使司机could make the correct de

6、cision when driving on the road.能够做出正确的决定在道路上行驶时。也可以检测车的一侧是否出现车辆BLIS couldmonitor whether the vehicles appear in the side of也也或在驾驶员不知情的情况下突然变道。雷达是为另一个解决方案。然而，the cost is much higher than the camera. Consequently,成本比相机更高。因此，vision-based blind spot detection becomes popular in this基于盲点检测成为了希望的田野eld.。

7、有许多基于视觉的障碍物检测系统，我们在文献中提出的。他们中的大部分集中在检测车道，车道在主机车前面的障碍偏离预警 13 ，还有避免碰撞的应用方法47等。车道检测是利用行车安全在早期的助理。此外，一个完整的调查addressed in 1. Besides, the front obstacle detection was处理 。此外，前方障碍物检测discussed enthusiastically in the past decade.Online boosting在过去的十年里被广泛地用来algorithm is proposed to detect the vehicle in fron

8、t of 检测前面的车辆host car 2. The online learning algorithm can conquer the主机车。在线学习算法可以克服online tuning problem for a practical system. OMalley et al.对实际系统的在线调整问题。OMalley 等人3 presented a rear-lamp vehicle detection and tracking for 提出了用车尾灯来检测和跟踪night condition. The rear-lamp pairs are used to recognize夜车辆的

9、想法。尾灯组是用来识别the front vehicle and track lamp pairs by Kalman lter. Liu前面的车辆和轨道，并采用卡尔曼滤波器。刘and Fujimura 4 proposed a pedestrian detection system和藤村提出了一种行人检测系统by stereo night vision. Human became hot spot in night立体声夜视。当人成为热点的夜晚，vision and would be tracked by blob matching. Labayrade视觉将由团块匹配追踪。labayrad

10、eet al. 5 integrated 3D cameras and laser scanner to detect等人集成三维相机和激光扫描用来检测multiobstacles in front of the vehicle. The width, height, and前面的车辆的宽度，高度，和深度障碍物的立体视觉，还要精确的估计precise obstacle position can be provided by laser scanner.障碍物的位置，但也可以通过激光扫描仪来进行扫描。This cooperative fusion approach achieved an acc

11、urate and这种合作的融合方法实现了更精确robust detection强大的检测得目的。Wong和Qidwai安装了六个超声波传感器和三图像传感器在车上。他们在算法上应用模糊推理来警告司机并且减少汽车发生事故的可能性，并用全向摄像机来监视周围地区的车辆。因此，可将盲目的障碍物现场和双方的障碍物现场同时的检测。轮子后面的过滤型滤波器系统是用来确定车辆是否存的。罗德等人设计了lane change assistance systemwith far range radar, side radar,车道变更辅助系统，射程远侧雷达，雷达，and stereo vision. The sens

12、or fusion and Kalman lter were立体视觉。传感器融合和卡尔曼滤波器被used to track the vehicle stably. D´ az et al. 9 applied optical用于轨道车辆稳定。D´AZ等人应用光学ow algorithm to segment the vehicle in the blind spot, andOW算法在盲区车辆段，对several scale templates were established for tracking. Batavia多尺度模板进行了跟踪。巴达维亚et al. 10 al

13、so monitored the vehicle in rear image with等人还监测车后图像optical ow algorithm and edge feature. Stuckman et al.光学OW算法和边缘特征。stuckman等人 11 used infrared sensor to get the information of the bl利用红外线传感器来获取盲信息spot area. This method was implemented on digital signal斑面积。该方法在对数字信号processor (DSP) successfully. Ad

14、aptive template matching处理器（DSP）上成功实现。自适应模板匹配(AdTM) algorithm 12 was proposed to detect the vehicle（高级数据传送模块）算法提出了在盲点区域检测车辆，并且以算法的级别来确定跟踪车辆的行径。如果这车辆是接近，水平会增加，否则由Yoshiokawas equipped by Yoshioka et al. 13 to monitor the blind等人安装多线ccd的平稳度将会降低。监测盲spot area. This sensor could obtain the height of a pix

15、el斑面积，该传感器可以获得一个像素高度的in the image because of the parallax between two lenses.在图像由于是两个镜片之间的视差。Thus, this method could obtain the height of the vehicle.因此，该方法可以获得车辆的高度。Te chme r 14 utilized inverse perspective mapping (IPM)Te chmer 利用逆透视映射（IPM）and edge extraction algorithm to match the pattern and与边缘提

16、取算法的匹配模式to determine whether a vehicle exists in the blind spot or来确定车辆是否存在盲点not. Furukawa et al. 15 applied three cameras to monit。Furukawa等人将三摄像头监控安装在front area, left behind area and right behind area. Horizontal前门的部分，左边的部分和右边的部分。水平segment by edge was used and the template matching was段的边缘和模板opera

17、ted by orientation code matching, which is one of通过定位编码匹配操作，这是一个the robust matching techniques. Most importantly, these鲁棒匹配技术。最重要的是，这些algorithms required less resources for operation and were算法实现在一个需要较少的资源运作implemented in one embedded system. Jeong et al. 16在嵌入式系统。Jeong等人分开几个确定的输入图像，定判断几个determined

18、these segmentations belonging to the foreground确定这些分割图像是属于前景or background by gray level. Afterwards, scale invariant还是背景的灰度级。后来，尺度不变feature transform (SIFT) was implemented to generate特征变换（SIFT）的实施产生robust features to check whether the vehicle exists or not.强大的功能来检查车辆是否存在。Finally, modied mean-shift

19、was used to track the detected最后，该均值漂移用于跟踪检测vehicle. C. T. Chen and Y. S. Chen 17 estimated the车辆。C. T.陈和Y.S陈估计image entropy of the road scene in the near lane. The在车道的道路场景熵的图像。通过分析lane information. Although they could track the obstacles车道信息，可以检测到道路信息。虽然他们可以实时的跟踪的障碍in real-time, they only judged wh

20、ether the tracked vehicle，但他们只判断履带车辆was approaching or not by considering the location in the接近或不考虑的位置previous frame and current frame. Consequently, the false前一帧和当前帧。因此，假alarm would be easily triggered. Four prespecied regions容易引发警报。四个地区were dened to identify the dangerous level in Sobel识别危险，并提取生物形

21、态操作的方法可以to generate clearer edge image. However, if only considering清晰的产生边缘图像。然而，如果只考虑the edge information, the system would easily alarm falsely边缘信息，系统将很容易by shadow and safety island.通过阴影和安全岛误报警。此外，在这领域最重要的问题之一是执行效率的系统。如果效率不够高了，个系统就不能实时的检测出结果，它无法立即提醒司机，就是没有使用价值的系统。在最近几年已经提出了很多方法来防止在盲点区域的碰撞，但大多数meth

22、ods are implemented on PC, which are not suitable as方法都是在PC机上实现的，不合适的an automobile electronics. Although there are somemethods汽车电子产品。虽然有一些方法which were implemented on DSP platform, low frame rate在DSP平台上实现了，但低帧速率and robustness became the serious problems to them. In和鲁棒性成为严重的问题，在this paper, edge, shado

23、w, and lamps in spatial domain本文，边缘，阴影，和在空间域上的are applied to increase execution eciency. Therefore, the应用电子灯提高了执行效率。因此，本performance of vehicle detection in this system is the main在该系统中的车辆检测性能是重要的topic here, especially overcoming the complex problems in主题，特别是解决在复杂harsh environments, such as driving o

24、n urban roads. Using恶劣环境中的问题，如在城市道路上行驶。使用the general features in spatial domain and keeping high在空间域的一般特征和保持较高的performance has been implemented through the method性能已经通过实地的验证，次方法introduced in this paper. Developing on DSP platform, the本文已介绍了。在DSP平台的发展frame rate of this system could still achieve 59 f

25、ps at most.本系统的帧速率仍然可以最多达到59 fps。For CIF images, this eciency is high enough that theCIF图像，效率是足够高的system can provide real-time information with the drivers,系统可提供实时信息给司机，so that drivers can make the most correct decisions in time.因此，驾驶员可以及时做出最正确的决定。The system has high frame rate on TI DM6437 platform

26、,该系统对TI DM6437平台具有高帧率，and through the long verication with on-road eld tests道路现场试验可以on highways, expressways, and urban roadways, and works在公路或城市道路，高速公路。well on sunny, cloudy, and rainy conditions in the daytime在晴天，阴天，或雨天的条件下也可以检测。这表明，系统robustness, so that it can work anytime at everywhere and的鲁棒性是很

27、好的，所以它可以随时随处provide warning function with the drivers. The warning提供警示功能。警告functions which could be a buzzer or LED light would be系统由一个蜂鸣器和LED灯组成triggered to alarm to the dri，在发生触动时发出报警来提醒司机。Section 2 briey introduces the working ow of the第2节简要介绍了presented method in this paper. The algorithms of veh

28、icle本文提出的方法。在3,4节中detection in the daytime and nighttime are introduced介绍了车辆在白天和夜间的检测部分in Sections 3 and 4, respectively. The experiment results。实验结果and comparisons would be shown in Section 5. Finally, the和各种情况之间的相互比较，将在第5节中说明。最后，本conclusions would be addressed in Section 6.结论将在第6节中。因为在白天车辆检测的特点明显地

29、不同于在夜间，可利用得特征和验证应用程序将会不同。此外，考虑到实际应用， BSD要夜以继日的工作。因为它非常地困难去区分白天和夜间，两车辆检测算法在白天和在夜间应在每帧处处理。白天和夜间的算法检测和跟踪不同但是features with the same workow in Figure 1 have been具有相同的工作图。这项功能已被implemented into our system and make this system more实施在我们的系统中，使该系统更有practical and robustness. The algorithm for the nighttime实用性

30、和鲁棒性。用于夜间的车辆检测算法follows the algorithm for the daytime in our system, so that和白天的算法，there is no need to determine what time it is now.没有需要确定它现在是什么时间。There are three main detecting modes in the algorithm该算法主要有三种检测模式对of vehicle detection. Those are the full searching mode, the车辆进行检测。这些模式分别是全搜索模式，tracki

31、ng mode, and the partial searching mode, respectively.Image preprocessing is performed to extract the edge,跟踪模式，和局部搜索模式。图像进行预处理时，提取边缘，shadow, lamp features for the vehicle detection. First, there阴影，为车辆检测灯的特性。首先，进行检测和轨迹跟踪。Therefore, the system would search the possible vehicle因此，系统会搜索可能的车辆candidates

32、in the whole region of the interest (ROI) of在整个地区the image in full search mode. If the vehicle is detected的全搜索模式的图像中。如果车辆检测and tracked successfully in the successive video frames, the并成功的跟踪连续的视频帧，该过程会vehicle trajectory is generated, and the system would process生成车辆轨迹，和系统the tracking mode in the next

33、 frame. Because of the data在下一帧的跟踪模式。由于数据saved from the full searching mode, we already know where是在完整的搜索方式下保存的，我们已经知道了the vehicles exist; thus, there is no need to search the车辆的存在；因此，不需要再次搜索的whole ROI again. We can only search the region where the整个ROI。我们可以只搜索区域vehicles exist and determine their be

34、havior in the tracking车辆的存在和确定追踪他们的行为mode. According to the locations of vehicles which had模式。根据车辆的位置been saved in the last frame, the searching region would be保存在最后一帧，将搜索区域set adaptively. After detecting, candidate matching and the自适应地确定。经过检测，候选匹配和vehicle behavior would be judged. In the end, the s

35、ystem车辆的行为将被评估。最后，系统triggers the warning signal to remind the driver. Partial触发预警信号，提醒司机。部分searching mode always follows the tracking mode to search if搜索模式总是尾随跟踪模式搜索，如果there is any other vehicle or motorcycle in ROI. However, the有任何其他车辆或摩托车的ROI，该partial searching mode would not search the zone in wh

36、ere局部搜索模式将不会在区域搜索there is already a vehicle appearing and has been detected by一个已经出现的汽车，和已检测到的the tracking mode.跟踪模式。首先有七个主题在下面的小节，我们介绍我们的投资回报率的定义和给与ISO规格阳离子分段3.1比较分段3.2和3.3阴影搜索。通过检测车辆，我们搜索 在3.4这款车正确的边界。然后，候选人都在分段3.5。用连续的帧来生成the vehicle trajectory would be discussed in Subsection 3.6.车辆轨迹将在第3.6节讨论了。

37、Finally, the vehicle behavior judgment is performed in the最后，车辆行为进行判断在Subsection 3.7.3.1. Dene the ROI in the Daytime. We have to dene ROI in第3.7。在白天的投资回报率，我们要去ROIthis system clearly at rst. Referring to the denition of lane系统查找。道德定义change decision aid systems (LCDASs) in ISO document, we改变决策辅助系统（lc

38、dass）ISO文件，我们obtain the denition of blind spot area and delimit ROI in获取盲点区的定义来划分our system. As shown in Figure 2, the ISO denition of blind我们的系统。如图2所示，ISO 性盲spot area is 3 meters to the side of the car and 3 meters斑面积3米到车侧3米behind the car. The ROI in our system is larger than the ISO后面的车。在我们的系统中的投资

39、回报率大于ISOdenition and completely covers it.并且完全覆盖它。The specication of detecting region in this system is 4.5在这个系统的检测区域特定的阳离子4.5meters to the side of the car and 15 meters behind the car.米到车侧15米。后面的车The specication of the warning region is 4 meters to side的警戒区的阳离子在4米侧到of the car and 7 meters behind of

40、 the car. When a vehicle is该车后7米。车辆启动时approaching to the warning region, the system would send接近警戒区域，系统会发送a warning signal to the driver. The detecting region and一个警告信号驱动。探测区域the warning region are drawn in blue and red in Figure 3,警戒区域绘制在图3中的蓝色和红色部分。3.2. Image Preprocessing for Vehicle Detection in

41、 the Daytime.3.2在白天车辆检测图像预处理，Shadow and edge features are chosen for vehicle detection阴影和边缘特征选择的车辆检测in the proposed system in the daytime. Shadow under the该系统在白天，阴影下car could illustrate the location of the vehicle. Therefore,可以说明车辆的位置。因此，extracting the shadow region is the rst step to detect the阴影区域

42、提取的第一步是检测vehicle.Wheels always provide great amount of vertical edge,车辆车轮的垂直边缘。总是提供大量，and there are a lot of horizontal edges on the air dam of有很多的气坝水平边缘most vehicles. The information would be fairly useful when大多数的车辆。该信息将是相当有用的时候detecting vehicles.车辆检测。Several weather conditions may happen when dri

43、ving onthe road, so a xed threshold to extract shadow region may路，所以固定阈值提取阴影区fail in outdoor scene. However, no matter what the weather室外场景的失败。然而，不管什么天气is, it is supposed that the color of shadowmust be the darkest，颜色是最黑暗的part in ROI. Therefore, we establish a gray level histogram of部分的投资回报率。因此，我们建立

44、了一个灰度直方图the pixels in ROI for adaptive shadow threshold. As shown自适应阈值的像素的ROI的影子。如图所示in Figure 4, we assume that 10% of the darkest part of this在图4中，我们假设这最黑暗的部分10%histogram might be shadow, thus adaptive threshold g for直方图可能是阴影，从而自适应阈值为Gshadow detection in this frame is calculated by (1). N is the在这

45、个框架中的阴影检测的计算方法是（1）。N是number of total pixels in ROI, h(g) is the number of pixels感兴趣区域总的像素数，h（G）的像素的数量at gray level g, and 0.1 is chosen for here. According to the在灰度级和0.1 g，选择这里。根states of the road surface, the threshold for shadow extraction路的表面状态，阴影提取阈值could be set dynamically to 92 in the sunny d

46、ay, 78 in the可以动态地设置了在阳光明媚的日子，在rainy day, and 56 under the bridge by this method:下雨天，和桥下通过这种方法：此外，边缘是一个有用的功能在空间域vehicle detection, and Sobel mask is used to extract the edge车辆检测时，从Sobel遮罩提取边缘feature here. Therefore, there are three kinds of features that这里的特征。因此，有三种功能could be used to recognize vehic

47、les.可用于识别车辆。As in Figure 5(a), shadow pixels are drawn in white, and如图5（a），阴影像素画在白色，和the horizontal edge pixels are drawn in gray. Otherwise, the水平边缘，像素画成灰色。否则，该pixels are set in black. The extracted vertical edge pixels像素被设置在黑。提取的垂直边缘像素would be set to 0 in another plane, and the nonvertical edge将被设

48、置为在另一个平面，和垂直边缘pixels are set in 255 in Figure 5(b).像素图5（b）在255集。3.3. Shadow Searching. The rst step of vehicle detection is3.3，阴影搜索。车辆检测的第一步是shadow searching. Every pixel between point SL and point阴影搜索。点与点之间的每一个像素的SlSR in each row is checked. Although every pixel is checked,Sr在每一行检查。虽然每个像素的检查，the co

49、mputation loading would not increase very much. The计算量不会增加太多。但denitions of points are calculated by (2) and illustrated in点计算的和说明Figure 6,where LP and RP are the positions of the left and图6，在LP和RP是左边的位置right boundaries of ROI in each row, and v is the row index:和每一行的右边界区域。When one of these pixels is

50、 the shadow pixel, vertical当其中一个像素的阴影像素，垂直projection would be processed in this row to nd whether投影会在这行的处理查看是否there are continuous shadow pixels in this row, as shown 有连续的阴影像素，如图所示Figure 7. The length of shadow pixels and the length of ROI图7。阴影像素的ROI和长度in this row are denoted by S and L,respectively

51、.If S is在这行是由和L表示，larger than 1/8 L, it would be considered as shadow under大于1/2After that, there might be several shadow candidates that。在那之后，可能会有一些阴影的候选人要控制确认后。3.4,正确的车辆的边界，虽然定位可以发现车辆阴影，但一些严重的条件将会导致错误的检测。早在清晨或傍晚，太阳照射而不是直接阳光会有一定角度的偏差，这将导致阴影下的车辆成细长型。在下雨天，严重的路面状况会使上述情况重新发生same situation as described

52、above, as exhibited in Figure 8.，即上面所描述的相同的情况，如图8中展出。Therefore, the boundaries of vehicle should be conrmed因此，车辆的界线应again using average intensity and vertical edges.再次使用平均强度和垂直边缘。In Figure 9, the length of shadow pixels S is found in在图9中，阴影像素的长度被发现Subsection 3.3. The row of shadow found in Subsectio

53、n 3.3 is。阴影的发现是来判断not the real bottom of the car. Therefore, we have to correct汽车有没有真正的的底部。因此，我们要正确the location from Subsection 3.3. The searching zone is从3.3小节的位置搜索区extended 1/2 S upward fromthe row RS, and the real bottom向上延伸到R和真正的底is searched in this zone LRRL. Shadow should be the darkest区搜索。这应该

54、是最黑暗的阴影intensity because of the shadow property. In this zone, the强度因为阴影属性在这个区域，该average gray level value in each row is calculated, and the计算每行的平均灰度值，和row with the minimum gray level would be considered as行的最小灰度将被视为new bottom of the vehicle. The new bottom of the vehicle车辆的新的底部。车辆的新的底部is obtained i

55、n (3), where average gray level value in the jth得到（3），那里的平均灰度值在7row is denoted as I( j). Hence, we could update the bottom行由（J）。因此，我们可以更新的底部location of the vehicle to v where is much closer to the车辆定位的V那里很接近bottom of the car汽车的底部 如果连续垂直边缘存在，它会作为车辆的候选人。如图10所示，该水平投影会检查是否有车辆高度问题，默认H，这是3 / 4的，即在3.3小节的阴影

56、宽度。位置左边界的计算，（4）：在水平投影的垂直边缘的量h(u). 对垂直边缘的最大数量的列UL。如果H（UL）大于1 / 4H，这会被认为是e阳离子。有很多的水平边air dam of the most vehicles, so we search the horizontal车辆气坝，所以我们搜索水平edge for vehicle verication. As shown in Figure 11,we车辆的真实阳离子的边缘。如图11所示，我们extend a region for searching continuous horizontal edges.扩展区域搜索连续水平边缘。Ve

57、rtical projection would be processed to check if there are垂直投影会进行检查，如果有continuous horizontal edges of air dim in this region. The空气在该地区连续的水平边缘模糊的vehicleisveriedby(5)and(6):The amount of horizontal edge after vertical projection（5）和（6）：在垂直投影量水平边缘is h(v). The row index is denoted as v, H is the height

58、 of是H（V）。行下标记为V，h是高度the detected vehicle, S is the width of the detected vehicle,检测到的车辆，是检测到的车辆的宽度and the row of the maximum amount of horizontal edge is和水平边缘的最大数量的行v. Vehicle identication is denoted as . If the conditionV。车辆识别阳离子被表示为。如果条is met in (6), this candidate would be considered as a real会（6

59、），该候选人将被视为一个真正的car and would be tracked in the next frame. The verication汽车将在下一帧跟踪。验证阳离子of motorcycle is the same as the vehicle verication. The摩托车是与车辆阳离子相同的criterion in (6) is important so that most false alarms could标准，大部分的假报警可能be avoided. However, the tradeo is that some motorcycles避免的。然而，折衷摩托车with less strong horizontal edges would be deleted.不强的水平边缘将被删除。3.6. Candidate Matching. So far, the real cars have been3.6，候选匹配。到目前为止，真正的汽车已retained.When the fu