IROS2019国际学术会议论文集 0403

Abstract Infrared imaging spectrometer IRIS often suffers from overlapped bands and random noises which limit the precision of subsequent processing in robot vision sensing To address this problem we propose a novel Gabor transform based infrared spectrum restoration method by successfully exploring the intrinsic structure of the clean IR spectrum from the degraded one At first a total variation TV regularized Gabor coefficients adjustment descriptor is designed and incorporated into the spectrum restoration model Then the proposed model is inferred via an efficient optimization approach based on split Bregman iteration method Comprehensive experiments illustrate the significant and consistent improvements of the developed model over state of the art approaches The restored high resolution spectrum can be utilized for detecting the different materials in the robot visual tracking systems I INTRODUCTION Infrared imaging spectrometers IRIS serve as a powerful tool for multispecies detection of materials 1 2 with the industrial applications ranging from the modern defense 3 4 to biological materials 5 6 7 Typically every material and compound hold their unique spectral signatures due to different reaction with IR light shown Fig 1 Unfortunately the IR spectrum has intrinsic imaging limitations including the overlap effect resolution degradation and optical noise These unfavorable factors tend to reduce the overall quality of spectral resolution and feature extraction Thus it is important for material classification tasks to recover the IR spectrum structure prior to which aims to infer the high quality spectrum signal from the observed IR spectrum This work was supported in part by the National Key Research and Development Program of China under Grant 2017YFB1401300 and Grant 2017YFB1401303 National Natural Science Foundation of China under Grant 61875068 Grant 61873220 Grant 61673329 Grant 61505064 Research Grants Council of Hong Kong Project No CityU 11205015 and CityU 11255716 Hong Kong Scholars Programs under Grant XJ2016063 and the Fundamental Research Funds for the Central Universities under Grant CCNU18ZDPY10 and Grant 2017YBZZ009 Corresponding author You Fu Li e mail meyfli cityu edu hk Hai Liu is with the National Engineering Research Center for E Learning Central China Normal University China and also with the Department of Mechanical Engineering City University of Hong Kong Hong Kong Youfu Li and Dan Su are with the Department of Mechanical Engineering City University of Hong Kong Hong Kong and also with the City University of Hong Kong Shenzhen Research Institute Shenzhen 518057 China Zhaoli Zhang Sannyuya Liu and Tingting Liu are with the National Engineering Research Center for E Learning Central China Normal University Wuhan 430079 China Moving Robot Material Classification Sensor Polypro pylene foam Polyvinyl chloride foam Cardboard Polythene foam Video Figure 1 Material detection system by the IR spectral imaging spectrometer and camera which could distinguish the different materials on the robot conveyor belt 8 9 IR spectrum restoration algorithms 10 15 can be classified as two groups analysis prior based restoration algorithm APR and synthesis prior based restoration algorithm SPR For the APR algorithm one of the most prevalent techniques is the Savitzky Golay filter based the spectrum signal restoration It performs a least square fitting of IR spectrum bands by several polynomial basis functions However it needs to manually determine the parameters of the window size and polynomial order Wiener filtering 10 is another popular spectral restoration method in respect of considering its high efficiency In 15 a Kalman filter based method is developed for enhancing the resolution of IR spectrum 16 17 In 18 Du et al proposes a convolutional sparse learning based spectrum signal deconvolution method for impulsive spectral feature detection But it fails to work well in real world scenarios with shot noises To circumvent this problem KatraSnik et al 19 proposes an optical acoustic adjusted deconvolution algorithm to suppress the shot noises which has obtained the high spectral resolution and good signal to noise ratio SNR However it is unable to handle the low SNR noisy spectrum Recently the Wiener estimation approach 20 has been applied for spectra with heavy noise which can mitigate the effect of noises by utilizing the IR spectrum along the wavenumber dimension Among the SPR methods the Richardson Lucy RL method 21 is widely adopted in the spectrum curve estimation But the noises will be amplified by the RL method with the increased iterations Noise amplification can be eschewed by integrating some prior knowledge conditions For example Liu et al 22 integrates the total variation TV regularization it to the RL method which can split the overlap bands and preserve the details in spectrum curves In 23 Clupek et al proposes a TV based finite impulse response restoration method for the suppression of spectral noises and background from the IR spectrum Zhang et al 24 improves the RL blind restoration method by introducing the adaptive total variation regularization to suppress noises in plain areas and reconstruct the spectral band structure details DISR Deep Infrared Spectral Restoration Algorithm for Robot Sensing and Intelligent Visual Tracking Systems Hai Liu You Fu Li Senior Member IEEE Dan Su Zhaoli Zhang Sannyuya Liu and Tingting Liu 2019 IEEE RSJ International Conference on Intelligent Robots and Systems IROS Macau China November 4 8 2019 978 1 7281 4003 2 19 31 00 2019 IEEE8006 The major contributions of this paper could be summarized as follows i Gabor transforms are applied to analyze the essential difference between the clean spectrum and raw observation which can be capable of fully digging the distribution among the low middle and high frequency ii A designed TV regularized Gabor coefficients adjustment GCATV descriptor is incorporated into the IR spectrum restoration model for the first time iii The split Bregman iteration SBI method is introduced to optimize the GCATV regularized IR spectrum restoration model with a light computation cost which allows real time materials recognition in robot vision applications 0 0 0 5 1 0 0 0 0 5 1 0 0 2000 4000 0 03 0 02 0 0100 01 0 02 0 03 a b c d 0 03 0 02 0 0100 010 02 0 03 hist Go hist Gg sparisty Random noises Overlap area High frequency Low frequency Plain area Structure area Middle frequency Overlap 2 a 2 b Figure 2 Coefficients comparison between the real spectrum and ground truth a Observed spectrum b High resolution spectrum c Gabor transform coefficient distribution of spectrum in 2 a d Coefficient distribution of spectrum in 2 b I CHARACTERISTIC ANALYSIS FOR IR SPECTRUM A Problem Formulation IR spectrum formation process 8 9 can be described as Poisson Poisson Fgfgo vvv 1 where g v and o v denote the clean and observed IR spectrum respectively Here f v is the INF Poisson represents Poisson noises process The objective of spectrum restoration is to compute g v and f v from the observed o v B Gabor Transform for IR Spectrum Gabor transform 25 has been widely used for nature image denoising and super resolution tasks Inspired by this an attempt was made to analyze the IR spectrum It can be seen in Fig 2 that there is an illustrative difference of the Gabor transforms coefficients of the degraded IR spectrum Fig 2 a compared with the ground truth one Fig 2 b Actually as depicted in the Fig 2 c and Fig 2 d there also exits a strong sparsity distribution of a high resolution IR spectrum In other words most of the high frequency components are nearly equal to zero and the number of zeros is clearly larger than that in the degraded IR spectrum Considering the fact that the coefficient sparsity distribution of the IR spectrum it could be useful to propose a novelty coefficients adjustment regularizer to utilize the piecewise smoothness and the sparsity in IR spectrum 00 0050 0100 0150 0200 025 Coefficients Number LowMidHigh Groundtruth distribution in Fig 2 d Degradation distribution in Fig 2 c transform Plain area Noise area Overlap area Structure area MWIR spectrum Gabor transform coefficients transform Figure 3 Gabor coefficients adjustment from degradation distribution in Fig 2 c to the ground truth one Fig 2 d C GCATV Regularization for IR Spectrum As analysis aforementioned exploiting the prior knowledge is an important consideration for IR spectrum noise suppression Combination the Gabor transform 25 and TV properties in IR spectral processing we propose a GCATV into the model which can be described as follows TV Ggg R 2 where TV denotes the L1 norm of the Gabor coefficients To keep it simple the symbol Gg is utilized to represent the Gabor coefficients In Fig 3 it is shown the adjustment sketch for the Gabor coefficients transform For the plain area the low frequency coefficients blue region should be adjusted bigger For the overlap area the middle frequency coefficients green region should be adjusted smaller i e the blue dot line is adjusted to the green line For the structure area the high frequency coefficients red region should be adjusted smaller since the noises contaminate the IR spectrum II PROPOSED METHOD A IR Spectrum Restoration Considering the cases where the IR spectrum is corrupted by Poisson noises the intensity values of all spectral data v in the degraded IR spectrum o are the random variables that obey the independent Poissonian distributions Thus the probability p o g f can be rewritten as L v v vv v vv p exp fg o fg fgo o 3 where L denotes the IR spectrum size To compute u and f iterative restoration method can be utilized One selection is to employ the log p o g f in Ref 19 which minimizes the following function v vvvvvE log fgofgg 4 Because the IR spectrum deconvolution process is an ill posed problem thus the regularization term is introduced to address this According to the Bayes theorem p g f o p o g f p g p f the restoration model for a IR spectrum can be illustrated as the following framework log v fgFgoFgfgR R E 5 where Fg denotes the matrix vector of the vvfg Both R g and R f are the regularization terms which give a prior knowledge for the clean IR spectrum and INF The symbols and represented the weight parameters which can control the balance among the fidelity term and two regularization terms 8007 B DISR Model The second term R g in 5 is the smoothness constraint for IR spectrum According to the analysis the Gabor transforms coefficients the GCATV regularization is introduced to enforce the solution space We substitute 2 into 5 the GCATV regularization based IR spectrum restoration model can be formulated as log fGgFgoFgfgR Q TV v 6 For the third term R f in 5 the difference matrix is employed to describe its smoothness i e 2 ff R 7 where is linear operator corresponding to the first order differences at spectral data v i e 4 2 11vv ffff v Since the IR spectrum size is L the first order difference of the spectral line will be of size of L 1 With the GCATV model the final MAP based restoration model proposed in this article can be formulated as 2 TV log fGgFgoFgfg E v 8 To summarize the motivation of the proposed method is to enforce the coefficient distribution of the degraded IR spectrum see Fig 2 c into the similar distribution pattern as the ground truth one see Fig 2 d while penalizing the smoothness of the INF to suppress the noise III SBI OPTIMIZATION The cost functional E g f is needed to optimize and estimate the optimal latent spectrum g and INF f A Calculating the Restored IR Spectrum g The split Bregman algorithm has been put forward in addressing the regularization problem based on the L1 norm and shows an adequate result 26 The basic idea of this optimization approach is applied by converting the unconstrained minimization of g to a constrained spectrum through adopting complementary constraints namely d1 Fg d2 Gg First we introduce the auxiliary variable d1 and d2 to the optimization process in the following way GgdFgd GgFgoFg d dg 21 log min 21 tosubject v TV 9 Then the Bregman iteration is applied leading to the result of transforming the constrained minimization 9 into an unconstrained one which is shown as follows 2 1 log min 2 222 2 211 111 21 bGgdbFgd ddod d dg v TVvv 10 where is the Bregman penalty parameter and the variables b1 b2 are optimized by the Bregman iteration The function 10 is adapted to three simplified minimization subproblems The first problem regarding of g can be expressed as 2 1 min 2 222 2 211 bGgdbFgd g 11 which is a convex function and equals a linear equation 2 2211 T1 bdGbdFgIFF TkT 12 The solution to 12 is a closed design with the appliance of the fast Fourier transform FFT 15 is then presented as fft 2 fft ifft 2211 T 1 FF bdGbdF g T T k 13 where fft represents the FFT while the ifft means its inversed form The superscript T denotes the matrix transpose with k stands for the time of iterating The subproblem in terms of d1 can be described as 2 21111 2 1 log min 1 bFgddod d v vv 14 A soft shrinkage operation is adopted to solve the problem 14 which is od 4 2 1 21 1 kkk 15 where kkk 1 1 1 Fgb Similarly 1 2 k d can be also derived simultaneously 12 1 2 1 2 11 2 0 max kk kk kkk bGg bGg bGgd 16 Finally the variables 1 1 k band 1 2 k b are updated parallelly in each iteration as follows 1 2 11 2 1 2 1 1 11 1 1 1 kkkk kkkk dGgbb dFgbb 17 The advantage of adopting the split Bregman iteration is the challenging optimization task for 10 can be separated into the three simply tasks for optimization We utilize the FFT to accelerate the g related task optimization with high efficiency B Calculating the INF f When the restored IR spectrum g is fixed as a known constant u0 o the INF f can be formulated as 2 v log fFgoFgf E 18 The calculation of the INF f can be derivate as kk k k k 1k fg o g f1 f f T 19 where the symbol denotes the adjoint operation IV EXPERIMENTS A Experiments for Noise free Case To verify the benefits of proposed method comparisons are conducted between our approach and the three state of the art algorithms SRA 27 A spectral restoration approach with high spectral specificity SRM 20 A spectral reconstruction method with stepwise and Wiener estimation SBD 28 A spectral semi blind deconvolution algorithm with HS regularization We choose the best performed values of the parameters for each method according to their articles The comparisons are carried out on the measured IR spectra under different noise levels We compare them by three frequently used indicators namely WCC 29 RFWHM and RMSE 30 different materials are selected with each added a new random noise and the 8008 average results of the 30 trials are shown in this section The RFWHM is a no reference metric no need for a ground truth spectrum thus it can serve as an index on real IR spectra for experimental purpose These indices can indicate the narrowing ability and noise suppression 19 600800100012001400 0 0 2 0 4 0 6 0 8 1436 912 767 1216 1162 a 600800100012001400 0 0 2 0 4 0 6 0 8 911 768 1205 1443 c 600800100012001400 0 0 2 0 4 0 6 0 8 d 0 10 20 30 0 0 005 0 010 0 015 0 020 0 025 0 030 0 035 30 20 10 b 1 band overlap band overlap spectral noises lower IDF Noises Figure 4 Data simulation for IR spectrum a IR spectrum from 1500 to 400 cm 1 b INF Gaussian function with 8 cm 1 c Degraded spectrum convoluted by the INF d Adding the Poisson noise a b artifact too deep details missing details missing Original spectrum Degraded spectrum Recovery spectrum 0 8 0 6 0 4 0 2 1 1 600800100012001400 0 0 0 2 0 4 0 6 0 8 c 600800100012001400 d more smooth ness too shallow details missingdetails 769 911 1161 1213 1344 0 0 Figure 5 Deconvolution results of IR spectrum under the noise free condition a SRA 27 b SRM 20 c SBD 28 d Proposed algorithm Following 1 a degraded IR spectrum is simulated based on the experimental spectrum of an industrial material methyl formate from 1500 to 400 cm 1 Figure 4 c can be simulated by the convolution operation between the original spectrum in Fig 4 a and INF in Fig 4 b After degradation the spectrum shows more smoothness and low resolution The peaks also present much more in width as well as lowness in value see peaks at 767 912 and 1436 cm 1 To investigate the performance of the noise suppression the overlap spectrum was contaminated by the Poisson noise with SNR 100 Fig 4 d Figure 5 illustrates the restoration results of an industrial material spectrum noise free case by different algorithms SRA see Fig 5 a and SRM see Fig 5 b produce rather abstract results which miss many spectral details In Fig 5 b the SRM has greatly improved the description the plain area but it also produces much artifact ringing peaks Both SBD see Fig 5 c and DISR see Fig 5 d obtain much purer results than the other algorithms but SBD produces too shallow valley shown by the red arrow By contrast Fig 5 d achieves much clean and faithful spectral details which is more similar with the ground truth spectrum red line Table I Band Distortions of IR Spectra Deconvolution by Different Algorithms ParamMeth 767 912 1162 1216 1436 RMSE Heig ht SRA 0 143 0 191 0 152 0 242 0 031 0 186 SRM 0 162 0 182 0 181 0 091 0 128 0 170 SBD 0 102 0 201 0 198 0 083 0 113 0 157 DISR 0 032 0 102 0 042 0 071 0 0930 098 Posit ion SRA 2 1 1 2 1 1 643 SRM 1 0 1 2 2 1 516 SBD 1 1 1 2 0 1 303 DISR0 0 1 1 0 0 837 Moreover we investigate the spectral signature change between the restored spectrum in Fig 5 and the ground truth one Fig 4 a Five bands are selected as the references such as 767 912 1162 1216 1436 cm 1 These bands distorted parameters are listed in Table I as well as the heights along with the wavenumbers between the restored blue lines and ground truth spectrum red lines RMSE are examined and shown in Table I it is obvious that the RMSE values obtained by the DISR method are smaller than those achieved by the compared algorithms b 0 0 0 2 0 4 0 6 0 8 a residual noise residu