数字图像处理作业2 图像拼接.doc

Image Processing & Computer VisionExercise 2: Geometrical TransformsZhongli LiuS1018531M-EMSYSAddress: Calslaan 26-004 Email: z.liustudent.utwente.nl28-2-2010The problem addressed in this exercise is image stitching. Figure 1 shows two images of a map. The images partly overlap. The purpose of this exercise is to glue the images together to obtain a single image covering the full width of the map. We do this by first applying a geometrical transform to the second image such that the transformed image can be seamlessly overlaid by the first image.Figure 1.1 Figu1.21. Matlab supports a number of geometrical transforms, i.e. box, projective, affine, etc. See maketform. The position and orientation of the camera relative to first map differs from the position and orientation relative to second map. What is theoretically the appropriate transform for this application? Motivate your choice.According to the above information, it is assumed the left image and right image are not in a same plane, which means a line can be shorter or longer and two parallel lines may not be parallel in the corresponding location in the other image. Hence the transform projective is used for its mathematical relation between new and old elements as below:x = (ax + by + c)/(gx + hy + 1) (1)y = (dx + ey + f)/(gx + hy + 1) (2)With the transform, a rectangle like in one image can be converted to be a to match the corresponding points in another image. Meanwhile, two images are not in a same plane is also the reason why the transform affine is not that good. Because affine transform can only convert a rectangle to be a parallelogram , some points in the two images may not be well matched when the images are expressed in different 2D plane. However in this assignment, the difference of focusing distances and angles between two original images is not that much which means affine is not a very bad choice. Anyway, projective is a better decision.2. In order to find the parameters of this transform, we manually select a number of points in the first image and select the corresponding points in the right image. What is the minimal number of corresponding points that is needed to define this transform?As seen in the equation (1) and (2), in total 8 unknown coefficients need to get. Therefore, 4 pair corresponding points are needed sufficiently to obtain 8 equations in order to calculate 8 coefficients. The minimal number of corresponding points needed is 4.3. Create and execute an m-file that defines this minimal set of corresponding points. (Hint: use cpselect).The command and the figure are shown below:cpselect(rgb2gray(img2), rgb2gray(img1);Figure 2.1 Figure 2.2In order to get a good transform, the points should be chosen that the spans between points are enough to obtain the correct ratio of two images.4. Create and execute an m-file that uses the set of corresponding points and creates a transformation structure (use maketform or cp2tform).The relative code is shown below, and the parameter input_points and base_points are defined in the last step by command cpselect.TFORM = cp2tform(input_points, base_points, projective);5. Extend the m-file of 4: apply the transform to the second image by using the transformation structure in imtransform. Hint: use the options XData, YData and XYscale to assure that the output image is large enough to contain both images (read the help of imtransform!). In order to find the right position you may want to calculate where the corners of the second image should be located in the transformed image. You can calculate that using tformfwd.The code is shown as below, the figure of transformed image2 is shown with x and y axis in Figure 3, and the figure of image1 with axis displays as a reference. img2_tr xdata ydata = imtransform(img2, TFORM);figure, imshow(img2_tr,XData,xdata,YData,ydata), axis onfigure, imshow(img1), axis onFigure 3.1 Figure 3.2In order to get the sufficient size of the final image and find the correct location of the image1 and transformed image2 in it, the following parameters are obtained.i j a = size(img1);m n b = size(img2_tr);x = round (xdata(1);y = round (-ydata(1);Because xdata(1)0 and ydata(1)0 is also added for more general use.column_size = max(m, y+i);row_size = max(x+n, j);Figure 4.1 Figure 4.2Figure 4.3 Figure 4.4Also according to the Figure 4.1Figure 4.4, in spite of which case it is, the correct location of im1 in the extended image should be from Y=y+1 to Y=y+I and X=1 to X=j; the correct location of im2_tr in the extended image should be from Y=1 to Y=m and X=x+1 to X=x+n. Following is the code building the new matrix of the extended size and locating the two images in each extended image.img_out = uint8(zeros(column_size, row_size, 3);img1_large = uint8(zeros(column_size, row_size, 3);img2_large = uint8(zeros(column_size, row_size, 3);img1_large(y+1):(y+i), 1:(j), 1:3) = img1;img2_large(1:(m), (x+1):(x+n), 1:3) = img2_tr;The result for image1 and image2 are shown in Figure 5.1 and Figure 5.2 respectively.Figure 5.1Figure 5.26. Copy the first image to the transformed image obtained in 5. Note: copy it to the right position as indicated by your choice of XData, YData and XYscale in 5.In the last step, the image1 and the transformed image2 are correctly located in the sufficient large images respectively. Now it is necessary to add 2 images (in fact it is addition of matrixes), and the command imgadd is used as below and the result is shown is Figure 6.imshow(imadd(img1_large,img2_large);Figure 67. Show the result on the screen and evaluate the result. Can you identify possible problems?As seen in the Figure 6, the overlapped area of the sum of two images is much brighter than before. That is because the RGB value of any point in this area is the sum of the two RGB values of two original images. In order to solve the problem, the command imsubtract is used to remove some part of the RGB value, in my code, the new RGB value of any point in this area is mainly decided by that of image1 (new value V = V(img1) V(img2)+ V(img2) = V(img1). The result of images substraction and the final image are shown in Figure 7 and Figure 8. The code to generate the final image is also included with the command imwrite.img_sub = imsubtract(img1_large, img2_large);figure, imshow(img_sub);img_out = imadd(img_sub,img2_large);imwrite(img_out, output.jpg);Figure 7Figure 8Appendix: Matlab Code (used in Matlab7.1)% 23-2-2010 zhongli% read the image and displayimg1 = imread(schier_left.jpg);figure(1);imshow(img1,);img2 = imread(schier_right.jpg);figure(2);imshow(img2,); % find the corresponding points in the two images for transformcpselect(rgb2gray(img2), rgb2gray(img1);pause; % build the transform structureTFORM = cp2tform(input_points, base_points, projective);% apply the transform on the image and displayimg2_tr xdata ydata = imtransform(img2, TFORM);figure(3), imshow(img2_tr,XData,xdata,YData,ydata), axis onfigure(4), imshow(img1), axis on% find the size of image1 and transformed image2i j a = size(img1);m n b = size(img2_tr); % extend the two images, ensuring the new image can contain the stitched image, and correctly locate the img1 and img2_trx = round (xdata(1); if(ydata(1)0) y = round(-ydata(1); column_size = max(m, y+i); row_size = max(x+n, j); img_out = uint8(zeros(column_size, row_size, 3); img1_large = uint8(zeros(column_size, row_size, 3); img2_large = uint8(zeros(column_size, row_size