Reportofimproving3DFaceReconstructionGeometry-based

1、Report of improving 3D FaceReconstruction:Geometry-based ApproachBy Jiang Shan? June, 2007Jiang Shan1AbstractThis short report is about the improvement achieved during June, 2007 on theexisting software 3D Face Reconstruction. The improvement has been donemainly on three issues: 1) Graphic User Inte

2、rface; 2) input and output; 3)accuracy. This report only refers improved features and new functions. For fullinformation about the existing software, refers to the references.Jiang Shan2Table of ContentsAbstract.2Table of Contents.31Improvement on Graphic User Interface.41.1Layout.41.2Distinguish Pa

3、rts of Face by Colours.61.3Displaying and Operating Two Images Simultaneously.62Improvement on Input and Output.82.1Exporting 3D Model as *.obj File.82.2Saving and Loading Feature Points.83Improvement on Accuracy.103.1New Interpolation Method.103.2Improving Shape of Nose.103.3Texture Mapping.12Refer

4、ences.13Jiang Shan31 Improvement on Graphic User Interface1.1 LayoutIn order to adapt to new features and new functions appended, the layout ofthe GUI has been adjusted properly. Comparing to the previous version, thefollowing adjustment has been done:(1) Larger space is reserved to display source i

5、mages and feature points.(2) Font size of GUI components in panels Translationand Scalingareturned as small as other GUI components.(3) The options in Quality panel are reduced to two from five.(4) A new option Edit both Imagesis added in Select Imagepanel.(5) A new panel Textureis appended. It cont

6、ains two options:Frontal ImageOnlyand Frontal and Side Images.(6) A new checkbox Single Selectionand a new command button Add NewNose Pointsare added inOperated Partspanel.(7) New menu items Load Feature Points,Save Feature Points,and Export3D Modelsare added in menuFile.(8) Adjust Points by Mousebu

7、tton is moved from top to bottom.The following figures show the comparison between the previous version andthe new one:Figure 1.1.1: Left Menu of Previous Version; Right New MenuJiang Shan4Figure 1.1.2: GUI Layout of the Previous VersionFigure 1.1.3: GUI Layout of the New VersionJiang Shan51.2 Disti

8、nguish Parts of Face by ColoursIn the previous version, all feature points are in blue. In order to distinguish eachpart from others, users should remember which part every single feature pointbelongs to. In the new version, all feature points are coloured group by group. Fortwo neighbouring parts,

9、different colours are used. Also, checkboxes used toselect different parts of face in Operated Partspanel are colouredcorrespondingly. Thus, users can distinguish each part based on colour directly.Besides this feature, a new mechanism called single selection is added. Itis controlled by checkbox Si

10、ngle Selection. When this checkbox is checked,users can select only one part of the face at once, i.e. all checkboxes used toselect parts will work as radio buttons. When Single Selectionis unchecked,multiple selections are allowed as the previous version.1.3 DisplayingandOperatingTwoImagesSimultane

11、ouslyIn the previous version, users can work on either frontal or side image, butnot both of them. In this version, option Edit Both Imagesallows displayingand operation both frontal and side images simultaneously.When a user does one operation,if it is an operation allowed in both twoimages, both o

12、f them will be affected; if it is only allowed in one of them, theimage allowing it will be affected and the other image will ignore it.Mouse operation is not allowed in this situation, because there are twosubplots, and it is difficult to distinguish which one of them is affected by themouse operat

13、ion.The shortage of displaying two images together is both of the two imagesturn smaller, so that feature points turn closer to each other. Figure 1.3.1shows it. Thus, locating a single feature point is not very efficient and accurate.For this reason, displaying and operating single image are inheri

14、ted from theprevious version. In order to monitor which state is in, a global variablehandles.front_sideis used. When the value of it is 0, it means frontal image isJiang Shan6being edited. When it is 1, it means side image is being edited, and -1 is forboth images.Figure 1.3.1: Displaying Frontal a

15、nd Side Images SimultaneouslyJiang Shan72 Improvement on Input and Output2.1 Exporting 3D Model as *.obj FileIn the previous version, reconstructed 3D face model can be saved in *.Matformat. However, *.mat format can be used only by Matlab and programs builtby Matlab. In order to allow other softwar

16、e loading the reconstructed modelsbuilt by this system, a new is appended in this version to export 3D model as*.obj file.This task is completed by a new function calledtools_3D_exportOBJ_File.This function accepts four arguments: path, whichis string; points3D, which is an n-by-3 matrix; faces, whi

17、ch is an m-by-3 matrix;vertices_colour, which is a p-by-3 matrix. What it does is basically using afunction provided by Matlab called fopen to create a file under the path passedas argument with write attribute. Then writing this file by function fprintf basedon the format of *.obj and the content o

18、f other arguments passed.The path of the file is selected by user under a standard Windows savingfile GUI dialog box.2.2 Saving and Loading Feature PointsFor particular tasks, the amount of work of reconstructing an acceptable 3Dface model can be huge. Users may become tired during the work. In orde

19、r toprovide the possibility of having breaks during reconstructing, the newfunctions of saving and loading feature points are appended in this version.The format of feature point file is the following:Front image pathof points in frontimage x yx y. .Jiang Shan8Side image pathof points in sideimage z

20、 yz y. .adjuster_radjuster_nFace listvertice1 vertice2 vertice3vertice1 vertice2 vertice3. . .Two possible file formats can be selected. One is *.mat file, which canbe used by all programs built by Matlab. And the other one is *.mpts file, whichis written in ASCII. It can be used by any program as l

21、ong as a standard filereader is provided.Jiang Shan93 Improvement on Accuracy3.1 New Interpolation MethodIn order to improve the accuracy of the reconstructed 3D face models, a newinterpolation method is used. This method is based on curved PN triangles. It is anon-linear interpolation algorithm. Fo

22、r each triangle face in the basic 3D model,seven vertices are added, two on each edge, one in the centre of the face, andnine triangle faces are formed. Because there are 113 vertices (104 useful) and184 faces in the CANDIDE model, after one interpolation, 1401 vertices (1394useful) and 1656 faces a

23、re involved. For full information about curved PNtriangles, refer to the document on it. In page 160, the layout of new vertices andfaces are demonstrated. And in page 162, the mathematics formulas oncalculating each vertex, as well as normal vectors are explained.Based on current research, this int

24、erpolation method cannot be nestedmore than once. The reason is not clear enough right now. Thus, if higherresolution is required, interpolation algorithms employed in the previous versionare recommended rather than this one.3.2 Improving Shape of NoseFor a 3D face model, the shape of nose is one of

25、 the most important issues onaccuracy. However, due to the shortage of the CANDIDE model, it is difficultto capture the shape of nose accurately in the previous version. In order to solvethis problem, some researchhas been done.Generally speaking, the plan is thefollowing:(1)Some new feature points

26、are provided near the nose. Eitherautomatically or manually done by users.(2)A mechanism is employed to capture the shape of nose by newfeature points.(3)Re-triangulation is done between new feature points and existing 3Dmodel.Jiang Shan10For the first step, theoretically speaking, adding feature po

27、ints manually isan ideal way. It provides the most flexibility and can reduce the work in step 2.However, it is not so practical, becauseactions performed by users cannot becontrolled by the software. Thus, if a user adds new points near the earinsteadof nose, the performance is damaged and the soft

28、ware can do nothing tocorrect it.Adding a user-defined number of feature points with fixed location is thesecond most flexibleoption.However,itisrejected because ofthere-triangulation algorithm. Details will be explained in step 3.At last, we choose to add four new feature points automatically. It i

29、s lack offlexibility, but very reliable. And based on experiments so far, it works very well.Those four feature points are not brandnew, though. They are some of uselessvertices provided by the CANDIDE model without any connection with othervertices and faces. The indices of those points are 5, 36,

30、37, and 38 in theCANDIDE.For step 2, a snake algorithm is rejected for its time consummation andcomplexity in implementation and testing. For now, those feature points arelocated automatically based on the position of other feature points on the nose.Then users can move them to the right position by

31、 mouse operation as otherfeature points.For step 3, Delaunay triangulation algorithm is considered. The problem ofit is it is an algorithm for general usage,but not specific on a face model. Inmany instancesof experiments, it does not consider the nose as a whole unit,and breaks it into two or more

32、parts. For this reason, it is rejected. For now,the re-triangulation is done manually, and recorded as a part of the code of thesoftware. Thus, there are two CANDIDE models employed, one is the originalone, and the other is re-triangulated involving four new feature points. Thefollowing figure shows the re-triangulation. Fifteen faces are involved totally,the indices are: 176 182, which belong to the or