视网膜生物化学国外英语科技论文资讯.doc

Foveated Vision Sensor and Image Processing A Review Mohammed Yeasin, Rajeev Sharma1. Department of Electrical and Computer Engineering, University of Memphis, TN 38152-3180Email: 2. Department of Computer Science and Engineering The Pennsyl-vania State University,University Park, PA-16802 Abstract. The term foveated vision refers to sensor architectures based on smooth variation of resolution across the visual field, like that of the human visual system.The foveated vision, however, is usually treated concurrently with the eye motor system, where fovea focuses on regions of interest (ROI). Such visual sensors expected to have wide range of machine vision applications in situations where the constraint of performance, size, weight, data reduction and cost must be jointly optimized. Arguably, foveated sensors along with a purposefully planned acquisition strategy can considerably reduce the complexity of processing and help in designing superior vision algorithms to extract meaningful information from visual data. Hence, understanding foveated vision sensors is critical for designing a better machine vision algorithm and understanding biological vision system. This chapter will review the state-of-the-art of the retino-cortical (foveated) mapping models and sensor implementations based on these models. Despite some notable advantages foveated sensors have not been widely used due to the lack of elegant image processing tools. Traditional image processing algorithms are inadequate when applied directly to a space-variant image representation. A careful design of low level image processing operators (both the spatial and frequency domain) can offer a meaningful solution to the above mentioned problems. The utility of such approach was explefied through the computation of optical flow on log-mapped images. Key words. Foveated vision, Retino-cortical mapping, Optical flow, Stereo disparity, Conformal mapping, and Chirp transform.IntroductionThe amount of data that needs to be processed to extract meaningful in-formation using uniform sampling cameras is often enormous and also redundant in many machine vision applications. For example, in case of autonomous navigation 1, 2, vergence control 3, 4, 5, estimation of time-to-impact 6, 7, 8, object recognition 9, 10, 11 and object tracking 12, 13, 14, one usually needs a real-time coordination between sensory perception and motor control 15. A biologically motivated sensor along with purposefully planned acquisition strategy can considerably reduce thecomplexity of processing. Hence, the main theme behind developing a space-variant sensor is to establish an artificial vision and sensory-motor coordination. The aim could also be to understand how the brain of living systems sense the environment also transform sensory input into motor andcognitive functions by implementing physical models of sensory-motor behaviors. Studies on primate visual system reveal that there is a compromise which simultaneously provides a wide field of view and a high spatial resolution in the fovea. The basis of this compromise is the use of variable resolution or Foveated vision system 16. The term Foveated vision refers to sensor architectures based on smooth variation of resolution across the visual field, like that of the human visual system. Like the biological ret ina, sensor with a high resolution fovea and a periphery whose resolution decreases as a function of eccentricity can sample, integrate, and map the receptor input to a new image plane. This architecture is an efficient meansof data compression and has other advantages as well. The larger receptive fields in the periphery integrate contrast changes, and provide a larger separation for sampling the higher velocities. Their elegant mathematical properties for certain visual tasks also motivated the development of Fove-ated sensors. Foveated architectures also have multi-resolution property but is different from the pyramid architecture 17. Despite some notable advantages space-variant sensors have not been widely used due to the lack of elegant image processing tools. Nevertheless, the use of space-variant visual sensor is an important fac-tor when the constraint of performance, size, weight, data reduction and cost must be jointly optimized while preserving both high resolution and a wide field of view. Applications scenarios of such sensors include:Image communication over limited bandwidth channels such as voice-band telephony 18 and telepresence 19, 20.Surveillance applications for public spaces (e.g., intelligent highway applications, factories, etc.) 21 and private spaces (e.g., monitoring vehicles, homes and etc.)22.Applications in which visible or infra-red camera system is used to analyze a large work area 23, and communicate the scene interpretation to a human observer via non-visual cues.Field applications (for example, agriculture, forestry, and etc.)in which identification and classification from a wide field of view must be performed by a small, low power portable system and communicated to the human user.Autonomous and tele-operated vehicle control. The broad range of applications mentioned above is by no means ex-haustive, rather, an indication of the potential advantages that a biologi-cally motivated design can offer to the large segment of machine vision and image communication. Although many difficult problems are con-fronted in the application of a space-variant sensor, one is motivated by the example of biological vision, the useful geometric characteristics and ele-gant mathematical properties of the sensor mapping, favorable space-complexity and synergistic benefits which follows from the geometry as well as from the mapping. The physiological and perceptual evidence indicates that the log-map image representations approximates the higher vertebrate visual system quite well and have been investigated by several researchers during the past several decades (for example, 24, 25, 26, 27, 28). Apart from these,a variety of other space-variant sensors has been successfully developed (for example, ESCHeR 29, 30), and has been used for many machine vi-sion tasks with a proven record of success and acceptable robustness 31,32. The problem of image understanding takes a new form with foveated sensor as the translation symmetry and the neighborhood structure in the spatial domain is broken by the non-linear logarithmic mapping. A careful design of low level image processing operator (both the spatial and fre-quency domain) can offer a meaningful solution to the above problems.Unfortunately, there has been little systematic development of image un-derstanding tools designed for analyzing space-variant sensor images. A major objective of this chapter is (i) to review the state-of-the-art of foveated sensor models and their practical realizations, and (ii) to review image processing techniques to re-define image understanding tools to process space-variant images. A review catadioptric sensor 33, 34, 35, 36,37 and panoramic camera 38, 39 which also share similar characteristics i.e., variable resolution and wide field of view were not included. The rest of the chapter is organized as follows. Section 2 review the retino-cortical mapping models reported in the literature. The synergistic benefits of log-polar mapping were presented in Section 3.Following this; Section 4 pre-sents the sensor implementations to date to provide a picture of the present state-of-the-art of the technology. Subsequently, discussions on the space-variant form of the spatial and frequency-domain image processing opera-tors to process space-variant images were presented in Section 5 Section 6 presents the space-variant form of classic vision algorithms (for example,optical flow on log-mapped image plane). The utility of the biologically motivated sensors were discussed in Section 7 and finally, Section 8 con-cludes the chapter with few concluding remarks. A Review of Retino-cortical Mapping Models The visual system has the most complex neural circuitry of all sensory sys-tems. The flow of visual information occurs in two stages 40: first from the retina to the mid-brain and thalamus, then from thalamus to the pri-mary visual cortex. Although, the primate eye has components serving functions similar to those of standard video cameras the eyes light transduction component, the retina, differs greatly from its electronic coun-terpart. Primate visual field has both binocular and monocular zones. Light from the binocular zone strikes the retina in both eyes, whereas light from the monocular zone strikes the retina only in the eye on the same side. The retina responds to the light intensities over a range of at least 5 orders of magnitude which is much more then standard cameras. Structurally, the retina is a three layer membrane constructed from six types of cells (for de-tails please see 41). The light transduction is performed at the photore-ceptors level, and the retinal output signals are carried by the optic nerve which consists of the ganglion cell axons. The ganglion cell signals are connected to the first visual area of the cortex (V1) via an intermediary body. The investigation of the space-variant properties of the mammalian retino-cortical mapping dates back to the early 1940s. In 1960s Daniel et. al. 42 introduced the concept of cortical magnification factor c,measured in millimeters of cortex per degree of visual angle, in order to characterize the transformation of visual data for retinal coordinates to primary visual cortex. The magnification factor is not constant across the retina, but rather varies as a function of eccentricity. Empirically, the corti-cal magnification factor has been found to be approximated by 43where ? is the retinal eccentricity measured in degrees, and C1 and C2 are experimentally determined constants related to the foveal magnification and the rate at which magnification falls off with the eccentricity, respectively. Integrating Equation (1) yields a relationship between the retinal eccentricity and cortical distance .To obtain an understanding of the variable resolution mechanism involved in the retina-to-cortex data reduction, one needs to understand the different aspects of the primate visual path ways (see 40 for details). Researchers from inter-disciplinary fields have been investigating this issue for quite some time and Schwartz 43 has pointed out that the retino cortical mapping can be conveniently and concisely expressed as a conformal transformation11, i.e., the log(z) mapping.This evidence does not by any means paint a complete picture about the processing and extent of data reduction performed by the retina. Nevertheless, it lays the foundation for the retino-cortical mapping models reviewed in this paper. Conceptually, the log(z) retino-cortical model consists of considering the retina as a complex plane with the center of fovea corresponding to the origin and the visual cortex as another complex plane. Retinal positions are represented by a complex variable z, and the cortical position, by a complex variable.The correspondence between these two planes is dictated by the function = log(z). The mapping model = log(z), has a singularity at the origin i.e. at z = 0, which complicates the sensor fabrication.To avoid the singularity at origin and to fabricate a physical sensor,Sandini et. al. 27, 44, 45 have proposed a separate mapping models for the fovea and the periphery. These mappings are given by equations (3) and (4) for continuous and discrete case, respectively:Where are the polar coordinates and are the log-polar coordinates. In the above expressions0 is the radius of the innermost circle,1/q corresponds to the minimum angular resolution of the log-polar layout,and p, q and a are constants determined by the physical layout of the CCD sensor that are related to the conventional Cartesian reference system by:. Though this method provides an easy way to construct a physical sensor, the fovea-periphery discontinuity is a serious drawback. In addition, the mapping is not conformal over the range of the sensor, which is an important factor in developing tools to process space-variant images.Alternatively, Schwartz 46 proposes a modified mapping,and shows that by selecting an appropriate value for a (a is real number in the range of 0.3 - 0.7 47), a better fit to retino topic mapping data of monkeys and cats can be obtained 48. As opposed to the provides a single output image. With modified mapping, the singularity problem, the need for uniform resolution patch in the fovea and the fovea-periphery boundary problems, is eliminated. To perform the mapping, the input image is divided into two half-planes along the vertical mid-line. The mapping for the two hemi-fields can be concisely given by the equationWhere is the retinal position and is the corresponding cortical point, while indicates left or right hemisphere. The combined mapping is conformal within each half plane22.In a strict mathematical sense, the properties of scale and rotation invariance are not present in the mapping. However, if then , and therefore, these properties hold. Also, since The template has a slice missing in the middle, circles concentric with and rays through the foveal center do not map to straight lines. To the best of our knowledge, no physical sensor exists which exactly mimics this model, but there are emulated sensor that approximates this model24.Another attempt to combine peripheral and foveal vision has been reported in 49 using specially designed lens. The lens characteristics are principally represented by the projection curve expressed in Equation (6),which maps the incident angle of a sight ray entering the camera to r(),the distance of the projected point on the image plane from the image center. This curve has been modeled in three distinct parts to provide wide and high resolution images: a standard projection in the fovea, a spherical one in the periphery and a logarithmic one to do a smooth transition between the two:where q, p and a are constants computed by solving continuity conditions on zeroth and first order derivatives, f1, f2 and f3 are the respective focal length (in pixels) of the three projections and 1, 2 and max are angular bounds.It combines a wide field of view of 120 degree with a very high angular resolution of 20 pixels per degree in the fovea. Those properties were achieved by carefully assembling concave and convex optics sharing the same axis. Despite the complexity of its optical design, the physical implementation of the lens is very light and compact, and therefore suitable for active camera movement such as saccade and pursuit.Synergistic BenefitsThere are a number of synergistic benefits which follows from a biologically motivated (i.e., complex log-mapping, log-polar, etc.) sensor. Like the human eye, a foveated sensor does not require a high quality optics off axis, as conventional cameras do, since peripheral pixels are in effect low pass filters. The complex log-mapping also provides a smooth multi resolution architecture 47 which is in contrast with the truncated pyramid architecture33 that is common in machine vision 17. The scale and rotation invariant properties of the mapping simplifies the calculation of radial optical flow of approaching objects, allowing the system to quickly calculate the time to impact. The selective data reduction is helpful in reducing the computation time and is useful in many image analysis and computer vision application.A mentioned earlier, retino-cortical mapping model provides a scale and rotation invariant representation of an object. The scale and rotation in variance of the transformation is illustrated (see Fig 2.1) by mapping bars of various size and orientation from standard Cartesian representation to a cortical representation. Figure 2.1 shows the results of the on center bars and the off-center bars. Clearly, the mapping (cortical representation) produce results which is independent of the size and orientation of the bar. It is important to note that the above properties hold if the rotation and scaling are centered about the origin of the complex-plane. This is due to the fact that the inertial axis is not unique, and can be ambiguous. The scale and rotation invariance property is of paramount importance and can be used to improve form invariant shape/object recognition. Traditional shape/object recognition schemes (i.e., template-matching and etc.) suffer from the variance of the size and the orientation of an object. The retinotopic mapping model of the form-invariant shape recognition approach may help in recognition of two-dimensional shapes independently from their position on the visual field, spatial orientation, and distance from the sensing device. The complex log-mapping has some favorable computational properties. It embodies a useful isomorphism between multiplication in its domain and addition in its range. It has line-circle duality4, which may be an interesting property for finding invariant features in the processing of space-variant images. For an image sensor having a pixel geometry image scaling is equivalent to radial shifting and image rotation is equivalent to annular shifting. Let us assume that the image is scaled by some real amount S, which can be written as Applying the log-mapping one would obtain,Similarly, rotating the image by an angle ? can be written as Applying the log-mapping one would obtain,Similarly, rotating the image by an angle ? can be written asA log-mapping leads to a relation,From equations (7) and (8) it is clear that scaling and rotation produces a shift alon