高分辨率刺激促进深度知觉：MT+在单眼深度知觉中起重要作用.docx

高分辨率刺激促进深度知觉：MT+在单眼深度知觉中起重要作用Yoshiaki Tsushima1,2, Kazuteru Komine1, Yasuhito Sawahata1& Nobuyuki Hiruma31Three-Dimensional Image Research Division, NHK Science and Technology Labs, Tokyo, Japan,2Sutokuin Lab., Osaka, Japan,3Human Interface Research Division, NHK Science and Technology Labs, Tokyo, Japan.摘 要 今天，我们人类面临着一个高质量的、完全新种类的虚拟世界。比如，我们有一个由足够高分辨率组成的数字显示，以致我们无法与真实世界区别开来。然而，很少有人知道高质量的表现对真实的感觉起了多少作用，尤其是在深度知觉（depth perception）方面。加工如此优良但虚拟的表现的神经机制（neural mechanism）是什么？这里，我们心理物理学（psychophysically）和生物学（physiologically）利用亮度对比（luminance-contrast）（明暗法）作为单眼深度线索调查了刺激分辨率与深度知觉之间的关系。结果，我们发现高分辨率刺激会促进深度知觉，即使这个刺激分辨率差别是无法察觉的。这个发现是反对传统认知层次的视觉信息加工。传统观点认为视觉输入（visual input）在一个自下而上（bottom-up数据驱动加工）层叠的大脑皮层（cortical）部位被连续不断地加工，对越来越多地复杂信息进行分析，比如深度信息。此外，功能核磁共振成像（functional magnetic resonance imaging, fMRI）结果显示人类颞中回（the human middle temporal, MT+）在单眼深度知觉（monocular depth perception）中起重要作用。这些结果可能不仅为我们深度知觉神经机制方面新的洞悉，而且为我们神经系统伴随着最先进技术的未来发展做准备。关 键 词 分辨率,深度知觉,神经机制,MT+,fMRI,1引言在现今时代，人类生活在我们以前从未见过的、由许多技术产生的高质量的虚拟世界里。例如，我们有一个由足够高分辨率组成的数字显示，以致我们无法与真实世界区别开来。一些看到高分辨率显示图像的人报告说他们比那些看低分辨率显示的人感觉更真实。然而，很少有人知道知道高质量的表现给真实的感觉出了多少力。加工如此优良但虚拟的表现的神经机制是什么？2方法两个空间图像（dimensional image）的深度信息可以大大地帮助我们感觉图像的真实性(Masaoka,Nishida,Sugawara,Nakasu,Nojiri,2013)。感知深度，换句话说，在我们的视网膜上创造两个空间图像的三维视野（the third dimension of a view）是我们神经系统最不可思议的功能之一。大量的研究显示了人们利用各种各样的方法去理解深度，比如双眼视差（binocular disparity）(Barlow, Blakemore, Pettigrew,1967; Pettigrew, Nikara,Bishop,1968)，运动视差（motion parallax）(Ferris, 1972; Reichardt,Poggio,1979)，明暗成形（shape from shading）(Horn,1975; Ramachandran,1988; Hou,2006; Sawada, Kaneko,2007)等，或者结合这些因素(Welchman,2005)。然而，我们依然不清楚刺激分辨率是怎样和神经联系一样，是怎样与深度知觉联系起来的（图1）。图1|相同的图像以不同的空间质量，低和高分辨率，展示出来。高分辨率图像（右）比低分辨率有更好的亮度对比表现。哪一个图像我们更能感觉到深度？（图画由Y.S与Y.T.创作，并用Adobe Illustrator CC, RRID: nlx_157287图形软件进行了修改。）这里，我们假设高分辨率显示的视觉图像（visual image）能促进深度知觉，并研究了刺激分辨率与深度知觉之间的关联。为了调查刺激分辨率与深度知觉的关系，这里我们利用亮度对比差异/改变（明暗法）作为深度线索(Ramachandran,1988; OShea, Blackburn, Ono, 1994)，因为显示分辨率显著地影响亮度对比差异/改变形势。另外，它还允许我们研究了深度知觉的一个简单机制，不包括复杂的、高水平的机制(Ramachandran,1988; Hou,2006; Sawada, Kaneko, 2007; Welchman,2005; OShea, Blackburn, Ono, 1994)。我们进行了一系列的心理物理学和功能核磁共振成像实验。在心理物理学实验中，我们运行了深度任务并要求被试报告用单眼观察时他们觉得哪个刺激在更远的地方（图2）。另外，为了研究被试是否注意到了刺激的分辨率不同，我们进行了分辨率任务并要求被试报告哪一个刺激有更高的分辨率。为了探究单眼深度知觉的神经机制，我们测量了那些进行深度任务的被试的功能核磁共振成像活动。另外，为了检验被试被要求做深度任务与集中于其他刺激特征时做的是不同的任务的可能性，比如亮度能量的完全或局部的数量在两个刺激物之间有所不同，因此我们进行了亮度任务，并比较了深度任务与亮度任务情况的功能核磁共振成像活动。图2|实验和刺激组。(a)，两个相同大小但不同分辨率的栅栏垂直呈现1秒。两个刺激之间的间隔是0.12度。为避免被试轻易地察觉出刺激分辨率的不同，它们很小，只有0.360.067度。刺激消失之后，被试报告他们觉得哪个刺激在更远的地方。这些刺激的组合在实验之间是多变的。(b)，利用了五种分辨率，分别是15.0, 30.0, 52.5, 105.0, 210.0 cpd。同时，有三种不同的刺激组，分别是1,2和3。Y轴代表了每个刺激的实际亮度值。粉色的虚线表示灰色背景的亮度值(14.4 cd/m2)。呈现的刺激在真实的实验中用相同的算法被运行。3结果3.1心理物理学实验的深度任务与分辨率任务结果，被试报告说在所有的刺激集合中，他们在高分辨率刺激下感觉更深（图3的红线）。另外，我们发现这种现象在高分辨率之间饱和了（举例说明，在两个3高分辨率刺激之间布莱德-特里分数(Bradley-Terry score，BTS) 的平均数没有显著差异）。根据分辨率任务的结果，没有被试意识到在刺激之间的分辨率不同（图3的蓝线）。从视觉敏锐试验（见方法）中，我们发现在深度任务与分辨率任务组之间没有显著的视觉敏锐度的差异。综合在一起，被试在高分辨率刺激下感觉更深，并且没有注意到分辨率的不同。然而，有可能是被要求做深度任务的被试做了一个不同的任务在面对其他刺激特征时，比如亮度能量在两个刺激之间有所不同（见图2）。为了检验这种可能性并检验心理物理学结果与神经的关联，我们要求被试在功能核磁共振成像中做深度任务与分辨率任务，并且比较功能核磁共振成像活动在深度任务与分辨率任务之间的情况。图3|深度任务与分辨率任务的结果。每个刺激在深度任务中的平均布拉德利-特里分数(Bradley, 1984)（红线）作为分辨率的函数(cpd) (n=10)。每个刺激在分辨率任务中的平均布拉德利-特里分数（蓝线）作为分辨率的函数(n=10)。绿色虚线代表机会水平的选择率（两个供选择的选项，有50%的选择率）。垂直误差线，1 标准误。高分辨率在所有的数据组中都显著促进被试的深度知觉（举个例子，最高分辨率(210.0 cpd)的平均布拉德利-特里分数显著高于最低分辨率(15.0 cpd)，在1, n=10, p.001下t检验和Bonferroni校正）。分辨率任务（蓝线）和机会水平选择率（绿色虚线）的执行结果之间没有显著差异，就是说，他们无法察觉到分辨率的不同。3.2深度任务与亮度任务的功能核磁共振成像实验我们获得了与以前没有功能核磁共振成像扫描仪时进行的深度任务相同的行为结果模式。同时，我们在深度任务与亮度任务环境时都在视觉区域发现了功能核磁共振成像活动。另一方面，功能核磁共振成像结果显示在深度任务环境下人类颞中回的活动数量显著高于在亮度任务环境下（图4，n = 10, p ,.0001，未校正），虽然重要的活动在其他视觉皮层不同（例如V1或V2），且其他区域没有被发现。这个结果揭示了两个重要的事情：首先，功能核磁共振成像活动在深度任务环境不同于那些在亮度任务环境的，也就是说，那些被指定去做深度任务的被试实际上做的是亮度任务这种事是不可能的。第二，表明人类颞中回在参与深度任务时起到了重要的作用。图4|深度任务与亮度任务的功能核磁共振成像数据的结果。在相同的刺激组，被试被要求在单独的一段时间内做深度任务或亮度任务。平均功能核磁共振成像活动，根据把亮度任务环境（基线）从深度任务环境中剔除来计算（n = 10, p ,.0001，未校正，同见补充信息），对深度任务特殊的情况被展示了出来。人类颞中回在深度任务中活动的数量显著高于在亮度任务。4讨论目前研究的结果证明了两个重要的点。第一，高分辨率刺激促进深度知觉，甚至当刺激分辨率的差别是无法察觉的。这个发现可能与认为视觉投入在一个自下而上层叠的大脑皮层部位被连续不断地加工，对越来越多地复杂信息进行分析的传统观点相矛盾，因为被试在高分辨率刺激下感觉更深，且没有注意到分辨率的不同。更特别的是，被试没有意识到原始的视觉信息（即分辨率的不同），却识别出了更高的认知信息，如深度信息。最重要的是，这种现象表明了两种视觉信息类型在我们视觉系统中的存在，有意识可用的和不可用的信息。为了揭示视觉意识的方面，进一步的行为和生理的调查是必须的(Tong, 2003)。第二，功能核磁共振成像结果揭示了人类颞中回活动的重要作用。众所周知，人类颞中回跟猴子颞中回是专门用来处理运动的(Newsome, Pare, 1988; Movshon,Newsome,1996; Rees, Friston,Koch,2000)，并且他们被认为关键是通过参与双眼深度视差来提取信息的(DeAngelis,Cumming,Newsome,1998; Uka,DeAngelis,2004)。这里，我们发现人类颞中回活动在深度任务环境中的数量明显高于在亮度任务中。它可能预示了一个关于人类颞中回的作用的新观点，即人类颞中回不仅对双眼而且对单眼亮度对比不同/改变的深度知觉起重要作用。而且，人类颞中回活动被认为比知觉加工（perceptual process）更多的参与认知加工（cognitive process）(Uka,DeAngelis,2004;Uka,DeAngelis,2006; Treue,Maunsell,1996; Seidemann,Newsome,1999)，因为进行深度任务的被试做决定不仅仅根据知觉信息，比如分辨率或亮度对比不同，而且任务要求或/和先验知识（prior knowledge）在亮度对比不同/改变时经常作为一个深度线索(Gibson, J. J. 1979)。实际上，它根据的依然是我们发现的信息，在低水平视觉领域的深度和亮度任务环境，功能核磁共振成像活动没有显著差异，且加工更多知觉信息。然而，我们实验结果的一种其他的可能解释是人类颞中回活动在深度任务环境可能通过自上而下的调节方式来更有效地调节，如注意(Treue,Maunsell,1996; Seidemann,Newsome,1999; Eger, Henson, Driver, Dolan, 2007)。举个例子，在深度任务中如果被试比在亮度任务中更集中注意力于刺激或任务，人类颞中回活动的数量会更高（因为深度任务比亮度任务更需要认知加工）。因此，进一步的研究必需澄清这一假设的观点。我们的研究结果表明，高分辨率刺激有助于深度知觉。然而，我们实验中使用的刺激对研究刺激分辨率与深度知觉之间的关系来说是很简单的，在未来的研究中，如果可以应该在视觉信息加工特征（例如Gabor patch研究范式）的基础上用刺激进行测试。同时，进一步的生理研究应该对这一现象的机理进行更详细的了解，特别是对神经在人类颞中回和其他皮质（cortical）区域之间的相互作用，因为颞中回神经元（neuron）在功能和解剖结构上联接着其他区域(Uka,DeAngelis,2004; 2006; Parker, Newsome, 1998; Tsushima, Sasaki, Watanabe, 2006)。虽然在目前的研究中，重要的功能核磁共振成像活动在其他皮质区域还没有被找到，但依然有极大的可能高水平的皮质区域如眶额皮层参与了这种认知能力(Bar, 2003, 2006)。如果眶额皮层的活动与我们的深度任务显著相关，由显示分辨率控制的视觉刺激的空间频率将成为一个影响我们行为数据的重要因素(Bar, 2003, 2006)。最后，我们相信神经系统的研究伴随着新技术的发展，将会向我们展现我们神经系统未来的进步以及我们对神经机制的新见解。参考文献Masaoka, K., Nishida, Y., Sugawara, M., Nakasu, E. & Nojiri, Y. (2013).Sensation of Realness from High-Resolution Images of Real Objects. Trans.Broadcast., IEEE 59, 7283.Barlow,H. B., Blakemore, C.&Pettigrew, J.D. (1967).The neural mechanism of binocular depth discrimination. J. Physiol. (Lond.) 193, 327342.Pettigrew, J. D., Nikara, T. & Bishop, P. O. (1968).Binocular interaction on single units in cat striate cortex simultaneous stimulation by single moving slit with receptive fields in correspondence. Exp. Brain Res. 6, 391410.Ferris, S. H. (1972).Motion parallax and absolute distance. J. Exp. Psychol. 95, 25863.Reichardt, W. & Poggio, T. (1979).Figure-ground discrimination by relative movement in the visual system of the fly. Biol. Cybern. 35, 91100.Horn, B. K. P. (1975).Obtaining shape from shading information. In: The Psychology of Computer Vision, ed. P.H. Winston. McGraw-Hill.Ramachandran VS. (1988).Perception of shape from shading. Nature 331, 1636.Hou et al. (2006) .Neural correlates of shape-from-shading. Vision Res. 46, 108090.Sawada, T. & Kaneko H. (2007). Smooth-shape assumption for perceiving shapes from shading. Perception 36, 40315.Welchman, A. E. et al. (2005).3D shape perception from combined depth cues in human visual cortex. Nat. Neurosci. 8, 82027.OShea, R., Blackburn, S. & Ono, H. (1994) .Contrast as a depth cue. Vision Res. 34, 15951604.Bradley, R. A. (1984).Paired comparisons: Some basic procedures and examples, in Handbook of Statistics, vol. 4, Krishnaiah, P. R. & Sen, P. K. Eds. Amsterdam, The Netherlands: Elsevier, pp. 299326.Tong, F. (2003).Primary visual cortex and visual awareness, Nat. RevNeurosci. 4, 219229.Newsome, W. T. & Pare, E. B. (1988).A selective impairment of motion perception following lesions of the middle temporal visual area (MT). J. Neurosci. 8, 2201211.Movshon, J. A. & Newsome, W. T. (1996).Visual response properties of striate cortical neurons projecting to area MT in macaque monkeys. J. Neurosci. 16, 773341.Rees, G., Friston, K. & Koch, C. (2000).A direct quantitative relationship between the functional properties of human and macaque V5. Nat. Neurosci. 3, 71623.DeAngelis, G. C., Cumming, B. G. & Newsome, W. T. (1998).Cortical area MT and the perception of stereoscopic depth. Science 394, 67780.Uka, T. & DeAngelis, G. C. (2004).Contribution of Area MT to stereoscopic Depth Perception: Choice-Related Response Modulation Reflect Task Strategy. Neuron Vol.42, 297310.Uka, T. & DeAngelis, G. C. (2006).Linking neural representation to function in stereoscopic depth perception: roles of the middle temporal area in coarse versus fine disparity discrimination. J. Neurosci. 26, 6791802.Treue, S. &Maunsell, J. H. R. (1996).Attentional modulation of visual motion processing in cortical area MT and MST. Nature 382, 53954.Seidemann, E.&Newsome, W. T. (1999).Effect of spatial attention on the responses of are MT neurons. J. Neurophysiol. 81, 17831794.Gibson, J. J. (1979).The Ecological Approach to Visual Perception. Boston: Houghton Mifflin.Eger, E., Henson, R. N., Driver, J. & Dolan, R. J. (2007).Mechanisms of top-down facilitation in perception of visual objects studied by fMRI. Cereb Cortex. 17, 2123133.Parker, A. J. & Newsome, W. T. (1998).Sense and the single neuron: probing the physiology of perception. Annu. Rev. Neurosci. 21, 22777.Tsushima, Y., Sasaki, Y. & Watanabe, T. (2006).Greater disruption due to failure of inhibitory control on an ambiguous distractor. Science 314, 1786788.Bar, M. (2003).A cortical Mechanism for Triggering Top-Down facilitation in Visual Object Recognition. J. Cognit. Neurosci. 15, 600609.Bar et al. (2006).Top-down facilitation of visual recognition. Proc. Natl Acad. Sci. U.S.A. 103, 44954.Thurstone, L. L. (1927).Psychophysical analysis. American J.Psychol. 38, 36889.Thurstone, L. L. (1994).Alow of comparative judgment. Psychol. Rev. 101, 26670.Handley, J. C. (2001).Comparative analysis of Bradley-Terry and Thurstone-Mosteller paired comparison models for image quality assessment. Proc. IS&T PICS Conf. Montereal, Qubec, Canada, 10812.Morovic, J. (2008). Psychovisual methods Color Gamut Mapping 5870 (Chichester, England: Wiley).Malkovic, A. et al. (2007).Cytoarchitechtonic analysis of the human extrastriate cortex in the region of V5/MT1: a probabilistic, stereotaxic map of area hOc5. Cereb Cortex 17, 56274.方法Psychophysical experiments of Depth task and Resolution Task. Participants.20 participants, aged from 20 to 39 (13 females and 7 males), with normal or corrected vision, participated in a series of experiments. All participants gave written informed consent and the study was approved by the Ethics Committee of the NHK Science and Technology Research Laboratory, and in compliance with Declaration of Helsinki. 10 participants were assigned for Depth Task (6 females and 4 males), the other ten participants for Resolution Task (7 females and 3 males). Apparatus. A 27 IPS-TFT color LCD Monitor (ColorEdge CG275W,EIZO Nanao Corp.) was used to present stimuli. The display had an area of 256031440 pixels with the pixel size of 0.2331 0.2331 mm and the contrast ratio of 8501. Color calibration was performed before experiments to correct color balance and display gamma. We used 256 gray levels (8-bit color depth) to present the stimuli. Visual stimuli were presented by using Psychophysics Toolbox 3 (Psychophysics Toolbox, RRID:rid_000041) on Windows 7.Stimuli and procedure Visual Acuity Test. To confirm that participants have normal vision, we examined each participants visual acuity with using Snellen eye chart. We used only E shape presented on the same setting (equipments and environment) as the main experiment.Depth Task. In order to make the same images shown in different resolutions (Figure 1), we presented two same-size but different-resolution visual stimuli on the computer display (Figure 2a), which had the gradual luminance-contrast change from one side to the other for containing depth information (from right to left, or left to right)7,11. Although Masaoka et al. suggest less effects of realness with more than 60.0 cycle/degree (cpd) resolution stimulus1, according to the result of our preliminary experiment, we presented five kinds of resolution stimuli, 15.0, 30.0, 52.5, 105.0, and 210.0 cpd, and three different contrast-step of stimuli sets, 1, 2, and3(Figure 2b). We set the highest resolution stimulus as the base stimulus (the 210.0 cpd stimulus consisting of 28 sub-bars, each sub-bar was the same in size; right stimuli in each stimulus set at Figure 2b), and then it was downconverted using linear interpolation of the contrast, with fixing the highest luminance-contrast (left edge of each stimulus in Figure 2b). Here, we denoted the number of sub-bar by n, the contrast-step by a (from 1 to 3, 1 to 3 respectively), and the contrast value of i-th sub-bar by Ci=256-(i-1)(28a/n). For example, the highest resolution stimulus in 1was (right in D1): the contrast change was from 256 to 229 gray levels (from 66.3 to 52.0 cd/m2). The lowest resolution stimulus in 1 was (left in 1): the contrast change was from 256 to 242 gray levels (from 66.3 to 58.6 cd/m2) (Figure 2b). In each trial, two kinds of resolution stimuli were randomly chosen within each stimuli set, 1, 2, or 3. Before starting the experiment, the experimenter told the participants the idea of shading as a depth cue, and we confirmed that they understood it. The participants were instructed to fixate the center and report which stimulus they perceived more depth with monocular viewing (Figure 2a). In a complete experiment, each stimulus set was repeated 32 times, so that a total experiment consisted of 5C2 three stimulus set (1, 2, and 3) 32 repetitions = 960 trials. The order of presentation of these conditions was randomly determined for each participant. In order to avoid that participants noticed the resolution difference between two stimuli, we did not tell them that there was resolution difference. No feedback was given to the participants.Resolution Task. It was identical to Depth Task except that the participants were asked to report which stimulus had higher resolution. Before starting the experiment,the experimenter told what resolution is, and we confirmed that they understood it.Measurement of the actual luminance. We used a photometer, Luminance Colorimeter (BM-7, Topcon Technohouse, Tokyo Japan) and measured the luminance of each component of the stimulus used in the experiment for five times, then calculated the mean value for each point of observation. Those values are shown in Figure 2b.Psychophysical data analysis. To examine the relationship between depth perception and stimulus resolution, we used the method of paired comparison in which each stimulus is matched one-on-one with each of the other stimulus in our experiment. Thurstone-Mosteller (TM) model28,29 (Case V) and Bradley-Terry (BT) model29 are well-known paired comparison model that can convert the paired comparison data to psychophysical scale rating. Since BT model is more mathematically developed30 and produces more robust estimates of confidence intervals than Thustones Case V model31, we used BT model to analyze the psychophysical data in this study.MRI experiments of Depth task and Luminance task. Participants. 10 participants, aged from 20 to 39 (6 females and 4 males), with normal or corrected vision, participated in a series of experiments. All participants gave written informed consent and the study was approved by the Ethics Committee of the NHK Science and Technology Research Laboratory, and in compliance with Declaration of Helsinki. All participants did both depth and luminance tasks.Apparatus. The experimental equipments were the same as the previous psychophysical experiments. However, the monitor was placed outside of the fMRI scanner, and the subjects viewed the stimuli by the mirror.Stimuli and procedureDepth Task. It was identical to the Depth Task used out of fMRI scanner, except that only 15.0, 30.0, 210.0 cpd in 1 and 3 were used. In a complete experiment, each stimulus set was repeated 60 times, so that a total experiment consisted of 3C2 two stimulus set (1 and 3) 60 repetitions = 360 trials.Luminance Task. It was identical to the Depth Task in fMRI scanner except that the participants were asked to report which stimulus was darker.fMRI Data acquisition. MRI data were obtained using a 3T MRI scanner (MAGNETOM Trio A Tim; Siemens, Erlangen, Germany) using a 12 channels head coil at the ATR Brain Activity Imaging Center (Kyoto, Japan). An interleaved T2*-weighted gradient-echo planar imaging (EPI) scan was performed to acquire functional images to cover the entire brain (TR, 2000 ms; TE, 30 ms; flip angle, 80; FOV, 192 192 mm; voxel size, 3.5 3.5 4; slice gap, 1 mm; number of slices, 30). T2-weighted turbo spin echo images were scanned to acquire high-resolution anatomical images of the same slices used for the EPI (TR, 6000 ms; TE, 57 ms; flip angle, 90;FOV, 256 256 mm; voxel size, 0.88 0.88 4.0 mm). T1-weighted magnetization- prepared rapid-acquisition gradient echo (MP-RAGE) fine-structural images of the whole head were also acquired (TR, 2250 ms; TE, 3.06 ms; TI, 900 ms; flip angle, 9; FOV, 256 256 mm; voxel size, 1.0 1.0 1.0 mm).fMRI Data Analyses. Image preprocessing and statistical analyses were run by SPM8 (The Wellcome Trust Centre for Neuroimaging, UCL, SPM, RRID:nif-0000-00