关 键 词：

基于 PCNN 图像 分割 算法 研究

资源描述：

基于PCNN图像分割的算法研究,基于,PCNN,图像,分割,算法,研究

内容简介：

480IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 10, NO. 3, MAY 1999PCNN Models and ApplicationsJohn L. Johnson and Mary Lou Padgett,Member, IEEEAbstractThe pulse coupled neural network (PCNN) modelsare described. The linking field modulation term is shown to bea universal feature of any biologically grounded dendritic model.Applications and implementations of PCNNs are reviewed. Ap-plication based variations and simplifications are summarized.The PCNN image decomposition (factoring) model is describedin new detail.Index TermsDendritic model, PCNN factoring, pulse coupledneural networks.I. INTRODUCTIONTHE STUDY of the electrochemical dynamics of neurons,from the seminal work of Hodgkin and Huxley 1 tothe most recent work on internal dendritic pulse generation2, has led to increasingly accurate and more detailed models3. The transcription of the biological models to algorithmicmodels has led to an extensive literature of data processingsystems, most of which have been primarily concerned withutilizing adaptive algorithms for data classifiers. The studyof systems of pulsed neurons as dynamical networks, withand without adaptation, is more recent. One of the earlypapers 4 described a dynamic linking architecture based onan excitatory-inhibitory pair of coupled oscillators. Researchinto more biologically grounded pulsed network dynamics wasspurred by the experimental observations of synchronous pulsebursts in the cat visual cortex 5, 6.The 1990 Eckhorn linking field network 7 was intro-duced as a phenomenological model of a system exhibitingsynchronous pulse bursts. It used a pulse generator calleda neuromime 8, a modulatory coupling term, and synapticconnections modeled as leaky capacitors. Its central newconcept was the introduction of a secondary receptive field,the linking field, whose integrated input modulated the primaryfeeding receptive field input by means of an internal cellularcircuit. It provided a simple, effective simulation tool forstudying synchronous pulse dynamics in networks, and wassoon recognized as having significant applications in imageprocessing 911. A number of modifications and variationswere introduced to the linking field model in order to tailorits performance as image processing algorithms, and theseManuscript received November 8, 1997; revised December 27, 1998. Thiswork was supported in part by AF under Contract 8530-94-1-0002, AFOSRunder Contract SREP F9620-C-0063, and NCCOSC under Contract N66001-97-C-8612.J. L. Johnson is with the U.S. Army, MICOM, Photonics and OpticalScience, Redstone Arsenal, AL 35898-5248 USA.M. L. Padgett can be contacted at Publisher Item Identifier S 1045-9227(99)03191-4.became known collectively as pulse coupled neural networks,or PCNNs 12. The linking modulation was shown to enablehigher order networks 13 and a new form of image fusion(Broussard and Rodgers, this issue), and further to allow theconstruction of arbitrarily complex fuzzy logical rule systemson a single neuron 14.The two fundamental properties of the PCNN are theuse of pulses and pulse products. The latter property comesdirectly from the original linking field model as the basiccoupling mechanism. It is an asymmetric modulation of oneneuronal input by another. The choice of a modulatory cou-pling rather than the more-common additive coupling has theadvantage that a neuron with no primary input can not beactivated by the coupling input, a feature that is importantin image processing. While additive coupling is a primarybiologically grounded mechanism in that synaptic currentsare in parallel and thus additive, there is some experimentalevidence that pulse products and temporal encoding of spatialinformation can be of equal importance 15. A candidatemechanism yielding multiplicative coupling is derived belowfrom the dendritic computational dynamics of a compartmentalmodel.Section II begins with a detailed inspection of a compart-mental model having two inputs, the minimum number fora PCNN. It shows the modulatory coupling of the inputs,and how even very simple model cells can provide an elegantmeans of uniquely encoding information as time signals. Thepulse generator mechanism is then discussed, and it is shownthat there is a significant difference between the neuromimeand the integrate-and-fire (I&F) models in the generationof pulse bursts. Section III reviews the basic linking fieldmodel on which most PCNNs are based. Multiple pulseand single pulse regimes are described, and a number ofuseful aspects of PCNNs given. These include logical rules,image fusion, scale definition by linking strength, iconic timesignals, PCNN histograms, and chaotic structure. Section IVis a short review that describes PCNN techniques and givesdefinitions of different variations, and Section V is devotedto PCNN applications and implementations. Because PCNNsare generally used as nonadaptive processors, the connectivityrequirements are low, and it is practical to build them as high-speed electronic chips. Section VI, an algorithmic descriptionof PCNNs, is included for those interested in developing sim-ple software code versions. It shows the major simplificationsand shortcuts used in many image processing applications. Thepseudocode presented includes new details for the factoringmodel. Section VII gives examples of the application of thesevariations, and features PCNN factoring.10459227/99$10.00 1999 IEEEJOHNSON AND PADGETT: PCNN MODELS AND APPLICATIONS481(a)(b)Fig. 1.The two-compartment model neuron: (a) compartmental model and (b) equivalent circuit.II. THEPARTS OF THEBASICMODEL, ITSORIGIN,ANDRELATION TOBIOLOGICALMODELSA. Pulse Products in a Dendritic ModelThe purpose of this analysis is to show that the shunting-induced modulatory coupling is a generic and pervasive bi-ological pulse-coupling mechanism. It is not the exclusivemechanism for pulse products. A nonlinear enhancement effectamong closely spaced synaptic inputs along a dendrite hasrecently been experimentally confirmed. It is one of severalactive-channel interactions in which the conductance has avoltage dependence. It provides a multiplicative effect amonginputs to complex cells, and is related to the sodium spikesin active dendrites 16. Like the shunting effect, it allowsfor additive as well as multiplicative coupling. Rather thanchoosing either additive coupling or multiplicative coupling,nature has chosen both.A major distinction between PCNNs and the standardHodgkinHuxley 1 neuronal model is in the choice of thepulse generator. The neuromime used in the original Eckhornmodel is very similar in overall performance to the biologicalintegrate-and-fire pulse generator, but it has some fundamentaldifferences. These differences are discussed later in moredetail.The PCNN is based on having at least two distinct in-puts. A compartmental model cell 17 consisting of a singledendrite compartment and a second compartment containingthe integrate-and-fire pulse generator, with a single synapticinput to each compartment, provides the right combination ofmathematical transparency and adequacy. This model containsnumerous simplifications and approximations of the morebiologically accurate models, and they in turn are incompletemodels of real neurons 3. Some of the factors not consideredhere are multiple ionic synaptic channels, active channels,cellular aging, long and short term adaptation, continuousdendritic geometries and branches, refractory period (exceptas an add-on term), and temperature effects.Each compartment is a leaky capacitor. The leakage hasthree major parts: intrinsic, synaptic, and active. Only the firsttwo are used here. A synapse can receive either a pulse inputor a steady-state input. The somatic compartment generatesthe output pulse. The compartments are connected by a seriesresistance. There is no axon compartment. Fig. 1(a) shows thecell body and the dendrite compartments, and their synapticinput conductances labeledand, respectively. Fig. 1(b)shows the equivalent circuit. The box labeledrepresentsthe pulse generator.is the resting potential inside the cell,nominally about70 mV,is the synaptic back poten-tial, typically20 mV, andare the compartmentalcapacitances, on the order of nanoFarads, while,andare the membrane intrinsic leakage conductances andthe longitudinal conductance, respectively. The synaptic andintrinsic conductances are of the order of megaMhos, whilethe longitudinal conductance can range from ten to a thousandtimes larger 18.The compartmental voltages are described by the relation(1)whereandSynaptic conductance waveforms are often modeled as thealpha function 19, where. This isrepresentative of an input pulse. The conductance can haveother forms as well. It can be a constant, a square pulse, ora delta function. Likewise, the output pulse functioncanhave several forms. The step functionStepis useful for constant-step discrete-time simulations, formingthe pulse in two time steps.is the threshold for pulse482IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 10, NO. 3, MAY 1999initiation. A variation is to use the step function as a gate fora pulse train which is set at the maximum allowed pulse rate,as suggested by Eckhorn 7. A third representation is to usea sum of delta functions taken over the firing times, whichare defined as the times whenexceeds(2)The modulation terms due to the synaptic shunting are mosteasily seen in the steady-state case where the internal activityhas reached an equilibrium level below the threshold for firing.For constant inputsand, and no pulses,the modulation products are immediately obtained from thenullcline () of (1) which is of the formwhereThis is a saturable activity function. It has an intrinsic leakageterm, linear terms, and a product term. Becauseis negativeandis positive, the leakage term is negative and sets athreshold for firing. Since, the coefficient of thelinear linking term in the numerator is positive, as are theremaining terms. In size, theconductance dominates theintrinsic and input conductances. This makes the constant andlinear terms of the numerator and the denominator all of thesame order of magnitude, while the product coefficients arein the ratio of the leakage conductance to the longitudinalconductance. These correspond to the linking strength factor, and are of the order of 0.1 to 0.001 or less. It is seen that thesynaptic shunting terms lead to both linear and multiplicativeterms for the internal activity, and that the latter can be ofa significant strength. Linking strength factors of 0.1 give asubstantial modification to a pulse activity pattern 20. Themodulation is symmetric in the inputs (unlike that of theoriginal linking field model), and the activity will saturate forlarge inputs.The nullcline of (1) for a general dendritic system withnumerous inputs, many branches, channels, and trees willlikewise give a combination of linear and multiplicative termsin a saturable ratio. However, it will be much more complex.It will contain all possible combinations of products and sums,and in fact will comprise a general higher order model neuronwhose order equals the total number of inputs to the entireneuron, which can be several thousand inputs. The highest-level products will be attenuated due to the separation of themost distant synapses on the dendritic tree, with modulationsamong local groups being more pronounced.Now consider the case for which the cell generates pulses, still with constant inputsand. The solutionis easily obtained by diagonalization of the 22 matrix.The diagonalizing transform matrixfor the general 22matrixiswhereNote that, so.The solution of (1) is then(3)where the time dependence ofis in the pulse functionof (2).andare the eigenvalues of the diagonalizedmatrix.Despite the simplicity of the two-compartment, two-inputsystem, it is not usually solved analytically. Bressloff and Tay-lor 21 review the general problem, including shunting effects,and an iterative approach involving time-ordered operators isgiven. In the example section, however, only a one-input, two-compartment nonshunting solution is presented. Accordingly,the existence of the modulation products is not discussed there,nor by Bressloff 22, an extension of the latter reference21. Rall 17 and others 16, 18 place the emphasis onnumerical methods. For this case in which modulation productsare obtained, the analytical solution is again ignored. This isdespite the experimental evidence 15 for products of pulsetrain time signals.These modulation products provide the critical function ofthe logical AND conjunction, thus allowing the constructionof large, complex logical rule systems on a single neuron andthe biological implementation of sigma-pi nets, higher ordernets, and the PCNN nets.The internal activity due to the Eckhorn linking fieldsmodulatory coupling is seen to be an asymmetric version ofthe steady-state internal activity of the standard compartmentalmodel, thus providing a basic biological grounding of thelinking field and all the PCNNs adapted from it. It furthersupports the experimental findings of McClurkin et al. 15,by showing that there is an explicit generic mechanism forforming products of input time signals in the standard cellmodel.Even the restricted case of constant inputs shows someinteresting dynamical properties. This case would correspondto the retinal ganglion cells that receive nonpulsed inputs fromthe earlier retinal layers, the inputs being roughly constantduring periods of visual fixation. Only the outline of thealgebraic solution is given, as the details, while straightforward, are tedious and lengthy.Integrate (3) from just after theth pulse to just aftertheth pulse. Apply the condition that the voltage inJOHNSON AND PADGETT: PCNN MODELS AND APPLICATIONS483compartment 1 just after reset is at its minimum.This gives two distinct equations for the voltagein thedendrite compartment, each a function of the inputsand,and of theth interspike time interval.One equation is at time; the other is at timeThese can be combined to give each interspike time intervalin terms of the inputs, cellular geometry and electrochemicalstructure, and the previous interspike time interval(4)For the conductance and voltage value regimes given earlier,these functions are approximatelyandwhere the-functions are terms involving the inputs and thecompartmental geometry. Fig. 2 illustrates the possible cases.For case (a), if, the curves do notintersect and there is only one pulse. For cases (b) and (c),ifthe curves intersect. In that case, iffurther (b), the pulse sequence can reach apoint attractor, a limit cycle, or can terminate after a finitenumber of pulses, while if (c), the sequencewill be either a point attractor or a limit cycle.Consideration of this recurrence relation shows that evenfor constant inputs the cell will generate a complex outputspike train pattern with information encoded in the interspikestructure. This agrees with the findings of McClurkin etal. 15that each color hue has its own pulse time signal structure,an experimental result that is difficult to explain otherwise.Further, the time signal also contains information about thestructure of the cell from which it originated, information thatcan be viewed as an identifying code for that cell or cell type.Finally, since as noted by Bressloff 22 the cellular responseis the temporal convolution of the input time signal and thecells response function it is an inescapable conclusion thatreal neurons have the capability to generate, transmit, receive,and respond to information-bearing time signals.B. Pulse Generators: The Neuromime VersusIntegrate and Fire (I&F)1) Coupled Oscillators: These are more general and com-plex oscillatory signal generators that contain the other twoas special cases. The output signals can be nearly pure har-monic functions, skewed periodic functions that are boundary-matched exponential decays/excitation, or sharp, spike-likefunctions, all depending on the choice of the model parametersand response functions. They consist of two “cells,” eachhaving its own response function. The first cell receives thesystem input, then transmits its output to the second cell whichin turn sends a signal back to the first cell. Generally, thesecond cell inhibits the first cell, and has a slower, longer-lasting effect on it than the first cell had on it.(a)(b)(c)Fig. 2.Recursive relationship of successive pulse periods for a single neuronwith constant inputs. (a) Null case. (b) and (c) The pulse periods can approacha point attractor, a limit cycle, or terminate after a finite number of pulsesdepending on the values of the cell structural parameters.If the response of the first cell is assumed to be instantaneous(a step function) and the response of the second cell is madeto be that of a leaky capacitor, the system is a neuromime.Similarly, a coupled oscillator with a linear excitatory rateto a leaky capacitor for the first cell and a fast step-likeinhibitory output from the second cell is an integrate-and-fire-pulse generator.A coupled oscillator signal generator, installed inside amodel neuron, can exhibit coupled synchronous activity. Itis more complex and general than the synchronizations ofthe other pulse generators. For example, it can be perturbedto oscillate about its synchronous lock-in phase point with asimilar coupled-oscillator signal, while a neuromime cannot(once two neuromimes with the same frequency lock together,they cannot be perturbed by low-level fluctuations).2) The Neuromime: The neuromime is described by theinteraction of an inhibitory thresholdand an excitatory stepfunctionstep484IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 10, NO. 3, MAY 1999For an input, the neuromime produces pulses at a rate(5)Strictly speaking, it is a second-order system with an implicitlyunderstood causal action between the threshold, the stepfunction, and then back to the threshold. More detailed modelsare discussed in Eckhorn et al. 7 and Dicke 23.3) The Integrate-and-Fire Generator: This is a two-partsystem that accumulates power from the strength of the inputto the first part, and a second part that abruptly dischargesthe first one after it reaches a preset constant threshold. Itis an excitatory-inhibitory pair. For completeness, the pulserate expression is shown below. It is obtained from (3) bysettingand usingas the constant input to the firstcompartment, then integrating over a single pulse period toobtain(6)and the pulse rate is found from. The inputmustbe aboveto causea pulse, as would be expected for a leaky capacitor. Note thatthe key parameters are all ratios: ratios of various potentialdifferences, and the ratio of the synaptic and membraneconductances.4) Comparison of the Neuromime and the I&F: Boththeneuromime and the integrate-and-fire generators are simplerthan the coupled oscillator. Likewise, they are less versatile.The neuromime has two adjustable parameters; the integrate-and-fire has four. An integrate and fire generator has a built-inbias level that must be overcome before it begins to fire, whilethe neuromime does not. A bias (and in general any inhibitorysignal) can be inserted as a subtractive constantin theneuromime. Neither model has a saturation limit. Biologicalpulse generators saturate when the interpulse time intervalapproaches the refractory periodof the cell. This can beapproximated in both generators by adding a small refractorytime period to the analytically derived pulse period prior toinverting it to obtain the firing frequency 23, or by using agated pulse train 7Step(neuromime)(7)Step(I&F)(8)The feature for which the neuromime and the integrate-and-fire pulse generators have the most significant difference iswith regard to the generation of bursts of pulses. See Figs. 3and 4. When the neuromimes input receives a sufficientlyhigh input it enters the multiple pulse regime 11. This isshown in Fig. 5. When a group of closely connected cells fire,Fig. 3.The neuromime as an excitatory-inhibitory pair.Fig. 4.The basic model.Fig. 5.Multiple pulses: (a) test figs., (b) time signals, and (c) scaled, rotated,and distorted versions.their combined linking pulse can greatly augment their usuallocal feeding input level. The amount of reset from a singlepulse is then inadequate to reset each cells threshold abovethe total input, so it continues to pulse until the thresholdfinally resets above the input. Due to the postulated differencein their relaxation times, the linking input saturates before thethreshold saturates, ensuring that reset will eventually occur.This happens for all the cells in the group, so the mutuallinking input disappears upon reset, leaving the threshold farabove the much smaller feeding input level. The cells willremain quiet until the thresholds can decay back below thefeeding. Once that happens, a new round of large linking pulsesis generated, causing another synchronous burst.Under the same circumstances, however, the integrate-and-fire pulse generator immediately saturates. Once the combinedlocal linking pulse can raise the internal activity above thefiring threshold within the refractory period time, then it willdo so again the next time, all else remaining the same. Oncestarted, it never stops. The model neuron continues firingforever at the maximum possible firing frequency, with noquiet periods to delineate bursts. There are no bursts, onlyconstant streams of pulses at the maximum firing rate. Sincebiological neurons do not use a neuromime pulse generator butrather use the integrate-and-fire model, these processes requireserious attention. Biological pulse bursts cannot be due to alocal synergistic effect, but must rather be explained by a moreJOHNSON AND PADGETT: PCNN MODELS AND APPLICATIONS485complex dynamical network process and/or a more realisticpulse generator model.A straightforward mechanism is to add a reversible sat-uration effect to the model neuron, essentially the same asbuilding a coupled oscillator whose excitatory part is theoriginal pulse generator. This gives a fixed burst size andrepetition rate if the burst spindle frequency is fixed at themaximum rate.Another way to obtain pulse bursts with the integrate-and-fire generator is to use the global inhibitory feedback of agroup of several spatially disjoint patches of activity, where allthe patches have the same feeding input 24. In this situation,the patches will automatically phase-shift their pulse times soas to fire in rapid sequence. If left to continue, the patches willeventually set themselves apart with equal phase separation.This is also discussed in Eckhorns article in this issue.If, however, the network is not allowed to establish phaseequilibrium, for example by using local saturation, the globalinhibitory feedback arrangement will produce a burst of pulses.The information carried by the bursts in this case is a measureof the degree of fragmentation of the feeding pattern. Suchinformation could arise, for example, in the cortical areain regions of binocular disparity. The pulse bursts could thenbecome a depth or range feature to be processed in the higherlevels. In general, the pulse bursts could indicate a temporarystate of disorganization and fragmentation, something thatmight be expected to occur between fixations in the visualsystem or during periods of transition of attention from onemultidimensional signal to another. The pulse bursts in thiscase would be a network effect rather than being due to achange in the cellular model itself.Finally, although the basic integrate-and-fire pulse generatordiscussed above is not capable of generating a burst, biologicalneurons are known to have the ability to produce bursts ofseveral pulses with a spacing of a few milliseconds (complexpulse) as well as generating single pulses. Further, pulsesare known to be generated in active dendritic structures aswell as near the axon, something not considered in the abovemodel. Any pulse bursts obtained in this way would be froma single cell and thus would not have a high degree of globalinformation. On the other hand, the purpose of a synchronouspulse burst may be for network activation or control, as ameans of intensity-to-pulse phase conversion, as suggested byHopfield 25; or, the burst may be a global cortical readoutsignal as proposed by Koch 26 and later by Taylor 27.If so, then the synchronous signal itself need not necessarilycontain information, and the bursts could well be generatedby single cells.III. A REVIEW OF THEBASICMODEL ANDITSPROPERTIESA. Basic ModelThe basic models main parts are the receptive fields,the modulation product, and the pulse generator 7. Whilemathematical descriptions are in several of the papers in thisissue, none of them are identical, and so a set of describingequations are listed below for clarity of discussion here.The neuron receives input signals from other neurons andfrom external sources through the receptive fields. The inputsignals are pulses, analog time-varying signals, constants, orany combination. While PCNNs are fully compatible withand complementary to adaptation processes, the main focusof interest is in the networks nonadaptive spatiotemporaldynamics. The model is shown in Fig. 4.After the inputs have been collected by the receptive fields,they are divided into two or more internal channels. Onechannel is the feeding input and the others are the linkinginputs. The distinction between the feeding and the linkingis that the feeding connections are required to have a slowercharacteristic response time constant than those of the linkinginputs. The linking inputs are biased and then multipliedtogether, and further multiplied with the feeding input to formthe total internal activity(9a)(9b)(9c)Step(9d)(9e)whereconvolution, andis thetemporal response kernel of theth synapse.andare the synaptic weight strengths defining the receptive fields,andare constant inputs,is the feeding input,is thelinking input,is the total internal activity,is the outputpulse, andis the neuromime threshold. The subscripts , andapportion the terms to their respective synapses andcells, thes ands characterize the synapses,andsetthe operation of the neuromime, andis the linking strength.For a discrete-time model, (9a)(9e) are computed in the listedorder, and, anduse their values as evaluated at theprevious time step.B. Pulse RegimesFor low feeding levels and limited linking inputs (low,small receptive fields) a single pulse will be adequate to resetthe threshold. This is the single pulse regime. Most imageprocessing applications operate in the single pulse regime.When the internal activity becomes very large, the amountof reset per pulse to the PCNN threshold is inadequate toraise it above the internal activity. The system will generateadditional pulses until the threshold is finally driven above theinternal activity. The difference in the neurons behavior in thetwo regimes is extreme. If the firing frequency is plotted as afunction of the internal activity, it increases until the multiplepulse regime is reached. A rule of thumb is that when thesingle-pulse period, which is a function of the feeding input,is less than twice the capture zone period 14, which doesnot depend directly on the feeding, the neuron will begin togenerate multiple pulses. The bursts of pulses then occur at afrequency that decreases with increasing internal activity, andthe system soon reaches a point where it is constantly pulsingwith no separation of the pulses into bursts.486IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 10, NO. 3, MAY 1999When groups of interacting neurons are considered, thesituation becomes even more complex. If a single isolatedneuron is receiving an aggregate linking pulse from a group, itwill repeatedly pulse until it drives its threshold over the largelinking input. The height of its threshold is then determined bythe size of the linking input, and since the size of the linkinginput is proportional to the size of the group, the interburstperiod of the isolated driven neuron is a function of the areaof the transmitting group as well as still being a function ofits own feeding input.For a neuron within a group of interacting neurons theamount of linking input is further increased by the number ofpulses generated by the group itself. Thus the harder a neuronworks to overcome the linking input by generating additionalpulses, the greater becomes the linking input itself, since allthe neurons in its group are doing the same thing. The neuronsin such a group can eventually “catch up with themselves” be-cause of the postulated difference in the response times of thelinking receptive fields and that of the feeding receptive field.The linking input saturates first, allowing the repeatedly pulsedthreshold to finally increase above it as indicated in Fig. 5.Each neuron in a connected group receives a large compositelinking input from the others, causing it to repeatedly pulse.This in turn increases the linking input further, causing morepulses. The linking pulse saturates, allowing the neuronsthresholds to finally increase above the linking pulse andterminate the burst of pulses.The number of pulses in a burst depends on the productof the area of the bursting group and its feeding intensity,that is, the total incident power. The time period betweenbursts is approximately proportional to a logarithmic functionof the area of the group. When two such groups interact,the capture zone time period for each group is a functionof the others area as attenuated by the separation distancebetween the groups. The ratio of the group capture time to theburst repetition period can be small, giving an effective weaklinking between groups even though the groups themselvesare well within the strong linking multiple pulse regime. Thesystem parameters must be chosen such that: 1) the thresholdreset gain is small compared to the linking pulse amplitude(for multiple pulses); 2) that there is a relatively long decaytime between bursts; and 3) that the intergroup linking pulseamplitude is adequately attenuated while still maintaining goodintragroup linking strength. For the latter, areceptive fieldfalloff is a good compromise.In this situation, internally synchronized groups interact to-gether as though they were single giant neurons. The synchro-nization is accordance to their areas rather than the incidentintensities, providing information (on a larger time scale) abouta completely different feature (area) than is obtained on thescale of individual neural linking (intensity). Group linkingin the multiple pulse regime will perform segmentation andsmoothing based on area rather than intensity.C. Logical Rules and Image FusionThe modulation products of the linking fields provide ameans of implementing logical rules with PCNNs. Informally,an algebraic product corresponds to a logical “AND.” Since thereceptive fields are weighted sums, and a sum corresponds toa logical “OR,” PCNNs can implement logical statements. Asingle PCNN neuron can support an arbitrarily complex fuzzyor crisp logical rule set. One such application is image anddata fusion. Several receptive fields sample the input imageand respond in proportion to the presence of the particularimage structure or feature to which each receptive field ismost sensitive. Some examples of geometrical receptive fieldkernels are those sensitive to edges, vertices, orientations,and parity. Other features include color, disparity, and opticalflow. Equally, the inputs could be from different sensorssuch as visible, infrared, SAR and millimeter wave imagers,from medical diagnostics such as X-rays, MRI, and CATscans (note that PCNNs can equally well be applied tosegmentation problems in three dimensions as well as two),or from computer-derived map patterns such as topologicalfeatures, terrain type indicators, population density, or weatherinformation.D. Scale and Linking StrengthThe level of the linking strength controls the amount ofmodulation of the feeding input. Accordingly, it controls theratio of intensities that can be linked. Ordinarily this wouldbe interpreted as the amount of grayscale quantization inthe segmented output. However, since images of interest tohumans generally are of a continuous nature, this also controlsthe size of the grouped regions. A large value of the linkingstrength can cause the entire image to be segmented as asingle group, whereas a somewhat smaller value can causeit to be grouped into two or three regions. A very smallvalue will result in over-fragmentation. In this view, then, thelinking strength controls the scale on which the input image isspatially resolved. A large value produces a coarse versionof an image while a small value produces a version withfine geometrical detail. This is a new way to approach theproblem of multiple scales in imagery, as it does not rely ona spatial scale parameter such as is done in wavelet theoryand in pyramidal image decomposition techniques. Instead ituses an intensity scale parameter (the linking strength), and theamount of image detail is dependent entirely on the nature ofthe image rather than a prechosen spatial scale. A hierarchicalimage factoring system 28 based on this property of PCNNsis described in which the image is decomposed into an orderedset of image product factors in sequence from coarse to finedetail.E. Time SignalsThe time signal from a PCNN is the pulse train as a functionof time. The time signal of a specified object in the feedinginput image is the pulse train emanating from all the neuronsbeing directly driven by that object. The time signals have thedesirable property of being object-specific, yet insensitive tonot only all known geometrical distortions due to object poseand an imaging system but also insensitive to overall sceneillumination and object articulation 14. A single referencetime signal generates a basin of attraction for its correspondingJOHNSON AND PADGETT: PCNN MODELS AND APPLICATIONS487(a)(b)(c)Fig. 6.Time signal invariance. The time signals of a Tee, Cross, and an Elare insensitive to translation, rotation, scale, distortion, and deformation.object, and serves as an identification icon for its object 29.This is analogous to the human experience where a singleexample of an object allows an individual to recognize allthe objects in that class. For example, a person can easilydistinguish cars versus trucks after having seen only oneexample (grotesque and pathological exceptions such as sportsutility vehicles excluded). A person does not need to undergoextensive training on large numbers of images of many cars;one is enough to determine whether an object is a car. Anexample of the use of time signals is shown in Fig. 6. Anobject consisting of a bar and a post generates a time signal foreach position of the bar on the post. The time signals of threereference samples, a Tee, a Cross, and an El, are compared tothe time signals of the many variations. The images are furthersubjected to changes of scale, rotation, and perspective.A time signal is generated for each location of the bar onthe post, and also for scaled, rotated, and distorted versions.Comparison with the time signal of the first, third, and lastfigures of the first row (Tee, Cross, and El) shows basins ofattraction where the differences from the references are zero.The time signal comparison functionshown in Fig. 6is defined so that it is zero when the integral of the absolutevalue of the difference of objects time signaland anyof the three reference objectsis zeroHere,indicates the location of the bar on the post as it movesfirst down and then to the right, forming in turn at Tee (),a Cross (), and inverted Tee (), and finally an Elat. A basic single-pass PCNN with an edge-enhancedfeeding image was used.Invariance against overall scene illumination is easily shownby noting that a global rescaling of the scene intensity doesnot affect the ratio of successively quantized PCNN grayscalelevels(10)whereis theth feeding level to trigger a pulse andis the next lower level.Other aspects of PCNN time signals include: 1) compositetime signals; 2) PCNN histograms; and 3) chaotic structure.1) Composite Time Signals: The individual time signalsdue to separate objects can be combined via the linkingmodulations into composite time signals. It is elementary toshow that the composite time signal could include syntacticalinformation as to the relative geometrical locations of theindividual objects, due to the relative times of arrival and/oroverall amplitude attenuations of the individual signals atthe point of their combination. This leads to an overall timesignal representation of a full scene that conceptually “turnsa painting into a symphony.” The analogy is underscoredby noting that any color hue generates its own time signalas discussed earlier and further that synchronicity is morelikely to occur for time signals containing harmonic andquasiharmonic ratios.2) PCNN Histograms: The intensity histogram for thetime-averaged pulse image is identical to the frequencyspectrum of the time signal. The intensity of the time-averagedpulse image is linearly proportional to the average number ofpulses from each point in the image, which in turn dependson the average pulse frequency. The number of cells withthat frequency is the spectrum amplitude, and is also the areacovered by the intensity generating that pulse frequency. Theintensity maps into pulse frequency, and the area maps intospectrum amplitude. The histogram is the number of cells ata given intensity, and the spectrum is the number of cells atthe corresponding pulse frequency.3) Chaotic Structure: Farhat 30 has shown that a spikingneuronal model (the bifurcating neuron) has a rich chaoticstructure. The spiking neuron model has a basic pulse fre-quency plus provision for an additional input. When thesecondary input is a harmonic function, the model exhibitsregions of chaotic structure interspersed with regions of limitcycle activity. Comparison of the models phase recurrencerelations with those of the PCNN, below, shows clearly thatthe model can be interpreted as an Eckhorn model with aconstant feeding input ofand a linking input of. In view of the natural tendency of the PCNN toprefer quasiharmonic ratios, the appearance of the limit cyclesin Farhats phase diagram is not surprising. Farhats work isa powerful argument supporting the generality and utility ofPCNNs.The bifurcating neuron:Eq. (1) 30Eq. (2) 30The PCNN:from which488IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 10, NO. 3, MAY 1999allows the correspondence:IV. APPLICATION-BASEDVARIATIONS ANDSIMPLIFICATIONSThere are several variations and modifications often usedin PCNN applications. The following descriptions show howthey operate and why they are useful.A. Single PassOne aspect of the PCNNs is that the segmentation, smooth-ing, and grouping action occurs in the first pass through theimage. A continuously running PCNN repeats this processingmany times. Processing time is reduced by restricting thealgorithm to a single pass through the image, either bycounting the number of pixels that have fired and stoppingwhen all of them have fired, or by limiting the total number oftime steps so that the pixels do not fire a second time. In orderto compute the equivalent output of the long time average overmany pulse firings, the timeat which a pixel or pixel groupfires is noted and used to compute the output intensity valueas(a measure of the average frequency).B. Fast LinkingA linking wave propagates through a region much fasterthan the neurons in that region can produce additional pulses(multiple pulse regime excluded; its time scales have beendiscussed earlier). The fast linking approximation assumes thatthe linking waves fully propagate to their limits with no changein the value of the time-decaying threshold. Its benefit is thatit gives a smoother regional segmentation result in the timeaveraged pulse image. Contrary to its name, the fast linkingapproximation does not make the software code run faster. Itruns significantly slower because the code must go throughadditional internal loops and “while” steps as it accumulatesall the pixels in a region that can be induced to fire by theaction of the linking wave sweeping through the region.C. Sigmoidal LinkingAnother simplification is in the linking. There are twochanges. One is that the leaky capacitor connection is notused. The total linking input is then the instantaneous weightedsum of the pulses in the linking receptive field. (The primarybenefit of having a decay tail on the linking input is that itwill eventually drive equal-frequency linked neurons into zero-phase synchronization even when they are initially outside ofeach others capture time zone. This takes a long time, andis unimportant for single-pass versions.) The second changecomes from observing that a significantly more uniform seg-mentation is obtained when the linking modulation is moreuniform in its value. In the basic model, the weighted sumof the local pulse activity strongly depends on the number ofneighboring neurons were active. By putting the weighted sumthrough a squashing function, the modulation product is mademore uniform, as desired. This is a heuristic variation. Usuallya step function is used so that the linking modulation is eitheron or off, and in practice it is found that a step function offsetof zero produces very good results.D. Linear DecayThis is often used in the single-pass versions. The ba-sic model uses an exponentially decaying variable thresholdbecause leaky capacitor mechanisms are common in the bio-logical neurons. However, the two essential features of thePCNN are pulses and products, and the exact means bywhich the threshold decays is not as important. Ranganath31 pointed out in 1993 that a simple linear decay thresholdcould be used. It is particularly appropriate for a single passPCNN, and in practice, the results are of the same processingquality as for an exponential threshold decay. It requires lesscomputational time and is an attractive alternate mechanismfor hardware implementations where the threshold decay canthen be implemented by a simple clock decrement circuit. Thiseliminates the large amount of chip area that would otherwisebe required for a leaky capacitor circuit.E. Edge-Inhibited LinkingControl over the linking waves can improve the sharpnessof the segmentation. If the linking strength is a function of thecharacteristics of the feeding image input, namely by requiringthe linking strength parameter to decrease at edges, then thelinking wave will be unable to flow across boundaries it wouldordinarily cross. This is edge inhibition. Note that in this casethe inhibition enters within the modulation product rather thanbeing subtracted from the total internal activity.F. Edge-Enhanced FeedingThe analysis of the compartmental model neuron showsthat the internal activity not only has the Eckhorn linkingmodulation products (although in an completely symmetricfashion in feeding and linking rather than as an asymmetricmodulation product) but also is normalized by the averagetotal input over the neurons receptive field. This is dueto the shunting terms in the biological model, and can beimplemented in the PCNN by using an edge-enhanced versionof the input image. Normalization is useful in that it optimizesthe effective dynamic range of the algorithm.G. Threshold ResetWhen using the PCNN for image segmentation and smooth-ing it is desirable to eliminate as many sources of complexityas possible. One source is in the threshold, where it is resetby a constant increment by the outgoing pulse. The incrementis added to the feeding level, and as a result, each neuronsthreshold begins its downward decay from a level generallydifferent than any other neuron. The simplification here is toreset all the thresholds to the same high level. This meansuse an absolute starting height rather than computing eachthreshold as an increment of the neurons particular feedinglevel. Mathematically, this corresponds to the basic model inthe limit for very small feeding inputs.JOHNSON AND PADGETT: PCNN MODELS AND APPLICATIONS489H. Low Feeding InputsOne problem the PCNN has is that the effective degree oflinking decreases at very low levels of the feeding input. Onesolution is to make two passes (Tarr and Samuelides 32,33). The first pass is done the usual way. For the secondpass the input image is contrast-reversed, and then the resultsof the two passes are combined either by product fusion or bytaking the best overall segmentation from both.Another solution 34 is to modify the input image bydividing it into a low-pass and a high-pass filtered version,then recombining them as absolute values. This causes thedark spots in the original image to be reversed into brightspots, enhancing their linking ability.I. ImplementationsA third method uses the threshold bias term. When it isnonzero it causes the capture zone period to increase forsmall feeding inputs, improving the effectiveness of the linkingaction at low feeding levels. A negative value has a greatereffect than a positive value, but also cuts off the smallestfeeding values.J. Group SeparationA problem often encountered in PCNN image segmentationis that several regions will have the same output intensityand pulse frequency yet will be spatially disjoint. Eckhornssolution uses global inhibitory feedback, as discussed earlier.Ranganath 35 solves this by taking the PCNN output for agiven intensity level, arbitrarily selecting a pixel and adding asmall increment to it, then rerunning that output. The disjointsegment with the incremented pixel will now pulse before theothers and can be distinguished from the others. The processis repeated until all the segments are isolated.V. APPLICATIONS ANDIMPLEMENTATIONSThe primary image processing applications of PCNNs todate are smoothing, feature binding, edge and peak cur-vature extraction, image fusion and image decomposition.Other applications are path optimization, invariant featuregeneration, and impulse movement detection. The latter takeadvantage of the spatiotemporal dynamics of PCNNs inmore direct way than the former. Future applications includeimplementation of logical rule sets, encoding and recallingtime sequences of patterns. Many applications are nonadaptiveand can be implemented in hybrid electro-optic arrangements,in electronic hardware as high-speed arrays of smart pixels, orby special-purpose accelerator boards and programmable chipssuch as the CNAPS board, the ZISC chip, and PGAs.A. SmoothingThe degree of smoothing by the basic PCNN is controlledby the linking strength parameter. It sets the size of thetop-down grayscale quantization as the local ratio of stepintensities equal to the linking modulation factor see (10).Noise reduction can be further enhanced by inhibiting thetotal internal activitywith a low-pass filtered sum of thetotal PCNN activity signal, as shown by Eckhorn et al.24 for single and multiple layersThis gives a dramatic decrease in both the spatial and temporalnoise when it is turned on during the steady-state operation ofthe PCNN by allowingto change from zero to one.Another smoothing method 35 is an iterative approachmost effectively implemented in software. The input imageis first run through a single-pass PCNN with zero linking. Ateach pixel, the firing sequence in its 33 nearest neighbors isrecorded. A pixel that fires early in the sequence accordinglyhas a locally high intensity, while one that fires late has arelatively low intensity. The original input image is modifiedaccordingly. The locally high and low pixels are adjusted bya small decrement or increment, respectively. It is found thatthis method preserves edges and converges after only a fewiterations, at which time the PCNN with nonzero linking isthen applied to the smoothed feeding input.B. BindingThe elementary segmentation, or binding, by similarity oflocal feature strength is the essence of the PCNN, and it isdone by conversion of feature strength to pulse frequency, andthen pulse synchronization via coupling terms. It is a generaleffect found in many models. The coupling can be additive ormultiplicative, linear or nonlinear, with or without temporalextension. The choice of a pulse generator mechanism is notcritical. The general coupled oscillator model of Wang 36,the bifurcating neuron 30, the standard integrate-and-firemodel and its many variation, and even the variable pulsewidths of the pulse-coded models all have been shown tohave synchronous capability. What is generally true is thatfeature binding by pulse coupling produces results that arebetter and are obtained more quickly than most, if not all, otherimage processing algorithms 35. Often the difference to theunaided eye is minor, but the improvement of the performanceof further processing algorithms is substantial.C. Edge and Peak Curvature ExtractionEdge extraction with a PCNN can be done two ways. Oneis to take each segmented region and apply a binary edgeoperator such as a Sobel kernel. The other is to wait until asegmented region has pulsed. Immediately after the pulse alinking wave is launched from the edge of the region, andis often an excellent approximation to the actual edge. Thisis particularly true for noncomputer generated imagery. Thepeak curvature points along the boundary of a region can befound by taking the binary template of the segmented areaand passing it through a high pass spatial filter. Regions ofmaximum curvature will be enhanced. The result is then sentthrough another PCNN. There, the enhanced regions will tendto fire first, signaling the location of the positions of peakcurvature in the original image.490IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 10, NO. 3, MAY 1999D. Image Fusion and Image DecompositionThe image fusion and image decomposition applicationsboth take advantage of the multiplicative coupling of the Eck-horn modulatory linking field. For image fusion, more than onemodulatory term is used. The additional factors are generatedby additional PCNNs, each processing one of the severalimages to be fused. PCNN fusion has been implementedusing the three video color channels of a digital color image,and by using as inputs different feature maps derived fromthe same grayscale image 37. The decomposition algorithm(Johnson and Padgett) is unique in that it factors an imageinto an ordered set of images as according to the level ofcontrast. The original image is recovered by multiplying allthe factor images together. It is the only known multiplicativedecomposition algorithm for images 28, and has been used toextract low-level contrast details from noisy images 3840.E. Path OptimizationPath optimization uses the linking waves. They flow fromhigh to low intensity, following the gradients of the image inmuch the same way as a stream seeks the path of greatest slope.Caulfield and Kinser (this issue) developed a maze-solvingalgorithm in which the linking waves explore all possible pathsin parallel, solving the maze in the time required to traverse theoptimum shortest path. Their method takes advantage of thecharacteristic of autowaves that two such waves will annihilateeach other, thus automatically preventing multiple traversalsof any part of the maze. The technique has some parallelswith quantum computing in that all possible states (candidatepaths) are active, but only the correct state (shortest path)survives when the measurement (the linking wave reachingthe goal point) is made. This has potential application toproblems in terrain navigation, routing network optimization,cryptography, and watershed erosion analysis.F. Invariant Feature GenerationThe generation of object-specific time signals that are in-sensitive to all known image deformations due to object poseand camera viewing geometry is a unique aspect of PCNNs.The time signal constitutes a natural, system-generated featurethat in many ways is an ideal platonic icon for the object. Bynesting several such icons in the mathematical format of thelinking modulation, it is possible to generate a simple proto-language in which the iconic time signals are the words andthe modulatory format provides the syntactical structure. Forexample, ifandare the iconic time signals for an objectand a color hue, respectively, then the two nonequivalentstatementsdenote a typeobject having the particular colorfor thestatement, and the colorincluding objects of the typefor the statement. Each statement is itself yet anothericonic time signal, and can be further used as another word inmore complex statements.G. Impulse Movement DetectionOne of the fundamental results of Eckhorns work wasthe finding that there are two basic types of synchronization,forced and induced. Forced synchronizations occur when anobject in the input feeding image changes in time, either bydirect movement or by some intensity time dependence. Theactual degree of change can be small and still produce alarge single-pulse forced synchronization signal because of thecommon linking action that quickly reinforces the individualpulses. Eckhorn 7 has shown that forced synchronizations inthe linking field model exhibit dynamical characteristics alsofound in biological reset events.H. Logical Rule SetsOther applications of PCNNs deserving of further studyinclude two of particular interest. One is the implementationof large logical rule systems using the biological shuntingmechanisms inherent in the cellular structure as opposed tothe familiar electronic gate techniques. It is more reasonable,if one is to build an artificial neural network, to use themethods and architectures found in nature rather that to attemptto adapt a digital computer for the job. By constructingartificial polarized membranes and incorporating a diode-like synaptic structure at the junctions of membranes, theshunting effect would automatically generate the necessarymultiplicative terms for the construction of logical rules.Another area is that of adaptive PCNNs. As noted earlier14, the use of any of the several widely accepted learninglaws with a PCNN will cause it to both learn and recall notonly the patterns of linking waves but also their movement.This allows the system to encode and recall the images thatgenerated the linking waves, and by the same mechanismwill link time sequences of images. The modification requiredfor this is to include a distribution of PCNN threshold timeconstants rather than using the same time constant for ev-ery neuron. Adaptive PCNNs are fully equivalent to somespatiotemporal memory models previously demonstrated 41.I.ImplementationsPCNNs have been implemented as electronic arrays, ashybrid electro-optical systems, and with the aid of specialaccelerator boards and programmable cards. The first imple-mentation of a PCNN was a hybrid electro-optical system 12.The input scene was imaged through a plane containing aspatial light modulator (SLM) so that the SLM was slightlyout of focus, then reimaged onto a video camera focal plane.The binary pulse output image on the SLM multiplied theinput image so that each pule pixel modulated a small regionof the input scene. This arrangement implemented the linkingreceptive field and the linking modulation product. The videocamera image, representing the total internal activity, wasthresholded in a computer and used to determine which pixelswould fire. The decaying threshold was computed in thecomputer and updated on each cycle. The new pulse imagewas then presented to the SLM for the next cycle.A small proof-of-principle PCNN chip was built as a 18-neuron array 42. It used an integrate-and-fire pulse generatorJOHNSON AND PADGETT: PCNN MODELS AND APPLICATIONS491and had nearest-neighbor linking. The chip showed linkingactivity, and ran at a maximum pulse rate of 1 MHz. A 2 MHzPCNN currently under evaluation by the U.S. Army has a 3232 square aspect focal plane array, and was designed andbuilt under contract by Johns Hopkins University 43. OtherPCNN array architectures have been reported 44, 45.The potential of PCNN chips in current technology is rep-resented by the goal of edge-bonded 0.2- m technology tiledarrays with optical inputoutput and a 100 MHz maximumpulse rate. This gives a system objective of 40004000 6-in-diameter FPAs for segmentation, smoothing, compression,fusion and the other nonadaptive PCNN functions at speedsin excess of 100000 equivalent frames per second. Thisassumes that 90% of the processing time is due to non-PCNNprocessing.VI. PCNN SOFTWARECODEThe following is a basic PCNN model and some variations.One popular version replaces the exponential threshold decaywith a linear decay, and is used as a single-pass model.Another variation deals with the linking input,. It variesaccording to the number of other pixels that fired in the localneighborhood. In order to makemore uniform, and thusmake the results more spatially uniform, the linking input isitself thresholded before being used in the internal activity.This is the sigmoidal linking variation discussed earlier. Thebasic model and some options are described below.A. Basic PCNN ModelThis version of the Basic PCNN model is expressed indiscrete time, and is intended for implementation on digitalcomputers.Given ainput image, say, initialize PCNN processingas follows.1) Normalizeto lie within 0, 1.2) Select a number of PCNN iterations, say.3) Initialize parameters and matrices as indicated below.Forto, perform the operations of Fig. 7, savinginteresting values at each time step, whereconvtwo-dimensional convolution;.*array multiplication;a step function givingwhere, andelsewhere.Sample parameter values are:tau LI.0decay term for linking;tau t5.0decay term for threshold (usually five to ten);beta0.2linking strength (0.01 weak and 1.0 strong);vL0.2magnitude scaling term for linking;vT20.0magnitude scaling term for threshold;rad3.0radius of linking field, K is 2*rad1;and initial values for matrices are:same dimensions as, ().same dimensions as, ().square matrix, dimension 2*rad1, center value is one; other pixelvalues arewhereis the dis-tance from the center pixel.Fig. 7.Basic model.Values of interest when accumulated at each time step mayinclude the pulse trainand the stabilizedimages.For example, for each time step, recordas the sumof pixels that fired at time.is thus the sum of all pixelvalues offor time. Plottinggives a time sequencecharacteristic of the objects in the image.The stabilizedimages may be accumulated in. Afterinitial variations inhave subsided, at time, the pulsesform a repeating pattern,time steps long. Letequal thesum offorto. The normalizedmay thenbe compared to the original input image.There are numerous ways to vary this basic model. Theseinclude: 1) selecting the multiple-pulse regime; 2) implement-ing adaptation; 3) using feeding as well as linking input; and4) making the output a nonbinary function of various inputs.To work in the multiple-pulse regime, setequal to thesmaller value of 1.0 or even 0.1. Alternatively, or in addition,setto a huge number, like 50. This magnifies the linkinginput so that more than a single reset is required for thethreshold to overcome it.To include adaptation, select a learning law. Hebbian,Hebbian with decay, competitive, saturable 43, and rate-covariance models are some examples. Alter the feedingand/or the linkingso that they each have an extra term thatis the adaptive matrix-vector input found in standard neuralnetworks.In Eckhorns general model, pulse inputs are included inthe feeding input as well as in the linking channel, and boththe feeding and linking channels are filtered with a first-orderrelaxation:Therelaxationtimesusuallyfollowtheinequalities. This is not a fundamentalrestriction, as the analysis given earlier shows that biologicallythere is no difference between the linking and feeding inputs.492IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 10, NO. 3, MAY 1999Fig. 8.Single-pass example.Fig. 9.Sigmoidal-linking example.The pulse output does not necessarily need to be binary. Itcould be some operator-chosen function of various inputs. Inthe PCNN maze-solver, whereisthe time when the cell pulse was made, andis theth timestep after it fired. Thus, if a string of cells fired, it would leavea comet-like decay tail.Other key PCNN variations include the single-pass model,the sigmoidal-linking model, the linear-decay model, and thefast-linking model. These can be combined to form the PCNNfactoring model. The following sections describe examples ofthese models.B. Single-Pass Example, Where Each Pixel Fires OnceTo distinguish the single pass model from the basic model,the pseudocode example of Fig. 8 is provided.In addition to the equations given in the basic model tableabove, define a firing time matrix, with the same dimensionsas the input image,. Initializeto zero. At each time step, update. For eachsuch thatis one andis zero, reset. Whenhas no more zeroelements, the single pass is complete, the initial firing time foreach pixel is preserved, andgives the output intensity.C. Sigmoidal-Linking ExampleThe sigmoidal linking model replaces the equation forgiven in the basic model. Define a matrix, with the samedimensions as the input image,. Initializeto zero. At eachlinking input update, setto one everywhere work is greaterthan(See Fig. 9).D. Linear Decay ExampleReplace the exponential decay of the threshold with a lineardecay ofat each timestep (See Fig. 10).E. Image Factoring Model: Fast-Linking, Single-Pass,Linear Decay PCNN with Sigmoidal LinkingA code outline for a fast-linking, single-pass, linear decayPCNN with sigmoidal linking is shown in Figs. 11 and 12. TheFig. 10.Linear decay example.Fig. 11.PCNN factoring model input.former give sinput definitions and suggested values, while thelatter shows the model logic.These variations have been implemented in Matlab 5.2 44.The outlines provided here are intended to provide pseudocodesuggestions for user-written applications.VII. PCNN APPLICATIONEXAMPLESTo demonstrate the power of these PCNN variations, andin particular the PCNN Factoring model, the following illus-trations are provided.A. Mammogram Example Comparing and Combiningthe Basic Model PCNN and the Factoring ModelUse of PCNN factoring as a preprocessor for the basicPCNN is shown in Fig. 13. In this medical application, amammogram X-ray is processed to provide both an outline ofthe area of interest, and the internal structure of a calcification.JOHNSON AND PADGETT: PCNN MODELS AND APPLICATIONS493Fig. 12.PCNN factoring model. Note that?indicates array multiplication,while?and?indicate element by element multiplication and division.B. Skyline Example of a One-Dimensional Image withShadow Removal by the Factoring ModelTo further clarify the action of the factoring option, thefollowing tables of images are provided. The first, a one-dimensional Skyline example in Fig. 14, illustrates the capa-bility of the factoring method for removing a shadow froman image. The left column shows processing of the originalSkyline image, and the right column shows processing ofthe Skyline image after a shadow is introduced. The shadowFig. 13.Mammogram: outline of region of interest (left) and skeletal struc-ture of the calcifications (right). Left column is PCNN on raw image (originalimage on top, output of a selected PCNN iteration at bottom). Right columnis PCNN on factored image (Ct1 factor on top, output of selected PCNNiteration at bottom) 48.is a 50% reduction in intensity over an interval. Examiningthe first five factors, Ct1Ct5 and Cts1Cts5, shows thatthe shadow impacts only Cts1. For the remaining n factors,there is virtually no difference between shadowed (Ctn) andunshadowed (Ctsn). Compare the product of factors twotenfor shadowed (Ctsnshad) and unshadowed (Ctnshad). The lastrow of the table shows that multiplication of all the factorsrestores the original image.For image processing applications, if the Ctsnshad product iscompared to the template image (Ctnshad), target recognitionis possible, even in the presence of shadows, such as that castby an overflying airplane wing.C. Retinal Images from a Diseased Eye Showinga Two-Dimensional Image with Feature Separationand Shadow Removal by the Factoring ModelFor a two-dimensional example of the shadow removalcapabilities of PCNN factoring, a retinal image from a diseasedeye was processed. The left column of Fig. 15 shows thefactoring of the original image, Y1bar. The right column showswhat happens when a shadow is added. The shadow is a50% reduction in intensity of a square near the center of theimage. The first factor shows the square (Cts1), and someringing around it. This is common when the shadow overlapsa sharp edge. Notice that the remaining factors (Cts2Cts5) arevirtually identical to the unshadowed image factors (Ct2Ct5).This is clearly illustrated in the comparison of Ctnshad andCtsnshad. Again, the recombination of all the factors (Ct andCts) shows duplication of the original and shadowed images.Several retinal images were obtained and examined insearch of automatable techniques for isolating arteries fromveins, and diagnostic spots from minute noise in images. Theleft column shows the potential of PCNN factoring for thisapplication. The image in question contains arteries, veins andnumerous spots resulting from disease processes. These canbe separated from the minute noise in the image by selectingappropriate factors. Ct1 and Ct2 show interesting views. TheCt1 factor suggests a separation of arteries from veins. Ct2contains outlines of all the vessels, and retains some diagnostic494IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 10, NO. 3, MAY 1999Fig. 14.PCNN factoring of a skyline image, with and without a shadow.spots. These large lumps should be noted and monitored overtime. For the purposes of printing these images, histogramequalization was performed. This processing highlights thecomponents of, in particular, Ct3, Ct4, and Ct5.D. Military Example of a Tank on Sand with Separation ofFeatures and Shadow Removal by the Factoring ModelThe next example of interest uses a tank running over asandy surface. The tank and its tread marks are targets, here.Fig. 16 shows, in the first column, (1.a) the original, unshad-owed template image, (1.b) the reconstructed image (productof the first ten factors), and (1.c) the residue. The residue canbe completely eliminated if necessary by separating out morefactors. The rest of column 1 shows the first five factors. Ct1shows the tank. Ct2 shows the tanks shadow, and some distantcurving treadmarks. Ct3 still hints at the tanks shadow, butCt4 and Ct5 seem to be noise.Fig. 15.Diseased retinal image with arteries, veins and diagnostic spots.Left column is unshadowed, right is shadowed.The right column of Fig. 16 shows the impact of a largeshadow added to the template image. Here, as in the previousexamples, most of the shadow falls into Ct1. The effect of theshadow decreases in Ct2. The second factor still retains somefeatures of interest, but nearly eliminates the shadow.E. Time Series Example of the Factoring Model FlaggingChanges of State in a System Being MonitoredAn important extension of the image factoring techniqueis the use of PCNN factoring on time series data. In thedevelopment of monitoring systems for flagging change ofstate in a system being observed, PCNN factoring can helpeliminate false alarms due to random noise, and, in general,improve the accuracy of the control system.In Fig. 17, simulated time signals from detector arrays arearranged in a matrix. The four corners are sources. The current25 sensor inputs from each source are arranged in 55JOHNSON AND PADGETT: PCNN MODELS AND APPLICATIONS495Fig. 16.Tank in sand example of the factor model shadow removal. Leftcolumn (1) shows a template image and its factors. (1.a) is the original imageof a tank. (1.b) The product of its first ten factors. (1.c) The residue imageafter the first ten factors. (1.d)(1.h) The first five factors. Right column (2)shows the same image with a shadow added (2.a)(2.h) 28.Fig. 17.Time signals from detector arrays 3840.matrices at the corners. The previous 25 measurements are du-plicated and moved to each side of the current measurements.Background signals remain in the center, and separate eachsource. Row 1 of Fig. 17 shows the original background signaland its factors. Row 2 shows the pattern resulting from a newodor appearing from all four sources. Row 3 then illustrates thepattern from a lasting odor. Comparing the top row of originalimages with the second row of Ct1 factors shows how muchclearer the separation of odor and background becomes. Notethat in the Ct2 and Ct3 factors, the lower right most corneris a bit blacker than other areas. Investigation showed thatthe random number generator used for the lower right odorsource noise varied from the ideal distribution. This discoveryillustrates the value of examining later factors to see if theyare indeed just noise as expected. Patterns appearing in theplace of noise may signal equipment failure, or the presenceof other interesting anomalies.Wavelet analysis of the images 49 is useful for auto-matically identifying the image factor of interest. In theseillustrations, the Ct1 factor has been full of detail, and mostsimilar to the original image. With variations in parameters,this may not hold true. However, the largest wavelet coeffi-cients of the original and the detailed Ct factor will be greatlysimilar. An example of this is found in 39.In an electronic nose application, sensor readings overtime are processed, and the output of a radial basis function496IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 10, NO. 3, MAY 1999Fig. 18.Olfactory neuron pathways.Fig. 19.Binocular structure for electronic nose.classifies the odor source. The radial basis function output is,however, seldom completely binary. Instead, the ten candidateodor nodes have values ranging from 0 to 1. Modifyingthe simulated data monitoring system above, an accuracyimproving process is obtained from PCNN factor analysis.See Figs. 1820.Fig. 20 uses a structure derived from analysis of theanatomy of neurons sectioned from the olfactory bulb of rats.The olfactory region structure has been shown, by Josephsonand Padgett 50, to channel input from neurons on the leftside back to neurons on the right side, and vice versa. Somestimuli from each side are channeled to the center back (seeFig. 18).These pathways are similar to the binocular structure at-tributed to the vision system. Therefore, this structure (Fig. 19)is adopted for the electronic nose application of Fig. 20.The 4040 odor-image matrix is partitioned as follows.Four geographically distributed sensor sources are assumed(UL, UR, BL, and BR). Each sensor source is comprised often rows of radial basis function output, each ten columnslong. This assumes ten candidate odors and 10 redundantmeasurements and outputs from each location. Each rowofthe 1020 submatrix UL and UR interweaves the elementsof the th row of UL with those of the th row of UR. Othermixed submatrices are similarly interwoven. If the sourcesensor outputs are similar, a large patch of similar output isproduced. Otherwise, spotty output is produced. The PCNNfactoring procedure is thus suitable for separating out the noiseand spotty results from the more consistent ones. Using the Ct1factor of the odor-image increases the accuracy of detection,as illustrated below.In Fig. 20, a normal background odor is assumed to bepresent before Odor2 drifts in from the top (originating fromUL and UR). The top image (20a) shows the transition fromNormal (bottom) to Odor2 (top).Notice the distinctive, more solid columns of Odor 2 in Ct1,as compared to the noisy Odor2 columns in Y1bar. Ct2 showsthe variation due to differences in sensors. It should be closeto random. Some sensor faults can be expected to produceFig. 20.Transition from Normal to Odor 2 39, 40.nonrandom patterns in Ct2. When detected, this can trigger arecalibration of sensors.Fig. 20(d)20(f) show the histograms derived from sum-ming each column of the odor image for various states.JOHNSON AND PADGETT: PCNN MODELS AND APPLICATIONS497Fig. 20(d), shows the transition state of Fig. 20(a)20(c). Next,the normal state Fig. 20(e) and the complete Odor2 stateFig. 20(f) are plotted. An odor is associated with eachcolumn of the histograms. For each column, circles on thehistogram indicate the probability of that odor being present,based on the original odor-image. The solid lines connect theprobabilities based on Ct1. The plusses at the top of the spikesindicate the ideal. Note that the solid lines from Ct1 outperformthe circles from the unfactored image.F. PCNN Parameter Selection and AutomationThe applications shown above were executed using thesame parameter values. This illustrates the robustness of thePCNN factoring approach. There are, of course, potentialadjustments for linking strength and number of threshold steps.The basic PCNN has even more possible parameter adap-tations. Approaches to automation of PCNN factoring havebeen suggested by Padgett et al. 28, 39, 40. Szekely andLindblad 51 have illustrated parameter tuning for a simplifiedPCNN. A wide range of applications using perturbations of thebasic PCNN model can be found in this issue.VIII. SUMMARYIn summary, the pulse coupled neural network and itsnumerous variations have been found to be useful in a widevariety of applications. Based on the study of biological neuralbehavior, these abstractions focus on practical utility andeventual hardware implementation of automated PCNNs.ACKNOWLEDGMENTOdor sensor simulations were based on information pro-vided by, AUs Institute for Biological Detection Systems(IBDS) courtesy of N. Cox and T. Roppel. The diseased eyeimage was provided by M. Wilcox.REFERENCES1 A. L. Hodgkin and A. F. Huxley, “A quantitative description ofmembrane current and its application to conduction and excitation innerve,” J. Physiol., vol. 117, pp. 500544, 1952.2 B. W. Mel, D. L. Ruderman, and K. A. Archie “Complex-cell responsesderived from center-surround inputs: The suprising power of intraden-dritic computation,” in Neural Information Processing Systems, vol. 9Cambridge, MA: MIT Press, 1997, pp. 8389.3 K. Koch and I. Segev, Eds., Methods in Neuronal Modeling: FromSynapses to Net