Behavior of a Model for Feedback-Controlled Reverberating Circuit and Immediate Memory Function

In this work we implement a mathematical model of synaptic transmission connecting neurons in a circuit of reverberating discharges in order to investigate its behavior in front of parametric variations. Using a program developed in C language, we verified if this model would behave as short-term memory circuit. In the simulation, we used neural parametric values from experimental measures in animal. Our model was able to reproduce polysynaptic activity of a neuronal group of rat brain (looping time of about 102 ms). On the other hand, we verified that the inhibitory feedback synapses in circuit with weight varying with presynaptic firing time and frequency are capable to change the circuit characteristic to reproduce the typical behavior of neural circuits. The results suggest that, differently from some recent considerations, the reverberating circuit model has great potential to reproduce the typical behavior of neural circuits and could be seen as a possible model for immediate memory.


INTRODUCTION
Mathematical and computer modeling has been widely employed in the biological sciences field, with important application in Neuroscience. In this way, experimental and theoretical hypotheses can be combined into one single model and tested [4,7,8,23,24,26].
Ramon y Cajal first introduced, in 1911, the concept of nerve cells as structural components of the brain, the germ of the morpho-functional unit theory, thus opening a horizon of correlations, not only in the biology context, but also in the physical and mathematical understanding of the nervous system, which provided the knowledge that led to the "silicon logic gates". These are non-reciprocal two-port devices, in the terminology of computational sciences, in which the propagation events occur on the nanoseconds time scale, while neurobiological events occur in the order of milliseconds. However, the difference in the processing rate in the brain is balanced by the existence of a large quantity of neurons (order of magnitude of (10 11 ) and their respective synapses (about 10 14 /neuron), constituting neural networks of efficient and outstanding complexity [15,25].
It is known that convergence (several presynaptic terminals synapsing with a single neuron) and divergence (one neuron synapsing with many other neurons) form the basis for data processing in the nervous system [21] and constitute the principle of functional organization that promotes the integrated functioning between the varied specialized structures in the central nervous system [14].
The existence of a local circuit pattern in several parts of the brain is already recognized [13], consisting of a population of excitatory and inhibitory neurons synapsing with each other. The action potential (AP) generated by the anterior neuron excites the following, which, in sequence, can reciprocally inhibit/excite the anterior, constituting a basic mechanism for the generation of the periodic brain activity. Such circuit is a neural oscillator [12]. A number of these circuits can be organized to function as repetitive discharge circuits, such as reverberating circuits. In these, recurrent discharges occur because of the presence of feedback loop, which can explain some of the most complex abilities of nervous function [3].
In reverberating circuit, signals can actively reverberate within the ensemble of neurons, in order to maintain its activity and coordinate the detection of additional coincidence of different sensory signals. Reverberation refers to the neural activity that circulates in a recurrent network to constantly feed some neural activity [27,28]. This type of circuit has already been pointed as a potential explanation for mechanisms of short-term memory and working or immediate memory, because it is able to restrain signals for some period of time [2,6,9,16,17,20,29].
The term "working memory" is generally used to refer to "short-term memory", but the two concepts are actually different. Short-term memory refers to the short-term storage of information, and does not entail the processing or organization of the memorized material. The immediate memory, or working memory, is a theoretical framework that refers to structures and processes used to temporarily store and manipulate information [6,17].
Reverberating circuits have also been proposed as a possible model to underlie the definitive memory formation in the central nervous system [13]. Thus, according to the Hebbian logic, memory formation would not involve only synaptic strengthening in feedforward circuits, but also signals reverberating in ensembles of neurons [20,30]. However, Niktarash [22] draws attention to the fact that, despite the maintenance of the activity for some time, neural reverberation can modify the information carried by a single neuron, since the signal frequency changes in each feedback cycle. Therefore, many structural and functional issues remain unexplained, and further studies are required to specify the degree to which the recurrent connectivity converges onto individual neurons, and how this reverberating activity interacts with the oscillatory behavior to facilitate inputs and retain the information for a given time period, preserving its meaning.
The objective of this work was to study the behavior of a mathematical and computer model for a reverberating neuronal circuit, verifying the neural spiking patterns produced at each node and at the circuit output, through simulations using a C++ language program. The adopted reverberating circuit is composed of three directly interconnected neurons forming recurrent loops. Based on the results, we discuss the capability of this type of circuit to reproduce the behavior of an immediate memory circuit.

Modeling the Synaptic Transmission
Considering that, in an excitatory synapse, the neurotransmitter-receptor interaction derived from a presynaptic action potential causes an excitatory or depolarizing postsynaptic effect (E) proportional to the postsynaptic membrane potential, one can write that dE/dt = −E/τ E , where τ E is the time constant. The solution of this equation for n action potentials arriving at the synapse j in the interval t i − t 0 is given by [4,8] where E 0 j is the characteristic value of the excitatory postsynaptic potential (EPSP) at synapse j. For a neuron with a set of k active synapses, considering w E j as the synaptic weight, the total postsynaptic effect will be Thus, this equation allows us to estimate the excitatory postsynaptic effect due to n action potentials arriving at k synapses of a neuron.
By analogy with the excitatory synapse, for the inhibitory synapse j, the inhibitory or hyperpolarizing postsynaptic effect will be given by and, for a neuron with q active inhibitory synapses, we have where τ I is the time constant of inhibitory synapses and m is the number of action potentials arriving at the inhibitory synapse j, whose synaptic weight is w I j .
The model considered the following boundary conditions: (1) the time of neurotransmitterreceptor interaction (time of receptor engagement-disengagement was limited and, once the receptor was free, it could be occupied again; (2) the postsynaptic membrane was highly permeable during 1.0 ms because of the neurotransmitter-receptor interaction; (3) the postsynaptic effect could persist for 8.0 ms, determining the existence of a refractory period.  Table 1). Figure 1 shows the temporal variations of the excitatory postsynaptic effect (Figure 1(a)), according to Eq. 2.1, and the variations of the inhibitory effect ( Figure 1(b)), according to Eq. 2.3, caused by a presynaptic discharge. These figures show that the postsynaptic effect on synapse j, either excitatory or inhibitory, caused by a presynaptic discharge (at t = 0) was maximum, and decreased with time. Such decrease was dependent on the time constant value. The values used in these figures are shown in Table 1. Thus, the synaptic dynamics was represented in our model, considering that each presynaptic discharge releases into the synaptic cleft a pool of neurotransmitter molecules that cross the cleft to access the postsynaptic receptor, and that the number of free receptors gradually decreases with their occupancy. Neurons were programmed so that, in a sequence of presynaptic discharges, each should produce a smaller effect than the previous one.
The program also simulated the disengagement of postsynaptic receptors, considering that they would be free after a few milliseconds after the presynaptic discharge, 7 ms in Figure 1(a), when the receptor can be occupied again. Note that in this figure, the synaptic effect returns to its maximum value with a new presynaptic discharge at t = 6 ms.
The dynamics of the inhibitory postsynaptic effect was similar to the excitatory one. However, because it involves hyperpolarization rather than excitation, the postsynaptic effect decreases in time is represented by a decrease in the negativity (return to the resting value). Figure 1(b) shows that, in the inhibitory synapse, the receptors became free after 11 ms. In this figure, the synaptic effect returns to its maximum value with a new presynaptic discharge at t = 10 ms.
As simultaneous inhibitory and excitatory postsynaptic potentials generated at widely distributed synapses can summate, the postsynaptic neuron in our model fired when a sufficiently large depolarizing effect was produced and the plasmatic membrane potential, V (t), became higher than or equal to the threshold potential, T (t), or V (t) > T (t). These potentials depended on the following intrinsic membrane characteristics: resting potential (P R ), reversal potential (P 0 ), threshold potential at rest (T R ) and excitation threshold after action potential (T P ). V (t) is the net variation in membrane conductance and involves both excitatory and inhibitory effects, in addition to the initial condition of the membrane potential and the characteristics P R and P 0 . The potential T (t) represents the natural resistance of the membrane potential to reach the threshold, and depends on T P and T R . The model considers that the action potentials before and after firing are different, since the action potential is followed by some milliseconds of refractory period. Thus, in our model, V (t) and T (t) were represented as where τ P is the time constant of the period after the hyperpolarization potential, τ H is the decay time constant for the relative refractory period and τ P is the absolute refractory period.
In Eq. 2.5, the second term refers to the variation in potential during the hyperpolarization period. This residual potential varies exponentially with time. In Eq. 2.6, the second term describes the evolution of the membrane potential after the firing of the action potential, until the membrane potential reaches the resting potential. Figure 2 illustrates that, after the firing of the postsynaptic neuron, V (t) (Eq. 2.5) gradually increased to less negative values, while T (t) (Eq. 2.6) decreased with time to more negative values, and the membrane changed from post-discharge hyperpolarized state to a resting state. In this way, both potentials reached their respective resting values, P R and T R , within 10-15 ms, and the neuron was prepared to fire again.  Figure 3 shows the schematic representation of the model adopted for the reverberating circuit. It was constituted by three neurons connected forming a complex feedback circuit. This circuit was able to execute a certain number of predetermined interactions and, in each of these, the neuron could stimulate its neighboring neurons.

Modeling the Reverberating Circuit
The propagation time (∆t) of the action potential from one neuron to other was given by where d is the neuron axon length, v is the propagation velocity along it, and t 0 is the synaptic delay (t 0 = 6 ms).
In feedback loop synapses (Figure 3), two different situations were considered: (1) fixed synaptic weight, and (2) synaptic weight varying with time. In situation 2, when the postsynaptic effect Figure 3: Model for a reverberating circuit. The signal arrives from E and propagates by axon terminal that synapses to neuron N1. Then this synapses to neuron N2, whose axon opens rams and makes a feedback synapse with neuron N1. S is the circuit output.
(Eq. 2.2 and 2.4) reached values E 0 j /2 or I 0 j /2, synaptic weights began to vary according to the following equations: where τ w is a decay constant of synaptic weight. Thus, the synaptic weight decreased inversely with both time and frequency of recurrent presynaptic discharges, adjusting the postsynaptic neuron firing.

RESULTS
Applying the numerical values given by Table 1 in the model (Figure 3), we could test it and verify its characteristics. (Figure 4(a)), B (Figure 4(b)) and C (Figure 4(c)) during 1.0 s, considering all synapses excitatory, with equal and time-independent synaptic weights. In this figure we can see that the three neurons exhibit a similar spiking pattern. For a single input, they fired repetitively and the input activity was changed with each reverberation cycle, becoming very high, until a maximal value, which persisted for a long time.  Figure 5(a), we can observe that neuron A fired several times for a single input, and the firing persisted for only 196 ms. A slightly different spiking pattern can be observed in Figure 5(b), neuron B began to fire  Table 1. only 7 ms after a single input, and stopped after 203 ms. Figure 5(c) shows a spiking pattern for neuron C similar to that of neuron A, stopping after 196 ms.  Figure 5, except for the feedback loop C − B. This was inhibitory and its synaptic weight (w I j ) varied as seen in Eq. 2.9. In Figure 6(a), we can observe that, for a single input, the neuron A displayed only two spikes in response to the feedback activity of neuron B on it. The second action potential occurred 14 ms after the first one was displayed. Neuron B (Figure 6(b)) generated a single action potential in result of the excitatory effect of neuron A on it. Figure 6(c) shows that neuron C was able to produce only one action potential at 14 ms.

DISCUSSION
Tests showed that the adopted synaptic transmission model was able to represent the functioning of a classical synapses, the computational model being prepared to simulate the engagementdisengagement dynamics of binding sites in postsynaptic receptors and the post-action potential refractory period.  Table 1.
As shown in Figure 2, after the neuronal firing, the membrane potential V (t) (Eq. 2.5) and the threshold potential T (t) (Eq. 2.6) gradually returned to their respective resting values, reproducing the refractory period normally found in membranes of neurons. After 10-15 ms, both the resting and threshold potentials have been restored.
Analyzing the spiking pattern of spectra shown in Figure 4, we can see that, if all synapses in the circuit were excitatory and their synaptic weights were constant over time, a single supraliminal input from E ( Figure 3) was able to initiate a reverberatory process, whose output frequency increased with each cycle. In that condition, one action potential generated in neuron A propagated to neuron B, which could in turn send a signal to neuron C and a feedback to neuron A. This one then generated secondary, tertiary signals, and so on, forming recurrent cycles. The same occurred with neuron B, because of its feedback connection arriving from neuron C. It is important to remember that the output frequency in each neuron of the circuit depended on the values of parameters involved in synaptic equations (Eq. 2.1-2.6) and on the boundary conditions (refractory period etc.) included in the program.  Table 1.
The theory of reverberating circuit considers that, when a high-frequency stimulus is applied directly on the surface of the cerebral cortex, the excited area continues sending rhythmic signals for a short period of time, even after this stimulus has been suspended. In a first analysis, this theory may be associated to the concept of immediate memory, since the fatigue effect in reverberating circuits could explain the weakening of memory and the temporal data replacement with the entry of new signals into the circuit [16,20,30].
The Hebb's postulate regarding reverberant networks has been subject of discussion in the Neurobiology of Learning and Memory [15,16]. The idea that reverberation circuits facilitate the association of similar data of sensory information has been admitted to explain the immediate memory dynamics [16]. However, for Niktarash [22], the circuit input activity can be completely changed with each reverberation cycle, and reverberatory circuits could not be thought to underlie the short-term memory. Thus, a circuit generating a firing pattern as shown in Figures 4 cannot be considered as a basis for an immediate or working memory. This type of memory involves transitory neural changes to keep the information for a period of time, and includes subsystems that temporarily store and manipulate information, as well as a central executive that coordinates the subsystems.
These observations lead us to the following question, among others: Is there in the biological synapse any way to control the increase in the postsynaptic effect in a feedback loop and set firing frequency of the circuit at a given value, which would persist for a certain period of time? If so, we might consider the reverberating circuit as the basis for a process of immediate memory.
In Figure 5 we can see that, when all synapses were excitatory and the synaptic weights (w E j ) of the feedback loops varied as in Eq. 2.8, the reverberation periods for neurons A, B and C were, for one single input, 196, 203 and 196 ms, respectively, displaying a temporally limited reverberation process with fixed frequency. We know that the reverberating time of an input signal and its power to modify an information have a strong dependency, not only on the synaptic weights, but also on the number of neurons composing the circuit, because of the delay time of each synapse. However, these results show that, for the variable-w model, the output pattern in the synaptic loops could be controlled and maintained at a given level, evidencing the synaptic weights capability to control the postsynaptic effect of the recurrent signal, even for an exclusively excitatory circuit.
It is important to note that the reverberating periods mentioned above ( Figure 5) are near to those found by Johnson et al. [16]. These authors have shown that the reverberating activity in lateral amygdala (a basal nucleus within the temporal lobe) could be prolonged for more than 40 ms in vitro and the apparent polysynaptic activity of in the awake rat may reach 240 ms. According to these authors, the neural activity identified throughout that basal brain nuclei included an apparent recurrent feedback from the inferior to superior parts and, for them, reverberation mechanisms are involved in the immediate maintenance information memory.
We are considering plausible the vision of the reverberation circuit as a device participating in the process of immediate memory, functioning as a collateral circuit that facilitates maintaining the information, if there is a mechanism to control the output pattern in the synaptic loops. It represents a possibility to retain information for enough time to allow the occurrence of associations between sequential data for training and pattern recognition. According to our model of variable synaptic weight ( Figure 5), information can reverberate normally in an active set of neurons and remain temporarily unchanged, because of the synaptic transitory changes, while it is manipulated by other circuits under the coordination of a central executive system.
When we considered feedback connection C-B as inhibitory ( Figure 6), with w I j varying according to Eq. 2.9, being excitatory the other feedback connections, with weights w E j (Eq. 2.8), a significant reduction was observed in the reverberation time and the number of fired spikes for each neuron (Figure 6). Neuron A firing generated only two spikes, and the second occurred 14 ms after the first. In neurons B and C (Figure 6(b)), only one spike was fired. These results show that the replacement an excitatory feedback connection by an inhibitory one in a circuit of the same type as that shown in Figure 3 is able to suppress its reverberating character, evidencing the power of this inversion.
This shows that, for a reverberation circuit with excitatory and inhibitory synapses, at least three parameters may play a key role in reverberation time: the input frequency (already known), the number of inhibitory synapses and the modulation level of each synapse. It seems, however, that the presence of inhibitory synapses in reverberating circuit is not enough to explain the slight transitory changes characteristic of immediate memory, although they might be present in the circuit. According to our results, a mechanism capable of modulating the synaptic weight over time and the input frequency may be more adequate to explain these transient changes.
Thus, we can raise important questions for future research on the structure and behavior of reverberating neuronal circuits. For instance, which biological mechanism could regulate the time of synapse fatigue? Which biological mechanism could explain the temporal modulatory dynamics of the synaptic weight as a function of the presynaptic frequency in consecutive cycles, as suggested by our model? Certainly, these are very complex questions within the biological vision of synapse, which is different from the vision of artificial neural networks.
In artificial neural networks, the synaptic weight defines the strength or efficiency of synaptic coupling between two nodes of the network [10]. When the synaptic weight is positive, the connection is called excitatory and, when negative, it is named inhibitory. Mathematical and computer studies have suggested that there is a large number of neurobiologically relevant types of feedback mechanisms that can be generated [11]. However, connections in neural networks do not have exactly the behavior of the biological interneuronal connection, because a neuron in this network might be representing a group or several groups of biological neurons, or even an entire anatomical structure. This is very different from what we are representing here.
In our model, each neuron means a biological neuron and each synapse can be seen as one single synapse or as multiple synapses between a single axon and one neuron. The purpose of this work was to model a real neurophysiological circuit to reach understanding of its characteristics and behavior.
From the biological point of view, the synaptic weight refers to the contribution that a synapse can offer within the set of synapses acting on a neuron [12]. We can understand the synaptic weight as a parameter related to the capacity of the receptor-neurotransmitter interaction to change the postsynaptic potential. The higher the capacity, the greater the postsynaptic potential produced (depolarizing or hyperpolarizing), thus being higher the contribution of this synapse to the electrical potential across the neuron membrane, and to its firing or inhibition. The synaptic weight, in turn, depends on certain parameters of the postsynaptic membrane, such as the number of available receptors, being related to the up-or down-regulation mechanisms and to the response level (opening of ion channels) of each of them.
Considering a single synapse acting on a neuron, there are some mechanisms that we can be think of to explain the synaptic weight varying with the presynaptic discharge and subsequently modulating the postsynaptic membrane response, allowing the adjustment required to maintain the information unchanged in a reverberating circuit. The first would be the variation in the time of occupancy of the synaptic receptors. This variation can be thought as resulting from a conforma-tional modification of the receptor able to promote a change in the membrane ionic permeability. This could prolong the postsynaptic potential growth time, delaying the neuron discharge and therefore controlling firing frequency. Another possible mechanism would be through a process of facilitating or enhancing the reuptake or degradation of the neurotransmitter excess released in the synaptic cleft in result of the high presynaptic frequency. These two mechanisms associated with variant synaptic weights could even be considered part of the arsenal of mechanisms involved in neuronal plasticity.
The transient synaptic changes mentioned above, related to the preservation of information in reverberating circuits, may be seen as a model of anticipatory mechanism capable of generating long-term changes, and could also involve the development of modulatory mechanisms of synaptic weights.
These hypotheses lead us to think about neuroevolution and structuring of neuronal circuits. Perhaps the transient variation in postsynaptic receptors engagement-disengagement or in cleft enzymes dynamics of action as a function of the presynaptic discharge is a special feature of neurons found in certain areas of the central nervous system, since they correspond to mechanisms of refined neural control. Such mechanisms could be the result of the development and improvement of the higher functions of the nervous system and, as such, they would be rare or absent in rudimentary organisms.
It is important to mention that it is not our intention to build the idea that our suggestions are the only way to understand the results presented here and those types of neural feedback mechanisms capable of providing synaptic stabilization, in order to approach the reverberating circuit to the immediate memory.

CONCLUSIONS
Results show that the characteristics of the mathematical-computational model adopted for reverberating circuit allowed the simulation of an immediate memory circuit, starting from the idea that reverberating networks facilitate the association of coincident data and the creation of patterns, based on Hebb's postulates and recent evidences.
Our model showed that the modulation of synaptic weight with time and frequency of presynaptic discharges altered the reverberating activity of the circuit (in which we used parametric values from experiments with animals), rendering it able to reproduce frequency values of the polysynaptic activity of a set of neurons of an awake rat, with looping time of three neurons of order of magnitude of 10 2 ms.
On the other hand, we verified that the existence of an inhibitory feedback synapse in the circuit, with synaptic weight varying with time and frequency of presynaptic firing, was competent to change the reverberating character, displaying the importance of balance between the input frequency and the inhibition level of the temporal response of the circuit.
Considering a single synapse acting on a neuron, some mechanisms may be proposed to explain the synaptic weight varying with the presynaptic discharging to preserve information in a circuit. One of them is the variation in the occupancy time of receptors at the synapse, and other would be the modulation on the process of reuptake or degradation of neurotransmitter at the synaptic cleft, through protein conformational change.
The circuit presented here can be used to study several additional situations, with different rhythms and frequencies, verifying the influence of each parameter on the circuit output and the behavior of the model as a short-term memory circuit.
Moreover, it can be amplified and transformed into a reverberating network, associating it to nonreverberating circuits, looking for a model to simulate the thalamocortical system, which would allow to studying the characteristics of interconnection between basic cellular components and their involvement with the definitive memory formation.