![[Figure 1. Event-Predictive Emulation]](events-processing.gif)
Figure 1. Event-predictive emulation.(Click image for larger version.)
| mind uploading research group |
Please note: Event-Predictive Emulation is a spline component function based approach to the efficient and biologically plausible emulation of dynamic continuous and discontinuous functions derived from neuronal circuitry. A paper presenting and demonstrating the technique and its applicability in software and hardware implementations is currently in stages of production and review. Details of the method will not be described here, pending submission of the full paper to a peer-reviewed journal. An abstract description of the method will suffice here: Rather than using a brute-force approach to computing temporal instances of complex interacting model functions, the functions are converted such that events of information-theoretic significance are predicted to any desired degree of precision with computational gains over the brute-force method that increase with complexity and precision.
[abstract]
Several paths can be taken in an effort to provide a processing substrate that can emulate a fully operational brain at speeds that at least equal those of the biological brain. At one end of the spectrum, the substrate can be chosen for its structural similarity with a biological brain, hardware specialization for that task. Alternatively, one may opt for a substrate with a biological network design, but with digital emulators for the local components, perhaps so that it may easily interface with conventional digital technology. Finallly, it is possible to use a general processing architecture based on conventional von Neumann principles, within which both the structure and the operations of a brain are emulated in software. While it is not the most efficient stubstrate for a specific whole brain emulation, a conventional architecture can have advantages for whole brain emulation research, since a structure and component functions in software are highly modifiable. The requirements for a member of each of the three approaches are investigated here.
[previous requirement calculations]
[different notions of what constitutes an emulation and what level of reproduction is required to achieve functional system-level emulation, use of event-predictive emulation in digital emulation, refer to Figure 1a and 1c]
Figure 1. Event-predictive emulation.
(Click image for larger version.)
The following is a tentative list of components that are needed to achieve a correct emulation and the reproduction of all system-level features:
Space
every components has state, possibly with an event window cache;
every function has parameters;
it is possible for the sake of approximation to treat the requirements of
the soma as equal to those of one synapse, and to assume an averaged cache
size, 10 doubles perhaps;
Processing cycles
using event-predictive algorithms, plausible functions may
require around 5 to 15 prediction steps, an average of 10; (i)
it is possible to interleave the processing of prediction stages in many
components, since they are independent in intervals between events (where
events are any events that can impact an ongoing prediction);
while interleaving may require more queing and calls to object-specific
prediction functions, it can save a great deal of computation, by avoiding
the computation of steps that describe functions at times beyond the next
event, i.e. steps that would otherwise be invalidated;
prediction steps can therefore be assumed to be ongoing in all functions from
A to K; (ii)
an event in a soma causes events at terminals of its axon(s), as well as
modulation events throughout its dendritic tree, i.e. the average frequency of
somatic events determines the frequency at which those events must be
processed; (iii)
an event at a receptor channel impacts ongoing prediction in the target,
the soma if events are directly incorporated into its predictive function,
or chains of local predictive functions leading to the soma; the average
frequency of synaptic events determines the frequency at which those events
must be processed; (iv)
unless complex intersection-event functions are required for synaptic
transmission or modulation, a synapse need schedule only one processing
request per forward or backward propagated response; the transfer of forward
propagated responses results in the scheduling of a somatic event (input);
somatic events (inputs or spikes) are assumed to involve complex
intersection-event functions, and therefore result in chains of scheduled
prediction steps, possibly leading to another somatic event (spike);
Assume (currently without adequate justification) that oscillations such as gamma are typical for internal cortical processes, and that approximately four patterns therefore occur every 150 ms (i.e. within the positive phase of cycles such as theta), i.e. about 27 each second.
Whenever the soma undergoes a spike event, a new sequence of intrinsic
processes begins, requiring a set of 5 to 7 (?) prediction steps. This occurs
for 2% of all patterns, using the sparseness suggested by Treves and Rolls
(1994), for lack of more region-specific data at this time.
There must actually be fewer than this, since not all neurons are connected
in a single large oscillating network. Propagation may not take place, e.g.
if cells in the propagative path do not spike), In addition to this, not all
prediction steps need be computed if an event is predicted in a step prior
to the last in the sequence, or if an available short-cut is identified
(see above).
A better estimate than the Treves and Rolls figure for sparseness may be obtained when regional information is available about the average temporal window size and average number of potentiating synaptic input contributions that are necessary to elicit an action potential. With a good estimate of the average frequency of synaptic inputs, the proportion of potentiating versus depotentiating input contributions (including pervasive modulations such as sub-threshold oscillations) and average intrinsic cell dynamics, this information can be used to derive an approximated average frequency at which each neuron in a given region produces action potentials.
Each such event also causes both forward and backward propagation of the effects of the action potential, leading to modulation at dendritic and target synapses (F and B events). These are currently computed in a single step each. Assume 1000 (?) unique connections (after having grouped as many connections between the same pre- and postsynaptic neurons as possible) from presynaptic neurons, as well as to target neurons via axonal terminals.
For each (probabilistic) transmission via axonal terminals, a target neuron receives an input (I) event, that results in state modification and the addition to the queue of scheduled prediction event of several prediction events, perhaps 5 (?), for the contribution of the input response.
Sufficiently simultaneous inputs from similar synapses can be grouped into a single contribution, e.g. theta is the result of a grouping - preprocessing of common groupings of simultaneous or periodic stimuli can lead to such plausible simplifications. Examples are theta modulation and synchronous GABAergic activity throughout an interneuronal network. With a specific desired precision it is also possible to group together prediction steps scheduled at times that are less distinct than is useful for that precision, choosing an average time at which to schedule prediction for the next spline component function in each member of the group.
Per second: N = average number of neurons M = average number of independently approximated intrinsic functions K = average number of independently approximated response functions per input L = average frequency of LTM (modification, recruitment, selection) steps N * 0.02 * 5 to 7 * 27 * M contribution spline function prediction steps (it may be possible to include functions for items in D and K above in M) N * 0.02 * (1000 + 1000) * 27 synaptic F and B modulations (these may be different modulation functions, depending on the phase that a synapse is in, e.g. late-phase LTP, as indicated in I above) N * 0.02 * 1000 * 27 input state modifications N * 0.02 * 1000 * 27 * 5 * K input response contribution prediction steps (grouped input contributions, such as theta and GABA should be treated as input response contributions, i.e. as individual synaptic processes, instead of as intrinsic function, eventhough they may affect every neuron, since they do not depend directly on the spike events of their target neuron) N * 1000 * L prediction steps for synaptic LTM changes per second (L can be small, since this is slow, and prediction uses state information such as potentiation)Assume (for now) that prediction steps, modulation operations, state modifications and LTM process steps each require a minimum of 100 computation cycles when coded for maximum speed efficiency. Also assume that N = 10 billion operational neurons (?this could also be described as 100 billion neurons, 10% significantly involved in processing - or perhaps 50% involved, but 0.004 rather than 0.02 ratio per pattern cycle per region --- clarify?), M = 3, K = 1 and L = 0.0003. (Using N=10^11 or N=10^12 clearly leads to correspondingly greater processing requirements.)
Using these numbers, a full-scale real-time emulation of a brain requires
100 * 10^10 * 0.02 * 6 * 3 + 100 * 10^10 * 0.02 * 2000 * 27 + 100 * 10^10 * 0.02 * 1000 * 27 + 100 * 10^10 * 0.02 * 1000 * 27 * 5 * 1 + 100 * 10^10 * 1000 * 0.0003 = 4.3207e+15 cycles per second (4320 trillion cycles per second)Using superconductor circuits with a cycle frequency of 750 GHz, real-time emulation for this estimate requires 5760 parallel CPUs. Put another way, using this translation from brain operations to computer instructions, the human brain can be said to handle the equivalent of about five quadrillion instructions per second. While this is quite formidable by current computing standards5, it is feasible with the technology that will enter mass production in the next five years.
If multi-step predictive processes are also needed in synapses
[...]
If A is included, and compartments are modeled as neurons, then while the sparseness would increase, multiplied with the number of compartments, the average number of synapses would decrease, since compartments would be linked with very few connections
B may be integrated in the specific contribution function that a synapse uses to apply input to the somatic response, but the addition of such a function to the response function may necessitate smaller prediction steps, unless the effect of the path-to-soma is plausibly expressed by a simple adjustment of parameters, such as amplitude or characteristic timing of an input contribution
separate functions indicated in C are sometimes more efficiently predicted using distinct spline approximations, and sometimes with a single approximation for the sum of several functions (as is currently done for AMPA and NMDA channel responses)
Requirements for Digital Hardware Emulation
[...]
Requirements for Analog Hardware Emulation
[...]
If a lesser degree of abstraction is desired in component B, compartments can be modeled as individual neural nodes with electrical synapses (no chemical transmission across a synaptic gap). A similar modeling philosophy can be used to include more exotic requirements as they appear, such as the potential contribution of communication services via astrocytes [cite]. The function in A can involve a single synapse-to-soma computation, or it may be incorporated in a piecewise propagation from the synapse through neighbours to the soma. With high resolution modularity, only individual channels and their corresponding response functions mentioned in C may occur in a component. Additional functions may be needed in E, if a dynamic threshold must be computed independent of the membrane potential [cite Petersen]. The use of event-predictive emulation based on spline component functions actually increases the ability to combine (possibly independent and orthogonal) functions with different features in a single prediction process.
Endnotes
References