Logarithmic distributions prove that intrinsic learning is Hebbian

In this paper, we present data for the lognormal distributions of spike rates, synaptic weights and intrinsic excitability (gain) for neurons in various brain areas, such as auditory or visual cortex, hippocampus, cerebellum, striatum, midbrain nuclei. We find a remarkable consistency of heavy-tailed, specifically lognormal, distributions for rates, weights and gains in all brain areas examined. The difference between strongly recurrent and feed-forward connectivity (cortex vs. striatum and cerebellum), neurotransmitter (GABA (striatum) or glutamate (cortex)) or the level of activation (low in cortex, high in Purkinje cells and midbrain nuclei) turns out to be irrelevant for this feature. Logarithmic scale distribution of weights and gains appears to be a general, functional property in all cases analyzed. We then created a generic neural model to investigate adaptive learning rules that create and maintain lognormal distributions. We conclusively demonstrate that not only weights, but also intrinsic gains, need to have strong Hebbian learning in order to produce and maintain the experimentally attested distributions. This provides a solution to the long-standing question about the type of plasticity exhibited by intrinsic excitability.

to incorporate these observations as primary constraints.More data are needed to consolidate the observations.

Introduction
Individual neurons show very different but mostly stable mean spike rates under a variety of conditions [26,12].To report on behavioral results, spike counts are often normalized with respect to the mean for each neuron.But this obscures an important question: Why do neurons within a tissue operate at radically different levels of output frequency?Here, we give examples for the distribution of mean spike rates for principal neurons under spontaneous conditions as well as in response to stimuli.We furthermore document distributions for intrinsic excitability ( [11,17,8] for cortical and striatal neurons, as well as conductance-based model neurons ( [13,8]) and synaptic weight distributions.This is a draft paper, since more and better data will be needed for solid theoretical conclusions.
The work refers back to data initially reported at [26].At the time, we only had data on cortical neurons available, plus independent evidence on intrinsic properties of striatal neurons.The observation on cortical data was taken up by [28,12], and led to a number of papers [24,4] focusing on the power-law distribution of spike rates as a cortical phenomenon, seeking explanations in the recurrent connectivity [12,24] and certain synaptic plasticity rules [16,24].We now show that fairly reliable data exist on similar spike rate distributions for midbrain nuclei, striatal neurons and cerebellar Purkinje cells.This makes the original explanation according to intrinsic properties more likely.We extend the presentation of data on intrinsic excitability as well.
With the existing data, we show that the distribution of spike rates for principal neurons within a neural tissue almost universally seem to follow a power-law distribution, i.e. a distribution with a 'heavy tail'.There are often also a smaller number of very low-frequency neurons, resulting in a 'lognormal distribution' [12].The power-law distribution is present in spontaneous as well as behavioral data.We show that for each neuron, the deviation from the mean rate attributable to a stimulus is small (CV ≈ 0.3 − 1, stddev=1-4 spikes/s), when compared to the variability in mean spike rate over the whole population (5-7-fold).
Similar lognormal distributions have also been found for synaptic effica-cies ( [30]).We have constructed a simple model for the interplay of intrinsic excitability, synaptic efficacies and spike rate distributions.Interestingly, we not only find that the similar distributions for these parameters fit into a common model, we can also show that specific properties to be expected from the model -the width of the distributions -are actually predictive for the three types of experimental data that we have available.

Methods and Experimental Data
In this section we report on data collection for the distribution of mean spike rates, intrinsic properties and synaptic properties.Secondly, we explain the simulation model we constructed for these data.

Mean spike rate distribution under behavioral conditions
We derived three data sets for cortical principal neurons under behavioral activation, from inferior temporal (IT) cortex [14], and primary auditory cortex (A1) [32] from monkeys, and primary auditory cortex (A1) of rats [12].In monkey IT, single unit activity was recorded over 200ms for passive viewing of 77 different natural stimuli for 100 neurons, each stimulus shown 10 times [14].This yielded 770 spike rate response data points per neuron.From these data, we calculated mean spike rate, standard deviation and extreme values (variance) (Fig. 1).We re-analysed data from primary auditory cortex (A1) from awake monkeys [32] for spike responses to a 50ms, 100ms, or 200ms pure tone (Fig. 2 and Fig. 14).For parameter values see Table 1.Data  have also been reported in [12] for spike rates for four different conditions of cortical neurons from cell-attached in vivo recordings from the primary auditory cortex of rats (Fig. 3).The exponential (power-law) distribution of  spike rates is evident in all conditions from low to high spike rates.Beyond cortical neurons, we also collected data from principal neurons in other brain areas.We have looked at Purkinje cells in cerebellum [5,23], and midbrain principal cells from inferior colliculus (IC).Beyond cortical neurons, we also collected data from principal neurons in other brain areas.We have looked at Purkinje cells in cerebellum [5,23], and midbrain principal cells from inferior colliculus (IC).Purkinje cells offer some difficulty since they have regular single spikes, and in addition, calcium-based complex spikes at low spike rates (≈ 1Hz).We regard this as a form of multiplexing and report only data for single spikes by [5] from in vivo recordings in rats (Fig. 4).Since complex  spike rates are low (1Hz or less), they would not skew the distribution, even though we have no data on the distribution of complex spikes, we may assume they are uniformly low.Data for two Purkinje cells from an entirely different experiment, [23], i.e. spike rates under behavioral stimulation (40 different conditions) have been added in order to show two values for standard deviation and variance (Fig. 4).For midbrain nuclei (IC of the guninea pig) we re-analysed spike rates in response to tones (for 200ms after stimulus onset) under variations of binaural correlation [33].The frequency ranking of neurons by mean spike rate, the standard deviation and variance are shown in Fig. 5A.The available data show another lognormal distribution of mean spike rates (Fig. 5B), and a CV (standard deviation/mean) almost identical to the cortical data (Fig. 5C).

Intrinsic excitability and Spontaneous Spike Rates
We show data on spontaneous spike rates in vitro, for isolated Purkinje cells, i.e., in the absence of synaptic stimulation, as reported in [21] and [9].The ranking of neurons by mean spike rate and the spike rate distribution are shown in Fig. 6.For striatal neurons, in awake rats, we show the spike rate during a +100pA current injection for neurons from globus pallidus (GP) (n = 145) ( [8], Fig. 7C).We present unpublished data for medium striatal neurons in slices from rats for nucleus accumbens (Nacb) shell and dorsal   Figure 7: Gain [Hz/nA] measured in response to intracellular current pulses in cat cortical area 17 in vivo for different types of cortical neurons [19].A: Gain for all neurons (n = 220).B: Gain for fast spiking neurons only (n = 33).C: Gain for neurons in GP in response to a +100pA current pulse (n = 145) [8].
striatum, where neuron-specific intrinsic diversity is clearly apparent (data W.Hopf, [11]).In Fig. 8A, we show the spike rate in response to current pulses, and calculate the gain for each neuron.Fig. 8B and C shows the distribution of rheobase (current-to-threshold) for dorsal striatal and Nacb shell MS neurons.Distributions appear mostly lognormal, with the exception of the 200pA current pulse response and data in Fig. 8C, where they appear normally distributed.
Intrinsic excitability data have been recorded from different types of cortical neurons in cat area 17 by intracellular current pulses in vivo [19].They show the gain, the spike rate in response to current, for all neurons (Fig. 7A), separately for regular spiking, bursting, and chattering cells, which are pyramidal neurons (Fig. 15) and fast spiking inhibitory neurons, which show quite a different, approximately normal distribution (Fig. 7B).It is interesting that the fast-spiking, presumably local inhibitory neurons, 'non-coding' neurons, have a different distribution.It is also important to realize that the subset of differently distributed fast-spiking neurons is not evident from the overall distribution -this shows that there may be subsets of neurons with different properties in other areas as well.The data are not sufficient at this point to draw firm conclusions.

Spike rate data from model neurons
We also report artificially generated data for model neurons.For model neurons which are periodically spiking ( [13]), the statistical distribution of spike rates is interesting (Fig. 9A).There is clearly a gap in the 10-40Hz range, possibly for neurons which require synaptic input.If we only report  the number of spikes for a periodically spiking neuron, when it is in the 'upstate', ('duty cycle'), this follows the exponential/lognormal distribution (Fig. 9B).There are also statistical data on conductance-based models for GP neurons ( [8]).The authors note that the models are not entirely representative of physiological neurons: Only about 19% of physiological neurons show no spontaneous spiking, but 56% of model neurons.Also, very low spike rates (less than 3Hz) are significantly underrepresented in the model neurons.Nonetheless, the model neurons show an almost perfect lognormal distribution (Fig. 9B).

Synaptic weight distributions
Synaptic weight distributions in cortex and hippocampus have also been investigated ( [30]), and EPSC magnitude measured (Fig. 10, Fig. 16).Consistently, lognormal distributions have been found.There are also synaptic weight data on granule cell to Purkinje cell connections [15,3,2], which show a similar distribution, with an order of magnitude weaker connections.Input EPSC distribution for each neuron are given in [31].

Simulation Model
Given are two neuron populations I, J with n = 1000 neurons.The input group I is modeled according to [12] for pyramidal cortical neurons with a spike rate distribution of µ = 1.6 and σ 2 = 0.47 (Table 1, Fig. 12A,B).The goal is to generate a spike rate distribution R(J) for J, given the gain G(J) for J, a weight distribution W and input I such that J is similar to I.For group J, the gain G(J) is given according to [19] as shown in Fig. 7 and Table 2 as µ = 4.87, σ 2 = 0.25.
First we need to calculate the weights W between I and J, assuming that the rates for I and J are identical and both groups are fully connected.Fig. 11 shows the principle architecture and we calculate Figure 11: Synaptic weight distribution follows from intrinsic frequency distribution for rates of both population R(I) and R(J), and a small scaling factor, here λ = 0.01.This corresponds to basic Hebbian learning rule, which is dependent on the input and output frequencies of I and J.To calculate the input to J, we assume that the system operates with a subset N ⊂ I of active neurons.This results in an effective connectivity of c = 10%, therefore, each neuron may receive input from about 100 neurons in our case.This seems like a large amount, but we need to be aware, that many of these neurons are 'silent' or fire with a low frequency (more than 50% of neurons from I have a rate of <5Hz)) and that the weights are low for low frequency neurons as well.For each neuron j ∈ J, we calculate its EP SP j as . We calculate the EPSC, by converting from mV into nA (0.3mV corresponds to 40pA) as EP SC j [pA] = (0.04/0.3)EPSP j .This EPSC distribution roughly corresponds to the quantal distribution of EPSC data in [31].Inhibitory input IP SP to J is modeled as Gaussian distributed population H with µ = 500 and σ 2 = 100 with IP SC j = (0.04/0.3)IPSP j we calculate the rate distribution R(J) as for all j ∈ J and all values suppressed where the IPSC is larger than the EPSC. Figure 12 shows the resulting distributions, which will be discussed in Section 3.3.

Numerical Fitting
In many cases, the data were only available in the form of histograms.The parameters of the log-normal distribution was obtained by fitting the histogram data using ezfit1 , which uses a Nelder-Mead optimization method (fminsearch).The data were fitted to the parameterized log-normal distribution [18] for the unknown parameters µ, σ 2 , and b.A scaling factor b 0 was used to keep b in the range of 0.1 . . . 10 in order to avoid numerical problems.

Universality of spike rate distributions
We have shown mean spike rates as a histogram over the distribution of spike rates (Fig. 1B).The distribution follows roughly an exponential or lognormal shape.The fold-change from low spiking neurons to high spiking neurons is high, 7-fold.The variance for each neuron is low in comparison such that a low spiking neuron never reaches the same rate as a high spiking neuron.In Fig. 1A, we show spike rates separately for each neuron, which are ranked by mean spike rate.The size of variation from the mean during behavioral stimulation is also shown.The standard deviation is fairly small, and fairly constant, for all neurons about (± 1-4Hz).Accordingly, the coefficient of variation (CV, standard deviation divided by mean) rises linearly from 0.3 and 1.2, less (4-fold) than the mean (7-fold) as shown in Fig. 1C.The fold-change for the CV is less than for the mean because the standard deviation is an equalizing factor.For instance, a low frequency neuron like Neuron 91, even with a high variance, has a peak rate (8Hz) that is below the mean rate for the 10 neurons with the highest frequencies.It is also below the lowest attested value for Neuron 1 (10Hz).We will address the question of what this means for neural rate coding below.
For A1 [32] the spike rate is much higher than in IT (both from monkeys), e.g., for the 200ms condition in both cases.The spike rate per second is higher for the shorter stimulus.But as we can see from Fig. 2A and B, the lognormal distribution and ranking of neurons by mean spike rate is consistent.The spontaneous spike rate for this set of neurons was reported as fitted by an exponential function (Fig. 2C).This emphasizes that the distribution of spike rates is remarkably consistent across conditions, such as spontaneous vs. stimulus-induced.The population response is higher under stimulation -even much higher, probably because of the short duration of measurement -than under spontaneous conditions, but it is proportional to the mean rate and, as shown in Fig. 1A keeps the frequency rank of each neuron essentially intact.
In all cases in cortex, the distribution of mean spike rate is essentially the same -or at least always follows a power law -both under spontaneous or under behavioral conditions [12,32,14].This was also shown for PFC [28], i.e., the distribution for the baseline spike rate and the increase during the delay period does not change.
A lognormal spike rate distribution appears as a defining feature of a collection of principal neurons, which is independent of the activation of the network.Rather, it seems to be an essential component of the functional set-up that is not affected by learning, plasticity, or activity of a (mature) network.
Since it has been previously assumed that this distribution is a feature of cortical neurons, we looked for data beyond cortex.Cerebellar Purkinje cells are known to have high spike rates compared to cortical neurons, by an order of magnitude.They are GABAergic projection neurons (in contrast to cortical glutamatergic cells) and they have a distinct protein expression profile.Their connectivity is very different from cortical pyramidal cells, in particular they do not have recurrent connections (direct connections between Purkinje cells).So it is interesting to find that the distribution of mean spike rates from cerebellar Purkinje cells is also lognormal and matches the distribution of cortical neurons [5,23].Finally, we found data from the inferior colliculus, a nucleus in the midbrain which is essentially an 'output' site for auditory and somatosensory cortex.Spike rates are high overall, nonetheless the frequency ranking of neurons by mean spike rate, the standard deviation and variance are comparable to the cortical data [33] (Fig. 5).The data are summarized in Table 1.
The lognormal spike rate distribution as an essential property of a neural tissue is therefore not specific to cortex, rather it extends at least into areas that are linked to cortex but have very different neuron types and connectivity, and different absolute spike frequencies.tissue mode mean µ width σ 2 n fold IT cortex [14] 2.2 1.5 0.71 100 0.5-14(28) A1 cortex [12] 3

Intrinsic excitability and synaptic weights
Beyond spike rates, we compiled data on intrinsic excitability, defined as the spike response to a constant current (gain), and synaptic weights.We found that both parameters are also lognormally distributed in different neural tissues.
We looked at data on striatal principal neurons (medium spiny neurons) as in nucleus accumbens, dorsal striatum or globus pallidus.These neurons are inhibitory (GABAergic) with little interconnectivity.They possess large dendrites with many inputs, and they project to targets in a relay fashion ('transfer neurons').If we examine the response of single neurons in slices to direct stimulation, we also find indications for heterogeneity, which is necessarily intrinsic, i.e., not attributable to differences in synaptic stimulation (Fig. 8).Even though the data are not entirely sufficient, the gain is lognormally distributed, but the rheobase (current-to-threshold) is not.In another set of data, also from striatum, but from a different substructure (globus pallidus), again, the spike response to current injection measured in vivo is lognormally distributed (Fig. 7C).
Cerebellar data also exist for spontaneous spike rates in isolated neurons [9,21] or again as the spike response to a constant current pulse [5].In both cases we still see a lognormal distribution of mean spike rate.The data for the intrinsic excitability of cortical neurons (Fig. 7) are interesting, since they show a difference between coding (pyramidal) and non-coding (inhibitory) neurons.The distribution of excitability, or gain, is lognormal for coding neurons, but near-Gaussian for interneurons.The number of interneurons is small (n = 33), but if this observation holds up, it could be an interesting exception to the findings on principal neurons.
Data for the intrinsic excitability (gain) from cortex, cerebellum and striatum are summarized in Table 2 Intrinsic excitability may be generally defined as response to current pulses, but the individual variables of resting potential, action potential threshold, and resistance define multidimensional response properties of neurons.According to [17], action potential threshold and resting potential are independent in cortical neurons, such that some neurons require only 15mV, others 30mV to spike.However, these variables are lognormally distributed and contribute to gain.The resistance is not related to the gain at all, and even seems not to be lognormally distributed.It may define the dynamic range of a neuron.This would mean that the dynamic range of a neuron and its frequency rank need not be coupled in a simple and direct way.
Finally, we were interested in data from conductance-based model neurons.For a large set of 10,000 models for globus pallidus neurons [8] (Fig. 9A), the characteristic shape of a lognormal distribution clearly emerges.For a set of spontaneously spiking neurons, there is a bipartite distribution where the component between 10-40Hz is lacking, but if we only count spikes during an 'upstate', we see the same lognormal distribution again ( [13], Fig. 9B).
Synaptic weight distributions, measured by EPSCs, are shown for cortex, hippocampus and cerebellum in Fig. 10 and are summarized in Table 3.

Simulation
Since not only mean spike rates but also both components, intrinsic excitability and synaptic weights, have lognormal distribution, this raises the question of how the functional system that we observe is set up.It is obvious, if the data are accurate, that these are basic parameters of any simulation and need tissue mode mean µ width σ 2 n cortexL23 [1] 0.17 -0.99 0.76 48 cortexL23 [6] 0.31 0.25 1.41 35 cortexL23 [10] 0.08 -0.94 1.54 61 cortexL5 [29] 0.13 -0.56 1.47 1004 cortexL5 [7] 0.41 -0.31 0.58 26 hippocampus [25] 0.05 -2.61 0.43 71 cerebellum GC-PC [3]  to be reproduced in a model to make it biologically realistic.We created a simple simulation model as described in Section 2.5.Two connected neuron populations, I and J are fully connected with a set of synaptic weights W .We also set the mean spike rates for I and J such that R(I) and R(J) have lognormal distributions.We calculate W assuming a basic Hebbian learning rule, which is dependent on input and output frequencies (Section 2.5): W = λ • R(I) × R(J).We then use W to re-calculate the output R(J) from the input R(I), using the gain G(J).There is N the active subset of I, from which all inputs to J are drawn.We know that N may be large, about 40% (20-50%) of neurons becoming active, and we have an excitatory connectivity of about 10%.For EPSP measurements in slices, usually low effective connectivities were reported (e.g., 11% in cortex), even though anatomical evidence has full connectivity (silent synapses hypothesis [2]).The actual inputs to a neuron j ∈ J are being used to sample I.If about 40% (20-50%) of neurons become active, yet neurons receive inputs from a small subset, about an order of magnitude less, what can account for the discrepancy?If there is full connectivity, that is not possible.It only works if neurons are sparsely connected, where each neuron only connects to about 10% of neurons, and for each activation receives input from 2-5%.
The set of neurons I could be any size, this only means to adjust λ, but we may want to calculate the numeric values.We can calculate this with numbers: synaptic strengths in cortex are between 0.2mV to 1mV.Currentto-threshold is between 15-40mV in cortex within a 10-50ms time gap [17].This means between 15 to 200 synaptic inputs are needed to induce a spike.For instance, 30 neurons at mean rate 40Hz would produce 15-60 inputs, 100 neurons at 20Hz produce 20-100 inputs, 500 neurons at 2Hz produce 50-100 inputs etc.The spike rate distribution is such that a single neuron can be 5 to 7 times as strong as most others (matching fold changes).How do J neurons sample N? The distribution is affected by a chance factor which is larger the fewer neurons are active.If the sampling was bad, the inputs would be worse than Gaussian, i.e., random, then the output spike rate distribution could become a noisy, unreliable lognormal distribution.However, the fact that there is Hebbian plasticity in our system: frequency rank of neurons is proportional to strength of connection, makes the system very robust.
So far, the output spike distribution is too wide (Fig. 12F, red).We notice that we have to add inhibition to make the data match more closely.We choose a Gaussian distribution for inhibition, which we subtract from the weighted input to each neuron.We may now achieve closer matches (Fig. 12F, blue) to experimental data (Fig. 12F, black).

How do spike rate distributions, synaptic weights and intrinsic excitability relate?
What is noticeable about the lognormal distributions gained from the experiments is that each type (gain, spike rate, weights) has fairly consistent CVs or widths of the distribution (Tables 1, 2, 3).For instance, it turns out that the gain distribution has smaller width than the spike rate distribution, and that weight distributions are the widest.
Numerically, we get data of approximately 1 for mean spike rate distributions, 0.5-0.7 for gain, 1.5-2 for weights.We get approximately: ) because of the learning rule.It is interesting that a basic Hebbian learning rule applied to lognormal spike rate distributions will produce synaptic weights with lognormal distributions [16].
For the model, we regard the simple case of a transmission from a set of neurons onto another set of neurons.We have a spike rate distribution R(I), which acts through a weight distribution W (I × J) onto a gain distribution G(J), where inhibition H is added, and a spike rate output distribution R(J) is produced (Fig. 11).
We estimate the width of the distributions from the model.Figure 12F shows the distribution R(J) (red) in contrast to I (black), when no inhibition is applied.The widening is caused by the application of the weight distribution W on R(I).In order to match R(J), a suitable Gaussian distribution for the inhibition H is used.A Gaussian will dilute the distribution by weakening or eliminating a certain range of higher frequencies in the EP SC distribution.The Gaussian effect of inihibition must be small enough to shrink the width of the lognormal distribution, but not make it disappear (Figure 12F, blue).
We derive an input EPSC distribution for each j, which we compare to the data from [31].The EPSCs to each single neuron show that they are Figure 13: Synaptic connectivity for frequency-ranked neurons also lognormally distributed.However, if the total input to each neuron j ∈ J is a constant C, the ranking of J neurons remains stable.Do we get an approximately constant mean input to each neuron?Each behavioral change to C is reflected by J by relative coding (relative to the frequency status of the neurons in J). (Fig. 13).The ranks of the postsynaptic neurons remain constant under spontaneous conditions.
For a model: how does is output rate relate to input?There is the gain filter in between?When changes occur at R(I), the output changes would be small.Can this model actually detect changes in input?Under which conditions?

Generalizations from the data
Spike rates of principal neurons seem to be universally distributed according to a lognormal distribution, i.e., a type of power-law with many neurons at low spike rates, and a small number at successively higher spike rates (heavy-tail) ( [26]).The data we report indicate that the observation extends beyond cortical neurons, to other types of principal neurons, such as medium spiny neurons in striatum, cerebellar Purkinje cells and midbrain nucleus (IC) neurons.The neurons observed are of very different types, and they are embedded in different kinds of connectivity.Medium spiny neurons and Purkinje cells are GABAergic (inhibitory), while cortical and IC neurons are glutamatergic (excitatory), but this is not reflected in a distinct spike rate distribution.Cortical principal neurons exist in a heavily recurrent environment, while medium spiny neurons, cerebellar Purkinje cells and IC neurons act mostly as transfer neurons, i.e., have no significant recurrent connectivity.IC neurons furthermore operate at very high frequencies, and Purkinje neurons at significantly higher frequencies than cortical neurons, yet the spike rate distribution is similar.In contrast, it seems that the 'universal' spike rate distribution does not extend to local interneurons, such as fast spiking cortical interneurons.This could mean that it is restricted to 'coding' neurons, and that the basis of spike rate coding is in fact logarithmic.We have also looked at the distribution of intrinsic properties associated with excitability, and synaptic weights, measured as EPSCs or EPSPs.Synaptic weights have been analyzed before [30], and we also find distributions for cortex and cerebellum that are lognormal, but with characteristic wide distributions.Striatal projection neurons and cortical neurons [19] show responses to constant current and current-to-threshold distributions which again appear lognormally distribted, but with smaller deviations than spike rate distributions.
We may want to point out that the dynamic range of a neuron is different from its overall gain.We have no data on the response type of a neuron relative to its frequency rank.In principle, linear range vs. switch type gain functions could occur with any frequency rank of a neuron.All data are from vertebrate, specifically mammalian brains, the properties of neurons in small networks in invertebrates may be different.Maybe only sufficiently large structures lead to the characteristic excitability distributions.

Network Structure and Plasticity
There is a general relationship between node activation and network structure: for a given network connectivity, a certain distribution of mean activation values can be expected [22].We have identified a network with a lognormal degree distribution as 'attraction-repulsion': a top-level of sparse 'hubs', many elements in distributed coding, (which together make up a 30% to 50% coding response), and a reservoir of very low connection nodes.
They arise from a combination of preferential attachment, which accounts for the heavy-tail distribution, and repulsion, as in disconnection at low connectivity, which accounts for the population of very low frequency elements [26].Any heavy-tail distribution of connections (weights) in a network of elements may imply a hierarchical structure where few strong elements dominate many weak elements, if there is additional strong modularity or clustering [22].Such networks may arise from copying modules, and retaining connections for the top elements.The end-result is both modular and scale-free ('hierarchical attraction-repulsion').For neurons, the spike rate distribution suggests a coding schemes with a few primary neurons at the top of the hierarchy that categorize for top-level features or subsume many features under a high-level 'concept' and many neurons of secondary scope which further distinguish primary features.In this case, the intrinsic excitability and synaptic weights correspond to node activation and network structure, and they may arise together in a process of training both gain and synaptic strengths in a Hebbian way.Bidirectional intrinsic plasticity [17], which is homeostatic for the spike rate distribution, may change the intrinsic excitability of the individual neuron.Over time, with plasticity, there is obviously pressure towards homeostatic adjustment such that the basic scheme remains.

Logarithmic coding schemes
Lognormal means that the values are normally distributed, as soon as we use a logarithmic scale.This is related to principles of sensory coding, where logarithmic scale signal processing enhances perception of weak signals while also being able to respond to large signals -effectively increasing the perceptual range compared to linear coding [4].This sensory coding principle may turn into a property of access of representations in a huge interconnected network.
This basic coding scheme could be expected to be stable.It would mean that feature clusters, or event traces could be accessed by a few connections to the top-level neurons, which then activate lower-level neurons in their immediate vicinity.
What this coding amounts to, which may explain why it is so universal, is that it solves the access problem.By accessing high frequency neurons preferentially, a whole feature area can be reached, and local spread will provide any additional computation.Similarly, the results of a local computation can be efficiently distributed by a high frequency neuron to other areas.Fast point-to-point communication using only high frequency neurons may be sufficient for fast responses in many cases.Scale-free networks in general support synchronization which is also a useful feature for rapid information transfer and access [27].From an engineering perspective, roughly equal connectivity, basic Hebbian plasticity and power-law distributed intrinsic excitability would be sufficient to generate stable power-law spike rate distributions.
Non-coding neurons such as fast spiking interneurons may not participate in this process and have a different spike rate distribution.
The lognormal distribution of spike rates has significant implications for neural coding.Logarithmic spike rates are coupled with linear standard deviations for responses to synaptic input.In other words, the greatest part of the coding results already from the frequency rank of the neuron itself, such that high frequency neurons have the largest impact.But in response to synaptic input, low frequency neurons show the greatest effect in relative terms, the response at high frequency neurons is much less compared in terms of relative increase or decrease.So logarithmic, hierarchical coding is not sparse.The low frequency neurons actually matter the most in terms of input response.This 'neural code' has the cost of small fluctuations becoming difficult to detect in a population rate -they need to be detected relative to the mean value for each neuron.The idea of summing synaptic inputs would not work well in this scenario, however, different types of dendritic integration could solve this.
Experimental data have often shown that sampling of neuronal responses from a large population (10 6 or more neurons), which become activated at 30% or more, yields accuracy for a stimulus already for small samples (100-200 neurons or 0.2%) (e.g., [20]) We may suggest that this happens when we sample from a highly modular structure, where all elements of a module will reflect the same stimulus information, even though those elements are heterogeneous in many other ways.

Summary
Lognormal spike rate distributions appear to reflect a structural property with high generality across neural tissues.They may be hallmarks of principal neurons in any tissue, i.e., of coding neurons as opposed to local interneurons.A fixed mean rate for each neuron allows stable expectation values for network computations.Synapses may get adapted to the intrinsic excitability of neurons to ensure functional information processing in neural tissues.

Figure 1 :
Figure 1: For cortical IT neurons (n = 100), passive viewing of 77 stimuli, 10 trials (770 data points per neuron).A: Mean spike rates (blue), standard deviation (red), and variance (green).B: Mean spike rates histogram shows a lognormal distribution.C: Distributions of std deviation and CV (=std dev/mean) show a linear slope, but small variation.

Figure 5 :
Figure 5: Inferior colliculus neuronal response to binaural stimulation.Data from [33] (n = 30) A: Mean spike rates, std deviation and variance.B: Spike rate histogram.C: CV sorted by mean spike rate as in A. CV increases linearly for smaller spike rates.

Figure 9 :
Figure9: Conductance-based models for periodically spiking neurons[13,8].A:The distribution of spike rates is clearly bimodal (lognormal and normal), filling in a gap for firing rates from 10-40Hz leads to a unimodal lognormal distribution B: The spike rate distribution per duty cycle, i.e., during the active period of the neuron follows a lognormal distribution.C: Spontaneous spiking of model GP neurons (n=100,602),[8]

Figure 12 :
Figure 12: Simulation Model: distribution of group I (A), target group J (B) and resulting weights (C).D: distribution of EPSPs.E: distribution of inhibition.F: distribution of output (blue) and output without inhibition (red).Target distribution of J is shown in black.

Table 1 :
Statistics of Spike Rate Distributions in Different Tissues

Table 2 :
. Statistics of Intrinsic Excitability (Gain) in Different Tissues

Table 3 :
Statistics of Synaptic Weight Distributions in Different Tissues