Dendritic neurons can perform linearly separable computations with low resolution synaptic weights

Biophysical neuron model

We performed simulations in a spatially extended neuron model, consisting of a spherical soma (diameter 10 µm) and two cylindrical dendrites (length 400 µm and diameter 0.4 µm). The two dendrites are each divided into four compartments and connect to the soma at their extremity. In contrast to a point-neuron model, each compartment has a distinct membrane potential.

The membrane potential dynamics of the individual compartments follow the Hodgkin-Huxley formalism with:

for the somatic compartment and

for the dendritic compartments.

Here, V_soma and V_dend are the respective membrane potentials, C_m = 1µFcm⁻² is the membrane capacitance, g_L, $\overline{g}$_K, and $\overline{g}$_Na stand for the leak, the maximum potassium and sodium conductances respectively, and E_L, E_K, and E_Na stand for the corresponding reversal potentials. The currents I_a represent the axial currents due to the membrane potential difference between connected compartments. The synaptic current I_s arises from a synapse placed at the respective compartment. It is described by

with E_s being the synaptic reversal potential and g_s the synaptic conductance. This conductance jumps up instantaneously for each incoming spike and decays exponentially with time constant τ_s = 1 ms otherwise:

The dynamics of the gating variables n, m, and h are adapted from 13 and omitted here for brevity.

The parameter values are summarized in Table 1.

Note that due to the absence of sodium and potassium channels in the dendrites, the dendrites are passive and cannot generate action potentials.

All simulations were performed with Brian 2¹⁴. The code is available from: http://doi.org/10.5281/zenodo.4023943¹⁵. It allows for reproducing the results presented in Figure 4, Figure 5 and Figure 6. To demonstrate that the details of the neuron model do not matter for the results presented here, the code can also be run with a simpler leaky integrate-and-fire model.

Elementary neuron model and Boolean functions

We first define as a reminder Boolean functions:

Definition 1.A Boolean function of n variables is a function on {0, 1}ⁿ into {0, 1}, where n is a positive integer.

Note that a Boolean function can also be seen as a binary classification or a computation.

A special class of Boolean functions, which are of particular relevance for neurons, are linearly separable computations:

Definition 2.f is a linearly separable computation of n variables if and only if there exists at least a vector w ∈ ℝⁿand a threshold Θ ∈ ℝ such that:

where X ∈ {0, 1}ⁿis the vector notation for the Boolean input variables.

Binary neurons are one of the simplest possible neuron models and closely related to the functions described above: their inputs are binary variables, representing the activity of their input pathways, and their output is a single binary variable, representing whether the neuron is active or not. The standard model is a linear threshold unit (LTU), defined as follows:

Definition 3.An LTU has a set of n weightsw_i ∈ 𝒲and a threshold Θ ∈ 𝒯so that:

where X = (X₁, . . . , X_m) are the binary inputs to the neuron, and 𝒲 and 𝒯 are the possible values for synaptic weights and the threshold, respectively.

This definition is virtually identical to Def. 2, however, w_i and Θ are no longer arbitrary real values, but chosen from a finite set of numbers depending on the implementation peculiarities and noise at which these value can be stabilised. It means that a neuron may not be able to implement all linearly separable functions. For instance, a neuron with non-negative weights can only compute positive linearly separable functions:

Definition 4.A threshold function f is positive if and only iff (X) ≥ f (Z) ∀(X , Z) ∈ {0, 1}ⁿsuch that X ≥ Z (meaning that ∀i: x_i ≥ z_i).

To account for saturation occurring in dendrites, we introduce the sub-linear threshold unit (SLTU):

Definition 5.A SLTU with d dendrites and n inputs has a set of d × n weights w_i,j∈ {0, 1} with n w_i such that${\displaystyle {\sum }_j^{}{w}_{i,j}}=w_i,$d dendritic threshold θ ∈ 𝒯 and a threshold Θ ∈ 𝒯, such that:

with

Such a neuron model can compute all positive Boolean functions (see Def. 4) provided a sufficient number of dendrites and synapses¹¹.

We used integer-valued and non-negative parameters both for the LTU and the SLTU without loss of generality. It allows to exactly determine the number of similar synapses necessary to implement a given computation. The code necessary to perform all simulations is available from: http://doi.org/10.5281/zenodo.4023943¹⁵

Function implementations for three input variables

In the following, we will look at computations that cannot be implemented in an LTU without using different strictly positive synaptic weights. The simplest case where this can occur is for three inputs. We list all such computations in Table 2. Of these computations, the first three (OR, AND/OR, and AND) do not require distinct synaptic weights; an LTU can implement these computations with the same weight for all synapses. This is not the case for the last two computations that we call the Dominant AND (D-AND) and the Dominant OR (D-OR): here, an LTU implementation needs to have one synaptic weight that is twice as big as the others (see Figure 1).

Figure 1. Minimal implementation of the Dominant AND computation (D-AND) and its dual by a linear threshold unit (LTU).

Implementations of the D-AND where X₀ is the dominant input. Squares represent synapses with their synaptic weight, and circles stand for transfer functions. Here, the transfer functions are threshold functions with the given value as their threshold. A: Implementation of the D-AND, note that X₀ has twice the synaptic weight compare to the others. B: Implementation of the D-OR, note that we keep the same synaptic architecture and we only change the threshold of the transfer function.

We have named the first new computation the D-AND because it requires the activation of a dominant (D) input AND the activation of another input. The D-OR is the Boolean dual of the D-AND, i.e. obtained by replacing AND operations by OR, and vice versa. In this function, an activation of the dominant input alone triggers an output. The other way to trigger an output is to activate both X₁ and X₂ at the same time. These functions have a “dominant input“ – an input that is sufficient to make the output true (D-OR), respectively necessary to make the output true (D-AND).

In the present paper, we chose X₀ as the dominant input, but we could have picked X₁ or X₂. There is nothing comparable in the other three computations which treat all inputs identically.

An LTU (Figure 1) implements the computation by using synaptic strength. We employed synaptic weights with integer values to reflect their finite precision. Even if synaptic weights can take real values, a finite precision means a finite number of values, where each integer represents the index of this value. The weight and threshold values to implement a function are obviously not unique. For example, we could multiply all the weights by 2 and set the threshold to 6 (D-AND), or 4 (D-OR) and obtain the same results. Here we always use the lowest possible integer values for synaptic weights, and the corresponding lowest possible threshold.

Then, we wanted to implement the D-AND and D-OR computation in threshold units with non-linear dendritic subunits, as an abstraction of neurons with dendrites⁷.

We consider two types of non-linearities: a threshold function to model supra-linear summation; and a saturating function to model sub-linear summation (SLTU; see Methods). Both types of summation have been observed in dendrites. Dendritic spikes are a well-known example of supra-linear summation¹², while sub-linear summation can be observed in completely passive dendrites due to the intrinsic saturation of synaptic conductance⁹.

Figure 2 shows a minimal implementation of D-AND in a dendritic neuron, with a supra-linear summation in Figure 2A and a sub-linear summation (SLTU) in Figure 2B. In both cases, all synapses are of identical strength. However, note that in the supra-linear implementation in Figure 2A, the X₀ input connects to both dendrites. Therefore, as we define an input’s synaptic weight as the total effect it has in the final summation stage (analogous to depolarisation measured in the soma of a neuron), we have to consider the weight of X₀ as twice as high as the other inputs. Note that this makes this implementation similar to the implementation in an LTU (Figure 1A): the dominance of X₀ is expressed by a stronger weight. This is not the case for the sub-linear implementation in Figure 2B, where the synaptic weights are identical. Here, the dominance of X₀ is only expressed by its placement.

Since the D-OR is the dual of the D-AND, its implementation also follows the same structure as the D-AND, but with the sub- and supra-linear implementations reversed. In this case, the supra-linear implementation uses placement to implement the dominance of X₀ whereas the sub-linear implementation uses strength.

In real neurons, spiking and saturating dendrites both integrate inputs sub-linearly in a given range^9,12. Therefore, we will focus on D-AND in the following section and further explore the potential of threshold units with SLTUs as an abstraction of dendritic integration in neurons.

Figure 2. Two implementations of the D-AND in threshold units with non-linear sub-units (“dendritic neurons”).

Squares represent synapses and circles represent transfer functions and their respective threshold/saturation values. Note that the final transfer functions (“somatic integration”) are always threshold units, whereas the transfer functions of the sub-units (“dendrites”) are threshold functions as an example for supra-linear summation in A, and saturating functions as examples of sub-linear summation in B. A: D-AND Implementation using supra-linear summation. Note that X₀ has two synapses and therefore a synaptic weight of two. B: D-AND Implementation using sub-linear summation. Here, all inputs have identical synaptic weights.

Implementing the D-AND for an arbitrary number of input variables

In the previous section, we have limited our analysis to computations with three input variables. We will now extend the definition of the D-AND to an arbitrary number of input variables. As in the three-variable case, we will consider one input to be the dominant input (assumed to be X₀, without loss of generality). This input has to be activated together with at least one of the non-dominant inputs. Formally, we therefore define f_n(X ) as follows:

where X is the n-dimensional input vector with elements X₀. . . X_n.

We can implement this function in an LTU (Figure 3A), as well as in a SLTU (Figure 3B)

Figure 3. Extending the D-AND implementation to n inputs.

Synaptic weights are in squares, and transfer functions are in circles. A: Minimal D-AND implementation in an LTU. Note that this implementation requires a synaptic weight that is n − 1 times bigger than the smallest weight. B: Implementation in an SLTU with sub-linear summation.

Figure 4. A biophysical model sensitive to synapses’ spatial distribution.

A: A biophysical model with two dendrites and a soma (lines: dendrites, circle: soma). Coloured squares depicts synapses. The model has three equivalent synapses. One comes from an input encoding the shape (black square), and two situated on the other dendrite encode the colour (blue or green). B: Membrane voltage traces responding to either clustered (the two synapses activate on the same dendrite; aquamarine) or dispersed (the two synapses activate on distinct dendrites; black) synaptic activation at five distinct locations (dendrite where two synapses cluster at 350 µm, 250 µm, 150 µm, 50 µm, and soma). Note that at the point where two synapses cluster the clustered activation is larger than the dispersed activation, whereas in the soma the opposite is true.

Figure 5. A biophysical model implementing the Dominant AND (rate interpretation) Top: activity of the three input synapses, the two first synapses impinge on the same dendrite while the black one impinges on another.

Bottom: Eight somatic membrane responses depending on the active inputs. (gray: no synapse/only black/green/blue, green: black + green, blue: black + blue, aquamarine: green + blue, black: all inputs active). Note that this activity reproduces the truth table from Table 2.

Figure 6. A biophysical model implementing the Dominant AND (spike interpretation) Top:

The biophysical model receives input from three sources, where activation happens at regular intervals of 25 ms, with a random jitter of ±1 ms for each spike. We translate this activity into a binary pattern for each time bin of 25 ms. Bottom: The model’s membrane potential as measured in the soma. The response spikes implement the output of the D-AND computation as described in Table 2.

In the LTU implementation (Figure 3A), the D-AND of n variables requires that an input has a synaptic weight at least n − 1 times bigger than the other inputs, and the threshold has to grow accordingly.

We can summarise these observations in a proposition.

Proposition 1.To implement the D-AND, an LTU requires that an input has a synaptic weight n − 1 times bigger than the smallest synaptic weight.

Proof. The LTU must stay silent when X₀ is not active, even if X₁, X₂, . . . , X_n−1 are active. Therefore w₁+w₂+...+w_n−1 < Θ, thus Θ must be greater or equal than n × w_min with w_min the smallest synaptic weight.

Conversely, the output should be active as soon as X₀ is co-active with any other input X_j (for j ≥ 1). So w₀ + w_min ≥ Θ, this means w₀ + w_min≥ n × w_min, thus w₀ ≥ w_min(n − 1).

In contrast, Figure 3B provides a constructive proof that an SLTU can implement the D-AND with equal synaptic weights. In this implementation, the distinguishing feature of the dominant input is that it targets the second dendrite; the synaptic weights and the threshold do not have to change with the number of inputs. If one would only measure the response to single inputs at the “soma” (last stage of summation), the dominant input would be indistinguishable from the other inputs, despite its dramatically different importance.

We will see next how these insights transfer to a more realistic biophysical model.

Implementation of the D-AND in a biophysical model

To use an illustrative example, consider a predator of a certain shape (triggering input X₀) and either coloured green or blue (inputs X₁ and X₂). For an animal that is a potential prey, a stimulus should only trigger a fleeing response if the predator’s shape is accompanied by one of the correct colours. Neither its shape alone nor either or both of the colours without the shape should trigger it.

Figure 4A presents a single neuron biophysical model implementing this function. One synapse comes from an input encoding for the shape of an object, and this synapse connects to the upper dendrite. The two others encode for two different colours, either green or blue, and they connect to the lower dendrite. All the synapses, taken individually, produce the exact same depolarisation at the soma because we place them at the same distance (350 µm) and give them the same maximal conductance (20 nS).

We first look at the sub-threshold behaviour by disabling the sodium channels ($g_{Na}^{max}$ = 0). Figure 4B plots the voltage response at distinct locations on the lower dendrite, and finally in the soma. To probe the response we either activate a shape and one of the colours (black line Figure 4B), or both colours (aquamarine line). In both cases, we activate two synapses, but in the first case this activation is dispersed over the two dendrites whereas in the second case it is clustered on the lower dendrite. Despite activating the same number of synapses in both cases, and despite them all having the same strength, the depolarisation is markedly different. At the tip of the dendrite – where the synaptic stimulation takes place – a dispersed activation leads to a smaller depolarisation than the clustered activation. In the soma, we observe the opposite: the dispersed activation leads to a bigger depolarisation than the clustered activation. For the dispersed activation, we record a depolarisation of 9.3mV at the soma, whereas it is only 6.2mV for the clustered activation.

We can explain this observation by considering the synaptic driving force¹⁶. The synaptic current induced by the activation of the synapse depends on the distance between the membrane potential and the synapses’ reversal potential; when several inputs drive the membrane potential closer to the reversal potential (here 0mV, this driving force diminishes). The combined effect of multiple synaptic inputs is therefore smaller then what is expected from summing the individual effects. In other words, the dendrite performs sub-linear summation.

Note that we could have expected a difference of up to 100% between the clustered and dispersed stimulation in the soma. In the actual simulation, the difference is closer to 50%, showing the non-trivial interaction taking place at the soma.

This means that even if all synapses have the same impact on the soma individually; even if we have a complete synaptic democracy¹⁷, the relative placement of the synapses strongly influences the somatic response.

Based on the sub-threshold behaviour presented above, we will now show that we can implement the D-AND in a spiking neuron model. It is crucial to look at the supra-threshold behaviour as it is how the neuron communicates with the rest of the network. Moreover, back propagated action potentials might undermine the dendritic non-linearity disrupting the implementation¹⁸.

We can interpret Boolean inputs and outputs in different ways when we apply them to a biophysical spiking neuron model. Here, we will consider two interpretations. Firstly, we can think of an active input as corresponding to a continuous stimulation where the individual spikes arrive at random times, and of an active output as some spiking activity of the neuron (“rate interpretation”). Alternatively, we can think of active inputs as coincidentally arriving spikes within a certain time window, and accordingly of an active output as a single spike emitted in response (“spike interpretation”).

We present the model implementing the rate interpretation in Figure 5. We introduced this model earlier (Figure 4), except that it has now active sodium channels ($g_{Na}^{max}$ = 650mScm^–2). Each of its inputs (colours corresponding to the colours in Figure 4) activates 25 times according to a Poisson process.

Figure 5 displays, from top to bottom, the model’s responses in five different situations:

This figure thus presents the response of the neuron model to all non-trivial cases, we have only omitted the case without any input activation (and therefore without any output activity).

Finally, we show an implementation of the spike interpretation in Figure 6. This model is identical to the model shown previously (Figure 5), except for a slightly lower activation threshold of the sodium channels (V_T = −55mV instead of V_T = −50mV) to make it spike more easily. We discretize time into bins of 25 ms and decide randomly for each input whether it is active in each bin. If it is active, it activates at the beginning of the bin with a small temporal jitter (1 ms); inputs activating in the same bin therefore spike coincidentally. We can directly link these activations to Boolean variables that are either 0 (no spike) or 1 (spike). As Figure 6 shows, the neuron implements the D-AND and only spikes whenever the “shape input” (black) is active together with at least one of the “colour inputs” (blue and green).

We have shown that a biophysical model can implement the D-AND computation using a different strategy than the LTU. Each input has the same synaptic weight producing the same depolarisation at the soma. To distinguish between the inputs, the biophysical model uses location instead of strength: the dominant input (black) targets its own dendrite, while the two other inputs cluster on the same dendrite. With this strategy, the model can implement the D-AND. This implementation also works for two interpretations of the Boolean inputs and outputs – as elevated rates of spiking without temporal alignment, or as precisely timed coincident spikes.

Underlying data

All data underlying the results are available as part of the article and no additional source data are required.

Extended data

Code necessary to perform all simulations is available from: http://doi.org/10.5281/zenodo.4023943¹⁵.

License: MIT