Hi manny,
The 2008 patent is of great value in that the main claim is very broad in scope and is directed to ML:
US2010076916A1 Autonomous Learning Dynamic Artificial Neural Computing Device and Brain Inspired System 20080921
A
n information processing system intended for use in artificial intelligence, consisting of a plurality of artificial neuron circuits connected in an array, comprising:
a first plurality of dynamic synapse circuits, comprising a means of learning and responding to input signals by producing a compounding strength value simulating a biological Post Synaptic Potential,
a temporal integrator circuit that integrates and combines individually simulated Post Synaptic Potential values over time, and thus constitutes an artificial membrane potential value,
a second plurality of dynamic soma circuits each capable of producing one or more pulses when the integrated membrane potential value has reached or exceeded a stored variable threshold value,
That said, there are a number of subsequent developments which greatly improve performance, chief of these (until TeNNs) is N-of-M coding which greatly increased sparsity and reduced power usage and latency.
Another landmark patent is ...
US11468299B2 Spiking neural network 20181101
View attachment 61140
A
neuromorphic integrated circuit, comprising:
a spike converter circuit configured to generate spikes from input data;
a reconfigurable neuron fabric comprising a neural processor comprising a plurality of spiking neuron circuits configured to perform a task based on the spikes and a neural network configuration; and
a memory comprising the neural network configuration, wherein the neural network configuration comprises a potential array and a plurality of synapses, and the neural network configuration defines connections between the plurality of spiking neuron circuits and the plurality of synapses, the potential array comprising membrane potential values for the plurality of spiking neuron circuits, and the plurality of synapses having corresponding synaptic weights,
wherein the neural processor is configured to:
select a spiking neuron circuit in the plurality of spiking neuron circuits based on the selected spiking neuron circuit having a membrane potential value that is a highest value among the membrane potential values for the plurality of spiking neuron circuits;
determine that the membrane potential value of the selected spiking neuron circuit reached a learning threshold value associated with the selected spiking neuron circuit; and
perform a Spike Time Dependent Plasticity (STDP) learning function based on the determination that the membrane potential value of the selected spiking neuron circuit reached the learning threshold value associated with the selected spiking neuron circuit.
... which showcases my all-time favourite drawing. As you can see, this has a priority of 2018, so it will last well into the 2030s.
CNN2SNN is certainly a vital commercial development, but, for me, not as exciting as N-of-M or TeNNS.
This patent introduced a number of "minor" improvements:
WO2020092691A1 AN IMPROVED SPIKING NEURAL NETWORK 20181101
[0038] But conventional SNNs can suffer from several technological problems. First, conventional SNNs are unable to switch between convolution and fully connected operation. For example, a conventional SNN may be configured at design time to use a fully-connected feedforward architecture to learn features and classify data. Embodiments herein (e.g., the neuromorphic integrated circuit) solve this technological problem by combining the features of a CNN and a SNN into a spiking convolutional neural network (SCNN) that can be configured to switch between a convolution operation or a fully- connected neural network function. The SCNN may also reduce the number of synapse weights for each neuron. This can also allow the SCNN to be deeper (e.g., have more layers) than a conventional SNN with fewer synapse weights for each neuron.
Embodiments herein further improve the convolution operation by using a winner-take-all (WTA) approach for each neuron acting as a filter at particular position of the input space. This can improve the selectivity and invariance of the network. In other words, this can improve the accuracy of an inference operation.
[0039] Second, conventional SNNs are not reconfigurable. Embodiments herein solve this technological problem by allowing the connections between neurons and synapses of a SNN to be reprogrammed based on a user defined configuration. For example, the connections between layers and neural processors can be reprogrammed using a user defined configuration file.
[0040] Third, conventional SNNs do not provide buffering between different layers of the SNN. But buffering can allow for a time delay for passing output spikes to a next layer. Embodiments herein solve this technological problem by adding input spike buffers and output spike buffers between layers of a SCNN.
[0041] Fourth, conventional SNNs do not support synapse weight sharing. Embodiments herein solve this technological problem by allowing kernels of a SCNN to share synapse weights when performing convolution. This can reduce memory requirements of the SCNN.
[0042] Fifth, conventional SNNs often use l-bit synapse weights. But the use of l-bit synapse weights does not provide a way to inhibit connections. Embodiments herein solve this technological problem by using ternary synapse weights. For example, embodiments herein can use two-bit synapse weights. These ternary synapse weights can have positive, zero, or negative values. The use of negative weights can provide a way to inhibit connections which can improve selectivity. In other words, this can improve the accuracy of an inference operation.
[0043] Sixth, conventional SNNs do not perform pooling. This results in increased memory requirements for conventional SNNs. Embodiments herein solve this technological problem by performing pooling on previous layer outputs. For example, embodiments herein can perform pooling on a potential array outputted by a previous layer. This pooling operation reduces the dimensionality of the potential array while retaining the most important information.
[0044] Seventh, conventional SNN often store spikes in a bit array. Embodiments herein provide an improved way to represent and process spikes. For example, embodiments herein can use a connection list instead of bit array. This connection list is optimized such that each input layer neuron has a set of offset indexes that it must update. This enables embodiments herein to only have to consider a single connection list to update all the membrane potential values of connected neurons in the current layer.
[0045] Eighth, conventional SNNs often process spike by spike. In contrast, embodiments herein can process packets of spikes. This can cause the potential array to be updated as soon as a spike is processed. This can allow for greater hardware parallelization.
[0046] Finally, conventional SNNs do not provide a way to import learning (e.g., synapse weights) from an external source. For example, SNNs do not provide a way to import learning performed offline using backpropagation. Embodiments herein solve this technological problem by allowing a user to import learning performed offline into the neuromorphic integrated circuit.
The TeNNs patents:
WO2023250092A1 METHOD AND SYSTEM FOR PROCESSING EVENT-BASED DATA IN EVENT-BASED SPATIOTEMPORAL NEURAL NETWORKS 20220622
WO2023250093A1 METHOD AND SYSTEM FOR IMPLEMENTING TEMPORAL CONVOLUTION IN SPATIOTEMPORAL NEURAL NETWORS 20220622
View attachment 61141
View attachment 61142
D
isclosed is a neural network system generally relates to the field of neural networks (NNs). In particular, the present disclosure relates to event-based convolutional neural networks (NNs) that are trained to process spatial and temporal data using kernels represented by polynomial expansion. The event-based convolutional neural networks (NNs) are spatiotemporal neural networks. According to an embodiment, an explicit temporal convolution capability is added through Temporal Event-based Neural Networks (TENN) models, or TENNs in the spatiotemporal neural networks. The TENNs includes a plurality of temporal and spatial convolution layers that combine spatial and temporal features of data for low-level and high-level features. The TENNs as disclosed herein are configured to perform in a buffer mode and recurrent mode that effectively learns both spatial and temporal correlations from the input data.
The FIFO (first-in - first-out) buffer acts like a data conveyor belt which facilitates comparison of incoming data with previously temporarily stored data.
. Promises were made, and they weren't achieved.
