Project Firestorm
Abstract
An artificial neural network utilizing spiking neurons is being developed, designed to interact with a real-world environment. Spikes travel from neuron to neuron and elicit naturally realistic postsynaptic potentials (PSPs), while the memory footprint and processing power usage are kept low. Hormonal substances, influencing the neurons on a larger time scale than spikes, are simulated through fluid dynamics on the graphical processing unit (GPU).
- Introduction
- Classic Neural Networks
- Spiking Neural Networks
- Real-time Potential
- Spikes and Postsynaptic Potentials
- Hormonal Substances
- Concluding: Input-Output Correlation
Introduction
Biological neurons encode information not only in if they fire, but also in the time when they fire. Spiking neural networks are designed to simulate this in order to take advantage of their ability to better recognize patterns. The goal of this research project is to develop an artificial neural network that uses spiking neurons that act in a naturally realistic way, yet can be simulated in real-time on a desktop computer. Several qualities of such a network are discussed. Spiking neurons have been chosen because they are more naturally realistic than classic artificial neurons, and are furthermore uniquely equipped to deal with time dependent data.
Classic Neural Networks
Classic neural networks emulate the functions of the brain by using very simple models of neurons. They typically use a basic artificial neuron that transforms the input by a weight and applies a function to generate an output. These artificial neurons are arranged in layers and for every input the corresponding output is calculated. After this, the weights are slightly altered to produce better outputs, either through supervised or unsupervised learning. Generally, datasets are used repeatedly on the same network to train it until a certain degree of learning is reached; after which the network can evaluate new data. As the timing of the input has no influence on the outcome of the network this type of neural network is in effect incapable to handle timed input.
Spiking Neural Networks
Spiking neural networks are a relatively new type of artificial neural networks, meant to more closely resemble the characteristics of real neurons. In reality a neuron only generates an output if its combined inputs make it cross a certain threshold; the output, called a spike, is either excitatory or inhibitory to the receiving neuron. While, similarly to the classic approach, the impact of the spikes on the receiving neuron is modulated by the 'weights' of the synapses, there is another source of information available, namely the timing of the spikes. This is what spiking neural networks are all about, and while the specifics of the simulation may differ (synapses for example may be simulated as simple weights, or complex neurotransmitter mechanisms), the overall idea remains the same: Spiking neurons use a continuous stream of inputs to generate a continuous stream of outputs where time (or timing) plays an important role.
Real-time Potential
Because time plays such an important role in spiking neural networks, it is vital that a spiking neural network is simulated real-time, so it can use the continuous inputs from a real-world environment to generate real-time outputs. However, since computers are inherently digital, continuity can only be simulated to a certain degree and often at the expense of large amounts of computing power. In our implementation of a spiking neural network we try to be as precise as possible for the things that are vital, and less precise for others, in order to achieve that the simulation can be real-time and powerful at the same time. The implementation requires a surprisingly small memory footprint, which makes the spiking neural network scale very well.
Spikes and Postsynaptic Potentials
Our spiking mechanism is simulated in a very natural way. Spikes elicit a natural postsynaptic potential (PSP) in the receiving neuron, modulated by the weight of the synapse. They also actually travel for a certain distance before reaching the connected neuron. This may seem obvious, but many spiking neural network implementations use 'spike trains', sets of time stamped spikes sent by a neuron. After which, the effect of these spike trains on the neuron are calculated by giving older spikes, having been sent out earlier by the neuron, a greater impact than newer spikes. Essentially an entire spike train is taken as input at once and continuous input is spoofed by weighting inputs based on timestamp, eliminating any real chance of real-time continuity.
Hormonal Substances
In addition to the spiking neurons in our artificial neural network, hormonal substances are simulated by using fluid dynamics. Neurons can secrete a differing amount of several types of substances (such as neuromodulators, neuropeptides and neuroactive hormones), which may for example affect other neurons locally, affect only certain neurons, or have an general effect on the entire network. These substances behave as a fluid and thus take time to flow around and elicit their desired effects. Fluid dynamics can be simulated on modern graphic cards using the unique architecture of graphical processing units (GPUs), and as such offload these computations from the CPU.
Concluding: Input-Output Correlation
Our more natural simulation makes the correlation between input and output harder to determine, but a good simulation is needed to produce meaningful results. The approach taken provides a better framework for research: an accurate, reliable and real-time simulation will help further our understanding of the workings of the brain, which will in turn allow the creation of better artificial neural networks, and so on.
Copyright © 2008 MeinMein®