Spiking vs Linear Neuron Models - Pros and Cons

Neural models have trade offs with speed, efficiency of computation, and ease of use.

There are two major domains for neural models: Pattern recognition and computation, and robot control.

Spiking neurons have the following advantages:

  • Spikes that only occur once over many time ticks do not require getting memory into cache for the target neurons, and thus are computationally efficient.
  • Spiking neurons can do bursts.
  • Spiking neurons make triggering behavior in a robot simple to implement.
  • Spiking Neural models are inherently forward additive, imply Neural Hiding, and subsumptive architectures.
  • Subtle timing of spikes can result in information encoding by spike arrival times.
Drawbacks to Spiking Neurons:

  • Integration over several time ticks are required to get intermediate levels of activation.
  • The math is more complex; Charge, Activation, Burst and other variables must be tracked.
Linear Neuron Advantages: (Neurons that transfer activation levels multiplied by weights to target downstream neurons on every tick.)

  • Fast response to intermediate levels of activation. Single tick response.
  • Matches current Tensorflow style packages in some cases.
  • Arbitrary transformation of one cluster (Concept Map) to another is straight forward.  E.g. Detect edges with direct computation.
  • Can be used in a subsumptive style.
  • Can be implemented in tensor fashion for memory paging efficiency.
  • Simple math and only need activation level.
  • Somewhat higher level abstraction than Spiking neurons. A time tick can represent the integration over time of spiking neurons.
Disadvantages:

  • Memory locality of reference can be poor.  The reason the usual neural nets use tensors and GPU processing in orderly array.
  • Poor at random and semi-random neuron webs.
  • Poor it neural webs that are changing and evolving over time.
Linear Neuron models usually imply direct transfer of values forward along the neural net array, like in a Convolutional Neural Net.  Another model is a sum forward being the implied style where a single 'layer' or Concept Map may have several inputs that may just sum in, or may be selected dynamically.

A mix of both of the above models my imply some spiking neurons for triggering style activation of circuits, and linear for fast analog response.  In robots some inputs are inherently spiking such as touch sensors, while others may be more linear, such as a camera input.
Same for outputs.

Comments

Post a Comment

Popular posts from this blog

UE4 Asset Directory Listing Blueprint

Image
So I needed to be able to, in game, get listing of asset directories and assets in a folder style.  The purpose is to let me make on-the-fly menu selections of something like 10000 assets in game and do it fast. Turns out in UE4.25 you can do all of this in blueprints. The low level problem is that assets get packaged into .pak files when you ship the game.  Regular C++ style file and directory operations will not find any assets.  Instead there is the Asset Manager. Assets are bundled as assets and are not in a directory structure in the pak file.  The path to an asset is in effect just a string.  So the over all flow is... Load all the assets under one or more asset directories.  When the game is packaged what was something like Content/DefaultObjects/Trees becomes /Game/DefaultObjects/Trees. When you want to ask "What sub directories and assets are in /Game/DefaultObject/Trees you call my blueprint node.  It returns a result structure with a list of string for subdir
Read more

Subsumption and Neural Hiding - The 'Object Oriented Paradigm' of Artificial Intelligence

In computer programming Object Oriented Programming (OOP) gave programmers a neat way to compartmentalize programs, rather than having a mess of disorganized code and data.  One can neatly make an object, treat it as a black-box and reduce the cognitive load on the programmer.  Nifty. In reactive neural networks where one is taking world inputs and generating response outputs Subsumption vastly simplifies the work of designing the neural processing. From dictionary.com: verb (used with object),   sub·sumed,   sub·sum·ing. to  consider  or  include  (an  idea,   term,   proposition,   etc.)  as  part  of a  more   comprehensive   one. to  bring  (a  case,   instance,   etc.)   under  a  rule. to  take  up  into  a  more   inclusive   classification. Subsumption in the context of neural networks is where some reflex is designed as a response to inputs or other neurons and overrides 'lower level' reflexes.  Then some other reflex is desired so that can then be des
Read more

Self Cycling Asynchronous Neural Systems

Current neural network systems have a general paradigm of being given some input, and then producing a resulting output. A true general AI should act like a biological system and be self-sequencing. The neural system should just free-run, responding to inputs and creating outputs and learning. One method I am experimenting with in Cognate and NuTank is having a hard-wired set of concept maps (a N-dimensional set of neurons used to embody a concept, e.g. foveolar edge angles.) that take inputs from the external world and produce hard-wired reactions as outputs to control a real or simulated robot. Such a system also uses subsumption to aid in much easier design. This is the Concept Map System (CMS) for the purposes of this post. For now lets ignore the idea that the hard-wired concept map system might also be adapting and evolving over time simulating neroplasticity. One then has a large 3D array of neurons that are in acting as a pattern matcher on the entire CMS.  What this 3D a
Read more