Posts

Showing posts with the label Convolutional Neural Networks

Evolution and Neural Systems

What is the purpose of a brain and how does that effect the evolution of design? Survival It is obvious that the reason any animals have neural systems is because the neural system provides a survival advantage. How does a brain provide an advantage? The simplest neural systems are in effect hard-wired with little or no learning. These simple systems provide reactions or reflexes to environment changes that may be harmful or beneficial.  Attraction toward food or mates, and fleeing from predators is very valuable. As neural systems become more complex and have learning on can ask: What is being learned? The answer is often stated as "better behaviors" and such. A better formulation would be: Brain impart a survival advantage by predicting the future state of the  environment  and pre-reacting to the environment. Here are several examples at varying levels of intelligence: A mouse is hungry and predicts a future state that includes food, and...

Cortical Neuron Model - SRS

In a Subsumptive Regular System (SRS  here )  the  regular array of neurons (cortex) that has the goal of predicting the future state of the subsumptive (cerebrum) system . If the match is strong on the present state of the subsumptive system then the stimulus of the subsumptive system into the predicted state is strong.  In the event of sudden changes in the actual external environment one wants to ignore the predicted state and react to the immediate environment. This has an evolutionary advantage of pre-reacting to the environment and getting a jump on surviving the future state. This document calls out the various axis of selection for algorithms.  Research has yet to show where the decision points are optimal. There are several decision points on the algorithms to select: Connectivity from the Subsumptive to the Cortex The subsumptive part of the SRS is hard-wired to take inputs, compute concept mappings (abstractions) of the external environment an...

Spiking vs Linear Neuron Models - Pros and Cons

Neural models have trade offs with speed, efficiency of computation, and ease of use. There are two major domains for neural models: Pattern recognition and computation, and robot control. Spiking neurons have the following advantages: Spikes that only occur once over many time ticks do not require getting memory into cache for the target neurons, and thus are computationally efficient. Spiking neurons can do bursts. Spiking neurons make triggering behavior in a robot simple to implement. Spiking Neural models are inherently forward additive, imply Neural Hiding, and subsumptive architectures. Subtle timing of spikes can result in information encoding by spike arrival times. Drawbacks to Spiking Neurons: Integration over several time ticks are required to get intermediate levels of activation. The math is more complex; Charge, Activation, Burst and other variables must be tracked. Linear Neuron Advantages: (Neurons that transfer activation levels multiplied by weight...

Avalanche Pattern Recognition

One feature of an SRS (see previous post) is that the 3D regular array of neurons (the cortex) can recognize a fragmented pattern in the subsumptive system, stimulate the subsumptive system into that pattern to a small degree, and then on the next time tick, recognize the now stronger pattern and repeat the process.  The entire system will rapidly avalanche into the state where the pattern is fully matched and the subsumptive system is fully in the match state. This system will then be locked into that state. Several factors can get the system out of the state and allow 'thinking' to continue. The pattern matcher will recognize the locked in state and begin predicting the next state. If one simulates the neurons getting 'tired' or 'bored' they can stop matching the locked in state and more easily match to the next state.  I call this Neural Fatigue.  An analog in biological neurons is the refractory period (maybe). Some external stimulus to the system li...