Home » Technology » Scientific Computing » More on Neural Networks

More on Neural Networks

© Adam Hughes

In the last article, the idea of neural networks was introduced with an eye toward understanding how they might eventually (or even currently) be used to tackle some of the more daunting challenges in scientific computing. Before we look at some of the proposed applications of this technology next week, it'll probably be helpful to become familiar with the basic architecture and characteristics of neural networks.

The human nervous system, which, of course, includes the brain, can be simplisticly viewed as consisting of neurons connected to each other to form a very complicated network. Connected neurons communicate with each other by sending signals back and forth. The impetus for a neuron to send a signal is the stimulation due to signals being received from other neurons. If the stimulation over a short period of time reaches some threshold value inherent to the neuron, it then sends its signal to all neurons it's connected to. The exact message seen by the receiving neurons depends on the nature of the message and the weight of the connection between the neurons. This weight determines how significant the incoming message is treated as being in the overall scheme of things.

This is the basic system which neural networks must be made to emulate, but there is a fundamental difficulty in determining the correct connectivity of a system. For a neural network to function correctly, it must have connections between the appropriate neurons, and these connections must be assigned the proper weights. As the problem to be solved becomes more complicated and the neural network necessarily grows larger, this can develop into a very tricky situation. To complicate matters, the connections (and their weights) in a viable network will undergo constant change to adapt to the problem.

Because of the complicated set-up described above, neural networks must be able to "learn." One of the common ways this is accomplished is to start the network in some initial configuration designed to solve a problem with a known solution. If the network makes a mistake, it is told so, and the connection weights are changed accordingly. This process is repeated until the desired outcome is achieved. At this point, the neural network has "learned" how to solve a certain type of problem and should be able to handle something similar in the future. (Note : a connection with a weight of zero is identical to there being no connection at all).

The general ideas presented above are, of course, not necessarily easily implemented. There have been a variety of neural network architectures proposed and implemented, each usually designed to treat one or more classes of problems.