Artificial Neural networks are composed of simple elements operating in parallel. These elements are inspired by biological nervous systems. As in nature, the network function is determined largely by the connections between elements. We can train a neural network to perform a particular function by adjusting the values of the connections (weights) between elements. Commonly neural networks are adjusted, or trained, so that a particular input leads to a specific target output. Such a situation is shown below. There, the network is adjusted, based on a comparison of the output and the target, until the network output matches the target. Typically many such input/target pairs are used, in this supervised learning, to train a network.
Batch training of a network proceeds by making weight and bias changes based on an entire set (batch) of input vectors. Incremental training changes the weights and biases of a network as needed after presentation of each individual input vector. Incremental training is sometimes referred to as “on line” or “adaptive” training.
An artificial neural network (ANN), usually called neural network (NN), is a mathematical model or computational model that is inspired by the structure and/or functional aspects of biological neural networks. A neural network consists of an interconnected group of artificial neurons, and it processes information using a connectionist approach to computation. In most cases an ANN is an adaptive system that changes its structure based on external or internal information that flows through the network during the learning phase. Modern neural networks are non-linear statistical data modeling tools. They are usually used to model complex relationships between inputs and outputs or to find patterns in data.
A literature review is part of a research project where a researcher researches on similar work to his or hers. This very important part of the research helps the researcher to find out how other researchers have tackled the problem he/she is attempting to solve. It gives insight on how to go about solving the problem at hand and provides information on available technologies and tools for solving the problem.
Perhaps the greatest advantage of ANNs is their ability to be used as an arbitrary function approximation mechanism that 'learns' from observed data. However, using them is not so straightforward and a relatively good understanding of the underlying theory is essential.
• Choice of model: This will depend on the data representation and the application. Overly complex models tend to lead to problems with learning.
• Learning algorithm: There are numerous trade-offs between learning algorithms. Almost any algorithm will work well with the correct hyperparameters for training on a particular fixed data set. However selecting and tuning an algorithm for training on unseen data requires a significant amount of experimentation.
• Robustness: If the model, cost function and learning algorithm are selected appropriately the resulting ANN can be extremely robust.
With the correct implementation, ANNs can be used naturally in online learning and large data set applications. Their simple implementation and the existence of mostly local dependencies exhibited in the structure allows for fast, parallel implementations in hardware.
The original inspiration for the term Artificial Neural Network came from examination of central nervous systems and their neurons, axons, dendrites, and synapses, which constitute the processing elements of biological neural networks investigated by neuroscience. In an artificial neural network, simple artificial nodes, variously called "neurons", "neurodes", "processing elements" (PEs) or "units", are connected together to form a network of nodes mimicking the biological neural networks — hence the term "artificial neural network".
Because neuroscience is still full of unanswered questions, and since there are many levels of abstraction and therefore many ways to take inspiration from the brain, there is no single formal definition of what an artificial neural network is. Generally, it involves a network of simple processing elements that exhibit complex global behavior determined by connections between processing elements and element parameters. While an artificial neural network does not have to be adaptive per se, its practical use comes with algorithms designed to alter the strength (weights) of the connections in the network to produce a desired signal flow.
These networks are also similar to the biological neural networks in the sense that functions are performed collectively and in parallel by the units, rather than there being a clear delineation of subtasks to which various units are assigned . Currently, the term Artificial Neural Network (ANN) tends to refer mostly to neural network models employed in statistics, cognitive psychology and artificial intelligence. Neural network models designed with emulation of the central nervous system (CNS) in mind are a subject of theoretical neuroscience and computational neuroscience.
In modern software implementations of artificial neural networks, the approach inspired by biology has been largely abandoned for a more practical approach based on statistics
and signal processing. In some of these systems, neural networks or parts of neural networks (such as artificial neurons) are used as components in larger systems that combine both adaptive and non-adaptive elements. While the more general approach of such adaptive systems is more suitable for real-world problem solving, it has far less to do with the traditional artificial intelligence connectionist models. What they do have in common, however, is the principle of non-linear, distributed, parallel and local processing and adaptation.
Submitted By :-
Introduction to Artificial Neural Networks
Neural networks : Introduction
Neural network: information processing paradigm inspired by biological nervous systems, such as our brain
Structure: large number of highly interconnected processing elements (neurons) working together
Like people, they learn from experience (by example)
Neural networks : Introduction
Neural networks are configured for a specific application, such as pattern recognition or data classification, through a learning process
In a biological system, learning involves adjustments to the synaptic connections between neurons
same for artificial neural networks (ANNs)
A new sort of computer
What are (everyday) computer systems good at... and not so good at?
Where can neural network systems help
when we can't formulate an algorithmic solution.
when we can get lots of examples of the behavior we require.
‘learning from experience’
when we need to pick out the structure from existing data.
Inspiration from Neurobiology
A neuron: many-inputs / one-output unit
output can be excited or not excited
incoming signals from other neurons determine if the neuron shall excite ("fire")
The synapse resistance to the incoming signal can be changed during a "learning" process 
The neuron calculates a weighted sum of inputs and compares it to a threshold. If the sum is higher than the threshold, the output is set to 1, otherwise to -1.
A simple perceptron
It’s a single-unit network
Change the weight by an amount proportional to the difference between the desired output and the actual output.
Δ Wi = η * (D-Y).Ii
Example: A simple single unit adaptive network
The network has 2 inputs, and one output. All are binary. The output is
1 if W0I0 + W1I1 > 0
0 if W0I0 + W1I1 ≤ 0
We want it to learn simple OR: output a 1 if either I0 or I1 is 1.
Artificial Neural Networks
Adaptive interaction between individual neurons
Power: collective behavior of interconnected neurons
Continuous process of:
Take new inputs
ANN evolving causes stable state of the weights, but neurons continue working: network has ‘learned’ dealing with the problem
From experience: examples / training data
Strength of connection between the neurons is stored as a weight-value for the specific connection
Learning the solution to a problem = changing the connection weights
No help from the outside
No training data, no information available on the desired output
Learning by doing
Example : -
Competitive learning: example
In this type, it is generally Winner takes all concept.
only update weights of winning neuron
Desired output of the training examples
Error = difference between actual & desired output
Change weight relative to error size
Calculate output layer error , then propagate back to previous layer
Improved performance, very common!
Where are NN used?
Recognizing and matching complicated, vague, or incomplete patterns
Data is unreliable
Problems with noisy data
Prediction: learning from past experience
pick the best stocks in the market
identify people with cancer risk
Predict bankruptcy for credit card companies
Pattern recognition: SNOOPE (bomb detector in U.S. airports)
Handwriting: processing checks
Not only identify the characters that were scanned but identify when the scanner is not working properly
infer grouping relationshipse.g. extract from a database the names of those most likely to buy a particular product.
e.g. take the noise out of a telephone signal, signal smoothing
Sensor data is noisy
Fairly new approach to planning
Strengths of a Neural Network
Power: Model complex functions, nonlinearity built into the network
Ease of use:
Learn by example
Very little user domain-specific expertise needed
Intuitively appealing: based on model of biology, will it lead to genuinely intelligent computers/robots?
Neural networks cannot do anything that cannot be done using traditional computing techniques, BUT they can do some things which would otherwise be very difficult.
Adapt to unknown situations
Robustness: fault tolerance due to network redundancy
Autonomous learning and generalization
Large complexity of the network structure
Future of Neural Networks
Most of the reported applications are still in research stage
No formal proofs, but they seem to have useful applications that work