How To Train Your Neural Network.

Courtesy of : DreamWorks Animation

Hey guys!!! Hope you are doing well.

We are back with our new article in Mind Tech. I think already you have a clue about the content of this article. Today we are going to tell you the process behind ‘training of a neural network’. Once you go through this article you will be able to know what happens in training of your neural network. Let’s start.

Learning process of NN…...

During the training time of neural network and after the training period, input data is fed to the input neurons of the input layer. Subsequently input data is passed to hidden neurons situated in between input neurons and output neurons. At that stage activation functions are applied and some neurons are activated by the activation function. These activated neurons function in the process of propagating the data throughout the network via connected channels. This is known to be ‘Forward propagation’ of neural net. If you need to know more about NNs, refer our article on NNs from here.

Above mentioned forward propagation continuously occurs until reaching the second last layer on the NN. In the last layer which is output layer, the neuron with largest probability value determines the predicted output of the NN.

Thereafter the actual output (y) expected from the NN is compared with the predicted output (y^) of the NN. At this stage error of prediction is determined when there is a difference among these 2 outputs. For this task of error prediction, a function named ‘Cost function’ is used.

Cost function is half of the squared difference between predicted output value and actual output value. Magnitude of the cost functions equal to the magnitude of error. Anyway our target is to reduce the cost function as much as possible. 

This information is transferred backwards through the network. The process that the network uses to teach itself can be termed as ‘Backward propagation’. Now by analyzing the computed error, the weights and biases are adjusted. This is just like a human learn from the mistakes done by himself/herself earlier and correct them by considering those lessons learned.

This cycle of forward propagation and backward propagation is continuously performed with multiple inputs. The process is continued until our weights are assigned such that the network can predict the output accurately. 

This whole process can be termed as learning of a neural network. 

Functioning of NN in nutshell…...

Neural network takes in labeled data. Then trains themselves to recognize the patterns in this data. After that predict the output for new set of similar unlabeled data.

Here the neural net should capable to generalize the specificity in data and apply that generalization in the next prediction. It can be described as follows.

What does our brain tell about these 2 pictures? Our brain tells that there is the same cat in both the pictures. That means our brain is capable of generalizing the 2 images although there are some sort of differences. So we expect that same kind of ‘generalization’ from our neural network. If it is so we know that performance of our neural network is in satisfactory level.

So next we are going to talk about some measures that can be taken in order to increase the performance of a neural network.

Improving performance of NN ……

•   Data Augmentation

Data Augmentation is a technique that is used to increase the size of data set such as by rotating, flipping, cropping, padding the same image. Such strategy increases the diversity of data and helps the neural net to get exposure to numerous versions of same image. 

• Introducing more layers in the network.

Instead of shallow networks, adding more layers results in improving the performance of the neural network. Learning with more layers brings an added advantage as it affects for the accuracy of final output too.

Conclusion

That’s the end of this article. Here we wanted to explain you about the learning process of a neural network along with other related concepts such as forward propagation, backward propagation, cost function. Without limiting to learning process of neural networks, we discussed how the neural network functions and also measures that can be taken to improve the performance of neural network.

More to come. Have nice days until next time. Good bye…….