Introduction to Neural Network

Understanding the concept of neural network

In this tutorial, we will guide you to what ‘Neural Network’ is. We will basically use the feed-forward network. Also there will be explanations such as why this mathematical model works, and what learning means when using this model.

What is ‘Neural Network’?

‘Neural Network’ is a simple mathematical representation of the human brain. In specific, it is a network of multiple neuron models that are mathematically modeled. The diagram below represents a general neural network model.

The round symbols are called ‘units’ (a.k.a perceptron), and the squares that are bundling the units are called ‘layers’. When units receive mupltiple 'inputs', each input signal is multiplied by the 'weights' for each input path, and then summed up. After summing up, the variables goes through the ‘activation function’, which decides how much 'output' is necessary, then outputs the signal, which becomes the input for the next unit. As mentioned, this flow of signals, which flows signals from the input to the output, is also known as ‘feed-forward’.

If you have layers and units inside your model, then that is neural network. There is no standard way of building the neural network, so how much layers and neurons exist can vary from designers. Like CNN (Convolutional Neural Network) and RNN (Recursive Neural Network), you can also put convolutional layers delaying operations inside the model and you can still call it a neural network.

Why is this mathematical model so beneficial?

Some of you may have thought ‘Why is this model so good? Its just a network of functions that sums and process signals!’ The advantage of this model is that by adjusting the weight parameters, you can obtain functions that are very difficult to define, such as drawing complex curve lines. You can even identify functions that have difficult math property. (Here, identify means to obtain the input and output relation, not clarification.) This process of identifying a certain function is also called ‘learning.’ When learning, the ‘weight’ parameters adjust so that the input and output of the target function are the same. As a result, we obtain functions that are necessary.

Will it be learning if we have no target function?

Mostly, when learning with neural network, you will normally use ‘input’ as learning variable and ‘output’ as answer variable. For example, suppose you have the ‘precipitation percentage’ as the output, and the ‘temperature’, ’atmospheric pressure’, ’humidity’ as the input data. From the definition of learning described above, learning is a process of identifing a (target) function. However, if there is a relation between the input and output data, even if there are no clear mathematical representation that describe the relationship between the input and output set, we can assume that there is a function. Thus, we can also call this process ‘learning’.

How does learning work?

How does weight adjust in order to learn certain functions? There is an update method that uses a powerful calculation method called ‘back propagation.’ Because of the unique structure that neural network has, we can apply this method to it. Detailed description will not be posted on this tutorial, but for now, just know that with this calculation, we are able to update the weights to learn certain functions.

Training and Testing sets

Just because we train using certain sets of data doesn’t mean its valid as a model. Training is the process where the weights are updated using certain datasets. In this case, training is the same as learning. For example, we can train a model with datasets between 1/1/2017 to 3/31/2017 and might score the ‘precipitation probability’ badly when validating with datasets between 5/1/2017 to 5/30/2017. In this case, the model trained for this case is not a worthy model for prediction, and the training method will have to be rethought of. For this reason, we always need a testing set when learning a model. This is how we validate if a model has learned or not.

Summary

‘Neural Network’ is a simple mathematical representation of the human brain constructed with ‘units’(perceptron) and ‘layers.’ When building a model, you can put convolution functions or delay functions between layers and will still be called a neural network. By adjusting the ‘weight’ parameters, we can achieve difficult functions, such as functions that can draw complex curve lines. By adding the ‘learning’ process, we can update the weights automatically and learn the target function. Learning is achieved by using ‘back propagation’ calculation. When learning, we will always need a training and testing function to evaluate whether the model is valid or not.