What is a neural network?
The term deep learning refers to training neural networks. In this post I’d like to provide some basic intuitions on neural networks.
Let us consider an example where we are trying to predict the price of a house given its size. Let’s say we have a dataset with size of the houses and their corresponding price and we want to fit a function to predict the price of the house as a function of its size. In Linear Regression we try to draw a straight line to the available dataset.
But we know that the straight line can eventually be negative whereas the price of a house is always greater than zero. So, we try to bend the curve so that it ends at 0. We can think of the function we just fit as a simple neural network where we have a node (neuron) with input as size of the house and output as price. The function is called RECTIFIED LINEAR UNIT (RELU).
A larger neural network can be formed by stacking all the single neurons. Let us extend the housing price example to understand more about this concept. Let’s say we want to predict the price of a house not just by its size but also with other features such as the number of bedrooms which implies that family size also helps in predicting the price of a house. We also have features like zip code (postal code) which determine walkability and wealth which determines the quality of facilities such as schools and colleges.
Each of the circles (neurons) can be a RELU function or some other non-linear function. Most of the people tend to pay more looking at the things that matter to them. In this case the walkability, family size and quality of the schools can help predict the price precisely. In this example, ‘X’ is the four input features and Y is the output which is the price of a house.
All we need to do is provide the neural network with the input ‘X’ and the output ‘Y’ while training the neural network with several training examples. The neural network will automatically identify all the middle layers by itself. Each of the neuron (circle) takes input of all the four input features. So instead of saying that the first node represents family size that family size depends only on the size of the house and the number of bedrooms, we give the neuron all the input features (x1, x2, x3, x4) and it should figure out whatever it wants the node to be.
We say that the input layer and the hidden layer are densely connected because every input is connected to every one of these neurons(nodes).
So, I hope I gave you a basic understanding of what a neural network is and is it used to make predictions. In the next post we will look at some more examples of neural networks and supervised learning.