Overview

• Each input and output is a single observation of the same dataset
• Input variables should be standardized or normalized and similar to the same range, these inputs represent one row of data as from different column/attributes
• Output values can be continuous, binary or categorical (with multiple output values)
• Weights get adjusted in synapses and they are crucial to adjust which signal has more importance when training the neural network
• All values get added up as a weighted sum and applies an activation function to decide if the signal should be passed on to the next neuron
• The cost function tells us the amount of error to update the weights, the lower the value of the cost function, the better

The difference between the output value and actual value is used to readjust the weights

Most Common Activation Functions

Threshold Activation Function - For binary answers

Sigmoid Activation Function - For probability answers

Rectifier Activation Function - For filtering answers (used a lot in hidden layers)