What is a neural network?

A NNNeural Network(Technology) is a mathematical object made of layers of simple functions.

Each unit takes a weighted sum of its inputs and passes the result through a non-linearity. Stack enough of those and you can approximate essentially any function.

Training adjusts the weights so the model's outputs match a target. The technique that makes this efficient is called backpropagation.

The surprise of deep learning is not the depth. It is that scaling the same idea keeps working.