What is gradient descent?

Gradient descent is a method for mathematical optimization. It is used to train neural networks, which lie at the heart of modern machine learning. In gradient descent, a network is repeatedly made to produce outputs and then adjusted in the direction that would most have improved those outputs.

3Blue1Brown’s videos offer a good illustration of the functioning of a neural network and of how gradient descent is used to train it.