Training neural networks in practice

What actually changes when you go from plain gradient descent to training a real network - better optimizers, batching, dropout, and batch normalization.