Why Do We Use Gradient Descent for Optimization in Machine Learning?

Choosing a computational optimization technique

Photo by Marius Haakestad on Unsplash

I’ve noticed that most guides to machine learning jump right into gradient descent without exploring why we actually use this approach to optimize our models. From a mathematical perspective, it seems odd. Gradient descent relies on knowledge of the gradient to ‘slide…