The Backpropagation Algorithm-PART(1): MLP and Sigmoid
What is this post about? The training process of deep Artificial Neural Networks (ANNs) is based on the backpropagation algorithm. Starting with this post, and…
Tag: gradientdescent
Deriving the Gradient Descent Rule (PART-2)
What Will You Learn? In our previous post, we have talked about the meaning of gradient descent and how it can help us update the…
The Derivative of Softmax(z) Function w.r.t z
What will you learn? Ask any machine learning expert! They will all have to google the answer to this question: “What was the derivative of…
Deriving the Gradient Descent Rule (PART-1)
The Gradient Descent Rule https://www.youtube.com/watch?v=gYqG4OT2Kj4 When training a model, we strive to minimize a certain error function (). This error function gives us an indication…
What is the Delta Rule? (Part-2)
What We Have Learned So Far … So far, we have learned that the Delta rule guarantees to converge to a model that fits our…
What is the Delta Rule? (Part-1)
The Beauty that is the Delta Rule In general, there are 2 main ways to train an Artificial Neural Network (ANN). In our previous post…