Dissecting Relu: A desceptively simple activation function
What is this post about? This is what you will be able to generate and understand by the end of this post. This is the…
Category: Mathematics of Machine Learning
Stochastic Approximation to Gradient Descent
What will you learn? The video below, delivers the main points of this blog post on Stochastic Gradient Descent (SGD): (GitHub code available in here) …
Back-propagation with Cross-Entropy and Softmax
What will you learn? This post is also available to you in this video, should you be interested 😉 https://www.youtube.com/watch?v=znqbtL0fRA0&feature=youtu.be In our previous post, we…
The Derivative of Softmax(z) Function w.r.t z
What will you learn? Ask any machine learning expert! They will all have to google the answer to this question: “What was the derivative of…