All theories and concepts of machine learning are found here.
What is this post about? The training process of deep Artificial Neural Networks (ANNs) is based on the backpropagation algorithm. Starting with this post, and a few subsequent posts, we will lay the foundation of what a Multilayer Perceptron (MLP) is. We learn how to choose a proper activation function for MLPs and how the …
The Backpropagation Algorithm-PART(1): MLP and Sigmoid Read More »
What Will You Learn? In our previous post, we have talked about the meaning of gradient descent and how it can help us update the parameters of our Artificial Neural Network (ANN). In this post we will actually mathematically derive the update rule using the concept of gradient descent. We will also look at a …
The Gradient Descent Rule https://www.youtube.com/watch?v=gYqG4OT2Kj4 When training a model, we strive to minimize a certain error function (). This error function gives us an indication as to how well is our model doing on our training data. So, in general, the lower it is, the better our model is doing on the training set. Make …
What We Have Learned So Far … So far, we have learned that the Delta rule guarantees to converge to a model that fits our data the best! It just so happens that the best fit might be a terrible model but still it is the best that the Delta rule has been able to …
The Beauty that is the Delta Rule In general, there are 2 main ways to train an Artificial Neural Network (ANN). In our previous post , I have told you about the popular perceptron rule that has been around for a long time. We also said that the perceptron training rule is guaranteed to converge …
The Perceptron Training Rule It is important to learn the training process of huge neural networks. However, we need to simplify this by first understanding how a simple perceptron is trained, and how its weights are updated! Only then, will we be able to understand the dynamics of complicated and monstrous neural networks, such as …
A Quick Recap Hello everyone and welcome! In our previous post, we talked about the first algorithm that uses the “more-general-thank-or-equal-to” operation to smooth out the search in the hypothesis space: Find-S Algorithm. As a reminder, below you can see the steps in this algorithm: Like all the other algorithms, and I mean it when …
Concept Learning and General to Specific Ordering-Part(5) Read More »
A Quick Recap on our Last Post In our last post, we talked about the more-general-than-or-equal-to operation, which we denoted with ≥g and we said that in order for hypothesis hj to be considered more general than or equal to hypothesis hk, the following has to hold: Today, we will talk about a famous algorithm that can …
Concept Learning and General to Specific Ordering-Part(4) Read More »
General-to-Specific Ordering of Hypotheses In the last post, we said that all concept learning problems share 1 thing in common regarding their structure, and it is the fact that we can order the hypotheses from the most specific one to the most general one. This will enable the machine learning algorithm to explore the hypothesis …
Concept Learning and General to Specific Ordering-Part(3) Read More »
Introduction If we really wanted to simplify the whole story behind “Learning” in machine learning, we could say that a machine learning algorithm strives to learn a general function out of a given limited training examples. In general, we can think of concept learning as a search problem. The learner searches through a space of hypotheses …
Concept Learning and General to Specific Ordering-Part(1) Read More »
