In the past three years, XLab has switched to Perceptron algorithms,
The entire infrastructure, products or companies owned by XLab are using the same algorithm
so what is Perceotron ?
Perceptron is a linear classifier (binary). Also, it is used in supervised learning. It helps to classify the given input data. But how the heck it works ?
A normal neural network looks like this as we all know
But how does it work?
The perceptron works on these simple steps
a. All the inputs x are multiplied with their weights w. Let’s call it k.
b. Add all the multiplied values and call them Weighted Sum.
c. Apply that weighted sum to the correct Activation Function.
For Example: Unit Step Activation Function.
so it's really amazing algorithm but the problem with it slowly , takes a lot of resources and we need to build new code for each company , projects or clients that we have
and Perceptron networks have several limitations. First, the output values of a perceptron can take on only one of two values (0 or 1) due to the hard-limit transfer function. Second, perceptrons can only classify linearly separable sets of vectors.
For that reason we developed some algorithms called Policy and Value Network
Consider any game in the world, input 🎮 given by user to the game is known as actions a. Every input (action) leads to a different output. These output are known as states s of the game.
From this, we can make different state-action pairs S = {(s0,a0),s1,a1),...,(sN,aN)} , representing which actions aN leads to which states sN. Also, we can say that S contains all the policies learned by the policy network.
The Network which learns to give a definite output by giving a particular Input to the game is known as Policy Network
For Example: Input a1 gives a state s1 (moving up) & Input a2 gives a state s2(going down) in the game.
Also, Some actions increase the points of the player lead to reward r .
Let’s look at some obvious symbols:
Usual Notations for RL environments
Why are we using Discount Factor γ
It is used as a precautionary measure (usually kept below 1). It prevent the reward r to reach infinite.
An infinite reward for a policy will overwhelm our agent & biased towards that specific action, killing the desire to explore unknown areas and actions of the game😵.
But how do we know which state to choose for your next move, eventually leading to the final round?
What is a Value Network?
The value network assigns value/score to the state of the game by calculating an expected cumulative score for the current state s . Every state goes through the value network. The states which gets more reward obviously get more value in the network.
Keep in mind that the reward is expected rewards, because we are choosing the right one from the set of states.
Now, the key objective is always to maximise the reward (aka Markov Decision Process). Actions that results in a good state obviously get greater reward than others.
Since any game is won by following a sequence of actions one after the other. The optimal policy π* of the game consists a number of state-action pairs that helps in winning the game.
The state-action pair that achieve most reward is considered as optimal policy.
The equation for optimal policy is formally written using arg max as:
Therefore, the optimal policy tells us which actions to take to maximises the cumulative discounted reward.
The optimal policy learned by the policy network knows which actions should be performed at the current state to get maximum reward.
Artificial Neural Network (ANN)
The term “Deep Learning” is a field involving Deep Neural Networks, meaning Artificial Neural Networks with one or more hidden layers. An Artificial Neural Network is a Directed Acyclic Graph (DAG) consists of connected layers of Artificial Neurons. These Artificial Neurons are also called “Perceptrons” and thus Artificial Neural Networks are sometimes called Multi-layer Perceptrons (MLP). The values will be fed into the Input Layer, which is the first layer of the network, and the result will come out from the Output Layer, which is the last layer.
We can think of Multi-layered Perceptrons as a voting scheme, in order to come out with a decision, each of the Perceptron in the input layer sends a weighted vote to Perceptrons in the next layer, and the next layer… until the vote is finalized in the perceptrons in the output layer. The activation functions are generally so steep in the middle to ensure the activation values lie in either end so that binary decision values are approximated. Here is a place where we can visualize how an Artificial Neural Network classifies data.
Forward Propagation & Activation Functions
In each of the layers, there are numbers of Perceptrons connected with each other. The mechanism between the Perceptrons can be defined by the following terms:
Activation (a): The Activation of a Perceptron is calculated by summing the Activations of Perceptron in the previous layer, added by a Bias, then transformed by an Activation Function.
Bias (b): Each time when a Perceptron receives a value, it will apply a bias to the value.
Weight (w): Activation value from the Perceptrons will be multiplied by a weight and summed together.
Activation Function (σ): Activation values of each perceptron are calculated from the Activation function.
There are several types of activation functions. Sigmoid Function (σ), often related to Learning Curves, is the most basic one. Each activation function has its specialties. Rectified Linear Units (ReLU) is the most popular one.
Each Perceptron will propagate its value into Perceptrons in the proceeding layer, and eventually, arrive at the output layer. This process is called Forward Propagation.
Back Propagation & Cost Functions
Forward Propagation is based on Weights, Biases and the Activation Function, but what determines these values? Activation Function is selected beforehand, but for a large neural network, it would be impossible to manually select the appropriate Weights and Biases.
In the field of Machine Learning, the models are supposed to “learn” from the data on its own, this learning process is also called “training”. Usually, the data is split into 2 different sets — the Training Set and the Test Set. The Training Set is used to “train” the model into a more mature state, and then the performance will be evaluated by the Test Set.
There are many different methods to “train” an Artificial Neural Network, but the most popular one is with Back Propagation.
Before Back Propagation, the Weights and Biases of the neural network are usually initialized randomly, in Normal Distribution. The neural network will then perform a Forward Propagation. Since the Weights and Biases are initialized at random, the result of the first Forward Propagation is usually way off. A Cost Function is then used to calculate the difference between the expected result and the output of the neural network. After the difference is calculated, it will be used to adjust the Weight and Biases of the previous layer. The process propagates backward in layers, and thus it is called “Back Propagation”.
Here is a more formal tutorial on Back Propagation, as it requires some advanced math to explain. An explanation of Neural Networks and code examples can be found here, where the author uses matrix operations to simulate a neural network in Python.
Convolutional Neural Network (CNN)
In order to process graphical data with better efficiency, Yann LeCun invented the Convolutional Neural Network in 1989. The network takes count of the spacial information on a 2D array. Convolutional Neural Network is also great for analyzing other 2D data where spacial information matters, which includes a chessboard.
Convolutional Neural Networks are constructed with 3 types of layers — Convolutional Layers, Pooling Layers, and Fully Connected Layers. These layers with different shape and sizes will have different performance on different subjects. Research on Convolutional Neural Networks usually involves adjusting those layers and their compositions to optimize performance on target data sets.
Convolutional Layer (conv)
Convolutional Layer usually appears as the first layer of a Convolutional Neural Network. These types of layers will scan through the source layer with a filter, and put the sum into the destination layer. Some filters are good at detecting edges and some are good in other tasks, more details on different kinds of convolutional filters and their applications in Computer Vision can be found here.
Pooling Layer (pool)
Pooling Layer iterate through the source layer and selects a specific value inside the bounded region. The value is typically maximum, minimum, average within the region. Reducing information into a smaller size is also called “downsampling”.
Fully Connected Layer (fc)
Fully Connected Layer is essentially a Multi-layer Perceptron, which is sometimes called “softmax” and essentially does something called “weighted sum”.
onvolutional Neural Network is most often used in the field of Computer Vision, which impacts many fields such as eCommerce, FinTech, Game Artificial Intelligence, Cancer Detection, Orthopedics, Psychiatry, WildFire Detection and many more. Here is an article on how Computer Vision is impacting eCommerce and another article on some Cool Frontend Frameworks in Computer Vision.
FASTER , BETTER , FLEXIBLE , ACCURATE
THIS IS OUR ALGORITHM
THIS IS US
THIS IS
Comentarios