Hello, I'm Ganesh. I'm building git-lrc, an AI code reviewer that runs on every commit. It is free, unlimited, and source-available on Github. Star git-lrc on GitHub to help more developers discover the project. Do give it a try and share your feedback for improving the product.
In the previous article, we discussed how backpropagation actually works. We also calculated weight and bias for all the layers.
We used soft plus activation function for our small neural network.
Now in this article let's use ReLU function for the neural network.
Defination of ReLU
It is a function which will return the input if the input is positive otherwise it will return 0.
f(x) = max(0,x)
In normal expression
f(x) = 1 if x > 0
0 if x <=0
Using ReLU in Neural Network
For First Layer we can calculate.
As we use 2 hidden neurons in the first layer we have to calculate for both the neurons separately.
x1 = ( input x weight1 ) + bias1
y1 = ReLu(x1)
Similarly calculating for second hidden neuron.
x2 = ( input x weight2 ) + bias2
y2 = ReLu(x2)
Conclusion
From both equation we can calulate the outputs of first layer and can pass it to the second layer.
Any feedback or contributors are welcome! It’s online, source-available, and ready for anyone to use.
⭐ Star git-lrc on GitHub