Hello, I'm Ganesh. I'm building git-lrc, an AI code reviewer that runs on every commit. It is free, unlimited, and source-available on Github. Star git-lrc on GitHub to help more developers discover the project. Do give it a try and share your feedback for improving the product.
In the previous article, we learned how to calculate the gradient of the last bias in a neural network.
Now we will explore how gradients flow through the entire network and how to calculate the weights of previous layers.
How to calculate weights
Now we will calculate weights of the previous layer
The challenge is that the loss function does not directly depend on these earlier weights.
For example, consider a weight (w1 and w2 are already calculated ) in a hidden layer.
Changing (w3 and w4):
- Changes the hidden neuron output.
- Changes the output neuron input.
- Changes the final prediction.
- Changes the loss.
So there is an indirect relationship between the weight and the loss.
This is exactly why we need the chain rule.
Conclusion
Similar to previous calculation we should also calculate for all weights and biases using chain rule and gradient descent.
Any feedback or contributors are welcome! It’s online, source-available, and ready for anyone to use.
⭐ Star git-lrc on GitHub