I've trained the same neural network by changing only the optimizers and got some really interesting results. Same Model, Same Data, Varying Optimizers: What Actually Changes? Check it out in my latest blog below: https://medium.com/p/498c4a37748c...