An overview of gradient descent optimization algorithms

Recently, a post that summarize almost every gradient descent optimization algorithms used in deep neural network has been created by Sebastian Ruder at:
http://sebastianruder.com/optimizing-gradient-descent/
Thanks for his effort, it is helpful.