Parallel sgd resources

Basic Stochastic Gradient Descent

• Andrew Ng's Coursera videos on SGD
  • https://class.coursera.org/ml/lecture/preview/index
• scikit's explanation of SGD
  • http://scikit-learn.org/stable/modules/sgd.html

Interesting Links

• McDonald, Hall, and Mann (2010), "Distributed Training Strategies for the Structured Perceptron"
  • http://www.aclweb.org/anthology-new/N/N10/N10-1069.pdf
• Slides on McDonald Paper
  • http://www.slideshare.net/matsubaray/distributed-perceptron
• Alex Smola slides on parallel SGD work at Google
  • http://isl.stanford.edu/colloquium/Smola.pdf
• Jeff Dean talk on parallel SGD work at Google
  • http://techtalks.tv/talks/57639/