Deep Learning Nanodegree Foundation Notes & Exercises

Notes, coding exercises and projects from Udacity's Deep Learning Nanodegree Foundation program.

Projects

PROJECT 1: Bike Sharing Neural Network (Repo)

PROJECT 2: Image Classification Convolutional Neural Network (Repo)

PROJECT 3: Simpsons Script Generation with Recurrent Neural Networks (Repo)

PROJECT 4: English to French Translation with Sequence to Sequence RNN (Repo)

PROJECT 5: Face Generation DCGAN (Repo)

Lessons

1 - Intro to Neural Networks

PROJECT 1: Bike Sharing Neural Network (Repo)

2 - Sentiment Analysis

Sentiment Analysis with Andrew Trask

Framing Problems for Neural Networks (Notebook)
Creating the Input/Output Data (Notebook)
Building the Neural Network (Notebook)
Making Learning Faster by Reducing Noise (Notebook)
Making our Network Train and Run Faster (Notebook)
Reducing Noise by Strategically Reducing Noise (Notebook)

Intro to TFLearn

Sentiment Analysis with TFLearn (Notebook)
Handwritten Digit Recognition with TFLearn and MNIST (Notebook)

3 - MiniFlow

Forward Propagation (Notebook)
Learning and Loss (Notebook)
Linear Transform (Notebook)
Sigmoid Function (Notebook)
Cost (Notebook)
Gradient Descent (Notebook)

4 - Intro to TensorFlow

Intro to TensorFlow Notes (Markdown) (Notebook)
TensorFlow Neural Network Lab (Notebook)

5 - Deep Neural Networks

Deep Neural Networks Notes (Markdown) (Notebook)

6 - Convolutional Neural Networks

Convolutional Neural Network Notes (Markdown) (Notebook)
PROJECT 2: Image Classification Convolutional Neural Network (Repo)

7 - Recurrent Neural Networks

Recurrent Neural Network Exercise (Anna Karenina Text Generation) (Notebook)
Sentiment Analysis Recurrent Neural Network (Notebook)
PROJECT 3: Simpsons Script Generation with Recurrent Neural Networks (Repo)

Resources

Lecture on RNNs and LSTM by Andrej Karpathy (CS231n - Stanford)
Understanding LTSM Networks by Christopher Olah (Blog)
The Unreasonable Effectiveness of Recurrent Neural Networks by Andrej Karpathy (Blog)
A Beginner's Guide to Recurrent Networks and LSTMs (Blog)
Recurrent Neural Networks in TensorFlow (Tutorial)
RNNs in TensorFlow (Blog)

8 - Embeddings and Word2Vec

Embeddings and Word2Vec (Skip-Gram) Exercise (Notebook)

Resources

Word2Vec Overview: The Skip-Gram Model by Chris McCormick (Blog)
Efficient Estimation of Word Respresentations in Vector Space by Mikolov et al. (Paper) [PDF] - (original Word2Vec Paper)
Distributed Representations of Words and Phrases and their Compositionality by Mikolov et al. (Paper) [PDF] - (improvements to orignial paper)
Word2Vec: NLP with Deep Learning with TensorFlow by Thushan Ganegedara (Blog)
TensorFlow Word2Vec Tutorial (Tutorial)

9 - TensorBoard

Viewing Graphs (Notebook)
Name Scopes (Notebook)
Inspecting Variables - TF Summaries (Notebook)
Choosing Hyperparameters (Notebook)

Resources

Hands-on TensorBoard Tutorial (TensorFlow Dev Summit 2017)
TensorBoard: Visualizing Learning (Tutorial)

10 - Weight Initialization

Comparing Different Weight Initializations on MNIST (Notebook)

Resources

11 - Transfer Learning

Transfer Learning with VGGNet (Notebook)

12 - Sequence to Sequence RNN

Character Sequence to Sequence (Notebook)
PROJECT 4: English to French Translation with Sequence to Sequence RNN (Repo)

13 - Reinforcement Learning

Playing Cart-Pole using Q-Learning (Notebook)

14 - Autoencoders

Simple Autoencoder on MNIST (Notebook)
Conventional Autoencoder on MNIST (Notebook)

15 - Generative Adversarial Networks

Intro to GANs

Generating Handwritten Digits with GANs (Notebook)

Deep Convolutional GANs

Batch Normalization Lesson (Notebook)
Batch Normalization Exercise (Notebook)
Deep Convolutional GAN on Street View House Numbers (SVHN) Dataset (Notebook)
PROJECT 5: Face Generation DCGAN (Repo)