README.md

Deep Natural Language Processing

Table of contents

• Course 1
  • Lesson
    • Course introduction
    • General machine learning terminology
    • Deep learning history
    • Linear regression example
    • Loss function
    • Gradient descent (with its stochastic variation)
  • Practical work
    • Anaconda environment setup
    • Manipulation of PyTorch tensors
      • Creation
      • Indexing
      • Slicing
      • Shape manipulation
      • Combination
      • Aggregation
      • Broadcasting
      • Boolean logic and indexing
• Course 2
  • Lesson
    • Neuron definition
    • Neural network definition
    • Softmax activation function
    • Cross entropy loss
    • Neural network example
  • Practical work
    • MNIST image classification using Multi-Layer Perceptron
    • Model evaluation
• Course 3
  • Lesson
    • Backpropagation algorithm
    • Convolution layer
    • Pooling layer
    • Convolutional neural network example
  • Practical work
    • MNIST image classification using Convolutional neural network
    • Optimizer change experimentation
    • CIFAR-10 image classification using Convolutional neural network
• Course 4
  • Lesson
    • CIFAR-10 Convolutional neural network solution
    • Dropout layer
    • Modular design of neural networks
    • Learning rate decay
    • Data augmentation
    • Concept of Transfer learning
  • Practical work
    • CIFAR-10 using modular design, dropout, data augmentation and learning rate decay
    • Resnet transfer learning
• Course 5
  • Lesson
    • Transfer learning in practice
    • Neural networks for Natural Language Processing
      • Word embeddings
      • Text preprocessing methodology
      • Multi-layer perceptron text classifier
      • 1D CNN text classifier
      • RNN text classifier
  • Practical work
    • Multi-layer perceptron image autoencoder
    • Convolutional image autoencoder
• Course 6
  • Lesson
    • Transfer learning for NLP
      • Language modeling
      • Next sentence prediction
    • BERT
      • Attention mechanism
      • BERT embeddings
      • Types of tasks BERT can handle
      • BERT architecture
      • BERT finetuning example on sentiment classification
  • Practical work
    • BERT application to horoscope classification
    • Horoscope language modeling

References

• Google machine learning crash course
• Python Data Science Handbook by Jake VanderPlas
• PyTorch tutorials
• Deep Learning, NLP, and Representations by Christopher Olah
• Oxford Deep NLP course
• University of Illinois CS 521: Statistical Natural Language Processing
• Stanford CS224n: Natural Language Processing with Deep Learning
• Carnegie Mellon University CS11-747: Neural networks for NLP