schedule.md

layout	title	permalink	schedule
default	MATH 60630A - Machine Learning II<br>Deep Learning	/schedule	date topics readings Aug.<br>26, 30 name Introduction [<a href="assets/slides/Week1-Intro-En.pdf">En</a>] [<a href="assets/slides/Week1-Intro-Fr.pdf">Fr</a>] name Mathematics [<a href="assets/slides/Week1-Maths-En.pdf">En</a>] [<a href="assets/slides/Week1-Maths-Fr.pdf">Fr</a>] name Machine Learning Basics [<a href="assets/slides/Week1-ML-En.pdf">En</a>] [<a href="assets/slides/Week1-ML-Fr.pdf">Fr</a>] name Deep Learning Book name url Chap. 2 http://www.deeplearningbook.org/contents/linear_algebra.html name url Chap. 3 http://www.deeplearningbook.org/contents/prob.html name url Chap. 5 http://www.deeplearningbook.org/contents/ml.html date topics readings Sep.<br>9, 6 name Feedforward Neural Networks & Optimization Tricks [<a href="assets/slides/Week2-FFN&Regularization-En.pdf">En</a>] [<a href="assets/slides/Week2-FFN&Regularization-Fr.pdf">Fr</a>] name Deep Learning Book name url Chap. 6 http://www.deeplearningbook.org/contents/mlp.html name url Chap. 7 http://www.deeplearningbook.org/contents/regularization.html name url Chap. 8 http://www.deeplearningbook.org/contents/optimization.html date topics readings homeworks Sep.<br>16, 13 name url Introduction to Pytorch assets/slides/Pytorch Tutorial 2024A.pdf name url PyTorch Part 1 https://colab.research.google.com/drive/18eGdGZ1x6jQ47l-VgTmPvxPzIDA8KYSA?usp=sharing name url PyTorch Part 2 https://colab.research.google.com/drive/1-pCDTNpRGDvQdXUEsoHtYuyl0WgceHEj?usp=sharing name url Python Numpy Tutorial http://cs231n.github.io/python-numpy-tutorial/ name url Pytorch's official tutorial https://pytorch.org/tutorials/beginner/basics/intro.html name url Neural Network from Scratch https://medium.com/dair-ai/a-simple-neural-network-from-scratch-with-pytorch-and-google-colab-c7f3830618e0 name url Dive into Deep Learning https://github.com/dsgiitr/d2l-pytorch name HW1 [<a href="assets/hws/HW1 Instructions-En.pdf">En</a>] [<a href="assets/hws/HW1 Instructions-Fr.pdf">Fr</a>] date topics readings references Sep.<br>23, 20 name Convolutional Neural Networks & Recurrent Neural Networks [<a href="assets/slides/Week4-CNN&RNN.pptx">En</a>] [<a href="assets/slides/Week4-CNN&RNN-Fr.pptx">Fr</a>] name Deep Learning Book name url Chap. 9 http://www.deeplearningbook.org/contents/convnets.html name url Chap. 10 http://www.deeplearningbook.org/contents/rnn.html name url ResNet http://arxiv.org/abs/1512.03385 name url GRU https://arxiv.org/abs/1412.3555 name url DenseNet https://arxiv.org/abs/1608.06993 date topics readings references Oct.<br>1, Sep 27 name NLP Basis [<a href="assets/slides/Week5-DL4NLP-part1-En.pptx">En</a>] [<a href="assets/slides/Week5-DL4NLP-part1-Fr.pptx">Fr</a>] name url Word2Vec https://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf name url GloVe https://aclanthology.org/D14-1162.pdf name url SGNS https://papers.nips.cc/paper/2014/file/feab05aa91085b7a8012516bc3533958-Paper.pdf date topics readings references Oct.<br>7, 4 name Attention, Transformers [<a href="assets/slides/Week7-DL4NLP-part2-Fr.pptx">Fr</a>] [<a href="assets/slides/Week7-DL4NLP-part2-En.pptx">En</a>] name url The annotated Transformer (blog) https://nlp.seas.harvard.edu/annotated-transformer/ name url Transformer https://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf name url Rotary Position Embedding https://arxiv.org/abs/2104.09864 name url Relative Position Embedding https://arxiv.org/abs/1803.02155 name url ViT https://arxiv.org/abs/2010.11929 name url Reformer https://arxiv.org/abs/2001.04451 name url FlashAttention https://arxiv.org/abs/2205.14135 date topics homeworks Oct.<br>16, 11 name Course Project QA name HW2 [<a href="assets/hws/MATH_60630A_HW2_2024.pdf">En</a>] [<a href="assets/hws/MATH_60630A_HW2_2024__French_.pdf">Fr</a>] date topics readings references Oct.<br>28, 25 name Large Language Models I [<a href="assets/slides/Week8-DL4NLP-part3-French.pptx">Part-1 Fr</a>] [<a href="assets/slides/Week8-DL4NLP-part4-French.pptx">Part-2 Fr</a>] [<a href="assets/slides/Week8-DL4NLP-part3.pptx">Part-1 En</a>] [<a href="assets/slides/Week8-DL4NLP-part4.pptx">Part-2 En</a>] name url BERT https://arxiv.org/pdf/1810.04805 name url GPT-3 https://arxiv.org/abs/2005.14165 name url LoRA https://arxiv.org/abs/2104.09864 name url Scaling Law https://arxiv.org/abs/2001.08361 name url GPT in 60 Lines of NumPy https://jaykmody.com/blog/gpt-from-scratch/ name url XLNet https://arxiv.org/abs/1906.08237 name url UL2 https://arxiv.org/pdf/2205.05131 date topics readings references Nov.<br>04, 01 name Large Language Models II [<a href="assets/slides/Week9-DL4NLP-part4.pptx">En</a>] [<a href="assets/slides/Week9-DL4NLP-part4-French.pptx">Fr</a>] name url Chain-of-Thought https://arxiv.org/pdf/2201.11903 name url Prompt Engineering (Blog by Lilian Weng) https://lilianweng.github.io/posts/2023-03-15-prompt-engineering/ name url InstructGPT https://arxiv.org/abs/2203.02155 name url Automatic Prompt Engineer https://arxiv.org/abs/2211.01910 date topics readings references Nov.<br>11, 08 name Generative Models [<a href="assets/slides/deep-generative-models-1.pptx">En</a>] [<a href="assets/slides/deep-generative-models-1-French.pptx">Fr</a>] name url VAE https://arxiv.org/abs/1312.6114 name url Evidence Lower Bound ELBO — What & Why (Blog) https://yunfanj.com/blog/2021/01/11/ELBO.html name url GAN https://arxiv.org/abs/1406.2661 name url Diffusion Probabilistic Model https://arxiv.org/abs/2006.11239 name url beta-VAE https://openreview.net/pdf?id=Sy2fzU9gl name url CycleGAN https://arxiv.org/pdf/1703.10593 name url Latent Diffusion https://arxiv.org/abs/2112.10752 date topics readings references Nov.<br>18, 15 name Multi-Modality Learning [<a href="assets/slides/MML-English.pptx">En</a>] [<a href="assets/slides/MML-French.pptx">Fr</a>] name url DALL·E3 https://cdn.openai.com/papers/dall-e-3.pdf name url Sora https://arxiv.org/pdf/2402.17177 name url Kling https://kling.kuaishou.com/en name url Flamingo https://arxiv.org/abs/2204.14198 name url LLaVA https://arxiv.org/abs/2304.08485 name url InstructBLIP https://arxiv.org/abs/2305.06500 date topics readings references Nov.<br>25, 22 name Graph Representation Learning [<a href="assets/slides/Week12-Graph-English.pdf">En</a>] [<a href="assets/slides/Week12-Graph-French.pdf">Fr</a>] name url GCN https://arxiv.org/pdf/1609.02907 name url Graph Neural Networks Implementation Tutorial https://uvadlc-notebooks.readthedocs.io/en/latest/tutorial_notebooks/tutorial7/GNN_overview.html name url DeepWalk https://arxiv.org/pdf/1403.6652 name url LINE https://arxiv.org/pdf/1503.03578 name url GIN https://arxiv.org/abs/1810.00826

Course Schedule

{% for week in page.schedule %} {% endfor %}

Date	Topic	Suggested Readings	Reference	Homework
{{ week.date }}	{% for topic in week.topics %} {% include href item=topic %} {% endfor %}	{% for reading in week.readings %} {% include href item=reading %} {% endfor %}	{% for presentation in week.references %} {% include href item=presentation %} {% endfor %}	{% for homework in week.homeworks %} {% include href item=homework %} {% endfor %}