- The Illustrated Transformer by Jay Allamar is an amazing tutorial if you want to understand how the transformer architecture works. I had great help from it while creating my own transformer network from scratch.
- Illustrated GPT 2 is another great tutorial by Jay Allamar. Although most of concepts that it explains come from the first tutorial, but nonetheless, the tutorial does a great job explaining the GPT 2 and "transformer decoder" model that only has decoder layers
- Natural Language Processing Demo from the Allen Institute
DeepMind's UCL course lectures went live on June 1. Apparently really good for intuition and recent research from the first looks of it as I can already see papers cited from 2020. Will keep you posted as I go through the videos one by one later this week Inshallah.
Technical University Munich's Introduction to Deep Learning Course (slides) went live in April. While I haven't really taken the lectures yet, but from the outlook of it and the topics covered, it seems that this is a beginner level course ideal for people starting their careers with deep learning. More like another version of Andrew Ng's deep learning specialization.
Introduction to Deep Leaning went live in February 2020. This series of lectures gives you an amazing perspective on cutting edge research with regards to reinforcement learning and some of the other relevant domains that are more au courant these days. More importantly, I found them to be really eye-opening in terms of the more daunting challenges such as Scent Detection, AGI and the likes. impact I wasn't able to spare time to watch the Reinforcement part though but I will return back to it once I have the time.
Customising your models with TensorFlow 2 is a Coursera which I believe is a great course that comes in handy for someone who wants to work with custom models and routines. The course has been optimized for Tensorflow 2 so that is a great advantage too. I would recommend this to someone who has been working with Tensorflow/Keras for sometime and now need an upgrade towards building custom-built models
Deep Learning A-Z is a comprehensive course for deep learning enthusiasts. I would say it is more for beginners rather than experts, but with a video content of over 20 hours, it touches many aspects of the practical side of deep learning with Tensorflow and related frameworks.
deeplearning.ai's flagship course that I feel is one of the best courses out there about deep learning. I think this course builds the foundations of deep leanring really well but one needs to do a tensorflow/keras/pytorch specialzation on top of it to really be able to implement this knowledge in the market.
I know that this is a lot to process here. One can never keep track of everything and the landscape keeps shifting but I am nonetheless, going to go ahead and create this list of some of the most amazing videos/channels on YouTube that have helped me throughout my difficult times.
While researching for some stuff, I stumbled accross these websites that provide with an excellent resource to track benchmark results down, sometimes with the code link on GitHub too.
This list lists some of the books that I find useful for refrencing when I am working with a Deep Learning problem and need to look back at some concepts for the task.
- Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems by Aurelien Geron
- Deep Learning by Ian Goodfellow and Yoshua Bengio and Aaron Courville
| Domain | Paper | Git Repo | Year | Comments |
|---|---|---|---|---|
| Speech Synthesis | End-to-End Adversarial Text-to-Speech | 2020 | Another Generative Modeling Speech synthesis paper after GAN-TTS. This time, its end-to-end. | |
| Image Recognition | Learning To Classify Images Without Labels | link, tutorial | 2020 | A seminal paper on Image Recognition that I believe will make waves throughout the research community in the coming years. They have beaten the SOTA methods by a margin of over 20% on CIFAR10 and CIFAR100-20 datasets |
| Image Recognition | FaceNet: A Unified Embedding for Face Recognition and Clustering | link | 2015 | A frontrunner (a bit older now) in the race of Face Recognition Systems |
| Speech Recognition | Generalized End-to-End Loss for Speaker Verification | link | 2017 | One of the best papers out there in the domain of speaker identification, verification. I think it is a bit old now, but its a very good read nonetheless |
| Image Recognition | High-Performance Large-Scale Image Recognition Without Normalization | link | 2021 | Major improvements (>8x) in reducing training time |
Tensorflow Tutorials are more for people who have just finished basic Tensorflow courses and are now ready to delve into more complex architectures that are seldom covered in those courses. They are really useful if you want to implement certain functionality out of the box. However, I did not find them easy to understand because they use Tensorflow almost completely (for performance reasons). Nonetheless, they are great for out-of-the-box implementations.
Tensorflow Advanced Tutorials gives you an insight into Gradient Tape, custom training loop and a bunch of things that you would need if you are working with advanced architectures. I find this tutorial really useful for when I want to be creative with neural nets and want to do more things custom than picking out-of-the-box implementatons.
Here is a list of the Forbes 50 AI companies