week09_policy_II

• slides #1 (trpo)
• slides #2 (dpg)

Materials

This section covers some steroids for policy gradient methods, along with a cool general trick called

• Lecture on NPG and TRPO by J. Schulman - video
• Alternative lecture on TRPO and open problems by... J. Schulman - video
• Our videos: lecture, seminar(PyTorch) (russian)
• Original articles - TRPO, NPG

Practice

• TRPO:

• PPO:

More: Reinforcement learning in large/continuous action spaces

While you already know algorithms that will work with continuously many actions, it can't hurt to learn something more specialized.

• Lecture by J. Schulman - video
• Q-learning with normalized advantage functions - article, code1, code2
• Deterministic policy gradient - article, post+code
• Stochastic value gradient - article
• Embedding large discrete action spaces for RL - article
• Lecture by A. Seleznev, 5vision (russian) - video