Skip to content

Latest commit

 

History

History
 
 

week05_explore

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
README.md
README.md
 
 
deep_see.png
deep_see.png
 
 
q_learning_agent.py
q_learning_agent.py
 
 
replay_buffer.py
replay_buffer.py
 
 
und1.mp4
und1.mp4
 
 
und2.mp4
und2.mp4
 
 
week5.ipynb
week5.ipynb
 
 

README.md

Open In Colab

Slides - here

Exploration and exploitation

  • [main] David Silver lecture on exploration and expoitation - video
  • Alternative lecture by J. Schulman - video
  • Alternative lecture by N. de Freitas (with bayesian opt) - video
  • Our lectures (russian)
    • "mathematical" lecture (by Alexander Vorobev) '17 - slides, video
    • "practical" lecture '18 - video
    • Seminar - video

More materials

  • Gittins Index - the less heuristical approach to bandit exploration - article

  • "Deep" version: variational information maximizing exploration - video

    • Same topics in russian - video

  • Lecture covering intrinsically motivated reinforcement learning - video

    • Slides
    • Same topics in russian - video
    • Note: UCB-1 is not for bernoulli rewards, but for arbitrary r in [0,1], so you can just scale any reward to [0,1] to obtain a peace of mind. It's derived directly from Hoeffding's inequality.

  • Very interesting blog post written by Lilian Weng that summarises this week's materials: The Multi-Armed Bandit Problem and Its Solutions

Seminar

In this seminar, you'll be solving basic and contextual bandits with uncertainty-based exploration like Bayesian UCB and Thompson Sampling. You will also get acquainted with Bayesian Neural Networks.

Everything else is in the notebook :)