| Subject | File | Colab |
|---|---|---|
| Exercise 1 | exercise_1_qlearning_frozenlake.ipynb |  |
| Exercise 2 | exercise_2_mep_env.ipynb | |
| Exercise 3 | exercise_3_mep_continued.ipynb | |
| Subject | File | Colab |
|---|---|---|
| Exercise 1 | exercise_1_qlearning_frozenlake_solution.ipynb | |
| Exercise 2 | exercise_2_mep_env_solution.ipynb | |
| Exercise 3 | exercise_3_mep_continued.ipynb | |
If you prefer to work elsewhere than Colab you can download the notebooks, create an environment and run
pip install "rlmep[local] @ git+https://github.com/Mads-PeterVC/rlmep.git"
To install all dependencies and the supporting code.
These are not required or necessary for the tutorials, but if you'd like to reinforce your learning they are good next steps
- The bible of reinforcement learning is Sutton & Barto
- An excellent youtube series on RL by Mutual Information
- Concise write-ups on Lil'Log by Lilian Weng covering an overview on RL, a long post on policy gradient methods and even some implementations off some algorithms.
- There exists a variety of Towards Data Science, Medium etc articles on RL - but they often skip the most difficult steps in their explanations so don't be surprised if you dont learn much from them.
- Spinning Up contains a nice but mathy introduction to policy gradient methods.
- Original papers for algorithms can be helpful, but honestly they often express themselves in the most complicated way possible for their given algorithm - perhaps this increases the chance of being accepted for NIPS. You can look them up if you want to.
- The internet, with the correct search terms it is often possible to find good questions an answers from e.g. stack exchange on RL topics.
There are several rather good code libraries containing implementations of many of the best RL algorithms, these are usually written to interface with gynamsium environments. These are important to keep in mind for several reasons, in some cases they might provide an implementation that can be taken off the shelf. In other cases it might be nice to benchmark your own implementation against a known good implementation, and have access to the code of their implementations is also nice for debugging to see if some
concept has been misunderstood.
- StableBaselines has implementations of many algorithms.
- CleanRL have particularly clean and self-contained codes for many algorithms, that makes it easier to read and perhaps use as a starting point for a tweaked version for your use case.
- Spinning up has implementations of several Policy gradient methods.
- Gymnasium The standard for implementing environments.
- RLlib This is from the AnyScale team, that also are the developers of Ray, and advertise 'Industrial grade RL', this library offers MANY algorithms but it is not too easy to get started with but probably one of the highest performing libraries.
- Acme developed by Deepmind provides a toolbox for building agents (Though I have not really tried it)
- TorchRL Official Pytorch implementations of various pieces of RL algorithms, and some good looking tutorials.