Acknowledgements

We gratefully acknowledge the contributions of Josh Achiam for creating the Spinning Up project. It provided us with an excellent template for building our algorithms that is easier to use for new RL researchers.