Overview
The RL Toolbox is a C++ based framework for Reinforcement Learning Algorithm. In the Design of the classes we focused our attention in the user-friendliness for the researcher so he can concentrate on the learning problem itself and not on the algorithm. The current version of the toolbox is 2.0b, which is a stable beta release. The class reference was updated for the most common classes, but for many new algorithm it is still missing. If you have questions to this undocumented algorithm write me an email. The manual is also still for the old version, but it explains a lot of the principle structures of the toolbox, which remained the same.Motivation
Our Motivation is to animate students, people of research and other interested people to use and play with reinforcement learning algorithm. Now you can easily generate your own learning problem, and test the different algorithm on them. The most difficult thing on reinforcement learning is to find the right parameters, state discretizations and reward models for the algorithm. So a tool is needed which provides you a comfortable framework for testing several state discretizations, reward models and algorithms without much effort.Basic Features
- The "Multi Learning" Principle: You can create one or more learning objects, which can use other algorithm, other state discretizations and other reward functions and all these learning objects can learn simultaneously from the same training trial.
- Learning from other Controllers: You can code you own controller for a learning proplem and then try to improve the controller with reinforcement learning.
- Different kind of Learning Algorithms There are many different kinds of learning algorithm implemented, TD-lambda Q-Learning learning, TD-Lambda V-Learning (TD learning also with continuous Time Residuals), Actor critic learning, Advantage Learning, model based reinforcement learning (prioritized sweeping, value iteration), policy search algorithm (PEGASUS and CONJPOMDP) and VAPS.
- An huge variety of possibilities to build your Q-Functions and states
- Tools for Logging and Error Recognition
- Learning of Semi MDP′s
- Hierarchical Reinforcement Learning
Introduction to the System
The manual of the new version is part of the Master Thesis : "Reinforcement Learning for optimal control tasks". It contains a comprehensive description of the class framework, a more detailed description of the implementation details is given in the class reference. Unfortunately not all classes are really good documented, but these classes are usually not needed by the "standard user". For those who don′t want to read the whole manual, there are some tutorials, see the example section.Getting and installing the RIL toolbox
You can download the RL Toolbox for Windows or for Linux here. The difference between the 2 versions is just the kind of library, the source code is supposed to work under windows and linux.Installing RL Toolbox
- VC++ .net: Settings for the project file: 数据挖掘研究院
- Project Properties / C/C++ / General / Additional Include Directories to: "RL Toolbox include dir"; "Torch3 include dir". e.g. "../external_include/RL Toolbox/";"../external_include/Torch3/"
- Project Properties / C/C++ / General / Detect 64-bit Portability Issues to: FALSE
- Project Properties / C/C++ / Language / Enable Run-Time Type Info to: YES /GR
- Project Properites / Linker / General / Additional Library Directories to "Library Path" e.g. "../external_library/"
- Project Properties / Linker / Input / Additional Depenencies to "Torch3.lib" "RL Toolbox.lib"
- Unix:
- add RL Toolbox includes to additional includes;
- add RL Toolbox path to additional library paths;
- add RL Toolbox to additional libraries
Contact
You have tested our Library? Please write me your opinion, your (good or bad) experenience with the toolbox or any suggestions. Feedback is always welcome in order improve the Library and also for a little motivation;). In case you find some bugs, it would be great if you could inform me. You can reach me at gerhard@igi.tu-graz.ac.atGetting involved
You are interested in using or have already used the RL toolbox for a learning problem? Then write us about your learning problem and your results so we can post it on our homepage.If you want to code some other learning algorithm not implemented in the RL Toolbox, then you can use the RL Toolbox framework and extend it with your learning algorithm. You will be able to use all other features or the RL toolbox for your algorithm. Write us if you are interested so we can integrate your code in the system. 数据挖掘研究院
