The first wave of deep learning applications has been characterised by hand-designed models and long, supervised training times, i.e. requiring manual data annotation. For illustration, neural networks used in image classification applications require millions of labelled example images.
The next generation of applications will need to learn and act with minimal human intervention. A key trend in current machine learning research is hence the question of unsupervised learning and autonomous decision making: How can software agents learn complex human tasks without having to program every step?
The mathematical framework for this task is reinforcement learning (RL). In RL, a software agent interacts with an environment and occasionally perceives rewards to learn an optimal behavioural policy. Deep reinforcement learning combines established reinforcement learning techniques with the ability of deep neural networks to capture structure in complex environments and generalise over large state spaces. While deep reinforcement learning is still in its infancy, researchers and practitioners are rapidly exploring new applications. We want to contribute to this development through open source components reflecting the specific requirements of reinforcement learning.
TensorForce is an open source reinforcement learning library on top of TensorFlow focused on providing clear APIs, readability and modularisation to deploy reinforcement learning solutions both in research and practice.
The main difference to existing libraries (such as rllab) is a strict separation of environments, agents and update logic that facilitates usage in non-simulation environments (see our architecture overview page for details). Further, research code often relies on fixed network architectures that have been used to tackle particular benchmarks. TensorForce is built with the idea that (almost) everything should be optionally configurable and in particular uses value function template configurations to be able to quickly experiment with new models. The goal of TensorForce is to provide a practitioner’s reinforcement learning framework that integrates into modern software service architectures.
TensorForce is actively being maintained and developed both to continuously improve the existing code as well as to reflect new developments as they arise.