By using industrial robots, processes can be automated that are repetitive, require high precision and/or involve high risks for humans. With increasing product diversity and shorter production and innovation cycles, the demands on automation solutions are rising. In order to remain economically viable in the future, robots must on the one hand be able to react robustly to increasing process variations. On the other hand, despite increasing complexity, commissioning that is as time- and cost-efficient as possible is essential. Machine learning, in particular Deep Reinforcement Learning (DRL), is a promising way to enable robots for complex tasks where manual programming of a solution strategy is impractical. In our research, we are investigating the use of DRL methods to automate robotic applications. A central object of investigation is transfer learning methods to increase the training efficiency and robustness of learning agents by pre-training them in a simulation and transferring solution strategies to new tasks and robot models.

Since training DRL agents with real data can be very costly, we are investigating ways to use simulation environments to efficiently pre-train agents. This not only allows for faster, more parallelized, and collision-free training runs. Simulations also allow the incorporation of process variations that are difficult to operate in real use cases. A major drawback of simulations is that they deviate from reality. In our research, we especially investigate how to deal with these deviations and how to bridge them with transfer learning strategies.

Cross-domain transfer in robotics refers to the ability of robotic systems to transfer experience and capabilities from one task to another, even when those tasks are performed in different domains. This means that a robot that is capable of performing a particular task in a particular environment should also be able to transfer these capabilities to another environment or even to a completely different task. In our research, we are investigating methods for learning cross-task capabilities that significantly reduce the training required to learn individual tasks. Furthermore, we are researching the transfer of skills between different robot models, which creates additional flexibility in the use of industrial robots.

In the course of our research, we have developed and published the Python-based DRL framework Karolos. With Karolos, it is possible to combine different robots and tasks to quickly generate transfer learning scenarios. Furthermore, the framework enables the parallelization of simulations and state-of-the-art DRL algorithms so that experiments can be performed efficiently.

A major application of our research is the automation of assembly processes. Here, a robot must first grasp a component and then assemble it at the desired location with a precise, collision-free movement. Key research goals are to learn precise and adaptive robot control as well as how to handle a high variety of parts. In addition, we are working on the modularization of individual components, for example in the form of a separate module for pose detection of components for the preprocessing of camera images in the initial gripping process.