Manuel Müller

Abstract

Autonomous systems gain more and more interest in research and society. However, they bring new challenges in safeguarding these systems. This contribution orders those new challenges and provides an overview of already existing concepts and approaches to solve those challenges of safeguarding autonomous systems. Moreover, existing metrics for safeguarding of autonomous systems are systematically reviewed. The presented concepts and approaches are of different domains, namely, ground, nautical and aerial vehicles, industrial robots and smart manufacturing, and medical and healthcare. Finally, the concepts and approaches are discussed concerning the following points: Main ideas, parallels existing in different domains, which ideas can be transferred from one domain to another, which high-level tasks were adressed and which assumptions were made.

Abstract

This paper presents a novel approach to ensure the quality of the Digital Twin models that modern Cyber-Physical Manufacturing Systems (CPMS) rely on. CPMS are configurable and intelligent. Environmental and system parameters change frequently. Thus, static models are inadequate. Autonomous mobile robots and the simulation of their movement are important elements of these CPMS. Based on our reinforcement learning-based methodology, we use these robots as an example to show how the Digital Twin automatically improves models that do not perfectly represent the physical asset, making it an intelligent Digital Twin. In our scenario, the behavior of the asset deviates from the simulated prediction, i.e., a simulation gap occurs. The presented approach closes this simulation gap through a three-step mechanism. First, it makes the simulated data and the real data comparable and synchronizes it. Second, it applies reinforcement learning to find patterns in the deviations between the simulated and real data. Third, it learns to compensate for them. The evaluation of this example shows promising results.

Abstract

In the production of the future, people and robots will work together in dynamic environments. Industrial mobile robots drive past workers and obstacles to the assemblers' workstations, pick up workpieces and deliver them to the next free workstation. To ensure reliable production, robots must carefully consider their actions based on the current situation to minimize losses. In order to reason about losses, it is necessary to identify loss scenarios that are likely to arise from the system's situation. In this paper, the authors propose a novel methodology for identifying these likely loss scenarios based on fault injectors. The fault injectors are trained using reinforcement learning. Based on a probability model of disruptive events (faults), the agents learn to destabilize the system by simulating such events. When a chain of these events leads to a loss, a loss scenario is found. The higher the probability of the loss scenario, the higher the reward for the agent. In this way, the fault injectors optimize for maximally likely loss scenarios. After describing the methodology, the authors propose a framework for systematically classifying different types of fault injectors and give advice on how to build them. Finally, the authors demonstrate reinforcement learning-based fault injectors on the path planning of the mobile robot platform Robotino.

Abstract

The emerging autonomous mobile robots promise a new level of efficiency and flexibility. However, because these types of systems operate in the same space as humans, mobile robots must cope with dynamic changes and heterogeneously structured environments. To ensure safety, new approaches are needed that model risk at runtime. This risk depends on the situation and is therefore a situational risk. In this paper, we propose a new methodology to model situational risk based on multi-agent adversarial reinforcement learning. In this methodology, two competing groups of reinforcement learning agents, namely the protagonists and the adversaries, fight against each other in the simulation. The adversaries represent the disruptive and destabilizing factors, while the protagonists try to compensate for them. The situational risk is then derived from the outcome of the simulated struggle. At this point, the system’s Digital Twin provides up-to-date and relevant models for simulation and synchronizes the simulation with the real asset. Our risk modeling differentiates the four steps of intelligent information processing: sense, analyze, process, and execute. To find the appropriate adversaries and actors for each of these steps, this methodology builds on Systems Theoretic Process Analysis (STPA). Using STPA, we identify critical signals that lead to losses when a disturbance under certain conditions or in certain situations occurs. At this point, the challenge of managing the complexity arises. We face this issue using training effort as a metric to evaluate it. Through statistical analysis of the identified signals, we derive a procedure for defining action spaces and rewards for the agents in question. We validate the methodology using the example of a Robotino 3 Premium from Festo, an autonomous mobile robot.

Abstract

Industrial mobile robots increasingly operate in dynamic and complex environments, resulting in risks that arise from the situation. Consequently, mobile robots need knowledge about the situative risk to make responsible decisions, e.g., when planning a trajectory. In this paper, the authors present a new approach to evaluate these situative risks at runtime. To compute the situative risk, the proposed approach uses multi-agent adversarial reinforcement learning. Unlike the usual approach that uses the self-play mechanism to harden the agents against disturbances, the approach directly uses the self-play to determine the confidence that the system can handle the current situation. Based on the evaluation of the strength of the perceived perturbations, the robot's digital twin calibrates the action space of the adversarial agents. In this way, the adversarial agents represent the disturbances that are likely to occur. From the simulation results of the competing agents, the digital twin infers the confidence with which the system can handle the situation. In the event that the system cannot handle the situation and crashes in the simulation, the digital twin calculates the probability that the simulated scenario will occur in reality. To calculate this probability, the authors propose the method of game-theoretic event trees. Combining the probability of an accident scenario with the damage caused by the simulated hazard, the results of situative risk are obtained. The approach is qualitatively evaluated using a case study. In this case study, a industrial mobile robot called Robotino is considered that connects workstations in a matrix production system. The case study shows that the approach is promising.

Abstract

The progress of digitalization and Internet of Things enables more and more complex, networked and powerful Cyber-physical Systems (CPS) operating in uncertain environments. This complexity and uncertainty, however, makes it unfeasible to model every aspect in advance. This causes the models to leave their scope and reach their capability limits. Specifically, in safe maintenance planning for highly-automated trucks, this fact causes waste of valuable resource, since maintenance models are often more rule-of-thumb (e.g. operation hours) than precise. In order to counteract this issue, we propose extending the digital twin concept by artificial intelligence such that the models become dynamic and adaptive. Having described the general approach and its architecture, we showcase and evaluate the approach in a highly-automated truck scenario.

Abstract

Advances in the domain of industrial automation systems and artificial intelligence research have brought autonomous systems into the focus of industrial automation research. However, as Hrabia et al. state, there is no common understanding of what autonomous systems are and what characterizes them: “there is still a lack of a widely accepted concept of autonomous systems or robots that contains a detailed definition according to different aspects and a clear distinction to other system concepts” 1. They conclude “the capability of adapting to the environment is strongly interconnected with autonomy”1. However, they do not derive a comprehensive definition. On the one hand, some systems are advertised as autonomous systems, although they are only highly automated systems. On the other hand, some autonomous systems are not labeled as such. In addition, a number of different properties have emerged that characterize an autonomous system as well as abilities attributed to autonomous systems. In order to provide a differentiated view on this topic, the relevant literature in the domain of industrial automation systems was investigated for this survey. The methodology of the investigation is described in Section 2. Starting with the historical development of the term autonomy, in Section 3, various definitions of autonomy are compared, and a summarizing definition is elaborated. From the definition in the narrow sense, the characteristics of industrial autonomous systems are derived (Section 3.1) and a distinction from intelligent industrial automation systems and classic automation systems is proposed (Section 3.2). Reviewing the abilities attributed to industrial autonomous systems in the literature, the abilities are categorized according to the characteristics (Section 4). In order to provide an example for the classification of systems as industrial autonomous systems based on the four derived characteristics, their manifestation is explored by three real-world examples (Section 5). Finally, Section 6 provides a conclusion and an outlook.

Abstract

Die Flexibilisierung von Produktionsanlagen ist notwendig, um in einem volatilen Marktumfeld erfolgreich sein zu können. Durch die Bereitstellung und Pflege geeigneter Schnittstellen und Betriebsmodi, sowie dem Umbau von Anlagen, kann auf geänderte Anforderungen an die Produktion reagiert werden. Für die Realisierung derartiger hochflexibler und rekonfigurierbarer Anlagen ist ein erhöhter Engineering- und Testaufwand notwendig. Jede zusätzliche Funktion samt aller damit verbundener Abhängigkeiten müssen vor Inbetriebnahme mehrfach von Herstellern, Integratoren und Betreibern überprüft werden. Damit der resultierende erhöhte Testaufwand auch zukünftig handhabbar bleibt, werden in dieser Veröffentlichung folgende Ansätze vorgestellt, um ein effizientes und wirkungsvolles Testen im Bereich der flexiblen Produktion zu ermöglichen: Zum einen lässt sich durch eine auf Richtlinien basierende Modularisierung von Anlagen und Komponenten der Testaufwand reduzieren. Auch sorgen Agile Teams für eine verbesserte Kommunikation zwischen Entwicklungs- und Testabteilungen. Überdies ermöglicht die Anwendung von Digitalen Zwillingen einen effektiveren Informationsaustausch zwischen den Stakeholdern und Modellbasiertes Testen ermöglicht Vorteile bei der Testautomatisierung. Alle Ansätze sind jeweils eigenständig wirksam, aber zusammengenommen ergeben sich zusätzliche Synergieeffekte.

Abstract

Digital Twins will change how systems and products are engineered and operated. Individual virtual representations of assets help to develop, maintain and change single components or whole factories. Aerospace engineering, product design and intelligent manufacturing are hot spots for the use of Digital Twins. Simultaneously, globalized markets lead to a growing awareness of dependability and quality, which increases the importance of verification and validation. The Digital Twin could prove to be key enabler for efficient verification and validation processes. This paper presents the results of the literature review of approaches that use Digital Twins for verification and validation purposes. Many solutions have been found for a wide range of challenges in various fields of application. This survey discusses the underlying methods and the elements of Digital Twins already in use. Most research approaches focus on simulations and three methodological clusters of approaches sharing similar ideas were identified.

Abstract

Die zunehmende Komplexität hochautomatisierter Systeme bringt neue Herausforderungen bei der Verwaltung ihrer Modelle entlang des gesamten Lebenszyklus des Systems mit sich - von der Kundenakquise über Engineering und Rekonfiguration bis hin zum Systemrecycling. Der Digitale Zwilling ist ein Konzept, welches über den gesamten Lebenszyklus eines Assets hinweg das Management dieser Modelle sicherstellen kann. Es unterstützt jedoch nicht die automatisierte Modellerweiterung. Hier setzt diese Arbeit an. Die Anreicherung des Digitalen Zwillings um Modellverständnis und KI-Algorithmen zur eigenständigen Modellerweiterung bildet die Grundlager des vorgestellten Konzepts. Über die intelligente Auswertung der Informationsmodelle -angereichert mit aktuellen Prozessdaten- erkennt der Digitale Zwilling, wenn Modelle an ihre Grenzen stoßen. Zwei mögliche Ursachen für diesen Sachverhalt werden genauer betrachtet: (1) es fehlt eine Fähigkeit oder Information (2) der Gültigkeitsbereich des Modells wurde verlassen. Für beide Zustände wird ein Verfahren vorgeschlagen, welches auf Basis kooperativer Information aus dem Wertschöpfungsnetzwerk automatisiert eine Lösung findet. Die Evaluierung des Konzepts anhand eines Szenarios aus der Logistik und aus der Produktion liefert vielversprechende Ergebnisse.