M. Müller, T. Jung, N. Jazdi, und M. Weyrich, „Safeguarding autonomous systems: emerging approaches, assumptions and metrics – a systematic literature review“, 11th IFAC Symposium on Fault Detection, Supervision and Safety for Technical Processes, 29 July 2022, 2022.
ZusammenfassungAutonomous systems gain more and more interest in research and society. However, they bring new challenges in safeguarding these systems. This contribution orders those new challenges and provides an overview of already existing concepts and approaches to solve those challenges of safeguarding autonomous systems. Moreover, existing metrics for safeguarding of autonomous systems are systematically reviewed. The presented concepts and approaches are of different domains, namely, ground, nautical and aerial vehicles, industrial robots and smart manufacturing, and medical and healthcare. Finally, the concepts and approaches are discussed concerning the following points: Main ideas, parallels existing in different domains, which ideas can be transferred from one domain to another, which high-level tasks were adressed and which assumptions were made.
M. Müller, N. Jazdi, und M. Weyrich, „Self-improving Models for the Intelligent Digital Twin: Towards Closing the Reality-to-Simulation Gap“, in 14th IFAC Workshop on Intelligent Manufacturing Systems, 5 May 2022, 2022.
ZusammenfassungThis paper presents a novel approach to ensure the quality of the Digital Twin models that modern Cyber-Physical Manufacturing Systems (CPMS) rely on. CPMS are configurable and intelligent. Environmental and system parameters change frequently. Thus, static models are inadequate. Autonomous mobile robots and the simulation of their movement are important elements of these CPMS. Based on our reinforcement learning-based methodology, we use these robots as an example to show how the Digital Twin automatically improves models that do not perfectly represent the physical asset, making it an intelligent Digital Twin. In our scenario, the behavior of the asset deviates from the simulated prediction, i.e., a simulation gap occurs. The presented approach closes this simulation gap through a three-step mechanism. First, it makes the simulated data and the real data comparable and synchronizes it. Second, it applies reinforcement learning to find patterns in the deviations between the simulated and real data. Third, it learns to compensate for them. The evaluation of this example shows promising results.
M. Müller, N. Jazdi, und M. Weyrich, „Situation-based Identification of Probable Loss Scenarios of Industrial Mobile Robots“, in 2022 IEEE 27th International Conference on Emerging Technologies and Factory Automation (ETFA), September, 2022, 2022.
ZusammenfassungIn the production of the future, people and robots will work together in dynamic environments. Industrial mobile robots drive past workers and obstacles to the assemblers' workstations, pick up workpieces and deliver them to the next free workstation. To ensure reliable production, robots must carefully consider their actions based on the current situation to minimize losses. In order to reason about losses, it is necessary to identify loss scenarios that are likely to arise from the system's situation. In this paper, the authors propose a novel methodology for identifying these likely loss scenarios based on fault injectors. The fault injectors are trained using reinforcement learning. Based on a probability model of disruptive events (faults), the agents learn to destabilize the system by simulating such events. When a chain of these events leads to a loss, a loss scenario is found. The higher the probability of the loss scenario, the higher the reward for the agent. In this way, the fault injectors optimize for maximally likely loss scenarios. After describing the methodology, the authors propose a framework for systematically classifying different types of fault injectors and give advice on how to build them. Finally, the authors demonstrate reinforcement learning-based fault injectors on the path planning of the mobile robot platform Robotino.
M. Müller, G. Ghasemi, N. Jazdi, und M. Weyrich, „Situational Risk Assessment Design for Autonomous Mobile Robots“, 32nd CIRP Design Conference 2022, Procedia CIRP, Vol. 109, pp. 72-77, 21. Juni 2022, 2022.
ZusammenfassungThe emerging autonomous mobile robots promise a new level of efficiency and flexibility. However, because these types of systems operate in the same space as humans, mobile robots must cope with dynamic changes and heterogeneously structured environments. To ensure safety, new approaches are needed that model risk at runtime. This risk depends on the situation and is therefore a situational risk. In this paper, we propose a new methodology to model situational risk based on multi-agent adversarial reinforcement learning. In this methodology, two competing groups of reinforcement learning agents, namely the protagonists and the adversaries, fight against each other in the simulation. The adversaries represent the disruptive and destabilizing factors, while the protagonists try to compensate for them. The situational risk is then derived from the outcome of the simulated struggle. At this point, the system’s Digital Twin provides up-to-date and relevant models for simulation and synchronizes the simulation with the real asset. Our risk modeling differentiates the four steps of intelligent information processing: sense, analyze, process, and execute. To find the appropriate adversaries and actors for each of these steps, this methodology builds on Systems Theoretic Process Analysis (STPA). Using STPA, we identify critical signals that lead to losses when a disturbance under certain conditions or in certain situations occurs. At this point, the challenge of managing the complexity arises. We face this issue using training effort as a metric to evaluate it. Through statistical analysis of the identified signals, we derive a procedure for defining action spaces and rewards for the agents in question. We validate the methodology using the example of a Robotino 3 Premium from Festo, an autonomous mobile robot.
M. Müller, N. Jazdi, und M. Weyrich, „Towards Situative Risk Assessment for Industrial Mobile Robots“, in 2022 IEEE 27th International Conference on Emerging Technologies and Factory Automation (ETFA), September, 2022, 2022.
ZusammenfassungIndustrial mobile robots increasingly operate in dynamic and complex environments, resulting in risks that arise from the situation. Consequently, mobile robots need knowledge about the situative risk to make responsible decisions, e.g., when planning a trajectory. In this paper, the authors present a new approach to evaluate these situative risks at runtime. To compute the situative risk, the proposed approach uses multi-agent adversarial reinforcement learning. Unlike the usual approach that uses the self-play mechanism to harden the agents against disturbances, the approach directly uses the self-play to determine the confidence that the system can handle the current situation. Based on the evaluation of the strength of the perceived perturbations, the robot's digital twin calibrates the action space of the adversarial agents. In this way, the adversarial agents represent the disturbances that are likely to occur. From the simulation results of the competing agents, the digital twin infers the confidence with which the system can handle the situation. In the event that the system cannot handle the situation and crashes in the simulation, the digital twin calculates the probability that the simulated scenario will occur in reality. To calculate this probability, the authors propose the method of game-theoretic event trees. Combining the probability of an accident scenario with the damage caused by the simulated hazard, the results of situative risk are obtained. The approach is qualitatively evaluated using a case study. In this case study, a industrial mobile robot called Robotino is considered that connects workstations in a matrix production system. The case study shows that the approach is promising.