Position within the page tree

Sheng Ding

M.Sc.

Academic staff
Institute of Industrial Automation and Software Engineering

Contact

+49 711 685 69198
+49 711 685 67302
Email

Pfaffenwaldring 47
70550 Stuttgart
Germany
Room: 3.252

Publications

Journals and conferences:

2021
1. Y. Ma, A. Morozov, and S. Ding, “Anomaly Detection for Cyber Physical Systems using Transformers,” The International Mechanical Engineering Congress and Exposition, IMECE, 2021.
  - Abstract
  - BibTeX
  Abstract
  Safety and reliability are two critical factors of modern Cyber-Physical Systems (CPS). However, the increasing structural and behavioral complexity of modern automation systems significantly increases the possibility of system errors and failures, which can easily lead to economic loss or even hazardous events. Anomaly Detection (AD) techniques provide a potential solution to this problem, and conventional methods, e.g., Autoregressive Integrated Moving Average model (ARIMA), are no longer the best choice for anomaly detection for modern complex CPS. Recently, Deep Learning (DL) and Machine Learning (ML) anomaly detection methods became more popular, and numerous practical applications have been presented in many industrial scenarios. Most of the modern DL-based anomaly detection methods use the prediction approach and LSTM architecture. The Transformer is a new neural network architecture that outperforms LSTM in natural language processing. In this paper, we show that the Transformer-based deep learning model, which has received much attention recently, can be applied to the anomaly detection of industrial automation systems. Specifically, we collect time-series data from a system of two industrial robots using a Simulink model. Then, we feed these data into our Transformer-based model and train it to be a time-series data predictor. The paper presents the experimental results that show the comparison of precision and speed of a Long-Short Time Memory (LSTM) predictor and our Transformer-based predictor.
  BibTeX
  @article{ma2021anomaly, abstract = {Safety and reliability are two critical factors of modern Cyber-Physical Systems (CPS). However, the increasing structural and behavioral complexity of modern automation systems significantly increases the possibility of system errors and failures, which can easily lead to economic loss or even hazardous events. Anomaly Detection (AD) techniques provide a potential solution to this problem, and conventional methods, e.g., Autoregressive Integrated Moving Average model (ARIMA), are no longer the best choice for anomaly detection for modern complex CPS. Recently, Deep Learning (DL) and Machine Learning (ML) anomaly detection methods became more popular, and numerous practical applications have been presented in many industrial scenarios. Most of the modern DL-based anomaly detection methods use the prediction approach and LSTM architecture. The Transformer is a new neural network architecture that outperforms LSTM in natural language processing. In this paper, we show that the Transformer-based deep learning model, which has received much attention recently, can be applied to the anomaly detection of industrial automation systems. Specifically, we collect time-series data from a system of two industrial robots using a Simulink model. Then, we feed these data into our Transformer-based model and train it to be a time-series data predictor. The paper presents the experimental results that show the comparison of precision and speed of a Long-Short Time Memory (LSTM) predictor and our Transformer-based predictor.}, author = {Ma, Y. and Morozov, A. and Ding, Sheng}, journal = {The International Mechanical Engineering Congress and Exposition, IMECE}, title = {Anomaly Detection for Cyber Physical Systems using Transformers}, year = 2021 }
2. S. Ding, A. Morozov, T. Fabarisov, and S. Vock, “KrakenBox: Deep Learning based Error Detector for Industrial Cyber-Physical Systems,” in The International Mechanical Engineering Congress and Exposition, IMECE, 2021.
  - Abstract
  - BibTeX
  Abstract
  Online error detection helps to reduce the risk of failure of safety-critical systems. However, due to the increasing complexity of modern Cyber-Physical Systems and the sophisticated interaction of their heterogeneous components, it becomes harder to apply traditional error detection methods. Nowadays, the popularity of Deep Learning-based error detection snowballs. DL-based methods achieved significant progress along with better results. This paper introduces the KrakenBox, a deep learning-based error detector for industrial Cyber-Physical Systems (CPS). It provides conceptual and technical details of the KrakenBox hardware, software, and a case study. The KrakenBox hardware is based on NVIDIA Jetson AGX Xavier, designed to empower the deep learning-based application and the extended alarm module. The KrakenBox software consists of several programs capable of collecting, processing, storing, and analyzing time-series data. The KrakenBox can be connected to the networked automation system either via Ethernet or wirelessly. The paper presents the KrakenBox architecture and results of experiments that allow the evaluation of the error detection performance for varying error magnitude. The results of these experiments demonstrate that the KrakenBox is able to improve the safety of a networked automation system.
  BibTeX
  @conference{noauthororeditor2021krakenbox, abstract = {Online error detection helps to reduce the risk of failure of safety-critical systems. However, due to the increasing complexity of modern Cyber-Physical Systems and the sophisticated interaction of their heterogeneous components, it becomes harder to apply traditional error detection methods. Nowadays, the popularity of Deep Learning-based error detection snowballs. DL-based methods achieved significant progress along with better results. This paper introduces the KrakenBox, a deep learning-based error detector for industrial Cyber-Physical Systems (CPS). It provides conceptual and technical details of the KrakenBox hardware, software, and a case study. The KrakenBox hardware is based on NVIDIA Jetson AGX Xavier, designed to empower the deep learning-based application and the extended alarm module. The KrakenBox software consists of several programs capable of collecting, processing, storing, and analyzing time-series data. The KrakenBox can be connected to the networked automation system either via Ethernet or wirelessly. The paper presents the KrakenBox architecture and results of experiments that allow the evaluation of the error detection performance for varying error magnitude. The results of these experiments demonstrate that the KrakenBox is able to improve the safety of a networked automation system.}, author = {Ding, Sheng and Morozov, A. and Fabarisov, T. and Vock, S.}, booktitle = {The International Mechanical Engineering Congress and Exposition, IMECE}, title = {KrakenBox: Deep Learning based Error Detector for Industrial Cyber-Physical Systems}, year = 2021 }
2020
1. S. Ding, A. Morozov, S. Vock, M. Weyrich, and K. Janschek, “Model-Based Error Detection for Industrial Automation Systems Using LSTM Networks,” In: International Symposium on Model-Based Safety and Assessment, pp. 212-226. Springer, Cham, 2020.
  Abstract
  The increasing complexity of modern automation systems leads to inevitable faults. At the same time, structural variability and untrivial interaction of the sophisticated components makes it harder and harder to apply traditional fault detection methods. Consequently, the popularity of Deep Learning (DL) fault detection methods grows. Model-based system design tools such as Simulink allow the development of executable system models. Besides the design flexibility, these models can provide the training data for DL-based error detectors. This paper describes the application of an LSTM-based error detector for a system of two industrial robotic manipulators. A detailed Simulink model provides the training data for an LSTM predictor. Error detection is achieved via intelligent processing of the residual between the original signal and the LSTM prediction using two methods. The first method is based on the non-parametric dynamic thresholding. The second method exploits the Gaussian distribution of the residual. The paper presents the results of extensive model-based fault injection experiments that allow the comparison of these methods and the evaluation of the error detection performance for varying error magnitude.
  BibTeX
  @article{ding2020modelbased, abstract = {The increasing complexity of modern automation systems leads to inevitable faults. At the same time, structural variability and untrivial interaction of the sophisticated components makes it harder and harder to apply traditional fault detection methods. Consequently, the popularity of Deep Learning (DL) fault detection methods grows. Model-based system design tools such as Simulink allow the development of executable system models. Besides the design flexibility, these models can provide the training data for DL-based error detectors. This paper describes the application of an LSTM-based error detector for a system of two industrial robotic manipulators. A detailed Simulink model provides the training data for an LSTM predictor. Error detection is achieved via intelligent processing of the residual between the original signal and the LSTM prediction using two methods. The first method is based on the non-parametric dynamic thresholding. The second method exploits the Gaussian distribution of the residual. The paper presents the results of extensive model-based fault injection experiments that allow the comparison of these methods and the evaluation of the error detection performance for varying error magnitude.}, author = {Ding, Sheng and Morozov, Andrey and Vock, Silvia and Weyrich, Michael and Janschek, Klaus}, journal = {In: International Symposium on Model-Based Safety and Assessment, pp. 212-226. Springer, Cham}, title = {Model-Based Error Detection for Industrial Automation Systems Using LSTM Networks}, url = {https://link.springer.com/chapter/10.1007/978-3-030-58920-2_14}, year = 2020 }
  Link
  https://link.springer.com/chapter/10.1007/978-3-030-58920-2_14
2019
1. K. Ding, S. Ding, A. Morozov, T. Fabarisov, and K. Janschek, “On-line error detection and mitigation for time-series data of cyber-physical systems using deep learning based methods,” in 2019 15th European Dependable Computing Conference (EDCC) (pp. 7-14). IEEE, 2019.
  Abstract
  A cyber-physical system consists of sensors, micro-controller, networks, and actuators that interact with each other, generate a substantial amount of data, and form extremely complex system operational profiles. These heterogeneous components are subject to errors, e.g. spikes, off-sets, or delays, that may result in system failures. As the complexity of modern systems increases, it becomes a challenge to apply traditional fault detection and isolation methods to such complex systems. Deep learning based methods have surpassed traditional methods in terms of performance as the data size and complexity increase. The signals of cyber-physical systems are mainly time-series data. In this paper, we propose a new on-line error detection and mitigation approach for common sensor, computing hardware, and network errors of cyber-physical systems using deep learning based methods. More specifically, we train a Long Short-Term Memory (LSTM) network as a single step prediction model for the detection and mitigation of errors, like spikes, or offsets. In order to detect the long-duration errors that show no sharp change (a sudden drop or rise) between two successive data samples when errors occurred, e.g. network delays, we train an LSTM encoder-decoder as a multi-step prediction model. We also introduce the on-line error mitigation approach. Automatic recovery is achieved by replacing the detected errors with the predicted values. Finally, we demonstrate on-line error detection and mitigation capabilities of the trained single step and multi-step predictors using representative case studies.
  BibTeX
  @conference{ding2019online, abstract = {A cyber-physical system consists of sensors, micro-controller, networks, and actuators that interact with each other, generate a substantial amount of data, and form extremely complex system operational profiles. These heterogeneous components are subject to errors, e.g. spikes, off-sets, or delays, that may result in system failures. As the complexity of modern systems increases, it becomes a challenge to apply traditional fault detection and isolation methods to such complex systems. Deep learning based methods have surpassed traditional methods in terms of performance as the data size and complexity increase. The signals of cyber-physical systems are mainly time-series data. In this paper, we propose a new on-line error detection and mitigation approach for common sensor, computing hardware, and network errors of cyber-physical systems using deep learning based methods. More specifically, we train a Long Short-Term Memory (LSTM) network as a single step prediction model for the detection and mitigation of errors, like spikes, or offsets. In order to detect the long-duration errors that show no sharp change (a sudden drop or rise) between two successive data samples when errors occurred, e.g. network delays, we train an LSTM encoder-decoder as a multi-step prediction model. We also introduce the on-line error mitigation approach. Automatic recovery is achieved by replacing the detected errors with the predicted values. Finally, we demonstrate on-line error detection and mitigation capabilities of the trained single step and multi-step predictors using representative case studies.}, author = {Ding, K. and Ding, Sheng and Morozov, A. and Fabarisov, T. and Janschek, K.}, booktitle = {2019 15th European Dependable Computing Conference (EDCC) (pp. 7-14). IEEE}, doi = {https://ieeexplore.ieee.org/abstract/document/8893390}, title = {On-line error detection and mitigation for time-series data of cyber-physical systems using deep learning based methods}, url = {https://ieeexplore.ieee.org/abstract/document/8893390}, year = 2019 }
  Link
  https://ieeexplore.ieee.org/abstract/document/8893390

Sheng Ding is a PhD Student and Research Associate at the University of Stuttgart, Institute of Automation Technology and Software Engineering. His main research interests are deep learning based error detection and reliability analysis of cyber-physical systems.

Research

Research focus:

Deep learning based error detection and reliability analysis of cyber-physical systems.

Description:

Anomaly detection is a well-known concept that is used in a wide variety of fields, including systems engineering, to detect or prevent errors. As deep learning (DL) approaches continue to improve, the applicability of DL techniques for error detection is being explored. This research will work to focus on time series computing and common CPS errors. First, the training data is generated using a combination of existing and own error injection methods and tools. For this purpose, data are generated using simulation in the area of networked automation systems and errors through injection with the appropriate label. So that the methods can be implemented in the real world, real case studies are considered and compared measured signals with the simulation.

The vast majority of existing research on anomaly detection does not consider real-time. However, this is very important to protect a CPS system. On the one hand, online real-time measurement and data traffic from the host system should be documented and evaluated. On the other hand, online real-time response is explored by comparing the computation costs of different DL models.

In the project, many types of methods or DL architectures are compared and integrated. Appropriate DL-based error detection approaches such as prediction, classification, and reconstruction are compared. The training and evaluation of the DL models take place offline with the collected data sets and comprise three phases. First, different network types, e.g. MLP, CNN, LSTM / GRU, Autoen-Coders, and transformers, as well as their structure and hyper-parameters are examined and identified. Second, the performance with accuracy and speed of various models is documented as a knowledge base. Third, to improve accuracy, dynamic reconfiguration is being researched. The aim is to develop a DL-based error detection with real-time reaction and high accuracy. The implemented method is tested online to demonstrate the concept. The results show that the errors can be successfully identified in real-time.

Research portal:

Google Scholar: https://scholar.google.de/citations?hl=en&user=jnvhuokAAAAJ

ResearchGate: https://www.researchgate.net/profile/Sheng-Ding-12

Sheng Ding

Contact

Publications

2021

Abstract

BibTeX

Abstract

BibTeX

2020

Abstract

BibTeX

Link

2019

Abstract

BibTeX

Link

Vita

Research

Audience

Formalities

Services

Organization

Sheng Ding

Contact

Publications

2021

Abstract

BibTeX

Abstract

BibTeX

2020

Abstract

BibTeX

Link

2019

Abstract

BibTeX

Link

Vita

Research

Here you can reach us

Audience

Formalities

Services

Organization