Visible to the public Multi-model Testbed for the Simulation-based Evaluation of Resilience (October '19)Conflict Detection Enabled

PI(s), Co-PI(s), Researchers:

  • Peter Volgyesi (PI)
  • Himanshu Neema (Co-PI)

This refers to Hard Problems, released in November 2012.

  • Security Metrics Driven Evaluation, Design, Development, and Deployment
  • Resilient Architectures

The goal of the Multi-model Testbed is to provide a collaborative design tool for evaluating various cyber attack/defense strategies and their effects on the physical infrastructure. The web-based, cloud-hosted environment integrates state-of-the-art simulation engines for the different CPS domains and presents interesting research challenges as ready to use scenarios. Input data, model parameters, and simulation results are archived, versioned with a strong emphasis on repeatability and provenance.


Xingyu Zhou, Yi Li, Carlos Barreto, Jiani Li, Peter Volgyesi, Himanshu Neema, Xenofon Koutsoukos, "Evaluating Resilience of Grid Load Predictions under Stealthy Adversarial Attacks". Resilience Week 2019 Symposium, San Antonio, TX, USA, November 4-7, 2019. (accepted)  -  received Best Paper Award

Carlos Barreto and Xenofon Koutsoukos, “Attacks on electricity markets.” In 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton). IEEE, 2019. (accepted)

Carlos Barreto, Alvaro A. Cárdenas, and Nicanor Quijano. "Design of Load Forecast Systems Resilient Against Cyber-Attacks." In International Conference on Decision and Game Theory for Security (GameSec). Springer, 2019. (accepted)

Carlos Barreto and Xenofon Koutsoukos, “Design of Electricity Markets Resilient Against False Data Injection Attacks.” (submitted to IEEE Computer).


In this quarter our Testbed development effort was focused on the initial integration of two existing design studios: (1) DeepForge, our collaborative deep neural network experimentation platform with TensorFlow/Keras backend support and (2) GridLAB-D Design Studio, for configuring and executing smart power grid simulation models through a web-based interface. Previously, we relied on both technologies to execute experiments for Load Forecasting and Adversarial Attacks against the forecasting algorithm (see Publications), however, we used the two components in isolation by transferring the simulation results manually into DeppForge. Although both design studios are built upon the same underlying technology (WebGME), these change the visualization and control interfaces of the core tool significantly, thus integrating them in a unified modeling tool is not trivial. Furthermore, both design studios rely on their complex domain-specific metamodels. Merging these models is part of our future work.

The architecture of our initial prototype integration is shown in the next figure. In the current implementation, both extension toolsets are hosted in the same WebGME server instance. However, the models (and metamodels) are kept separately in the Model Database. We rely on lightweight cross-project links from the DeepForge models into GridLAB-D simulation models to refer to those simulations and parameters of which output data is used as input data for the training and testing phases of the neural network pipelines.


Our immediate plan is to demonstrate an end-to-end integrated workflow between DeepForge and GridLAB-D, using one of the previously developed Load Forecasting models. The goal is to automatically invoke the power grid simulation step and collects its results in the DeepForge pipeline.

The predecessor  (SURE) of the current Testbed was built on the C2WT (C2 Windtunnel) simulation integration platform. This enabled us to execute heterogeneous multi-domain CPS simulation models, primarily targeting traffic networks with communication networks. The C2WT technology also provided a COA (Courses of Action) library to inject cyber-attacks based on time and event triggers. The current testbed---so far---followed a lighter approach with direct drivers towards the domain-specific simulation tools. This decision was in part influenced by the fact that the original C2WT integration framework became outdated.

There is a renewed effort to modernize the HLA-based simulation integration toolset, by (1) updating to Portico 2.2.0, (2) enabling support for the new/updated services developed in HLA-Evolved, (3) supporting specification, configuration, and execution of scalable federations by utilizing the hierarchical composition functionality in the centralized RTI architecture (4) supporting the specification, configuration, and execution of message filters that
restrict information flows only to intended recipients and thus provide security. We plan to leverage this effort and use the updated platform in our testbed. One (most) important immediate benefit will be the renewed capability of simulating communication network effects.


Collaboration and technical exchange with the Cybersecurity Research Group at Fujitsu System Integration Laboratories Ltd. This group uses WebGME, DeepForge and technology elements of our SURE testbed to develop their Cyberrange product.
Next visit: November 24-27, 2019 Tokyo, Japan.


None, during this reporting period.