Scenarios

In EPyT-Flow, a scenario refers to a water distribution network (WDN) that is to be simulated – i.e. performing a hydraulic and quality analysis. Besides the network itself, a scenario usually contains a sensor configuration and might also contain some events such as leakages, sensor faults, actuator events, etc. Furthermore, a scenario might also include some control modules.

Basics

There are two important classes for working with scenarios in EPyT-Flow. The class ScenarioConfig for describing the scenario, and the class ScenarioSimulator for simulating the scenario.

For creating new instance of ScenarioSimulator, either an .inp file (together with an optional .msx file) is needed, or an instance of ScenarioConfig describing and precisely specifying the scenario to be simulated.

Note

When using the ScenarioSimulator class, it is important to close it afterward so that EPANET is unloaded correctly and all temporary files are removed.

Closing a ScenarioSimulator instance can done automatically by using a with statement:

# Creates a new scenario based on the Hanoi network --
# it will be closed automatically after the with block is left!
with ScenarioSimulator(f_inp="Hanoi.inp") as sim:
    # Set any additional parameters and finalize the scenario configuration ....
    # Run simulation ...

Alternatively, you can close and unload everything manually by calling close(). Beware of any potential exceptions (e.g. raised by EPyT-Flow in case of incorrect parameters or simulation errors) that might occur during the process of setting up and running the simulation.

# Open/Create a new scenario based on the Hanoi network
sim = ScenarioSimulator(f_inp="Hanoi.inp")

# Set any additional parameters and finalize the scenario configuration ....
# Run simulation ...

# Do not forget to close the scenario
sim.close()

The simulation (i.e. hydraulics and quality analysis) itself is run by calling run_simulation(). Alternatively, the simulation can also be run step-by-step by calling run_simulation_as_generator(). In both cases, the result of the simulation is provided as a ScadaData object. In the latter case, the result is provided as a generator.

# Load Hanoi network
with ScenarioSimulator(f_inp="Hanoi.inp") as sim:
    # Run simulation
    scada_data = sim.run_simulation()

More details on ScadaData are given here.

Customize the Simulator

The behavior of the ScenarioSimulator class can be customized by deriving a child class and overriding or adding some methods.

Example of adding a method for removing all controls – smth. that ScenarioSimulator can not do because it viloates EPyT-Flow’s design principle of immutability:

class MyScenarioSimulator(ScenarioSimulator):
    def remove_all_custom_controls(self) -> None:
        """
        Removes all controls from the scenario.
        """
        self._controls = []

Parallel Simulation

EPyT-Flow also supports the parallel simulation of scenarios. This becomes handy in cases where many scenarios have to be simulated at once and multiple CPU cores are available.

Note

EPANET (in contrast to EPANET-MSX) does not make use multiple CPU cores – i.e. simualting the hydraulics of a single scenario will always use a single CPU core only.

For this, the function run() of the static class ParallelScenarioSimulation can be utilized.

# Load the first 10 LeakDB Net1 scenarios
scenarios = load_leakdb_scenarios(range(10), use_net1=True)

# Run simulations in parallel using as many CPU cores as possible
# SCADA data of each scenario will be stored in "my_leakdb_results" folder
ParallelScenarioSimulation.run(scenarios,
                               callback=callback_save_to_file(folder_out="my_leakdb_results"))

Network Topology

The topology (i.e. a graph) of the WDN is represented by a NetworkTopology instance and can be obtained by calling get_topology() of a ScenarioSimulator instance.

The topology NetworkTopology not only contains the WDN as a graph but also includes all relevant node and link/pipe attributes such as curves, elevation, diameter, length, etc., and can (optionally) also include all demand patterns as well. All those stored information can be written exported to an .inp file by calling the function to_inp_file(). This can become handy if the corresponding .inp file is not shared or not available any longer.

Furthermore, NetworkTopology also comes with some helper functions such as those for computing the adjacency matrix (get_adj_matrix()), the shortest path between two nodes (get_shortest_path()), or exporting the topology to GeoDataFrame instances (to_gis()).

Example of working with NetworkTopology:

# Create scenario based in Net1
with ScenarioSimulator(scenario_config=load_net1()) as sim:
    # Get network topology
    topo = sim.get_topology()

    # Show all edges
    print(topo.edges)

    # Show all nodes
    print(topo.nodes)

    # Shortest path between node "2" and node "22"
    print(topo.get_shortest_path("2", "22"))

    # Adjacency matrix of the graph
    # A sparse matrix is returned, which we convert it to a dense matrix
    print(topo.get_adj_matrix().todense())

    # Get nodes as a geopandas.GeoDataFrame
    print(topo.to_gis()["nodes"])

Low-level EPANET and EPANET-MSX Functions

Besides providing high-level functions for working with scenarios, EPyT-Flow also provides access to lower-level functions as provided by EPANET, and EPANET-MSX. Those functions can be accessed through the attribute epanet_api of a ScenarioSimulator instance.

Warning

Caution must be used when calling EPANET or EPANET-MSX functions as those might cause side-effects in EPyT-Flow.

Whenever possible, EPyT-Flow functions should be used!

Example of manually setting the emitter coefficient of a node by calling an EPANET function:

# Create scenario based in Net1
with ScenarioSimulator(scenario_config=load_net1()) as sim:
    # Calling an EPANET function for setting the emitter coefficient of the first node to zero
    sim.epanet_api.setnodevalue(1, EpanetConstants.EN_EMITTER, 0.)

    # ....

Units of Measurements

The units if measurement are automatically derived from the .inp and .msx files. However, it is also possible to change those before the simulation is run. In addition, all measurement units can be changed afterwards by post-processing the SCADA data – see here for more information.

The most convient way of changing/specifying the hydraulic units is by specifying the flow and pressure units when loading the .inp file – note that the flow and pressure units specify all other hydraulic units:

# Load Net1 with CMH (cubic meter per hour) as the flow unit
# and meters as the pressure unit
scenario_config = load_net1(flow_units_id=EpanetConstants.EN_CMH,
                            pressure_units_id=EpanetConstants.EN_METERS)

Alternatively, the flow and pressure units can be changed anytime by calling set_general_parameters() of a ScenarioSimulator instance:

# Open/Create a new scenario based on the Hanoi network
with ScenarioSimulator(f_inp="Hanoi.inp") as sim:
    # Change flow units to CMH (cubic meter per hour)
    # and meters as the pressure unit
    sim.set_general_parameters(flow_units_id=EpanetConstants.EN_CMH,
                               pressure_units_id=EpanetConstants.EN_METERS)

    # ...

Scenario Configurations

An alternative to passing the path to an .inp file (and .msx file) to ScenarioSimulator, is to use a ScenarioConfig instance which completely describes/specifies a scenario.

Note

A ScenarioConfig instance can also contain a NetworkTopology instance, completly specifying the water network. If the specified .inp file is not found, the ScenarioSimulator creates an .inp file based on the information from the NetworkTopology instance and uses then loads and uses this .inp file.

Because ScenarioConfig instances are immutable, there are usually not explicitly constructed by the user but loaded/parsed from a file (custom binary and JSON files are supported).

Example of loading a scenario from a JSON configuration file called myScenarioConfig.json:

# Load scenario configuration from JSON file
scenario_config = None
with open("myScenarioConfig.json", "r") as f:
    scenario_config = ScenarioConfig.load_from_json(f.read())

# Create scenario based on scenario configuration
with ScenarioSimulator(scenario_config=scenario_config) as sim:
    # Make some modifications to the scenario configuration ....
    # Run simulation ...

where myScenarioConfig.json contains a sensor placement (4 pressure and one flow sensor), two leakages (one abrupt and one incipient), one sensor fault, and uncertainties with respect to pipe length and roughness, as well as sensor noise:

{
    "general": {
        "file_inp": "Hanoi.inp",
        "simulation_duration": 100,
        "demand_model": {"type": "PDA", "pressure_min": 0, "pressure_required": 0.1,
                         "pressure_exponent": 0.5},
        "hydraulic_time_step": 1800,
        "reporting_time_step": 3600,
        "quality_time_step": 300
    },
    "uncertainties": {
        "pipe_length": {"type": "gaussian", "mean": 0, "scale": 1},
        "pipe_roughness": {"type": "uniform", "low": 0, "hight": 1},

        "sensor_noise": {"type": "gaussian", "mean": 0, "scale": 0.01}
    },
    "sensors": {
        "pressure_sensors": ["13", "16", "22", "30"],
        "flow_sensors": ["1"],
        "demand_sensors": [],
        "node_quality_sensors": [],
        "link_quality_sensors": []
    },
    "leakages": [
        {"type": "abrupt", "link_id": "12", "diameter": 0.1,
            "start_time": 7200, "end_time": 100800},
        {"type": "incipient", "link_id": "10", "diameter": 0.01,
            "start_time": 7200, "end_time": 100800, "peak_time": 54000}
    ],
    "sensor_faults": [
        {"type": "constant", "constant_shift": 2.0, "sensor_id": "16",
            "sensor_type": 1, "start_time": 5000, "end_time": 100000}
    ]
}

Note that the individual entries in the JSON file correspond to the classes as implemented in EPyT-Flow.

At every time, a complete ScenarioConfig can be obtained by calling get_scenario_config(). This scenario configuration could be then, for instance, stored in a file so that it can be reloaded in the future without having to make all the manual specifications again – see Serialization for details.

Example of obtaining and storing the current scenario configuration:

# Open/Create a new scenario based on the Hanoi network
with ScenarioSimulator(f_inp="Hanoi.inp") as sim:
    # Make some modifications to the scenario configuration ....

    # Get final scenario configuration
    scenario_config_final = sim.get_scenario_config()

    # Store scenario configuration in a file
    scenario_config_final.save_to_file("myHanoiConfig.epytflow_config")

# ....

# Load scenario configuration
scenario_config = ScenarioConfig.load("myHanoiConfig.epytflow_config")
with ScenarioSimulator(scenario_config) as sim:
    # ....

Predefined networks

EPyT-Flow comes with a set of popular benchmark water distribution networks already included. These networks are, if necessary, downloaded and wrapped inside a ScenarioConfig instance, so that they can be directly passed to ScenarioSimulator.

Also, note that in some cases (i.e. Hanoi and L-TOWN) a predefined sensor placement can be included as well.

Network	Function for loading
Net1	load_net1()
Net2	load_net2()
Net3	load_net3()
Net6	load_net6()
Richmond	load_richmond()
MICROPOLIS	load_micropolis()
Balerma	load_balerma()
Rural	load_rural()
BSWN-1	load_bwsn1()
BWSN-2	load_bwsn2()
Anytown	load_anytown()
D-Town	load_dtown()
C-Town	load_ctown()
Kentucky	load_kentucky()
Hanoi	load_hanoi()
L-TOWN	load_ltown()
L-TOWN-A	load_ltown_a()

Example of loading the Hanoi network:

network_config = load_hanoi()   # Load Hanoi network
with ScenarioSimulator(scenario_config=network_config) as sim:
    # Set any additional parameters and finalize the scenario configuration ....
    # Run simulation ...

Benchmarks scenarios

EPyT-Flow comes with a set of benchmark scenarios. Usually, those are pre-defined scenarios for different tasks such as leakage detection and localization.

Benchmark	Module
LeakDB [1]	leakdb
BattLeDIM [2]	battledim
BATADAL [3]	batadal

Benchmark data sets

In addition to benchmark scenarios (see previous section), EPyT-Flow also includes several (WDN related) benchmark data sets from the literature:

Benchmark	Function for loading
GECCO Water Quality 2017 [4]	load_gecco2017_water_quality_data()
GECCO Water Quality 2018 [5]	load_gecco2018_water_quality_data()
GECCO Water Quality 2019 [6]	load_gecco2019_water_quality_data()
Water Usage [7]	load_water_usage()

WaterBenchmarkHub

For more networks and benchmarks, check-out the WaterBenchmarkHub, a platform for accessing and sharing Water Distribution Network (WDN) benchmarks and data sets.

[1]

Vrachimis et al. (2018) – see https://github.com/KIOS-Research/LeakDB/

[2]

Vrachmimis et al. (2020) – see https://github.com/KIOS-Research/BattLeDIM

[3]

Taormina et al. (2017) – see https://www.batadal.net/

[4]

Friese et al. (2017) – see http://www.spotseven.de/gecco-challenge/gecco-challenge-2017/

[5]

Rehbach et al. (2018) – see http://www.spotseven.de/gecco/gecco-challenge/gecco-challenge-2018/

[6]

Rehbach et al. (2019) – see https://www.th-koeln.de/informatik-und-ingenieurwissenschaften/gecco-challenge-2019_63244.php

[7]

Pavlou et al. (2018) – see https://github.com/KIOS-Research/Water-Usage-Dataset/