Edit on GitHub

Introduction to the PowerModelsDistribution Data Models

In this notebook we introduce the engineering data model added to PowerModelsDistribution in version v0.9.0. We will give several examples of how to use this new data model directly, including new transformations that have become easier with its introduction, how to convert it to the the lower-level mathematical model that was previously the only user interface we offered, and how to get various types of results using this new model.

Imports

All commands in this document with no package namespace specified are directly exported by PowerModelsDistribution or already available in Julia base. Any commands that are only avaiable via an external package will be specified by including by using import, which will require specifying the originating package before the command, e.g. Ipopt.Optimizer as you will see below.

using PowerModelsDistribution

In these examples we will use the following optimization solvers, specified using optimizer_with_attributes from JuMP v0.21

import Ipopt

ipopt_solver = optimizer_with_attributes(Ipopt.Optimizer, "tol"=>1e-6, "print_level"=>0)
MathOptInterface.OptimizerWithAttributes(Ipopt.Optimizer, Pair{MathOptInterface.AbstractOptimizerAttribute, Any}[MathOpt
Interface.RawParameter("tol") => 1.0e-6, MathOptInterface.RawParameter("print_level") => 0])

Parsing Data

Here we give the first example of how to parse data into the ENGINEERING data model structure, which is the default data structure type that the user will see without passing additional arguments, as we demonstrate later.

We start with a 3 bus unbalanced load case provided as a dss file in the test folder of the PowerModelsDistribution.jl repository

eng = parse_file("../test/data/opendss/case3_unbalanced.dss")
Error: SystemError: opening file "../test/data/opendss/case3_unbalanced.dss": No such file or directory

Different information and warning messages will be given depending on the input file. In the case above, these messages all related to various parse notifications that arise during a parse of a dss file, and can be safely ignored

The resulting data structure is a Julia dictionary. The first notable field is "data_model" which specifies which data model this data structure corresponds to, in this case ENGINEERING. This value is expected to be an Enum of type DataModel

The next notable field is "settings", which contains some important default/starting values for the distribution network

eng["settings"]
Error: UndefVarError: eng not defined
  • "sbase_default" is the starting value for the power base,
  • "vbases_default" is the starting voltage base for the case, and multiple voltage bases can be specified, which would be useful in cases where there are multiple isolated islands with their own generation,
  • "voltage_scale_factor" is a scaling factor for all voltage values, which in the case of OpenDSS is in kV by default
  • "power_scale_factor" is a scaling factor for all power values
  • "base_frequency" is the base frequency of the network in Hz, which is useful to know for mixed frequency networks

Next we look at the "bus" components

eng["bus"]
Error: UndefVarError: eng not defined

We can see there are three buses in this system, identified by ids "primary", "sourcebus", and "loadbus".

NOTE: In Julia, order of Dictionary keys is not fixed, nor does it retain the order in which it was parsed like e.g. Vectors.

Identifying components by non-integer names is a new feature of the ENGINEERING model, and makes network debugging more straightforward.

NOTE: all names are converted to lowercase on parse from the originating dss file.

Each bus component has the following properties in the ENGINEERING model

eng["bus"]["sourcebus"]
Error: UndefVarError: eng not defined
  • "terminals" indicates which terminals on the bus have active connections
  • "grounded" indicates which terminals are grounded
  • "rg" and "xg" indicate the grounding resistance and reactance of the ground
  • "status" indicates whether a bus is ENABLED or DISABLED, and is specified for every component in the engineering model

Next, we look at the "line" components, which is a generic name for both overhead lines and underground cables, which we do not differentiate between in the nomenclature

eng["line"]
Error: UndefVarError: eng not defined
eng["line"]["quad"]
Error: UndefVarError: eng not defined

Again, we see components identified by their OpenDSS names. A "line" is an edge object, which will always have the following properties:

  • "f_bus"
  • "t_bus"
  • "f_connections" - list of terminals to which the line is connected on the from-side
  • "t_connections" - list of terminals to which the line is connected on the to-side

Here we are also introduced to two important concepts, the "source_id", which is an easy way to identify from where an object originates in the dss file, and a data type element, pointed to by "linecode" in this case.

A data type element is an element that does not represent a real engineering object, but only contains data that one of those real objects can refer to, in this case a linecode, which contains information like line resistance/reactance and conductance/susceptance.

eng["linecode"]["4/0quad"]
Error: UndefVarError: eng not defined

Next, we introduce a node element, the "load" object, where we also see the first example of a specification of less than three phases at a time

eng["load"]["l1"]
Error: UndefVarError: eng not defined

We can see that the length of the Vectors for "pd_nom" and "qd_nom" are only one, although the number of terminals listed in "connections" is two. This is because the connection is WYE, and therefore the final connection is a grounded neutral

Here we are also introduced to two new Enums, WYE, which gives the connection configuration, and NO under dispatchable, which indicates that if this case were used in an MLD problem, i.e. with run_mc_mld that this load would not be sheddable.

Finally, we show the generation source for this case, which in opendss is a voltage source named "source"

eng["voltage_source"]["source"]
Error: UndefVarError: eng not defined
  • "vm" - specifies the fixed voltage magnitudes per phase at the bus
  • "va" - specifies the fixed reference angles per phases at the bus
  • "rs" and "xs" specifies internal impedances of the voltage source

Importing raw dss properties

In case there are additional properties that you want to use from dss, it is possible to import those directly into the ENGINEERING (and MATHEMATICAL) data structure with the import_all keyword argument

eng_all = parse_file("../test/data/opendss/case3_unbalanced.dss"; import_all=true)

eng_all["line"]
Error: SystemError: opening file "../test/data/opendss/case3_unbalanced.dss": No such file or directory

You will note the presence of "dss" dictionaries under components, and "dss_options" at the root level

Time Series Parsing Example

In the ENGINEERING model, we have included the time_series data type, which holds all time series data and can be referred to similar to "linecode" as demonstrated above.

Below we can see an example of a parse that includes some time_series components

eng_ts = parse_file("../test/data/opendss/case3_balanced.dss"; time_series="daily")
Error: SystemError: opening file "../test/data/opendss/case3_balanced.dss": No such file or directory
eng_ts["load"]["l1"]["time_series"]
Error: UndefVarError: eng_ts not defined

You can see that under the actual component, in this case a "load", that there is a "time_series" dictionary that contains ENGINEERING model variable names and references to the identifiers of a root-level time_series object, 

eng_ts["time_series"]["ls1"]
Error: UndefVarError: eng_ts not defined

This feature is useful for building multinetwork data structures, which will be described below in the section on the MATHEMATICAL model

Running Optimal Power Flow

In this section we introduce how to run an optimal power flow (opf) in PowerModelsDistribution on an engineering data model

In order to run an OPF problem you will need

  1. a data model
  2. a formulation
  3. a solver

In these examples we will use the eng model we worked with above, the ACPPowerModel, which is a AC power flow formulation in polar coordinates, and the ipopt_solver we already defined above

result = solve_mc_opf(eng, ACPPowerModel, ipopt_solver)
Error: UndefVarError: eng not defined

The result of solve_mc_opf will be very familiar to those who are already familiar with PowerModels and PowerModelsDistribution. The notable difference will be in the "solution" dictionary

result["solution"]
Error: UndefVarError: result not defined

Here you can see that the solution comes back out by default into the same data model as is provided by the user to the run_ command, as well as being in SI units, as opposed to per unit, which is used during the solve. For example,

result["solution"]["bus"]["loadbus"]
Error: UndefVarError: result not defined

If for some reason you want to return the result in per-unit rather than SI, you can specify this in the run_ command by

result_pu = solve_mc_opf(eng, ACPPowerModel, ipopt_solver; make_si=false)

result_pu["solution"]["bus"]["loadbus"]
Error: UndefVarError: eng not defined

Branch Flow formulations

Previously, to use a branch flow formulation, such as SOCNLPUBFPowerModel, it was required to use a different run_ command, but now, by using multiple dispatch we have simplified this for the user

result_bf = solve_mc_opf(eng, SOCNLPUBFPowerModel, ipopt_solver)
Error: UndefVarError: eng not defined

Running Time Series Models

By default, time_series object will be ignored when running a model. To use the time series information you will need to have a multinetwork problem specification

In the example below we use a test case, which is not exported by default, and therefore requires the specification of the PowerModelsDistribution namespace

result_mn = PowerModelsDistribution._solve_mn_mc_opb(eng_ts, NFAPowerModel, ipopt_solver)
Error: UndefVarError: eng_ts not defined

Engineering Model Transformations

One of the power things about the engineering model is that data transformations are much more simple. Here we illustrate two examples that are currently included in PowerModelsDistribution, but writing your own data transformation functions will be trivial, as we will show.

Note: In v0.9, apply_kron_reduction! and apply_phase_projection! are applied by default, but can be disabled with the keyword arguments kron_reduced=false and project_phases=false, respectively in parse_file or transform_data_model.

First, there are several objects that have loss models by default when parsing from dss files, such as voltage sources, transformers, and switches. To remove these loss models, therefore making these components lossless, we can use the included make_lossess! function. Here we use a basic 2-winding wye-wye connected transformer case from test to illustrate this

eng_ut = parse_file("../test/data/opendss/ut_trans_2w_yy.dss")

eng_ut["transformer"]["tx1"]
Error: SystemError: opening file "../test/data/opendss/ut_trans_2w_yy.dss": No such file or directory

We can see that "noloadloss", "rw", and "imag" are non-zero, but if we apply the make_lossless! function we can see these parameters are set to zero, effectively eliminating the losses

make_lossless!(eng_ut)

eng_ut["transformer"]["tx1"]
Error: UndefVarError: eng_ut not defined

Alternatively, we can apply this function at parse

eng_ut = parse_file("../test/data/opendss/ut_trans_2w_yy.dss"; transformations=[make_lossless!])

eng_ut["transformer"]["tx1"]
Error: SystemError: opening file "../test/data/opendss/ut_trans_2w_yy.dss": No such file or directory

Another transformation function included in PowerModelsDistribution is the apply_voltage_bounds! function, which will apply some voltage bounds in SI units, given some percent value, e.g. if we want the lower bound on voltage to be 0.9 and upper bound 1.1 after per-unit conversion

apply_voltage_bounds!(eng_ut; vm_lb=0.9, vm_ub=1.1)

eng_ut["bus"]["2"]
Error: UndefVarError: eng_ut not defined

Alternatively, this can be specified at parse by

eng_ut = parse_file("../test/data/opendss/ut_trans_2w_yy.dss"; transformations=[make_lossless!, (apply_voltage_bounds!, "vm_lb"=>0.9, "vm_ub"=>1.1)])

eng_ut["bus"]["2"]
Error: SystemError: opening file "../test/data/opendss/ut_trans_2w_yy.dss": No such file or directory

Transformations on Multinetworks

Transformations on Multinetworks should happen before the network is converted into a MATHEMATICAL data model, so that they can generally follow the same pattern as shown above and can be seen in the make_lossless! and apply_voltage_bounds! functions already in PowerModelsDistribution

Mathematical Model

In this section we introduce the mathematical model, which was the previous user-facing model in PowerModelsDistribution, explain how conversions between the model happen in practice, and give an example of how to do this conversion manually

In practice, unless the user is interested, the conversion between the ENGINEERING and MATHEMATICAL models should be seemless and invisible to the user. By providing an ENGINEERING model to a run_ command the run_mc_model command will know to convert the model to MATHEMATICAL, which will be used to the generate the JuMP model that will actually be optimized. Similarly, the solution generated by this optimization will be automatically converted back to the format of the ENGINEERING model.

Let's first take a look at how to convert to the MATHEMATICAL model

math = transform_data_model(eng)
Error: UndefVarError: eng not defined

There are a couple of things to notice right away. First, the data model transform automatically converts the model to per-unit. Second, there are a lot of empty component sets, whereas in the ENGINEERING model, only component types that had components in them were listed. In the MATHEMATICAL model certain component dictionaries are always expected to exist, and the eng2math conversion functions will automatically populate these.

Next, there are a few unusal fields, such as "settings", which previously didn't exist in the MATHEMATICAL model. This is used for the per-unit conversion specifically in PowerModelsDistribution. Also, is the "map" field, which is a Vector of Dictionaries that enable the conversion back to ENGINEERING from MATHEMATICAL. Without this it would be impossible to convert back, and in fact only the solution can be converted, because some properties are combined destructively during the conversion to the MATHEMATICAL model, and therefore cannot be reverse engineered. However, since the conversion to MATHEMATICAL is not in-place, you will always have a copy of eng alongside math.

Here is an example of one of the "map" entries

math["map"][end]
Error: UndefVarError: math not defined

Alternatively, the MATHEMATICAL model can be returned directly from the parse_file command with the data_model keyword argument

math = parse_file("../test/data/opendss/case3_unbalanced.dss"; data_model=MATHEMATICAL)
Error: SystemError: opening file "../test/data/opendss/case3_unbalanced.dss": No such file or directory

Multinetworks

In this subsection we cover parsing into a multinetwork data structure, which is a structure that only exists in the MATHEMATICAL model

For those unfamiliar, the InfrastructureModels family of packages has a feature called multinetworks, which is useful for, among other things, running optimization problems on time series type problems.

Multinetwork data structures are formatted like so

mn = Dict{String,Any}( "multinetwork" => true, "nw" => Dict{String,Any}( "1" => Dict{String,Any}( "bus" => Dict{String,Any}(), ... ), ... ), ... )

To automatically create a multinetwork structure from an engineering model that contains time_series elements, we can use the build_multinetwork keyword argument in transform_data_model

math_mn = transform_data_model(eng_ts; build_multinetwork=true)
Error: UndefVarError: eng_ts not defined

Alternatively, we can use parse_file with the build_multinetwork keyword argument combined with data_model=MATHEMATICAL

math_mn = parse_file("../test/data/opendss/case3_balanced.dss"; build_multinetwork=true, data_model=MATHEMATICAL)
Error: SystemError: opening file "../test/data/opendss/case3_balanced.dss": No such file or directory

Running MATHEMATICAL models

There is very little difference from the user point-of-view in running MATHEMATICAL models other than the results will not be automatically converted back to the the format of the ENGINEERING model

result_math = solve_mc_opf(math, ACPPowerModel, ipopt_solver)

result_math["solution"]
Error: UndefVarError: math not defined

It is also possible to manually convert the solution back to the ENGINEERING format, provided you have the map

sol_eng = transform_solution(result_math["solution"], math)
Error: UndefVarError: result_math not defined

Running MATHEMATICAL Multinetworks

As with the ENGINEERING example of running a multinetwork problem, you will need a multinetwork problem specification, and as with the previous single MATHEMATICAL network example above, we only obtain the MATHEMATICAL solution, and can transform the solution in the same manner as before

result_math_mn = PowerModelsDistribution._solve_mn_mc_opb(math_mn, NFAPowerModel, ipopt_solver)

result_math_mn["solution"]["nw"]["1"]
Error: UndefVarError: math_mn not defined
sol_eng_mn = transform_solution(result_math_mn["solution"], math_mn)

sol_eng_mn["nw"]["1"]
Error: UndefVarError: result_math_mn not defined

Building the JuMP Model

In some cases the user will want to directly build the JuMP model, which would traditionally be done with instantiate_model from PowerModels. In order to facilitate using the ENGINEERING model we have introduced instantiate_mc_model to aid in the generation of the JuMP model. instantiate_mc_model will automatically convert the data model to MATHEMATICAL if necessary (notifying the user of the conversion), and pass the MATHEMATICAL model off to PowerModels' instantiate_model with ref_add_arcs_transformer! in ref_extensions, which is a required ref extension for PowerModelsDistribution.

pm_eng = instantiate_mc_model(eng, NFAPowerModel, build_mc_opf)

print(pm_eng.model)
Error: UndefVarError: eng not defined

This is equivalent to

import PowerModels

pm_math = PowerModels.instantiate_model(math, NFAPowerModel, build_mc_opf; ref_extensions=[ref_add_arcs_transformer!])

print(pm_math.model)
Error: UndefVarError: math not defined

Conclusion

This concludes the introduction to the ENGINEERING data model and conversion to the MATHEMATICAL model. We hope that you will find this new data model abstraction easy to use and simple to understand