Matgas Format (.m)

Here we detail the parameters that can be inputted using the matgas format. They can be inputted in the order they appear here, or selectively, in the case where some data is not required, by using the following header format.

%% junction data
% id p_min p_max p_nominal junction_type status pipeline_name edi_id lat lon

See case files in test/data/matgas or examples/data/matgas for examples of file syntax.

Junctions (mgc.junction)

These components model “point” locations in the system, i.e. locations of withdrawal or injection, or simply connection points between pipes. Each junction may have multiple pipes attached.

VariableTypeNameStandard Units (SI)United States Customary UnitsRequiredDescription
idIntJunction idYUnique id for junctions
p_minFloat64Pressure MinimumPascalPSIYOptimization constraint
p_maxFloat64Pressure MaximumPascalPSIYMaximum operating pressure (MOP, psig) used in line pack calculations, which is lower than the maximum allowable operating pressure (MAOP, psig)
p_nominalFloat64PressurePascalPSIYNominal pressure, can have a default that lies between the pressure min/max
junction_typeIntJunction TypeYClassification of the junction: 0 = standard node, 1 = slack node
statusIntJunction StatusYDetermines if the component is active in the model
pipeline_nameStringPipeline NameName of the pipeline to which this junction belongs
edi_idStringEDI IDEDI ID if applicable, to enabled easy input of near-real-time EDI data
latFloat64LatitudeDecimal degreesDecimal degreesLatitude of the junction
lonFloat64LongitudeDecimal degreesDecimal degreesLongitude of the junction

Pipes (mgc.pipe)

These components model pipelines which connect two junctions.

VariableTypeNameStandard Units (SI)United States Customary UnitsRequiredDescription
idIntPipe idYUnique id for pipes
fr_junctionIntFrom Junction idYUnique id of the junction on the from side
to_junctionIntTo Junction idYUnique id of the junction on the to side
diameterFloat64DiameterMetersInchesYPipe diameter
lengthFloat64LengthMetersMilesYPipe Length
friction_factorFloat64Friction FactorYparameter based on the relative roughness of the pipe and Reynolds Number
p_minFloat64Pressure MinimumPascalPSIYFloat value link minimum pressure used as optimization constraint, typically set equal to 14.7 psia, which is equivalent to 1.01325 bar
p_maxFloat64Pressure MaximumPascalPSIYMaximum allowable operating pressure (MAOP,psig) for a given pipeline segment
statusIntPipe statusYDetermines if the component is active in the model
is_bidirectionalIntBi-directionalitySpecifies whether the pipe supports bi-directional flow
pipeline_nameStringPipeline NameName of the pipeline to which this pipe belongs
num_spatial_discretization_pointsIntSpace pointsNumber of spatial discretization points in the pipe, used for Transient calculations

Compressors (mgc.compressor)

These components model infrastructure used to boost pressure between two nodes.

VariableTypeNameStandard Units (SI)United States Customary UnitsRequiredDescription
idIntCompressor idYUnique id for compressor
fr_junctionIntFrom Junction idYUnique id of the junction on the from side
to_junctionIntTo Junction idYUnique id of the junction on the to side
c_ratio_minFloat64Minimum compression ratioYMinimum compression ratio
c_ratio_maxFloat64Maximum compression ratioYMaximum compression ratio
power_maxFloat64Maximum powerWHPYFloat value compressor max power, used as optimization constraint
flow_minFloat64Minimum mass flowkg/sYMinimum mass flow
flow_maxFloat64Maximum mass flowkg/sYMaximum mass flow
inlet_p_minFloat64Minimum inlet pressurePascalPSIYMinimum pressure at compressor inlet
inlet_p_maxFloat64Maximum inlet pressurePascalPSIYMaximum pressure at compressor inlet
outlet_p_minFloat64Minimum outlet pressurePascalPSIYMinimum pressure at compressor outlet
outlet_p_maxFloat64Maximum outlet pressurePascalPSIYMaximum pressure at compressor outlet
statusIntCompressor statusYDetermines if the component is active in the model
operating_costFloat64Operating cost/W/kWCost to operate compressor
directionalityIntDirectionalityAn integer value that denotes the directionality of compression and flow: 0 = bi-directional compressor, 1 = unidirectional compressor with no reversal for flow allowed, 2 = unidirectional compressor which allows for uncompressed reversal of flows
compressorstationnameStringCompressor Station NameName of compressor stations
pipeline_nameStringPipeline nameName of Pipeline
total_installed_powerFloat64Total installed powerWWTotal installed horsepower – same as maximum power
num_compressor_unitsIntNumber of compressor unitsNumber of compressor units – needed to account for number of gas-fired compressors versus those driven by electric power
compressor_typeStringCompressor TypeCompressor type (reciprocal versus turbine) – used to determine compressor efficiency
design_suction_pressureFloat64Design Suction PressurePascalPSIUsed to estimate compressor max pressure delta
design_discharge_pressureFloat64Design discharge pressurePascalPSIUsed to determine maximum extent of line pack for downstream pipeline segments
max_compressed_volumeFloat64Maximum compressed volumeMaximum volume compressed at design conditions
design_fuel_requiredFloat64Design fuel requiredMMSCFDFuel required at design conditions
design_electric_power_requiredFloat64Design electric power requiredkWh/dayElectric power required at design conditions – determines which compressor stations could be affected by an electric outage
num_units_for_peak_serviceIntNumber of units for peak serviceNumber of units in service during peak conditions
peak_yearIntPeak yearYear of peak conditions

Short Pipes (mgc.short_pipe)

These components model, e.g., handling complicated contract situations at single entry points; they are modeled to have zero resistance.

VariableTypeNameStandard Units (SI)United States Customary UnitsRequiredDescription
idIntShort pipe idYUnique id for short pipe
fr_junctionIntFrom Junction idYUnique id of the junction on the from side
to_junctionIntTo Junction idYUnique id of the junction on the to side
statusIntShort Pipe statusYDetermines if the component is active in the model
is_bidirectionalIntBi-directionalitySpecifies whether the pipe supports bi-directional flow
pipeline_nameStringPipeline NameName of the pipeline to which this pipe belongs

Resistors (mgc.resistor)

These components model pressure drops for which no other data or models are available.

VariableTypeNameStandard Units (SI)United States Customary UnitsRequiredDescription
idIntResistor idYUnique id for resistor
fr_junctionIntFrom Junction idYUnique id of the junction on the from side
to_junctionIntTo Junction idYUnique id of the junction on the to side
dragFloat64Drag factorYthe drag factor of the resistors - non dimensional value
statusIntResistor statusYDetermines if the component is active in the model
is_bidirectionalIntBi-directionalitySpecifies whether the pipe supports bi-directional flow
pipeline_nameStringPipeline NameName of the pipeline to which this pipe belongs

Loss Resistors (mgc.loss_resistor)

These components model constant pressure drops along edges.

VariableTypeNameStandard Units (SI)United States Customary UnitsRequiredDescription
idIntLoss resistor idYUnique id for resistor
fr_junctionIntFrom Junction idYUnique id of the junction on the from side
to_junctionIntTo Junction idYUnique id of the junction on the to side
p_lossFloat64Pressure lossPascalPSIYConstant pressure loss across the loss resistor
statusIntLoss resistor statusYDetermines if the component is active in the model
is_bidirectionalIntBi-directionalitySpecifies whether the loss resistor supports bi-directional flow

Regulators (mgc.regulator)

These components model pressure reducing valves.

VariableTypeNameStandard Units (SI)United States Customary UnitsRequiredDescription
idIntRegulator idYUnique id for regulator
fr_junctionIntFrom Junction idYUnique id of the junction on the from side
to_junctionIntTo Junction idYUnique id of the junction on the to side
_factor_minFloat64Minimum reduction factorYThis value is necessarily < 1
reduction_factor_maxFloat64Maximum reduction factorYDefault value = 1, but can be < 1
flow_minFloat64Minimum flowkg/sMMSCFDYMinimum flow through the regulator
flow_maxFloat64Maximum flowkg/sMMSCFDYMaximum flow through the regulator
statusIntRegulator statusYDetermines if the component is active in the model
discharge_coefficientFloat64Discharge coefficientYMore commonly known as the "flow coefficient Kv"
design_flow_rateFloat64kg/sMMSCFDMaximum design flow
design_inlet_pressureFloat64PascalPSIPressure upstream of the regulator
design_outlet_pressureFloat64PascalPSIPressure downstream of the regulator
pipeline_nameStringPipeline NameName of the pipeline to which this pipe belongs

Valves (mgc.valve)

These model components which close off flow between two points in a natural gas network.

VariableTypeNameStandard Units (SI)United States Customary UnitsRequiredDescription
idIntValve idYUnique id for valve
fr_junctionIntFrom Junction idYUnique id of the junction on the from side
to_junctionIntTo Junction idYUnique id of the junction on the to side
statusIntValve statusYDetermines if the component is active in the model
flow_coefficientFloat64Flow CoefficientYCoefficient of flow (Cv) – used to determine a valve’s flow under various conditions (e.g., potentially partially open)
pipeline_nameStringPipeline NameName of the pipeline to which this pipe belongs

Transfers (mgc.transfer)

These components model interconnects, i.e. points in the network that can act both as a receipt point or delivery point at different times of the day (redundant for steady-state). In practice, a transfer could be modeled as a receipt point and a delivery point, however, modeling it as a transfer allows for explicitly stating that a boundary condition can act as either an injection or consumption–but not both.

VariableTypeNameStandard Units (SI)United States Customary UnitsRequiredDescription
idIntTransfer idYUnique id for transfer
junction_idIntJunction idYUnique id of Junction to which component is connected
withdrawal_minFloat64Minimum Withdrawalkg/sMMSCFDYThis number can be negative, in which case it means that the transfer point is injecting gas into the system
withdrawal_maxFloat64Maximum Withdrawalkg/sMMSCFDYthis variable can depend on flow direction
withdrawal_nominalFloat64Nominal Withdrawalkg/sMMSCFDYCan have a default of 0.0
is_dispatchableIntDispatchableYIf the component is marked as dispatchable, it means that it can vary its withdrawal between its minimum and maximum. If not, then the component is injecting or withdrawing exactly at the nominal withdrawal rate
statusIntTransfer statusYDetermines if the component is active in the model
bid_priceFloat64Bid Price/kg/volBid price
offer_priceFloat64Offer Price/kg/volOffer price
exchange_point_nameStringExchange Point NameName of Exchange point
pipeline_nameStringPipeline NameName of pipeline to which the transfer belongs
other_pipeline_nameStringOther Pipeline NameName of pipeline to which the transfer connects
design_pressureFloat64Design pressurePSIMaximum designed pressure
meter_capacityFloat64Meter CapacityMMSCFDProvided by real-time data collection (EDI), model constraint
daily_scheduled_flowFloat64Daily scheduled flowMMSCFDProvided by near-real-time data collection (EDI)

Receipts (mgc.receipt)

These components model consumers of natural gas.

VariableTypeNameStandard Units (SI)United States Customary UnitsRequiredDescription
idIntReceipt idYUnique id for receipt
junction_idIntJunction idYUnique id of Junction to which component is connected
injection_minFloat64Minimum Injectionkg/sMMSCFDYMinimum amount of gas that can be injected
injection_maxFloat64Maximum Injectionkg/sMMSCFDYMaximum amount of gas that can be injected
injection_nominalFloat64Nominal Injectionkg/sMMSCFDYNominal gas injection
is_dispatchableIntDispatchableYIf the component is marked as dispatchable, it means that it can vary its injection between its minimum and maximum. If not, then the component is injecting exactly at the nominal injection rate
statusIntReceipt statusYDetermines if the component is active in the model
offer_priceFloat64Offer Price/W/volOffer price
nameStringReceipt NameName of receipt point
company_nameStringCompany NameName of company that owns receipt point
daily_scheduled_flowFloat64Daily Scheduled FlowMMSCFDProvided by real-time data collection (EDI), model constraint
design_capacityFloat64Design CapacityMMSCFDProvided by real-time data collection (EDI), model constraint
operating_capacityFloat64Operating CapacityMMSCFDProvided by real-time data collection (EDI), model constraint
is_firmIntInterruptable vs FirmIdentifies the order by which the receipt flow could be curtailed (interruptible first, then firm)
edi_idIntEDI IDUnique ID to allow easy input of near-real-time EDI data

Deliveries (mgc.delivery)

These components model producers (providers) of natural gas.

VariableTypeNameStandard Units (SI)United States Customary UnitsRequiredDescription
idIntDelivery idYUnique id for delivery
junction_idIntJunction idYUnique id of Junction to which component is connected
withdrawal_minFloat64Minimum withdrawalkg/sMMSCFDYMinimum amount of gas that can be withdrawn
withdrawal_maxFloat64Maximum withdrawalkg/sMMSCFDYMaximum amount of gas that can be withdrawn
withdrawal_nominalFloat64Nominal withdrawalkg/sMMSCFDYNominal gas withdrawal
is_dispatchableIntDispatchableYIf the component is marked as dispatchable, it means that it can vary its injection between its minimum and maximum. If not, then the component is injecting exactly at the nominal injection rate
statusIntDelivery statusYDetermines if the component is active in the model
bid_priceFloat64Bid Price/W/volBid price
nameStringDelivery NameName of delivery point
company_nameStringCompany NameName of company that owns delivery point
daily_scheduled_flowFloat64Daily Scheduled FlowMMSCFDProvided by real-time data collection (EDI), model constraint
design_capacityFloat64Design CapacityMMSCFDProvided by real-time data collection (EDI), model constraint
operating_capacityFloat64Operating CapacityMMSCFDProvided by real-time data collection (EDI), model constraint
is_firmIntInterruptable vs FirmIdentifies the order by which the d flow could be curtailed (interruptible first, then firm)
edi_idIntEDI IDUnique ID to allow easy input of near-real-time EDI data

Storage (mgc.storage)

These components are used to model locations which can store and release natural gas.

VariableTypeNameStandard Units (SI)United States Customary UnitsRequiredDescription
idIntStorage idYUnique id for storage
junction_idIntJunction idYUnique id of Junction to which component is connected
pressure_nominalFloat64Nominal pressurePascalPSIYpressure at the storage
flow_injection_rate_minFloat64Minimum injection ratekg/sMMSCFDYMinimum flow injection rate
flow_injection_rate_maxFloat64Maximum injection ratekg/sMMSCFDYMaximum flow injection rate
flow_withdrawal_rate_minFloat64Minimum withdrawal ratekg/sMMSCFDYMinimum flow withdrawal rate
flow_withdrawal_rate_maxFloat64Maximum withdrawal ratekg/sMMSCFDYMaximum flow withdrawal rate
capacityFloat64CapacitykgMMSCFYCapacity of the storage
statusIntStorage statusYDetermines if the component is active in the model
nameStringStorage NameName of the Storage point
owner_nameStringOwner NameName of the Storage point owner
storage_typeStringStorage typeType of storage (aquifer, salt cavern, or depleted oil and gas)
daily_withdrawal_maxFloat64Daily withdrawal ratekg/sMMSCFDMaximum daily withdrawal
seasonal_withdrawal_maxFloat64Seasonal withdrawal ratekg/sMMSCFDMaximum Seasonal withdrawal
base_gas_capacityFloat64Base gas capacitykgMMSCFBase gas capacity
working_gas_capacityFloat64Working gas capacitykgMMSCFWorking gas capacity
total_field_capacityFloat64Total field capacitykgMMSCFTotal field gas capacity
edi_idIntEDI IDUnique ID to allow easy input of near-real-time EDI data

Network Parameters (mgc.parameter)

VariableTypeNameStandard Units (SI)United States Customary UnitsRequiredDescription
gas_specific_gravityFloat64Specific GravityGas gravity
specific_heat_capacity_ratioFloat64Specific heat capacityspecific heat capacity ratio
temperatureFloat64TemperatureKtemperature of the network
sound_speedFloat64Speed of Soundm/sspeed of sound in gas
RFloat64Universal Gas ConstantJ/mol/KUniversal gas constant
gas_molar_massFloat64Molar Masskg/molMolar Mass
compressibility_factorFloat64Compressibility factorCompressibility factor (unitless)
base_pressureFloat64Base PressurePascalPSIBase pressure in Pa or psi
base_lengthFloat64Base LengthMetersMilesBase length in m or miles
base_timeFloat64Base TimeHoursHoursBase time in hr
unitsStringUnits'si' for standard units or 'usc' for United States customary units
is_per_unitIntPer-unitIf data is already in per-unit (non-dimensionalized
nameStringCase NameName of Network Case
yearIntYearYear from which data originated

Matgas extensions

The matgas format supports extensions which allow users to define arbitrary components and data which are used in customized problems and formulations. For example, the syntax

  %column_names% data_field_name1, data_field_name2, ...
  mgc.component_data = [
  data1, data2
  data1, data2
  ...
  ]

is used to add data to standard gas components. In this example, the data dictionary for component will be augmented with fields called data_field_name1, data_field_name2, etc. The names trailing the keyword %column_name% are used as the keys in the data dictionary for each component. The key word mgc.component_data is used to indicate the component the new data should be associated with. For example, mgc.pipe_data adds the data to pipes. The data should be listed in the same order as used by the tables that specify the required data for the component. The syntax

  %column_names% data_field_name1, data_field_name2, ...
  mgc.component = [
  data1, data2
  data1, data2
  ...
  ]

is then used to specify completely new components which are not specified in the default format. This example would create an entry from components called component in the data dictionary.

Transient Data Format (CSV)

The transient/time-series data will come in as one .csv file (comma separated format). The first column in the CSV format will be a time stamp in the date-time format YYYY-MM-DDTHH:MM:SS+HH:MM, where the +HH:MM indicates the timezone offset, the second column will be the component type, the third column will be the component id, the forth column will be the parameter name, and the fifth column will be the value of the parameter. See below example for valid header names.

timestamp,component_type,component_id,parameter,value
1992-09-12T00:00:00.0+00:00,delivery,1,withdrawal_nominal,0.104678
1992-09-13T00:00:00.0+00:00,delivery,1,withdrawal_nominal,0.540400
1992-09-14T00:00:00.0+00:00,delivery,1,withdrawal_nominal,0.929477
1992-09-15T00:00:00.0+00:00,delivery,1,withdrawal_nominal,0.412720