16 August 2022 (1,815 words)
In this article we continue the Python Production mix series, using the Pyomo library. Specifically, we build Model 4, which changes Model 3 to:
- Import the data from an external json file.
- Read the data into the Model object, rather than into separate objects.
These changes reflect features that we may need to include in an operational model.
Articles in this series
Articles in the Python Production mix series:
- Python optimization Rosetta Stone
- Production mix - Model 1, Pyomo concrete
- Production mix - Model 2, Pyomo separate data
- Production mix - Model 3, Pyomo external data
- Production mix - Model 4, Pyomo json file
- Production mix - Model 5, Pyomo using def
- Production mix - Model 6, Pyomo abstract
- Production mix - Model 7, PuLP
- Production mix - Model 8, OR-Tools
- Production mix - Model 9, Gekko
- Production mix - Model 10, CVXPY
- Production mix - Model 11, SciPy
- Production mix - Conclusions
Download the model
The Python code and data for this model are available in the following files:
- Jupyter notebook: production-model-4.ipynb
- Python code: production-model-4.py
- Data: productiondata4.json
The "Jupyter notebook" file contains a formatted combination of Python code and markdown code – this file should be opened and run in Jupyter Lab. We describe setting up our Python environment, including Jupyter Lab and various Python libraries, in the article Set up a Python modelling environment.
The "Python code" file is a plain text file containing only the Python code of this model. Download this file if you have a non-Jupyter environment for running Python programs.
The "Data" file is a plain text file containing the model's data, in json format.
The model files are also available on GitHub.
Formulation for Model 4
For this model, we're using the same general formulation that we used for Model 3, as shown in Figure 1.
Model 4 Python code
The first task is to import the libraries that are needed for our program. As shown in Figure 2, in addition to the Pyomo and pandas libraries, we import the
json libraries – which we'll use to import the data from the json file.
The main difference between Model 4 and Model 3 is the data format. Specifically, as shown in Figure 3, we have the data in a json file. Json files are commonly used for storing and exchanging data, so it is useful to explore how to handle json data in a Python model. For more information about the json file format, see www.json.org.
Note that to edit a json file in Jupyter Lab, you'll need to right-click on the file and select Open with > Editor.
To load the json file, we use the
json libraries, as shown in Figure 4. This code loads all the json file data into a single object, which we'll parse in the next section.
The json file contains the same data as the Python file we used for Model 3. The difference is that we need to parse the json data into Python data structures before we can use it.
As shown in Figure 5, we continue to use a Pyomo concrete model. Note that we include the model's name in the declaration, so we can use
Model.name in the output, even though we don't explicitly define
Unlike Model 3, we assign each item of data to a Pyomo
pyo.Param object. We do this so that all the model's data is part of the
Model object. While this isn't strictly necessary, it does enable us to create more consistent model definitions (as we'll do in the next section).
To parse the single data values, we use the field names specified in the file. For example, to get the model's name, having read the json file into
Data, we use
Data['Name']. We access the other single data values in a similar way.
We read the coefficients structure in one step. Then we create a products index by getting the keys of the json
Coefficients structure (i.e., "Discs" and "Orbs"). We use the keys to define a Pyomo
pyo.Set. Note that a Pyomo Set (uppercase S) differs from a Python set (lowercase s).
Before we populate the objects that are indexed by product – i.e.,
Model.Margin – we declare them as "mutable", meaning that they can be changed. We need to do this as they are initially empty, with values assigned in the block that follows.
In contrast, the other values are immutable, by default. For example, if we add the line
Model.Hours = 1000 at the end of Figure 4, then we would get a runtime error telling us that
Model.Hours is immutable, so its value cannot be changed. To make the value of
Model.Hours changeable, we would need to add
, mutable = True to its definition.
Finally, most of the parameters are defined as
NonNegativeReals, meaning that they can take any value greater than or equal to zero. The exceptions are
Model.Engine, which is
pyo.Any because it contains text, and the values
Model.Margin, which we allow to be positive or negative.
Define the model
The model definition, as shown in Figure 6, is similar to Model 3. The differences are that we use the
Model object throughout, rather than using a mixure of Python data structures and the
For example, in Model 3 we used
Coefficients[p]['People'] * Model.Production[p], while in this model we use
Model.People[p] * Model.Production[p]. In a simple model, like this one, the distinction between these two approaches is not significant. For more complex models, the code is simpler, clearer, and easier to modify, when we use the
Model object consistently throughout the model definition.
As shown in Figure 7, the code for solving the model is almost the same as for Model 3 – except that we use the
Model object rather than individual Python data structures. Note that we need to use, for example,
pyo.value(Model.Engine) to get the value, rather than just using
The code for processing the solver result, as shown in Figure 8, is the same as for Model 3.
The code for writing the output, as shown in Figure 9, is almost the same as for Model 3 – again, except that we access the data values using, for example,
When we find an optimal solution, the output is shown in Figure 10. This output is the same as for Model 3, apart from the model's name.
Evaluation of this model
Model 4 explores an alternative external data file format, along with having a more consistent model definition.
Being able to handle different data formats is an important part of building operational models. Similarly, having a consistent model definition makes the building and maintenance of operational models easier and less error prone.
We have one more concrete model, which we'll explore in the next article of this series.
That is, we'll look at using a more flexible way of defining constraints and the objective, using
def function blocks. Using functions will give us more control over how a constraint or the objective function is defined, including making decisions about the parameters to use on a case-by-case basis, or whether we should skip defining a specific instance based on potentially complex criteria.
In addition, we'll output the slack values and dual prices (also known as shadow prices) for each constraint, to provide more information about the solution.
This article continues our exploration of optimization modelling in Python. Compared with Model 3, we loaded data from an external json file rather than using a Python data file, and we used Pyomo objects throughout the model, for greater consistency. In the next article, we'll continue improving the design.
If you would like to know more about this model, or you want help with your own models, then please contact us.