Open source interoperability: integrating OSeMOSYS and PyPSA for power system analysis

Summary

Different energy system models are built and used in isolation, preventing the sharing of inputs and outputs between them

TransitionZero has developed an interoperable data model that unifies the formats used for data and results, applying it to the case of TZ-OSeMOSYS and PyPSA

The interoperable data model uses a combination of records and dimensions to address issues of ambiguous terms, different standards, messy geographies, and domain-specific terminology

Lost in translation

Energy system modellers have long faced the dilemma of how to marry long-term investment strategy with short-term operational reality in their modelling. Tools like TZ-OSeMOSYS excel at long-term, integrated energy planning, optimising capacity expansion and technology choices across multiple sectors and decades. They answer fundamental questions about what technologies to build and when to build them to meet demand cost-effectively, considering resource availability and policy constraints. However, TZ-OSeMOSYS, in service of its long-term focus, often employs a simplified representation of the electricity grid, abstracting away hourly resolution and network constraints that are paramount for reliable power system operation.

Conversely, PyPSA shines in high-resolution power system analysis. It can model detailed electricity networks, hourly dispatch, generator unit commitment, and the complex interplay of renewable variability and storage across geographical regions. PyPSA is a powerful tool for understanding how a given power system operates, the implications of grid bottlenecks, and the precise hourly costs and emissions. Yet, PyPSA typically operates over shorter time horizons or often requires pre-defined capacity expansion pathways, making it less suited for initial long-term, holistic investment planning across an entire energy system.

Developed in isolation, these frameworks lack interoperability. Data collected and processed for one model framework needs to be re-processed and formatted for use in the other. The same applies to model results, preventing coherent and consistent model outputs from being reliably and effectively compared and difficulties feeding the outputs of one into the inputs of the other.

Bridging the modelling gap with interoperability

To solve this at TransitionZero, we created a unified, interoperable data model – a standardised framework to define, categorise, and communicate data across the organisation. It allows input data and results compatible with TZ-OSeMOSYS's long-term, economy-wide investment decisions to be used interoperably with PyPSA's high-resolution network and operational constraints. This means that data and results from one model framework can be easily shared between and across multiple other modelling frameworks.

A unified, interoperable model confronts the messy reality of energy data:

1. Ambiguous terms: Classifying technologies isn't always straightforward. For example, pumped hydro could be categorised as storage or as a hydroelectric technology, depending on the model's specific use case. The data model requires thoughtful decisions that affect model behaviour.
2. Differing standards: Integrating external datasets is difficult because different organisations have unique classification systems. This requires manual adjustments and transformation rules to align external data with TransitionZero's internal model.
3. Messy geographies: Real-world administrative and grid boundaries are often overlapping and complex (e.g., the US grid). The model's ideal hierarchy (Assets → Region → Grid → Country) doesn't always apply, requiring nuanced structural decisions.
4. Communication between teams: Because modelling involves generalising complex concepts, there can be multiple 'correct' ways to define a dataset based on the research question, country, or team experience. TransitionZero addresses this with a dedicated Data Model Committee to standardise data treatment across all projects.

Despite the challenges, the data model is essential for scalability. It dramatically reduces data processing time, improves data accuracy, and streamlines collaboration. For the end user, this means faster modelling, more reliable results, and comprehensive analyses powered by a broader range of high-quality data and modelling frameworks. It ensures transparency, reproducibility, and sets a higher standard for energy system modelling.

Under the hood: What is an interoperable data model and how does it work?

An interoperable data model provides a common language for every new dataset by defining what technology or commodity it describes, where and when the data is valid, and its category.

Records

These are the actual data inputs for models, categorised by properties like unit (e.g., GW), temporality (e.g., annual, full year, typical profile), and validity (which technologies or commodities it applies to). For example, a single demand profile can be added into the model that is then configured to be used by both PyPSA and TZ-OSeMOSYS 'under the hood', including adjusting its temporality to match each framework.

Dimensions

These define the context for each record, answering: what (technology, commodity, operating mode), where (node, edge), and when (year, year-part, day-part). Dimensions are hierarchical, allowing the teams to aggregate upward (e.g., combining onshore and offshore wind data into a single 'wind' record) or broadcast downwards (eg. we can set a single coal price for the Philippines that can be applied to its individual grid regions) to fill data gaps.

This structured approach ensures that all teams use the same naming conventions when loading data, allowing for smoother data integration and enabling the use of the same datasets for building complex models with both TZ-OSeMOSYS and PyPSA. Ultimately, this interoperability enhances the transparency and reproducibility of energy research. By linking two of the most prominent open-source tools, it empowers a global community of researchers, policymakers, and industry professionals to conduct more comprehensive, credible, and policy-relevant energy system analyses.

Open

Next steps

This release represents our work towards a fully interoperable energy system modelling landscape. Future developments will focus on enhancing the depth of data mapping, exploring two-way coupling mechanisms where PyPSA insights can directly influence TZ-OSeMOSYS's iterative optimisation, and incorporating more modelling frameworks that address related topics like load flow modelling.

You can interact with this interoperable data model via Scenario Builder, a free, no-code electricity grid modelling software for energy transition planning. It showcases how an interoperable data model unlocks the capability to seamlessly move between capacity expansion and dispatch models with TZ-OSeMOSYS and PyPSA, respectively, under the hood.