Git Product home page Git Product logo

mojo's Introduction

System-model (mojo)

This repository describes the making of the system model framework, i.e. turning the raw data into into a computational structure.

This system model deals with of all the modelling choices and data reconciliation between the different data sources. The very core takes the raw data from the rdf database and turn it into a computational structure (such as an A-matrix). Then, it exports it again to the rdf database.

Hackathon objective (initial version of Bonsai)

For the purpose of the 2019 Barcelona hackathon, the electricity information in EXIOBASE needs to be updated with the information from the B-entso project. For the first simple model we use pySUT to construct an input-output table from supply and use tables (SUTs). pySUT takes numpy arrays (SUTs) as an input and outputs a numpy array A-matrix.

For this we will need to following:

  • Function to query exiobase rdf data and turn it into supply use tables. (main work)
  • Function to query electricity mix rdf data.
  • Function to map/aggregate the regional electricity data to the exiobase region/activity/unit name classification.
  • Function to insert updated electricity mix information into the the SUTs
  • Function (wrapper) to give user options which constructs to apply to turn the SUTs into an A matrix.
  • Function to write the A-matrix to the rdf database.

Methodology of mojo

Choice of the construct

For the initial version of Bonsai, it has been decided to use the Exiobase hybrid version.

With regard to hybrid tables, the choice on the construct to be used Majeau-Bettez et al. (2014) is very limited. The industry technology assumption is not an option since an activity may produce outputs in different units. Product technology assumption and by-product technology assumptions give the same results Suh et al. (2010), unless negative values in the IOTs are manipulated. Usually, this manipulation of negatives is not implemented in LCA. Therefore, the by-product technology results to be the best choice for the current version of Bonsai. However, in order to give more choice to the user, the monetary version of Exiobase could still be an option of further versions. The monetary version of Exiobase provides the product by product (pxp) and industry by industry (ixi) IOTs. The monetary IOTs adopt the industry technology assumption, which is an allocation method.

Resuming, the Bonsai user may have three choices:

  1. hybrid tables with by-product technology assumption (substitution method)
  2. monetary ixi IOTs (with allocation method)
  3. monetary pxp IOTs (with allocation method)

However, the current version of mojo only deals with bullet 1.

Why flows do not have a location

In Supply & Use tables the flows and flow Objects do not need to be specified by location, since they obtain that from the location of the activity. In a Direct Requirements Table the Supply data and the Use data are combined by applying some specific assumptions about how activities supply each other. In other words, each flow is connected to both a supplying and a using activity. The flows still do not need to be specified by location, since this is given by the supplying and using activities. Several assumptions can be be made for linking of activities and flows. Different Requirement Tables can be produced simply by changing the linking assumptions.

Yet, in the EXIOBASE hybrid Use table, we find that Flows have been provided with (country of origin) location, because EXIOBASE hybrid Use table combines data from two original data sources: The national Supply & Use tables and a balanced version of the international bi-lateral trade data. For transparency reasons, as well as to allow flexibility of use, is important to keep the original Supply & Use data separate from the linking assumptions. To do so the EXIOBASE Hybrid tables shall be de-constructed into the original national Supply & Use tables and a trade matrix, both of which can be stored in the BONSAI format, without the need to assign a location to any flow. The original Supply & Use data is simply obtained by summing all uses of each flow object into one flow. The trade information is preserved by taking the off-diagonal trade flows from the EXIOBASE Use table and placing them in the separate trade matrix.

This is furhter explained in the docs attached to this repository, in the file Why do flows not have a location and in the related excel file.

Aggregation and generation of global markets for exclusive by-products

Exiobase HSUTs have a format products by activities (pXa). There are 200 products and 164 activities. The initial step consists of aggregating the rows in order to get squared matrices. Before doing that, exclusive by-products are selected. Given a country, an exclusive by-product is a product carried out as secondary production and not produced elsewhere as principal production (determining production). For example blast furnace gas is a by-product of steel production and it is not produced in any other activity as principal production. In the current version of mojo, a global market is generated for each exclusive by-product.

A global market of product (GMP) is an activity with outputs and inputs. Inputs consist of all the principal productions of that specific product by activities across the world. It is important to notice that only principal productions can be inputs to GMPs.The output of a GMP is the sum of all the inputs and is defined as product from the global market (pGM) (for example, 'gas from the global market'). The pGM is used by any activity insted of exclusive by-products. In our example, an input of blast furnace gas will be replaced by an input of gas from the global market, which consists of natural gas produced in all countries across the world. In this case it is implicitly assumed that an activity can use blast furnace gas or natural gas indifferently. At the same time, as for activity producing exclusive by-products, it is assumed that the by-product will replace the product from the global market. In our example, when the steel activity produces blast furnace gas, it will reduce the supply of gas from the global market. Notice that the substitution of exclusive by-products with product from the global market is not valid for internal uses.

When building markets, it is important to take into account the properties of produts. Indeed exclusive by-products and the substituting principal productions may have different properties, therefore an homogenization must be performed. This is implemented by mean of conversion coefficients. For example, in the current version of mojo a global market of gas is homogenized using calorific values.

Figure

Figure 1 - Generation of global markets for exclusive by-products

Integration of ENTSO-E data: generation of electricity markets

The introduction of national electricity markets (NEMs) is necessary to integrate the ENTSO-E data into Exiobase. For each country/region of Exiobase, a NEM is built. NEMs can be seen as the national electricity grids, which provide electricity supplied by different power plants, including imported electricity. In Bonsai NEMs are activities with outputs and inputs. Inputs are directly taken from the ENTSO-E data and output are just the sum of these inputs. The output of electricity market is the electricity from the grid (EfG). Because of the by-product technology assumption, electricity from waste in the ENTSO data is excluded from the inputs to the market. Electricity carried out by waste treatment activities as by-product is assumed to replace the electricity from NEMs. In this way mojo internalizes the electricity from waste tretment in the system model.

Al the activities are assumed to use the electricity from the grid. Therefore the inputs of different types in Exiobase are replaced by an unique input form the grid.

Figure2

*Figure 2 - ENTSO-E data incorporated into Exiobase HSUTs

mojo's People

Contributors

jakobsarthur avatar micdr avatar stefano-mrc avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

mojo's Issues

include logger

Include a logging function. Will do this after the module is functional.

System Model: first version

The system model frame work. Turning the raw data into into a computational structure.

This project will be take care of all the modelling choices and data reconciliation between the different data sources. As the very core it will take the raw data from the rdf database and turn it into a computational structure (such as an A-matrix). It will then export it again to the rdf database.

This module relies on the electricity and exiobase data to be in the rdf database. As well as the availability of concordances between entso and exiobase (both region and electricity source) as well as unit correspondences (e.g. kWh/Joules etc)

Need support with aggregation to create square SUTs

We have now annotated most of the Exiobase data into RDF, however we face some challenges in annotating all the concepts from the ontology. Essentially we need square Supply and Use tables to be able to pick the determining flows when querying the data. I noticed that the mojo system model does something similar. Hence I would appreciate if the developers can support @IKnowLogic with this task.

function to query the exiobase data from rdf data base and turn into numpy SUT

This function requires the EIXOBASE data to be in the rdf database.

It should query the data the output should be a two numpy arrays with the SUT values and the meta data (process names, units etc) in separate arrays/to be defined data structures. this meta data will need to be used to later turn the A-matrix (IO-table) back into rdf format to be pushed back to the database.

function to insert updated electricity mix data into exiobase SUTs

this function creates an extra row and column into the supply and use tables. The new row in the Use table should be the "grid electricity" sum of all electricity inputs of every activity. the new column should be a new activity "electricity grid" which uses the updated electricity mix from the entso data.
In the Supply table it there should be a new activity producing grid electricity (this should produce the same amount as of grid electricity as the activity electricity grid uses.

The output should be numpy SUT arrays with updated electricity information

function to map/aggregate the regional electricity data to the exiobase classification

This function relies on the concordances between the ENTSO electricity data and EXIOBASE classification as well as the regional mapping between the two datasets (although the latter should be relatively straightforward).

The ENTSO regional electricity data needs to be mapped/aggregated to the exiobase regions. Secondly the electricity production processes need to be mapped to the EXIOBASE classification.

output: a exiobase-region X exiobase-electricity-source matrix* with the updated electricity mixes.

*The format does not need to be a matrix, but can be different (e.g. list of dictionaries, pandas dataframe, etc), in coordination with the function that injects the data into the exiobase SUTs.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.