arfc / d3ploy Goto Github PK

A collection of Cyclus manager archetypes for demand driven deployment

License: BSD 3-Clause "New" or "Revised" License

Python 0.42% Dockerfile 0.01% Jupyter Notebook 99.58%

d3ploy's Introduction

d3ploy

A collection of Cyclus manager archetypes for demand driven deployment. It operates using global calls within a Cyclus simulation so that all agents within the simulation can communicate. The goal of this package is to provide three types of mathematical basis for predicting supply and demand of commodities within Cyclus; Non-optimizing (NO) deterministing optimization (DO), and Stochastic optimization (SO).

Dependencies

Cyclus: Fuel cycle simulation tool. Documentation

statsmodels: Python package for statistical analysis.Documentation

arch: Python package for conditional heteroskidasticity models.Documentation

Pmdarima: A python package for ARIMA/SARIMA methods. Documentation

Installation and Testing

Install from the d3ploy directory by typing python setup.py install and test with the command pytest. If tests fail due to module import errors, try pip3 install --upgrade scipy==1.2.0. Make sure the python version used to install cyclus and d3ploy is consistent. If Python 3 is being used, the python package dependencies should be installed with pip3.

demand_driven_deployment_inst and supply_driven_deployment_inst

demand_driven_deployment_inst

demand_driven_deployment_inst is a Cyclus Institution archetype that performs demand-driven deployment of Cyclus agents.

The institution works by using the chosen method to predict supply and demand for given commodities. Each timestep the institution calculates a prediction on the supply and demand for the next time step. If the demand exceeds the supply it deploys facilities to ensure supply exceeds demand. Predictions are not performed if the method chosen by the institution is 'moving average'. In this instance the institution will schedule facilities for deployment only if there is a deficit in the current time step.

This institution is used for facilities that exist in the front end of the fuel cycle.

supply_driven_deployment_inst

supply_driven_deployment_inst is a Cyclus Institution archetype that performs supply-driven deployment of Cyclus agents.

The institution works by using the chosen method to predict supply and capacity for given commodities. Each timestep the institution calculates a prediction on the supply and capacity for the next time step. If the supply exceeds the capacity it deploys facilities to ensure capacity exceeds supply. Predictions are not performed if the method chosen by the institution is 'moving average'. In this instance the institution will schedule facilities for deployment only if there is a deficit in the current time step.

This institution is used for facilities that exist in the back end of the fuel cycle.

Required Inputs for each institution

In the first four inputs, for demand_driven_deployment_inst, the facility included should be the facility that supplies the commodity and for supply_driven_deployment_inst, the facility included should be the facility that supplies capacity for that commodity.

facility_commod: This is a mapstringstring defining each facility and the output commodity to track. Every facility that the user wants to control using this institution must be included in this input.
facility_capacity: This is a mapstringdouble defining each facility and the (initial) capacity of the facility. Every facility that the user wants to control using this institution must be included in this input.
facility_pref: This is a mapstringstring defining each facility and the preference for that facility. The preference can be given as an equation, using t as the dependent variable (e.g. (1.01)**t). Only the facilities that the user wants to give preference values to need to be included in this input.
facility_constraintcommod: This is a mapstringstring defining each facility and the second commodity that constraints its deployment. Only the facilities that the user wants to constrain with a second commodity need to be included in this input.
facility_constraintval: This is a mapstringdouble defining each facility and the amount accumulated of the second commodity before the facility can be deployed. Only the facilities that the user wants to constrain by a second commodity need to be included in this input.
facility_sharing: This is a mapstringdouble defining the percentages with which more than one facility share the supply or capacity for one commodity.
driving_commod: The driving commodity for the institution.
demand_eq: The demand equation for the driving commodity, using t as the dependent variable.
calc_method: This is the method used to predict the supply and demand.
buffer_type: This is a mapstringstring defining each commodity and the type of supply/capacity buffer for it. For percentage, the user should input rel, for a absolute value, the user should input abs. The default is percentage.

Differing Inputs

DemandDrivenDeploymentInst:

supply_buffer: This is the amount above demand that the user wants the supply to meet. The user can define the buffer type in the state variable buffer_type. If the user wants a 20% value of supply higher than demand, they should input '0.2' and if the user wants a 100[whatever unit] value of supply higher than demand, they should input '100'.
installed_cap: This is a boolean to determine whether deployment is governed by supply of the commodity of installed capacity for that commodity.

SupplyDrivenDeploymentInst:

capacity_buffer: This is the amount above supply that the user wants the capacity to meet. The user can define the buffer type in the state variable buffer_type. If the user wants a 20% value of capacity higher than supply, they should input '0.2' and if the user wants a 100[whatever unit] value of capacity higher than supply, they should input '100'.
installed_cap: This is a boolean to determine whether deployment is governed by actual capacity of the commodity of installed capacity for that commodity.

Prediction Methods

Prediction methods are categorized in three - Non-optimizing, deterministic-optimizing, and stochastic-optimizing.

Non-Optimizing Methods

There are three methods implemented for the NO models. Autoregressive moving average (ARMA), and autoregressive conditional heteroskedasticity (ARCH). There are four parameters users can define:

steps: Number of timesteps forward to prdict supply and demand (default = 1)
back_steps: Number of steps backwards from the current timestep to use for the prediction (default = 5)
supply_std_dev = Standard deviation adjustment for supply (default = 0)
demand_std_dev = Standard deviation adjustment for demand (default = 0)

MA (`ma`)

ARMA (`arma`)

The autoregressive moving average method takes a time series and uses an auto regressive term and a moving average term.

ARCH (`arch`)

The Autoregressive Conditional Heteroskedasticity (ARCH) method predicts the future value by using the observed values of returns or residuals.

Deterministic Optimization

There are three methods implemented for the DO models. Polynomial fit regression, simple exponential smoothing, and triple exponential smoothing (holt-winters). There are two parameters users can define:

back_steps: Number of steps backwards from the current timestep to use for the prediction (default = 10)
degree : degree of polynomial fit (default = 1)

Polynomial fit regression (`poly`)

The polynomial fit regression method fits a polynomial equation of degree k (degree) for the n (back_steps) previous values to predict the next value. A polynomial equation of degree 1 is a linear equation (y = ax + b) This method is suitable for values with a clear trend.

Exponential smoothing (`exp_smoothing`)

The exponential smoothing method takes the weighted average of past n (back_steps), in which more weight is given to the last observation. This method is suitable for values with no clear trend or pattern.

Triple exponential smoothing, Holt-Winters (`holt_winters`)

The triple smoothing method combines three smoothing equations - one for the level, one for trend, and one for the seasonal component - to predict the next value. This method is suitable for values with seasonality.

Fast Fourier Transform (`fft`)

(EXPERIMENTAL) The method builds a function that represents the data as a sumation of harmonics of different order. In the case of having a set of data that presents oscilations the user should set the degree to 2.

Stochastic Optimization

Currently a work in progress

Demand Fac

This facility is a test facility for D3ploy. It generates a random amount of supply and demand for commodities, and then reports these using the RecordTimeSeries functions inside of Cyclus.Thus providing a supply and demand to the institutions.

Amount of commodity demanded and supplied can be determined randomly through their minimum and maximum values. If for instance you'd like variability in the production rate of your supply you can set these minimum and maximum values to reflect that.

Required Inputs

demand_commod: This is the commodity that the facility demands in order to operate.
demand_rate_min: Minimum amount of the demanded commodity needed.
demand_rate_max: Maximum amount of the demanded commodity needed.
supply_commod: The commodity that the facility supplies.
supply_rate_min: Minimum rate of production of the supplied commodity.
supply_rate_max: Maximum rate of production of the supplied commodity.

Optional Inputs

demand_ts: The amount of time steps between demanding material. For example a reactor may only demand material every 18 months.
supply_ts: The amount of time steps between supplying material.

d3ploy's People

Contributors

Stargazers

Watchers

Forkers

flanflanagan scopatz katyhuff jbae11 gwenchee kipk49 robfairh nuclearkatie abachma2

d3ploy's Issues

Goal oriented reporting

This issue can be closed when the NO report is updated to be more focused on the goals of the NO functionality of d3ploy.

Add DO prediction algorithms

Add deterministic prediction algorithms for predicting demand.

These functions should be implemented in a separate file, and called into the main solver file

addlibrecordtimeseries

addlibrecordtimeseries for the calculated supply and demand for each commodity

include the institution ID so we don't double count

add notion of technology preference

technology preference over time, possibilities:

equation for each technology preference
off/on switch for when new technology can begin being deployed

Error message if supply chain is broken

This is to catch instances of users making a mistake and not properly choosing commodities for their simulations.

Building logic update

Updating the logic from random choice to a more organized operation.

d3ploy library not found

putting solver into a separate file causes this error when trying to run a cyclus simulation:
No module found for path libd3ploy.no_inst.so

tag no_inst_fleet_unit_tests

Tag no_inst_fleet_unit_tests.py so that the test skips,to be incorporated into the continuous integration

map typesystem on python does not work

I am testing the a simple map typesystem for no_inst.py. An error is thrown when running a cyclus simulation.
State variable declaration in no_inst.py
test_out = ts.MapIntString( )

XML schema:
<test_out> <item> <key>3</key><val>POWER</val> </item> </test_out>

Cyclus output error:
AttributeError: 'list' object has no attribute 'encode' Exception ignored in: 'cyclus.typesystem.str_py_to_cpp' AttributeError: 'list' object has no attribute 'encode' XPath error : Invalid expression terminate called after throwing an instance of 'xmlpp::exception' what(): Invalid XPath: Aborted (core dumped)

Driving commodity

The demand driving commodity (the one governed by demand equation - growth rate and initial demand) is currently hard-coded as power. The user should be able to decide the demand-driving commodity.

This issue can be closed when the hard-coded demand-driving commodity is changed to a user input.

self.commodity_demand[commod] is a dictionary, but can be an array

currently, the variable self.commodity_demand[commod] (also self.commodity_supply[commod]) is a dictionary with:

key: time (integer)
value: supply or demand quantity

I personally think that this would be better if it was a numpy array. This would eliminate the need for this line because the arrays could be initialized as zeros in the beginning.

What do you think @FlanFlanagan ?

resolve pep8 issues

Threading.py error

Threading error:
Exception ignored in: <module 'threading' from '/home/gwenchee/anaconda3/lib/python3.6/threading.py'> Traceback (most recent call last): File "/home/gwenchee/anaconda3/lib/python3.6/threading.py", line 1292, in _shutdown assert tlock.locked() SystemError: <built-in method locked of _thread.lock object at 0x7fb09f34d6c0> returned a result with an error set
This error appears when time series listeners line is used in no_inst.py
lib.TIME_SERIES_LISTENERS[~].append(~)

Algorithm test performance metric

A more structured discussion (with documentation) needs to be made about methods to calculate the algorithm performance.

Currently, the metric is:
error = sum( supply_timeseries - demand_timeseries)

more plots for test file

plot calculated demand and supply in the test plot outputs

criteria for tests

Set tolerance as 1 capacity unit for tests with not many facilities

Create docker image with Cyclus, cycamore and d3ploy deps

Make cycamore enrichment facility compatible with d3ploy

put in a calculation that converts SWU to mass

prototype input for the no_inst

add prototype input to no_inst to ensure no prototype overlap.

deploy solver in separate file

move deploy solver in separate file

and no_inst should import from that file

build tests

write a test file to include in circle ci to check if all prediction algorithms are functioning

add DO tests

add tests for DO algorithms

add expected failure tests

Add expected failure tests for supply to be within 1 facility tolerance of demand

Fleet based unit tests updated to reflect d3ploy changes

The NO_Inst segments of the mock simulations in no_inst_fleet_unit_tests.py need to be updated to reflect new d3ploy NO_inst.py structure. Only after #18, #19 and #20 have been implemented.

Create a new Cyclus typesystem: mapstringmapstringdouble

Create a new Cyclus typesystem: mapstringmapstringdouble so that the input {commod:{proto:capacity}} can be done.

update tests to plot demand vs supply

Update no_inst_fleet_unit_tests.py to plot the supply and demand for fuel for each sqlite output

Implement a simple vector string state variable for commodities, prototypes and capacities

The ideal input type for NO_Inst commodities, prototypes and capacities information is #18 , however, we are running into an error implementing it: #33. To ensure that we can move forward with developing the back end while simultaneously fixing this bug, a simple vector string state variable will be used intermediately. It will look like this:
<commodinfo> <val>commod_prototypename_capacityvalue</val> </commodinfo>

tick or tock

In line 137 of no_inst.py the function is tick but the docstring states that it is a tock. Clarification required

processing capacity demand

@jbae11 ?

deploy solver test not importing from no_inst.py

The deploy_solver_test.py test file should just import no_inst.py and call the function from there rather than copying the function in the test file

Import new typesystem in no_inst.py for prototype, commod and capacity information

Import the typesystem: mapintmapstringdouble into no_inst.py and put commodity, prototype and capacity information into it in this format: {1:{prototype:capacity}} where the number corresponds to its supplied commodity. Then, move the information from this mapintmapstringdouble into a mapstringmapstringdouble in no_inst.py where the int is replaced by string name of the relevant commodity.

Travis to Circle

Convert from .travis.yml ci to circle ci. Use docker images cyclus/cyclus-dep to shorten ci testing time.

add --core-version 9999.99 to resolve issue #58

We need typsystem.MapStringPairStringMapStringString

The desired input structure for commodity and facility definition is:

<commodity>
    <val>power</val>
        <item>
            <facility_name>LWR</facility_name>
            <info>
                <cap>1000</cap>
                <pref>100-3*t</pref>
            </info>
        </item>
        <item>
            <facility_name>sfr</facility_name>
            <info>
                <cap>600</cap>
                <pref>3*t</pref>
            </info>
        </item>
</commodity>

Add smarter logic to deployment choices in d3ploy

Have d3ploy determine facility production capabilities in a more intelligent way.

Possible DO implementation prediction

delay coordinate embedding

large scale automated forecasting usually fractals

More rigorous tests

Rigorous tests that reflects real world nuclear fuel cycle more closely

normal reactor behaviour: refuel time --> 1, cycle time > 2, lifetime
enrichment facility
chain becomes: source -> enrichment -> reactor -> sink
other chains: source -> enrichment -> lwr reactor -> reprocessing -> sfr reactor -> sink

Decommission in NO archetype.

Remember to add decommissioning behavior to region NO archetype.

NO_Inst reactor not building

NO_Inst does not built facilities when demand eq is a constant value

Tests for large scale deployment

This issue can be closed when tests are designed and implemented for large scale deployment for both the non-optimizing and deterministic optimizing cases. The tests will compare a commodity demand and the supply produced by the algorithm. Based on a determined tolerance percentage, the test will pass or fail.

Demand equation variability

Currently the demand equation is given by the equation
initial_demand * ((1 + growth_rate)**time).

However, it would be nice if more options are provided, like the piecewise functions
available in cycamore::Growthregion.

Change tock logic to occur in decision phase

Change tock logic to occur in decision phase once decision phase is implemented in Cyclus

Random prototype selection

Currently, d3ploy finds a random prototype among the lists of prototypes in the input parameter,
and calculates the prod_rate by dividing the supply by the number of children of the institute.
( Or in the most recent release this is not the case)

Maybe it would be more beneficial to simply have one prototype (vector prototypes -> str prototype)
and calculate the prod_rate by doing supply / len(self.children).

Any thoughts on this @FlanFlanagan ?

update readme regarding demand equation

Update the README:

Make it clear to the user that the demand_eq should have the same time step length as the dt that they defined in the simulation control. Default dt is month-long and so, default demand_eq will also be a month.

Cyclus Cycamore docker image is outdated

Cyclus Cycamore docker image is outdated, causing the ci to fail.

This issue can be closed when
https://hub.docker.com/r/cyclus/cycamore/
is updated with the most recent cyclus master.

DO prediction algorithms implementation

Linear regression
Polynomial regression
Exponential smoothing (simple, Triple)

Add various growth rates for demand

Update the demand_calc function in no_inst.py to have various types of growth rate curves. Possible extensions could be linear and piece-wise.

set up continuous integration

The continuous integration should check if:

python install.py cyclus:FlanFlanagan:time_series
python install.py cycamore:FlanFlanagan:d3ploy_update
python setup.py install builds d3ploy
cyclus test_cycamore.xml
check if the sqlite output has all the timeseries tables

Bump up all tests that have deployment at time step 1 to deploy at time step 2

Facilities cannot be deployed until the first DRE and tock phases, therefore, tests can only pass if expected deployment occurs after the first time step. This issue can be closed when tests that have deployment at time step 1 are moved to only deploy at time step 2 or an initial facility list that includes the facility that wants to be commissioned at time step 1 is added.

Time issue with demand_eq

Currently, the eval function evaluates the demand_eq state variable in no_inst.py with the notion that time is in seconds. However, it is unclear to the user what time step length the t variable should be.

This issue can be closed when:

A state variable that defines what the demand eq’s time step length is: second/day/month/year. The default will be set as month.
The README is updated to tell the user that this new state variable must be the same as the dt variable defined in the control section of the cyclus input file.

Unit test for solvers

We need unit tests for all solvers,