Decide what to put in and make a readme file for the project. Have it use the .rst extension so github can render it pretty.

Consider ways of making simple cases of table creation (e.g. tables with no fk relationships) "easier" for developers that want a turn-key way of recording some basic module-specific simulation data.

In order for developers to follow a tutorial for making a new module, we need to provide a complete set of stubs for tests and input. We also need to provide some useful demonstration of including relaxNG schema.

Feature Description :
The Stub files intended to be used as module templates should be reviewed for clarity, updated to match current interfaces, and documented to indicate their use.

Subtasks :
Describe tasks that are necessary for this feature to be implemented. If
there is an associated issue describing the subtask, report it as a related
issue below.

1. Confirm Stub files exist for all types of module for which they would be helpful
2. Perhaps create testing stub files?
3. Update Stub files to match current interfaces
4. Document Stub files to indicate their use
5. Review Stub files for clarity    

If there is necessary documentation for this feature, make sure there is an
associated issue built for that documentation.

Next Action: select serialization to support more flexible initialization/Factory concepts

Currently the utility provided by the InputXML class only works if a cyclus input file has gone through the entire load process. This is undesirable for creating unit tests/fixtures. The InputXML class needs to be modified so that code that constructs objects from input file xml runs successfully given an xmlNodePtr that was created from a std::string test fixture. The code in InputXML class now (e.g. get_xpath_content, get_xpath_elements) only runs if curFilePtr among other things have been properly initialized (they are only initialized now if the InputXML::load_file method is invoked). Basically, the get_xpath_[stuff](xmlNodePtr node, std::string expression) methods need to work without any file loading initialization having taken place in the InputXML class. Sorry I can't be more clear - I am still quite unfamiliar with libxml2.

Current action: #243

This was discovered while trying to implement issue #30. Current Agent list recording paradigm:

• the simulation runs the entire specified duration
• at sim end, BookKeeper queries a unversal (static) model list kept in the Model class
• BookKeeper builds structured tables of model data
• BookKeeper writes data to file.

Problems with this:

• Model bookkeeping is only performed at the end of the simulation - no ability for 'chunking', buffering
• Forces us to keep all model objects in memory for duration of the simulation - we can never use destructors - at least not until after BookKeeper queries the model list and builds the structured data.

Proposed solution:

• BookKeeper keeps a map of structs: keys=model-ID, values=structs filled with the agent data (ID, name, impl, birth, death, isTemplate, etc.)
• Whenever a model undergoes any change, it invokes BI->registerAgentDatum(model_id, "[parameter name]", param_new_value).

This will allow the bookkeeper to write as often as it wants. And since it won't store/use the actual model objects (or pointers to them) themselves, the process is unaffected by the deallocation of model objects. This will eliminate the need to keep is/isn't template-segregated static lists in the Model class - just kidding - I think we need to keep the lists separate to facilitate fast/clean access to templates throughout the simulation regardless of how many non-template models there are.

Target Date :
ASAP

Feature Description :
Our instructions for getting and building Cyclus are very unix/linux centric. As our first potential developers on Windows platforms are trickling in, it has become clear that we need detailed instructions (STEP-BY-STEP) for getting and building Cyclus on a windows machine. This will require stepping through the entire process on a windows machine (or two).

Subtasks :
Describe tasks that are necessary for this feature to be implemented. If
there is an associated issue describing the subtask, report it as a related
issue below.

1. Step through the process of getting and building Cyclus on a windows machine (or two)
2. If there are build system fixes that will simplify the process, APPLY THOSE FIXES
3. Record each step of the process, and update the GettingAndBuildingCyclus wiki page
4. Suggest GUIs and tools that work well for GUI-oriented developers (TortoiseSVN, the CMake GUI, etc.)

Comment 2 by project member katyhuff, Nov 22, 2011
Update....

As this documentation is written, it is worth noting that windows makes every step of this process more difficult.

• SVN,, CMake hdf5, boost, and libxml all must be downloaded and put in a reasonable place.
• Clarifying the locations of these libraries within CMake's configuration script is made necessary and occaisionally complicated since windows OBEYS NO FILE HIERARCHY CONVENTION FOR LIBRARY PLACEMENT. Include directories full of header files might have been put anywhere by the user.
• command prompt instructions will not work for individuals avoiding the command line.
• Users of Visual Studio will have compiler errors that are not seen in the gnu or intel compilers.
• Some linking issues caused by windows incompatibility with dlfcn.h have been fixed by revision 483. . . probably.
  Labels: OpSys-Windows

Comment 6 by project member uwgonuke, Jan 11, 2012
Cygwin only for Release 0.1

Profile cyclus (ideally with simulations that have several facilities transacting). I recommend using callgrind with the kcachegrind gui. Preliminary investigations indicate that stl container usage and the IsoVector class are good candidates for optimization.

valgrind --tool=callgrind ./cyclus ./input-file

Use smart pointers to manage resource object memory. I will use intrusive_ptr again - it provides more 'safety' than shared pointers - we don't have to worry about ending up with two different reference counts on the same object among other things.

@gonuke @katyhuff @rwcarlsen

We currently have been bouncing about with different styles (myself included; mea culpa). Before release 0.1, we should review all of our code to make sure it adheres to our guidelines.

In the current use of facilities, a template is defined with some set of parameters, after which that template can be used to defined specific instances of facilities based on that template. There is currently no mechanism to define instance-specific parameters for a facility that are different from the values defined in the template (overriding the defaults) or a different set of values than are defined in the template (extending beyond the defaults).

Some recipe reactor facility model is ripe as a place to demonstrate this.

It will be difficult (impossible?) to define required instance-specific parameters in the XML grammar, so software-level error checking will be required.

Proposed solution

• Have a private method in the communicator class named trackMessage(msg_ptr). This method will add the passed msg_ptr to a member var list of messages that have passed through that communicator (on their way up).
• have another communicator class private method named untrackMessage(msg_ptr) which will remove a message from the same list (messages are untracked if they pass back through on their way 'down' the communicator/model hierarchy)
• the message class must be a friend of the communicator class (already the case).
• the Message class's sendOn method will appropriately call the trackMessage and untrackMessage methods of the communicators that it is being sent to/from.
• the message class will have a kill() method that will effectively render the message's sendOn method do nothing. - the message will effectively be unable to be passed anywhere - shortly after which, it should be automatically deallocated with no remaining refs to it.
• the communicator class destructor will kill() all messages that are in its tracked_messages_ member var. This will render all those messages unable to return to the now deallocated communicator/model.

Writing to the database is an operation that could be moved into its own thread relatively easily - and frequent disk access could slow cyclus down significantly.

When a user defines a SeparationMatrixFacility, it is possible that they don't account for 100% of every element. That is, the sum of all the separation fractions across all streams for some element may be less than 100%.

If so, an extra stream should automatically be generated that captures all the excess material.

Comment 1 by project member uwgonuke, Aug 3, 2011

Additional comments:

1. What if they sum to >100% - throw exception to user and halt?
2. User should get warning if/when extra stream is generated.
3. Unclear: what commodity/market should the extra stream be connected to?

Tests need to be written to check the model memory management code. Issue #30 and its comments can be used as a guide for what needs to be tested. Focus should be placed on testing the message tracking/killing.

Since we want resource object conservation anyway - i.e. only one model 'owns' a resource at a time - using auto_ptr instead of boost::intrusive_ptr might actually be a smarter choice.

Determine fundamental data required to be visualized to reproduce VISION once-through output.

Instead of initializing NULL member variables, and returning NULL pointers, we should throw an exception within the get method of not-fully-initialized member variables. Rather than dealing with these NULL pointers externally with if statements, this will mean wrapping the get statements in try/catch blocks. This should be implemented fully across the code for consistency.
Note the addition of specific exception categories in r435. The CycNullException should be used for null pointer cases.

I think this has been mostly taken care of. Someone still ought to comb through the code carefully and make sure there are no lingering NULL returns for pointers.

• input file: null/input/test1.xml
• simulation duration >= 2225
• frontend and backend inventories increased to continue transacting throughout the entire simulation duration.

Segfault occurs in different places in the bookkeeper during final data writing - location depends on how far above 2225 you set the simulation duration. I think we might be blowing past the stack array size limit.

@rwcarlsen @katyhuff @gonuke -- I'd enjoy some discussion/clarification on this topic.

While building new tables to log all of our information, I have noticed that we seem to create resources willy-nilly. There's not an idea of a permanent resource in the simulation that gets passed around, but rather resources are copied any time they're needed.

That's all well and good, but it messes up our logging paradigm with respect to resource IDs. Also, it implies that the the resource object that is created is NOT the same as the resource object that is being transacted. I do not think this is a good thing.

To make this issue more clear, I've pasted the below output from /null/test1.xml. Note the difference between the logs of what is inserted into Transactions versus what is inserted into Resources and Material History - The Resources and Material History log a resource ID of x and Transactions logs an ID of x+5.

(note this is only a snippet)

DEBUG5(table ): Added command to row commands: INSERT INTO Resources (ID, Type, Quantity, OriginatorID) VALUES (48, 0, 1000, 5);
DEBUG5(table ): Added command to row commands: INSERT INTO Material History (ID, StateID, Time) VALUES (48, 1, 8);
DEBUG5(table ): Added command to row commands: INSERT INTO Transactions (SenderID, ReceiverID, ResourceID, Time, Price) VALUES (5, 6, 53, 8, 0);
DEBUG5(table ): Added command to row commands: INSERT INTO Resources (ID, Type, Quantity, OriginatorID) VALUES (54, 0, 1000, 5);
DEBUG5(table ): Added command to row commands: INSERT INTO Material History (ID, StateID, Time) VALUES (54, 1, 9);
DEBUG5(table ): Added command to row commands: INSERT INTO Transactions (SenderID, ReceiverID, ResourceID, Time, Price) VALUES (5, 6, 59, 9, 0);

Currently, cyclus only looks for recipe book and facility catalog files in the directory from which cyclus is executed. Cyclus should search more intelligently for recipe files. I should look in one or more the following locations:

• The directory cyclus is executed from (searched currently)
• The directory the input file resides in.
• Some designated recipe/catalog that can be a standard place for catalogs to be "installed".

To achieve some waste form volume and mass tracking capabilities similar to those in vision, a lookup-table-based WasteConditioning FacilityModel needs to exist which pairs waste streams with appropriate waste forms according to their isotopics.

Notes on vision capability:

The capability in vision relies on tabulated loading density values for various waste streams. The table looks something like:

|| _________ || ________ || _______ || _______ || ________ || ______ || ______ ||
|| density-> || low      || low     || low     || med      || med    || med    ||   
|| _________ || ________ || _______ || _______ || ________ || ______ || ______ || 
|| stream    || ceramic  || glass   || ...     || ceramic  || glass  || ...    ||    
|| I         ||  -       || 0.2     || ...     || -        || 0.4    || ...    ||    
|| Cs        ||  0.05    || -       || ...     || 0.1      || -      || ...    ||   
|| Sr        ||  -       || 0.3     || ...     || -        || 0.6    || ...    ||  
|| Cs/Sr     ||  -       || 0.2     || ...     || -        || 0.4    || ...    || 

Each waste stream has only one allowed waste form.
This means that the user has only one decision to make, and that is whether they'd like the waste forms to be loaded with a "low", "medium", or "high" loading density.

Began work with r449 .

The ability to import "recipebook"s and "facilitycatalog"s has been implemented already, but there are other examples of where this might be useful. Most urgently is the ability to load a library of regions that represent the actual state of the world.

For regions (and maybe others?) it is probably also valuable to provide the ability to include a subset of an external library. If a library of regions included the whole world, a given simulation shouldn't need to always include the whole world.

Maybe regions should be handled differently by introducing a facility to read a single region from a single region file.

Possible ways to manage buffering:

• write to database after reaching a fixed threshold of buffered row data.
• write to database on some fixed time interval
• write to the database as often as connection can be made
• write to database as often as a concurrent process is ready.

An ideal solution would likely involve some intelligent combination of the ideas above.

Instead of creating a static list of recipes populated early on, allow facilities to search the XML for the recipes they want. A recipe will still be a notion in the input, allowing reuse of code in the input file. Recipes will still be referred to as strings.

On the Cyclus wiki, create a Getting Started guide that walks a new developer through the process of implementing a new facility class, starting from a stub.

It should be lightweight, i.e. not going into too much detail of "why" and focusing on "how", but with links to the Doxygen for more detail.

Write python scripts for graph generation from VISION once-through data.

Traditional software build work flows include

• a build step that uses source code in a source directory tree and generates object code and executables in a build directory tree
• an installs step that copies executables, libraries, and other support files from the source directory tree and the build directory tree to an install directory tree

While it is common for the code to be usable from the build tree, some users/organizations will prefer to have a clean install tree.

A desired/advised structure is necessary for these trees. The source tree is defined, in part in issue91 . Its requirements are defined by the needs of developers.

The build tree requirements are defined by the needs of the build process and only need to accommodate developers.

The install tree requirements are defined by the desire to have fully built modules delivered and installed in a usable installation.

To facilitate modularity, the master RelaxNG schema should include other files from modules. Perhaps there is a single modules.rng file that is a growing list of single line includes for other rng files.

Explore the validity of a "bare" include at the use of the <ref ...> to avoid 2 references to each facility, etc.

Merged from issue 118:

Feature Description :
In order to provide a clean and easy-to-understand system for adding module specific input parameter definitions, a paradigm for numerous rng files or something similar must be designed and implemented.

Subtasks :
Describe tasks that are necessary for this feature to be implemented. If
there is an associated issue describing the subtask, report it as a related
issue below.

1. Decide on a system for adding modular grammar definitions 
2. Implement the system
3. Document the use of the system and provide templates for use.

This would eliminate a great deal of down-casting. The only case where a class only inherits from one, but not the other is: Converters inherit from the model class, but don't inherit from the Communicator class (or maybe the other way around). I spent time looking at the doxygen inheritance diagrams and decided that we get almost no benefit from the parallel inheritance from Model and Communicator.

@rwcarlsen

Please add a section to the README describing the steps necessary to update the Cyclus home page.

If this is as simple as editing a file in _sources and pushing, please state that. However, if you must rebuild the site after, please describe that and any other related processes.

Run a cyclus simulation of somewhat long duration in order to generate a dataset on the order of (or near) millions of records. This dataset will be given to the U of U collaborators to evaluate database alternatives as well as begin prelim visualization development. HDF5 output is acceptable, but SQL data would be preferrable.

Next Action: Review current documentation that describes environment and building blocks.

This guide, based in the wiki, will provide more depth than the getting started guide ( issue 107 ) but still have the purpose of instruction, as opposed to reference. It should follow a carefully chosen path of introducing the reader to the necessary classes, their purposes and their interfaces, to logically introduce the necessary information for generating a new model.

Comment 2 by project member uwgonuke, Dec 3, 2011

Perhaps this should be LaTeX theory manual? Or some format that can be sent to both PDF and Wiki?

Develop a capability to dump the current state of a simulation and restart from such a dump. (More details should follow...)

To provide some form of economic analysis, adapt Arnaud's scripts for Cyclus output

Add more granularity levels to logger. Ideas to consider:

• adding more entries to the LogLevel enum
• creating 2 logging macros - one for a debug log and the other for a sim run info log
• adding a second argument (std::string) to the log macro that adds a unique prefix to each log statement. Each module's log statements will have their own unique prefix. This will allow easy filtering of logger output.

Suggested implementation:

• create a connect() method on the message class. This method will create a Channel object and feed a pointer to the channel to both the request message and the offer message. Markets will need to send both finalized request/offer messages back down the hierarchy (not just the offer messages). Messages will obviously need a member variable to carry the channel pointer.
• require the requester to invoke the approve() method on the returned matched message before the transaction is initiated.
• Create a Channel class. This class will need a public method that allows the message class to communicate when an approve() happens. When the channel has receive 2 approves, it immediately begins the material transaction. The channel pulls the material from the supplier. It then pushes the material to the requester. If the push to requester operation fails, the channel will return the pulled material back to the requester.

Allow models to decide how often they will respond to ticks/tocks based on the implicit time step of the model and the user-defined time step of the simulation.

Two notes:

1. this could be embedded in the TimerAgent base class??
2. parents will need to negotiate with their children to pass ticks at least as often as they want them

Comment 1 by project member Princessandsuperman, Nov 28, 2011

As I mentioned, my preference for solving this involves:

1. Having the arbitrary tic/toc call order convention
2. Having all models register for tics/tocs at a certain frequency (takes place in their default constructors)
   • the frequency is represented as some integer quantity of time change in some base time unit.
   • this removes the need for any tic/toc passing in concrete models (each model deals directly with the timer).
   • The timer will need to calculate when to call each model.

Benefits:

1. simplified concrete models (no tic/toc passing)
2. No code outside constructor - the tic/toc methods only contain code relevant to the concrete model - no need for code that is "core framework" code.
3. Better performance (no tic/toc method invocations unless concrete model wants them)
4. Even more performance if: after all registrations are complete, the timer builds a list of all tic/toc/resolve calls for the entire simulation before the sim begins. If there are gaps larger than the base time step with no facility calls, the timer auto-increments the time up to the next tic/toc required.

Disbenefits:

1. facility creation mid-simulation is not as straight forward. Developers need to create facilities during the toc (otherwise the facility may or may not miss its first tic).
2. Others?

In my opinion, this kind of implementation would simplify the core code overall - and simplification is one of my primary metrics for evaluating change desirability. Other opinions?

The decommission() method in Model may not be appropriate for all Models. Perhaps should be moved down the hierarchy to a derived class for which all instances/derived classes require the method.

Note that this applies to matt's db work that is not all fully pushed to blessed yet.

We don't want to record every state that an IsoVector is in - every time somebody is manipulating recipes or doing stuff within their concrete agents, we would have no choice but to record every little change. If someone were to create an intermediate/temp IsoVector object, it would be recorded. We only want to record compositions of IsoVector's that are "participating" in the simulation - and the defining characteristic of "participating" IsoVector is their inclusion in a Material object. The addToTable method can stay in the IsoVector class, but the functionality that exists in trackComposition needs to somehow be migrated into the material class.

If you wanted to keep stateID_ tracking in the cyclus core, it too should be shifted into the material class.

@gidden

Use Boost shared_ptr, weak_ptr (to prevent parent/child cycles), and possibly intrusive_ptr for cyclus memory management. Note that the == operator still works for pointer comparisons between smart pointers. For an example of what sorts of things might need to be done to make this change, see issue #6.

Use Boost shared_ptr and weak_ptr (to prevent parent/child cycles) for cyclus memory management. Eventually, the entire core needs to be converted from raw pointers to smart pointers. Note that the == operator still works for pointer comparisons between smart pointers.

The libxml based XML reading and parsing has been created with little/no regard for cleaning up the memory objects created during reading and parsing. In order to avoid freeing/deleting objects that are still needed, almost no memory cleanup was done.

Someone should more carefully understand the nature and scope of the memory objects and implement some memory cleanup in InputXML.

@katyhuff Does this already exist somewhere?

At first I went to decayInfo.dat, but that doesn't list any stable isotopes (duh..). Is this list predefined somewhere else or do we need to define it now?

I know other fuel cycle sims group isotopes based on certain characteristics, which we can, of course, do. Just thought we should discuss.

This will likely be done as a part of issue #2. I am in great anticipation of the day when this issue is closed.

Change from this:

...
old_model...
...
Model* new_model = model_creator();
new_model->copyFresh(old_model);

To this:

...
old_model...
...
Model* new_model = old_model.clone();

I remember my investigation into this several weeks ago revealed this sort of change to be more difficult than at first glance. If I remember correctly, the difficulty was primarily due to the inheritance coupling of the current model copying paradigm. There was a fair amount of discussion on this in 2 separate cyclus-dev googlegroup topics:

https://groups.google.com/d/topic/cyclus-dev/3F8fkSMtqtI/discussion

and

https://groups.google.com/d/topic/cyclus-dev/SuDmt4hv9w8/discussion

Comb through the doxygen-related code and do the following:

• check for accuracy
• make the marking-up consistent across files
• write a cyclus doxygen style guide

Target Date :
TBD

Feature Description :
I think there should be a single entry point for new developers. This entry point should direct the developer through a streamlined set of descriptions, explanations, instructions, and resources which fully equip them to be a cyclus developer. While we have a DevelopersDocumentation wiki page, the GettingAndBuildingCyclus wiki page, and other resources, a comprehensive list of useful resources and a directed path through them will help developers. Perhaps this will exist solely on the wiki in the Developers guide, or perhaps there should be a supplemental document. In any case, this task should ensure that there is a clear starting point and directed path through all introduction documentation on the wiki, documentation, Stub files, and paper resources intended for Developers.

Subtasks :
Describe tasks that are necessary for this feature to be implemented. If
there is an associated issue describing the subtask, report it as a related
issue below.

1. Decide on the essential information a developer needs to participate (resources, intros, stub files)
2. Put these resources in order of a coherent learning trajectory
3. Make certain that all elements of the website and documentation direct the new developer to a single starting point
4. Make certain that the starting point leads the developer through the learning trajectory in a straightforward way.

Comment 1 by project member Matthew.Gidden, Nov 21, 2011

I'm trying to think of basic things that a new developer would need to know. Immediately coming to mind is the process of building facilities at different times... I'm about to start trying to work on output, which only makes sense if we have that ability, so I can start writing as I go through that process.

Comment 2 by project member katyhuff, Nov 21, 2011

What do you mean by trying to work on output?
Do you mean pytables scripts to make graphs and whatnot or refactoring the BookKeeper?

Comment 3 by project member uwgonuke, Nov 21, 2011

Three different levels of documentation are indicated:

1. Getting Started: brief, lightweight tutorial for making a new facility (wiki)
2. Instructional guide: chart a path through the reference documentation that is designed to teach things in a carefully chosen order (wiki)
3. Doxygen as canonical reference documentation

Create a database abstraction layer.

The abstraction layer should be contain two high-level methods:

• a method to describe a new table layout
• a method to add data to that table

Ideally, the abstraction layer will be able to write to different database formats as per the user's choice. A key difficulty to overcome will be to describe compound data types (the oodbms-ness of ordbms-type data bases) in a generic way.

We should have a user tutorial on the wiki that steps a user through a simple once-through fuel cycle. I propose the tutorial show the following steps:

1. Create input file
2. Run cyclus
3. Make output files

As a goal, we may wish to use our previous GENIUSv2 once-through case as the tutorial example.

Once this issue is complete, we should be able to reproduce most VISION once-through capabilities.