Comments (9)
I see your point. This is tricky. We could handle that in SimResList.plot()
, but it makes that method more complicated and SimResList
isn't really intended to concatenate results anyway. We could add a merge() to SimRes
or as a standalone function, but the details to truely merge the data depend on the format it was loaded from. I wasn't planning to add a method to write or rewrite simulation result files (although SimRes.to_pandas()
can export to some standard formats).
The easiest way might be to create a dedicated class that extends from SimRes
. There are two abstract methods in SimRes
: values
and times
. As it stands these are populated by the Dymola file loader (_modelicares.io.dymola.readsim). We could create a SimResSequence
class that extends from SimRes
but has an __init__
method similar to that of SimResList
. It would define the values()
and times()
methods to pull from the SimRes
instances stored within it. In theory, everything else (its plot()
, min
, max
, etc.) would fall into place.
What do you think?
from modelicares.
Good thinking, that could be an easy solution!
from modelicares.
Would you consider taking the first shot at it?
from modelicares.
Sure, I was planning to ;)
from modelicares.
I followed your approach and everything works :) see 75b565b. However, I did had to change values in NamedTuple
instances which is something you should normally not do.
If you are okay with my implementation, we can close this.
PS: I think the continue_run()
method is a great addition to the module. It enables developers to use external control methods on Dymola models (as is also possible in Matlab). I think the FMU approach works best but I did not yet succeed in running FMUs with the Dymola solvers.
from modelicares.
Some considerations after I tried this with a large simulations:
- Each new simulation result of a continued run holds redundant data of the parameter names etc. When you do a lot of iterations of the
continue_run()
method, this creates a lot of overhead. ThePyFMI
module does provide the ability to create a custom result handler which might be useful here. - If there are too many files,
SimResSequence
is unable to process these files the way it is programmed now due to Memory issues. We might need to merge the result files first and delete the redundant data for the files to be able to be read by aSimRes
instance. - When exporting a FMU from Dymola, you have the ability to store the results in a
.mat
file. This .mat file does work for continued simulations by simply adding the new results to the file. If the FMU is able to create such a file, the simulation results should be stored in memory instead of creating extra.txt
files. However, it is impossible to know from thefmu
object if it has this option. I added a method ``move_mat_file()to the
fmi` simulator to move the .mat file to the result directory but currently I see no generic way to automate this. - I uploaded different types of FMUs to the directory and they all work for me except for the
CS-2.0/1.0-Dymola
FMU which is a Co-Simulation FMU with internal Dymola solvers.
Note that when working with continued simulations, the file writing is the bottleneck, especially when the iterations are short. This can cause the following overhead:
- Dymosim: When you have large models,
dsin.txt
can get big (~50 MB) and it can take a long time to write these files because Python needs to rewrite the entire file everytime you change even a single value. - PyFMI: Uses .txt files for output which are alot heavier than the compact .mat files of dymosim. Storing the results in memory or using a .mat output file can considerably speed up the simulation. I noticed a significant speed increase of FMUs with this configuration: ``fmu('model.fmu', log_level=0, result_handling='memory'`
from modelicares.
I changed the approach to generate new NamedTuple
instances for each variable (179571d). I'm not sure if this will reduce the memory load but it could.
from modelicares.
We could speed up the dsin.txt
writing, but it would take some work. Options:
- Update the file in place. This is probably not safe because the length of the new values may be longer than the old ones and the existing whitespace.
- Use binary files (dsin.mat) generated from dymosim with the -ib option. We'd need to create functions to read and write to those (using scipy's loadmat and savemat).
- Write only the modified parameters to a new dsin file. Dymosim apparently has an option to use two files where one overrides the other. Edit: Looking at the documentation now, I think I was wrong on this.
from modelicares.
I think I'll close this because we've solved the original issue. I'll open a new issue (#27) related to file size and processing time.
from modelicares.
Related Issues (20)
- Consider merge HOT 3
- 'SimRes' object has no attribute 'find' HOT 1
- Set and read simulation start and stop times from a Dymola dsin.txt HOT 4
- Slow processing for large (> 50 MB) continued simulations
- Raise an error if a non-tunable parameter is adjusted in the dymosim interface
- simulators not implemented? HOT 8
- ImportError: No module named _display HOT 2
- Using subplot and other matplotlib functionality HOT 3
- Support asynchronous scheduling of continued simulations
- to_pandas error
- Installation under Jython 2.7
- PyPI is way outdated
- ImportError: cannot import name read_options HOT 1
- Syntax errors in ModelicaRes_Master
- to_pandas() error when progressing modified simulation data
- Problems with natu library HOT 2
- Is it enough to use dymola_script class to run simulations?
- Compatiblity issue with matplotlib version >=3.1.0 HOT 6
- SyntaxError when importing SimRes HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from modelicares.