Comments (11)
Another thought:
We could pickle the cache out to a file and load it into the TournamentManager each time it's initiated. That way, we wouldn't have to run the first round robin before kicking off the parallel processes. We could just run all repetitions in parallel and even the first would have access to the cache.
We'd need to remember to rebuild and re-pickle each time a new strategy is added but, even if we forget, it's not a problem.
from axelrod.
I'll pick this up. It will be on branch 117 in my fork.
from axelrod.
I've hit a snag on this one and could do with some help:
I've refactored the mechanism to ensure that there is only ever one cache object which is then passed correctly to the various classes that use it. There are some temporary logging messages which show that the same object id is being used throughout.
That all works fine right up until the last commit (f5689b8) where the cache is shared across tournaments by the tournament manager.
As hoped, run_tournament executes slightly quicker than before the commit, but always crashes on me at the very end. The results are produced correctly, but I always have to kill the process to make it exit. This is true regardless of whether I run in parallel or not.
The change is on line 47 of tournament_manager.py. Commenting it out makes run_tournament execute without crashing.
The cache counts in the log messages show the expected numbers both with that line and without.
Can anyone else confirm that they see the same behaviour as me?
Anyone got a clue as to what is going on?!!
from axelrod.
Will take a look this pm, seems weird though :)
from axelrod.
I ran the code without problems.
from axelrod.
I ran the code without problems.
Thank you!! I think that's good news!! I did wonder if it's something obscure on my machine or my python installation
from axelrod.
Further good news - pickling the cache to a file and loading it into the tournament manager has now reduced running time on my machine from 70s to 47s!!
from axelrod.
Ran just fine for me also.
Further good news - pickling the cache to a file and loading it into the tournament manager has now reduced running time on my machine from 70s to 47s!!
How does pickling the cache speed things up? Is everything not just in memory anyway? I think I don't understand some of this...
from axelrod.
At the moment, the cache is built from scratch by each tournament during its first round robin. Once it's built, subsequent round robins execute far more quickly.
f5689b8 adds sharing of the cache from one tournament to another (via the TournamentManager) so that each tournament is building on top of the cache from the previous one instead of starting from scratch.
5d99340 creates a file which has pickled the cache at the end of the all_strategies tournament and so contains all of the deterministic scores which might exist. Loading that cache at the init of the TournamentManager and sharing it across tournaments means that no tournament has to build the cache at all and has all its possible deterministic scores available from the start.
Parallel processing: At the moment, only second and subsequent repetitions are run in parallel; the first is run before the parallel processes are started in order to build the cache which is then made available to those processes. 656674a doesn't bother with that first round robin if the cache is already pre-loaded - it goes straight into parallel processing.
That's the intention anyway!! Still testing to see if I've had the desired effect.
from axelrod.
That's awesome. I think I'm still missing why the cache needs to be pickled and can't just be passed directly but I can wait to read the code and find out.
(There's a decent temp file library in python https://docs.python.org/2/library/tempfile.html if we want to have the option of not keeping the pickle files around.)
from axelrod.
I think I'm still missing why the cache needs to be pickled and can't just be passed directly but I can wait to read the code and find out.
Essentially to share it between different runs of run_tournament so that the cache is available from the very start of all runs and never has to be built by any of the tournaments.
It's loaded fro the file into the TournamentManager and then passed, as normal, to each of the tournaments.
The pickled file only has to be re-created when new strategies are added.
from axelrod.
Related Issues (20)
- Reorganisation ? of cheating strategies
- Add a citation.cff file HOT 6
- Links to contributing guide broken by docs restructure
- CI failing due to typing issues HOT 3
- Implement asymmetric games HOT 6
- Game classification HOT 4
- Implement abstract games more fully (5.0.0)
- Restructure strategies folder HOT 4
- Documentation for 5.0.0
- Simplify/move the `ResultSet` HOT 1
- Expressing in a formal logical language HOT 2
- You may have missed some details in your code HOT 1
- I couldn't find the strategy submitted by Mauk in the competition for 19th place HOT 2
- If I want to test the first tournament, what should I do based on your code? HOT 2
- Do you know the source code of the first tournament? HOT 2
- axelrod.plot.Plot may be incompatible with recent Pandas udpates HOT 2
- Change the TFT in the first tournament HOT 3
- High-noise Tournament for comparison HOT 1
- Supporting Python 3.12 HOT 1
- Deal with new numpy version
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from axelrod.