Comments (4)
Specifically, I want to know how the performance of the algorithm can be evaluated. Similar to the evaluation of reinforcement learning algorithms can be represented by the reward obtained from the environment. Taking leduc poker as an example, how to prove the algorithm is effective? After the model training is completed, what we expect to save, rl model or the policy. I am not completely famaliar with this. I hope you can give some advice, thank you!
from open_spiel.
Hi @Root970103,
If you're using PSRO or some form of fictitious play, the thing you save is either the average strategy, or the entire set of policies coupled with the meta-strategy. The latter can be turned into one policy using the policy_aggregator (if the game is small enough).
A good place to start is this example: https://github.com/deepmind/open_spiel/blob/master/open_spiel/python/examples/psro_v2_example.py
Hope this helps, but please don't hesitate to ask more questions if it's not clear.
from open_spiel.
Hi @Root970103,
If you're using PSRO or some form of fictitious play, the thing you save is either the average strategy, or the entire set of policies coupled with the meta-strategy. The latter can be turned into one policy using the policy_aggregator (if the game is small enough).
A good place to start is this example: https://github.com/deepmind/open_spiel/blob/master/open_spiel/python/examples/psro_v2_example.py
Hope this helps, but please don't hesitate to ask more questions if it's not clear.
Thank you for your reply! I have run this example script. And I obeserved the changes in nash_conv.
I also wonder if I can used the trained model (or policy ) against other algorithms in leduc poker env. For example, I want to test the trained model against CFR, the entir policies or the aggregate policy should be used?
In addition, in an adversarial scenario, is it appropriate to use the q value to evaluate the algorithms?
It's very kind of you to give these advices.
from open_spiel.
I also wonder if I can used the trained model (or policy ) against other algorithms in leduc poker env. For example, I want to test the trained model against CFR, the entir policies or the aggregate policy should be used?
Yes, you can extract the policy (that is what the NashConv computation needs) and you can simulate the policy against CFR's policy.
In addition, in an adversarial scenario, is it appropriate to use the q value to evaluate the algorithms?
Q-values are just estimates of values for a state and action. You can turn that into a policy by choosing argmax Q(s,a) but these will be deterministic. So if the environment requires any kind of mixing, you would lose that if you argmax over the Q-values.
from open_spiel.
Related Issues (20)
- dqn.cc build error HOT 2
- Policy Gradient based Self Play HOT 23
- Recommended Alphazero training config parameters HOT 6
- Question: how to evaluate rnad algorithm HOT 3
- RNaD: Possible Error in calculation of Neurd Loss HOT 7
- Quoridor Movement Action IDs keep changing HOT 16
- Implementing alphamu algorithm? HOT 1
- bug in chess terminal determination? HOT 2
- Implementation of dilated form of MMD HOT 2
- Username HOT 1
- Question about Bridge observation tensor types. HOT 5
- About how to load subgame of Libratus HOT 3
- How to implement a python variation of alpha-beta and test using the framework in WSL HOT 2
- I can't compile open_spiel in c++ HOT 2
- Neurd Clip Instability [RNaD] HOT 1
- Success stories HOT 2
- Q-learning is a loser? HOT 6
- Does alphazero support reuse-tree? HOT 4
- Implement ReBeL with Public State API HOT 5
- Bug with nox HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from open_spiel.