Comments (2)
dear @angel-ayala ,
We do this transfer of spaces because we think the Tuple and Dict environments are not well-suited for reinforcement learning state space. Thus we decided not to support these state spaces for the two following reasons:
-
The tuple is nothing else than a finite MDP environment with coordinates. If the MDP is too big, you should not use tuples, but the Box environment (e.g. if you want a generalization in coordinates). NB: if you use tuples, you cannot solve finite MDPs using mushroom finite agents, as they use a single index to discriminate finite states. If you want to use a continuous algorithm, you can just put the tuples inside a vector using box state (the fact that the coordinates are discrete is completely irrelevant)
-
The dict state space is irrelevant, as you can stack together the elements of the environment and create a vector.
If you want to use dictionary output, you can simply return data using the info dictionary, which is already passed to the fit method of the environment.
Side note about gymnasium: we are still not sure when (and if) we will support it. The API of gym was breaking too much to be of any use (indeed we stopped the support at gym 0.24.1), and until gymnasium is stable and not breaking every release, we are not going to support it. However, making an interface based on the gym one is pretty trivial.
It would be instead interesting to support more complex state space, such as graphs. This requires however support for graph neural networks and a bit of software engineering, and is not planned in the short future.
from mushroom-rl.
I was looking the implementation of the Gym environment and there is a _convert_gym_space()
method which transfer the Gym spaces to the Mushroom spaces code and comparing with the Discrete and Box's from Gym library.
So, I was wondering the need to do this transfer spaces because the only enhancement achieve is to store the object as Serializable
. Can this be done using a wrapper adding the self._add_save_attr()
method use for the respective attributes presentes in the Gym spaces?
from mushroom-rl.
Related Issues (20)
- Can't install package HOT 4
- suspected memory leak HOT 8
- How to train an agent in one environment and use it on another slightly different envoirnment HOT 3
- dynaq agent HOT 1
- how to reproduce DQN nature paper? HOT 7
- compress frames HOT 2
- n_steps dqn performs worse. bug? HOT 1
- PPO for lunar lander [BUG] HOT 10
- Multi modal state support HOT 1
- Save and Load Agent for the Second Time HOT 2
- 'Taxi-v3' error: "ValueError: too many values to unpack (expected 4)" HOT 2
- TypeError: can't convert np.ndarray of type numpy.object_. The only supported types are: float64, float32, float16, complex64, complex128, int64, int32, int16, int8, uint8, and bool. HOT 2
- SAC postload optimizer for alpha HOT 2
- Unable to tun atari_dqn.py file in examples HOT 5
- Python 3.11 support HOT 2
- Suggestion: Add median to compute_metrics
- [solvers/dynamic_programming] Use np.linalg.solve instead of np.inv HOT 2
- [requirements.txt] Missing requirement for OpenAI gym HOT 4
- [Categorical DQN/Rainbow] Inconsistent behavior of Categorical DQN for an even number of atoms
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mushroom-rl.