Comments (1)
Upon further investigation I discovered what triggers the error. Below is a simple example of CartPole-v1
where this error shows up. If you add the input argument policies_to_train=["p0"]
to the PPOConfig
it will error out. If you do not add policies_to_train=["p0"]
, it will run.
ppo_config = (
PPOConfig()
.environment(MultiAgentCartPole, env_config={"num_agents": 2})
# Switch both the new API stack flags to True (both False by default).
# This enables the use of:
# a) RLModule (replaces ModelV2) and Learner (replaces Policy)
# b) and automatically picks the correct EnvRunner (single-agent vs multi-agent) and enables ConnectorV2 support.
.api_stack(
enable_rl_module_and_learner=True,
enable_env_runner_and_connector_v2=True,
)
.resources(
num_cpus_for_main_process=16,
)
# supports arbitrary scaling on the learner axis, feel free to set
# `num_learners` to the number of available GPUs for multi-GPU training (and `num_gpus_per_learner=1`).
.learners(
num_learners=0, # <- set this value to the number of GPUs
num_gpus_per_learner=0, # <- set this to 1, if you have a GPU
)
.training(train_batch_size_per_learner=5000)
# Because you are in a multi-agent env, you have to set up the usual multi-agent parameters:
.multi_agent(
policies={"p0", "p1"},
# Map agent 0 to p0 and agent 1 to p1.
policy_mapping_fn=lambda agent_id, episode, **kwargs: f"p{agent_id}",
policies_to_train=["p0"],
)
)
from ray.
Related Issues (20)
- CI test windows://python/ray/tests:test_actor_state_metrics is consistently_failing HOT 10
- [core][experimental] Remove separate asyncio path for accelerated DAGs
- Ray Serve: Fail to create Serve applications HOT 1
- [Data] Remove `InputBuffer` op from exported metrics / dashboard / Jobs page view
- CI test linux://rllib:learning_tests_multi_agent_cartpole_appo_multi_gpu is consistently_failing HOT 5
- [core] Cancelling tasks who is waiting for dependencies hangs result objects
- CI test linux://rllib:learning_tests_multi_agent_cartpole_appo_gpu is consistently_failing HOT 6
- [Data] Execution hangs when there are many operators in the pipeline
- Exception raised in creation task: The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__()
- [Ray Data] Improve the design and of `_MapWorker`
- [tune] Don't re-evaluate HyperOpt's points_to_evaluate HOT 1
- CI test linux://rllib:learning_tests_multi_agent_cartpole_appo_multi_cpu is flaky
- [Core] Actor supports job lifetime HOT 2
- CI test linux://rllib:learning_tests_multi_agent_cartpole_appo_multi_cpu is flaky
- [core][experimental] Accelerated DAG should execute work on actor's main thread
- [core][experimental] ray.get of accelerated DAG result may not throw exception for MultiOutputNode HOT 1
- CI test linux://rllib:examples/offline_rl/pretrain_bc_single_agent_evaluate_as_multi_agent is flaky HOT 1
- Release test single_node_oom.aws failed HOT 2
- [ Core] cannot serialize polars.LazyFrame
- EOFError error during remote_worker_envs flags
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ray.