maxpumperla / learning_ray Goto Github PK

View Code? Open in Web Editor NEW

212.0 12.0 56.0 117.91 MB

Notebooks for the O'Reilly book "Learning Ray"

Home Page: https://maxpumperla.com/learning_ray

License: MIT License

Jupyter Notebook 96.37% Python 3.63%

data-science deep-learning distributed-computing machine-learning notebook python ray

learning_ray's Introduction

Hi there, I'm Max 👋

I’m a Data Science & Engineering practitioner from Hamburg, Germany. I’m an avid open source contributor, author of machine learning & technology books, speaker and Coursera instructor.

I specialize in Deep Learning and its applications and can build machine learning solutions and data products from first prototype to production. As Ray contributor, DL4J core developer, Keras contributor, Hyperopt maintainer and author of a range of open source libraries, I have a distinct view on ML engineering and the data science ecosystem.

Here's what I'm up to right now

📖 Check out my new book with O'Reilly called "Learning Ray", which helps you get started with distributed Python with Ray. Most content is available on my homepage.
🔭 Some of the materials from my courses as Data Science professor at IU can be found here. My teaching centers around programming Python, big data technologies and reinforcement learning.
🔭 If you have interesting freelance opportunities, don't hesitate to get in touch with me.
💬 Ask me about anything Python or Data Science related. I'm happy to discuss!
⚡ Fun fact: I get asked for directions by total strangers all the time, even in foreign countries (and I have yet to figure out why).

Interesting things I've been doing in the past

📖 If you like the game of Go, or Python, or both, you might enjoy my book "Deep Learning and the Game of Go" (Manning). All the code for the book is free here on GitHub.
🐍 I've authored Elephas: Distributed Deep Learning with Keras & Spark
🐍 I'm also the author of Hyperas: simple hyperparameter optimization for Keras models.
🐍 We co-authored Logodetect: One-shot logo detection.
🐍 Hyperopt is not the most active project anymore, but I'm still trying my best to set aside some time here and there to maintain it.
🐍 In what feels like the very distant past, I've been a fairly active Keras contributor once.

And here's how you can get in touch with me

📈 GitHub Stats

learning_ray's People

Contributors

Stargazers

Watchers

learning_ray's Issues

pygame is required for pendulum demo in chap01

ModuleNotFoundError: No module named 'pygame'

location

details

rllib train -f pendulum.yml
2022-10-18 17:10:38,085 INFO worker.py:1333 -- Connecting to existing Ray cluster at address: 172.24.16.114:6379...
2022-10-18 17:10:38,091 INFO worker.py:1509 -- Connected to Ray cluster. View the dashboard at 127.0.0.1:8265
/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/util/placement_group.py:78: DeprecationWarning: placement_group parameter is deprecated. Use scheduling_strategy=PlacementGroupSchedulingStrategy(...) instead, see the usage at https://docs.ray.io/en/releases-2.0.0/ray-core/package-ref.html#ray-remote.
  return bundle_reservation_check.options(
/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/_private/ray_option_utils.py:266: DeprecationWarning: Setting 'object_store_memory' for actors is deprecated since it doesn't actually reserve the required object store memory. Use object spilling that's enabled by default (https://docs.ray.io/en/releases-2.0.0/ray-core/objects/object-spilling.html) instead to bypass the object store memory size limitation.
  warnings.warn(
/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/actor.py:637: DeprecationWarning: placement_group parameter is deprecated. Use scheduling_strategy=PlacementGroupSchedulingStrategy(...) instead, see the usage at https://docs.ray.io/en/releases-2.0.0/ray-core/package-ref.html#ray-remote.
  return actor_cls._remote(args=args, kwargs=kwargs, **updated_options)
/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/actor.py:637: DeprecationWarning: placement_group_bundle_index parameter is deprecated. Use scheduling_strategy=PlacementGroupSchedulingStrategy(...) instead, see the usage at https://docs.ray.io/en/releases-2.0.0/ray-core/package-ref.html#ray-remote.
  return actor_cls._remote(args=args, kwargs=kwargs, **updated_options)
/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/actor.py:637: DeprecationWarning: placement_group_capture_child_tasks parameter is deprecated. Use scheduling_strategy=PlacementGroupSchedulingStrategy(...) instead, see the usage at https://docs.ray.io/en/releases-2.0.0/ray-core/package-ref.html#ray-remote.
  return actor_cls._remote(args=args, kwargs=kwargs, **updated_options)
(PPO pid=52305) 2022-10-18 17:10:42,722 INFO algorithm.py:1871 -- Your framework setting is 'tf', meaning you are using static-graph mode. Set framework='tf2' to enable eager execution with tf2.x. You may also then want to set eager_tracing=True in order to reach similar execution speed as with static-graph mode.
(PPO pid=52305) 2022-10-18 17:10:42,722 INFO ppo.py:378 -- In multi-agent mode, policies will be optimized sequentray::PPO.__init__() (pid=52305, ip=172.24.16.114, repr=PPO)
  File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/rllib/algorithms/algorithm.py", line 308, in __init__
    super().__init__(config=config, logger_creator=logger_creator, **kwargs)
  File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/tune/trainable/trainable.py", line 157, in __init__
    self.setup(copy.deepcopy(self.config))
  File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/rllib/algorithms/algorithm.py", line 443, in setup
    raise e.args[0].args[2]
ModuleNotFoundError: No module named 'pygame'

(PPO pid=52305)     cls = load(self.entry_point)
(PPO pid=52305)   File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/gym/envs/registration.py", line 55, in load
(PPO pid=52305)     mod = importlib.import_module(mod_name)
(PPO pid=52305)   File "/root/miniconda3/lib/python3.9/importlib/__init__.py", line 127, in import_module
(PPO pid=52305)     return _bootstrap._gcd_import(name[level:], package, level)
(PPO pid=52305)   File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
(PPO pid=52305)   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
(PPO pid=52305)   File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
(PPO pid=52305)   File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
(PPO pid=52305)   File "<frozen importlib._bootstrap_external>", line 850, in exec_module
(PPO pid=52305)   File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
(PPO pid=52305)   File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/gym/envs/classic_control/__init__.py", line 1, in <module>
(PPO pid=52305)     from gym.envs.classic_control.cartpole import CartPoleEnv
(PPO pid=52305)   File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/gym/envs/classic_control/cartpole.py", line 10, in <module>
(PPO pid=52305)     import pygame
(PPO pid=52305) ModuleNotFoundError: No module named 'pygame'
(PPO pid=52305)
(PPO pid=52305) During handling of the above exception, another exception occurred:
(PPO pid=52305)
(PPO pid=52305) ray::PPO.__init__() (pid=52305, ip=172.24.16.114, repr=PPO)
(PPO pid=52305)   File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/rllib/algorithms/algorithm.py", line 308, in __init__
(PPO pid=52305)     super().__init__(config=config, logger_creator=logger_creator, **kwargs)
(PPO pid=52305)   File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/tune/trainable/trainable.py", line 157, in __init__
(PPO pid=52305)     self.setup(copy.deepcopy(self.config))
(PPO pid=52305)   File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/rllib/algorithms/algorithm.py", line 443, in setup
(PPO pid=52305)     raise e.args[0].args[2]
(PPO pid=52305) ModuleNotFoundError: No module named 'pygame'
(RolloutWorker pid=52448) 2022-10-18 17:10:46,627       ERROR worker.py:756 -- Exception raised in creation task: The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=52448, ip=172.24.16.114, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7edf6bb79250>)
(RolloutWorker pid=52448)   File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/rllib/evaluation/rollout_worker.py", line 490, in __init__
(RolloutWorker pid=52448)     self.env = env_creator(copy.deepcopy(self.env_context))
(RolloutWorker pid=52448)   File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/rllib/env/utils.py", line 50, in _gym_env_creator
(RolloutWorker pid=52448)     return gym.make(env_descriptor, **env_context)
(RolloutWorker pid=52448)   File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/gym/envs/registration.py", line 676, in make
== Status ==
Current time: 2022-10-18 17:10:46 (running for 00:00:08.53)
Memory usage on this node: 275.2/502.1 GiB
Using FIFO scheduling algorithm.
Resources requested: 0/64 CPUs, 0/0 GPUs, 0.0/254.8 GiB heap, 0.0/113.19 GiB objects
Result logdir: /root/ray_results/pendulumppo
Number of trials: 1/1 (1 ERROR)
Number of errored trials: 1
+-----------------------------+--------------+-------------------------------------------------------------------------------------------+
| Trial name                  |   # failures | error file                                                                                |
|-----------------------------+--------------+-------------------------------------------------------------------------------------------|
| PPO_Pendulum-v1_bb4a6_00000 |            1 | /root/ray_results/pendulumppo/PPO_Pendulum-v1_bb4a6_00000_0_2022-10-18_17-10-38/error.txt |
+-----------------------------+--------------+-------------------------------------------------------------------------------------------+
(RolloutWorker pid=52448)     return registry.make(id, **kwargs)

(RolloutWorker pid=52448)   File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/gym/envs/registration.py", line 520, in make
(RolloutWorker pid=52448)     return spec.make(**kwargs)
(RolloutWorker pid=52448)   File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/gym/envs/registration.py", line 139, in make
(RolloutWorker pid=52448)     cls = load(self.entry_point)
(RolloutWorker pid=52448)   File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/gym/envs/registration.py", line 55, in load
(RolloutWorker pid=52448)     mod = importlib.import_module(mod_name)
(RolloutWorker pid=52448)   File "/root/miniconda3/lib/python3.9/importlib/__init__.py", line 127, in import_module
(RolloutWorker pid=52448)     return _bootstrap._gcd_import(name[level:], package, level)
(RolloutWorker pid=52448)   File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
(RolloutWorker pid=52448)   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
(RolloutWorker pid=52448)   File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
(RolloutWorker pid=52448)   File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
(RolloutWorker pid=52448)   File "<frozen importlib._bootstrap_external>", line 850, in exec_module
(RolloutWorker pid=52448)   File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
(RolloutWorker pid=52448)   File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/gym/envs/classic_control/__init__.py", line 1, in <module>
(RolloutWorker pid=52448)     from gym.envs.classic_control.cartpole import CartPoleEnv
(RolloutWorker pid=52448)   File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/gym/envs/classic_control/cartpole.py", line 10, in <module>
(RolloutWorker pid=52448)     import pygame
(RolloutWorker pid=52448) ModuleNotFoundError: No module named 'pygame'
(RolloutWorker pid=52447) 2022-10-18 17:10:46,644       ERROR worker.py:756 -- Exception raised in creation task: The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=52447, ip=172.24.16.114, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7fcf394a5280>)
(RolloutWorker pid=52447)   File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/rllib/evaluation/rollout_worker.py", line 490, in __init__
(RolloutWorker pid=52447)     self.env = env_creator(copy.deepcopy(self.env_context))
(RolloutWorker pid=52447)   File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/rllib/env/utils.py", line 50, in _gym_env_creator
(RolloutWorker pid=52447)     return gym.make(env_descriptor, **env_context)
(RolloutWorker pid=52447)   File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/gym/envs/registration.py", line 676, in make
(RolloutWorker pid=52447)     return registry.make(id, **kwargs)
(RolloutWorker pid=52447)   File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/gym/envs/registration.py", line 520, in make
(RolloutWorker pid=52447)     return spec.make(**kwargs)
(RolloutWorker pid=52447)   File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/gym/envs/registration.py", line 139, in make
(RolloutWorker pid=52447)     cls = load(self.entry_point)
(RolloutWorker pid=52447)   File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/gym/envs/registration.py", line 55, in load
(RolloutWorker pid=52447)     mod = importlib.import_module(mod_name)
(RolloutWorker pid=52447)   File "/root/miniconda3/lib/python3.9/importlib/__init__.py", line 127, in import_module
(RolloutWorker pid=52447)     return _bootstrap._gcd_import(name[level:], package, level)
(RolloutWorker pid=52447)   File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
(RolloutWorker pid=52447)   File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
(RolloutWorker pid=52447)   File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
(RolloutWorker pid=52447)   File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
(RolloutWorker pid=52447)   File "<frozen importlib._bootstrap_external>", line 850, in exec_module
(RolloutWorker pid=52447)   File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
(RolloutWorker pid=52447)   File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/gym/envs/classic_control/__init__.py", line 1, in <module>
(RolloutWorker pid=52447)     from gym.envs.classic_control.cartpole import CartPoleEnv
(RolloutWorker pid=52447)   File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/gym/envs/classic_control/cartpole.py", line 10, in <module>
(RolloutWorker pid=52447)     import pygame
(RolloutWorker pid=52447) ModuleNotFoundError: No module named 'pygame'
== Status ==
Current time: 2022-10-18 17:10:46 (running for 00:00:08.53)
Memory usage on this node: 275.2/502.1 GiB
Using FIFO scheduling algorithm.
Resources requested: 0/64 CPUs, 0/0 GPUs, 0.0/254.8 GiB heap, 0.0/113.19 GiB objects
Result logdir: /root/ray_results/pendulumppo
Number of trials: 1/1 (1 ERROR)
Number of errored trials: 1
+-----------------------------+--------------+-------------------------------------------------------------------------------------------+
| Trial name                  |   # failures | error file                                                                                |
|-----------------------------+--------------+-------------------------------------------------------------------------------------------|
| PPO_Pendulum-v1_bb4a6_00000 |            1 | /root/ray_results/pendulumppo/PPO_Pendulum-v1_bb4a6_00000_0_2022-10-18_17-10-38/error.txt |
+-----------------------------+--------------+-------------------------------------------------------------------------------------------+

2022-10-18 17:10:46,656 ERROR ray_trial_executor.py:103 -- An exception occurred when trying to stop the Ray actor:Traceback (most recent call last):
  File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/tune/execution/ray_trial_executor.py", line 94, in _post_stop_cleanup
    ray.get(future, timeout=0)
  File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/_private/client_mode_hook.py", line 105, in wrapper
    return func(*args, **kwargs)
  File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/_private/worker.py", line 2277, in get
    raise value
ray.exceptions.RayActorError: The actor died because of an error raised in its creation task, ray::PPO.__init__() (pid=52305, ip=172.24.16.114, repr=PPO)
  File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/rllib/evaluation/worker_set.py", line 125, in __init__
    self.add_workers(
  File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/rllib/evaluation/worker_set.py", line 391, in foreach_worker
    remote_results = ray.get([w.apply.remote(func) for w in self.remote_workers()])
ray.exceptions.RayActorError: The actor died because of an error raised in its creation task, ray::RolloutWorker.__init__() (pid=52448, ip=172.24.16.114, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x7edf6bb79250>)
  File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/rllib/evaluation/rollout_worker.py", line 490, in __init__
    self.env = env_creator(copy.deepcopy(self.env_context))
  File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/rllib/env/utils.py", line 50, in _gym_env_creator
    return gym.make(env_descriptor, **env_context)
  File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/gym/envs/registration.py", line 676, in make
    return registry.make(id, **kwargs)
  File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/gym/envs/registration.py", line 520, in make
    return spec.make(**kwargs)
  File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/gym/envs/registration.py", line 139, in make
    cls = load(self.entry_point)
  File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/gym/envs/registration.py", line 55, in load
    mod = importlib.import_module(mod_name)
  File "/root/miniconda3/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/gym/envs/classic_control/__init__.py", line 1, in <module>
    from gym.envs.classic_control.cartpole import CartPoleEnv
  File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/gym/envs/classic_control/cartpole.py", line 10, in <module>
    import pygame
ModuleNotFoundError: No module named 'pygame'

During handling of the above exception, another exception occurred:

ray::PPO.__init__() (pid=52305, ip=172.24.16.114, repr=PPO)
  File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/rllib/algorithms/algorithm.py", line 308, in __init__
    super().__init__(config=config, logger_creator=logger_creator, **kwargs)
  File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/tune/trainable/trainable.py", line 157, in __init__
    self.setup(copy.deepcopy(self.config))
  File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/rllib/algorithms/algorithm.py", line 443, in setup
    raise e.args[0].args[2]
ModuleNotFoundError: No module named 'pygame'

Traceback (most recent call last):
  File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/bin/rllib", line 8, in <module>
    sys.exit(cli())
  File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/rllib/scripts.py", line 41, in cli
    train.run(options, train_parser)
  File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/rllib/train.py", line 269, in run
    run_experiments(
  File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/tune/tune.py", line 848, in run_experiments
    return run(
  File "/root/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/tune/tune.py", line 752, in run
    raise TuneError("Trials did not complete", incomplete_trials)
ray.tune.error.TuneError: ('Trials did not complete', [PPO_Pendulum-v1_bb4a6_00000])

May I translate your book to chinese version?

Who am I?

7+ years in Technical Writing in chinese.
8+ years Experience in AI Infrastructure, AI/ML.

I read early release of Learning Ray. I love your book, and I am looking forward to sharing it with others.
May I contribute chinese transilation to this awsome book?

Exception Errors in Chapter 4: Could be newbie error as well

I'm going through this book and running the code as I go. I am running into the following issue:

ALL DONE Exception ignored in: <function RolloutWorker.__del__ at 0x7fb2260368c0> Traceback (most recent call last): File "/home/rustykettle/.local/lib/python3.10/site-packages/ray/util/tracing/tracing_helper.py", line 461, in _resume_span TypeError: 'NoneType' object is not callable

The ALL DONE is my print:

ray.init()
gymEnv = GymEnvironment()
trainer = DQNTrainer(env=GymEnvironment, config={"num_workers": 4})  # <1>
config = trainer.get_config()  # <2>
#print(pretty_print(config))

for i in range(10):
    result = trainer.train()  # <3>

ray.shutdown()
print(pretty_print(result))  # <4>
print("ALL DONE")

It is to show that the exception is something to do with Ray, but it also might have something to do with my configuration. I followed the book very closely, but I'm not discounting a user-error on my part.
output.log

I'm also getting depreciation warnings that don't appear in the log. I'm imagining that this happens because it is happening on a different thread.

Example 4-5. ValueError: Cannot evaluate w/o an evaluation worker set in the Trainer or w/o an env on the local worker!

ValueError: Cannot evaluate w/o an evaluation worker set in the Trainer or w/o an env on the local worker!
Try one of the following:
1) Set `evaluation_interval` >= 0 to force creating a separate evaluation worker set.
2) Set `create_env_on_driver=True` to force the local (non-eval) worker to have an environment to evaluate on.

RLlib: env must be one of following supported types issue

Hello @maxpumperla ,

Thank you for the amazing book, it's been a real pleasure working through it so far!

I am, however, having a problem with evaluating the maze game using RLlib in chapter 4. When I get to the command

rllib evaluate ~/ray_results/maze_env/ --algo DQN --env maze_gym_env.Environment --steps 100

I get the following error:

ValueError: Env must be of one of the following supported types: BaseEnv, gym.Env, MultiAgentEnv, VectorEnv, 
RemoteBaseEnv, ExternalMultiAgentEnv, ExternalEnv, but instead is of type class <'maze_gym_env.Environment'>.

The above error has been found in your environment! We've added a module for checking your custom environments. It may 
cause your experiment to fail if your environment is not set up correctly. You can disable this behavior by setting 
'disable_env_checking=True' in your environment config dictionary. You can run the environment checking module 
standalone by calling ray.rllib.utils.check_env([env]).

I'm don't understand why I am getting this error as I followed your instructions to the letter. It's also not clear where the config dictionary is supposed to be stored.

I was wondering if you could help me resolve this issue?

Thank you!

Example 6-2. not imported

NameError: name 'Scaler' is not defined

NameError: name 'Repartitioner' is not defined

NameError: name 'Chain' is not defined

Pendulumv0 vs Pendulum-v0

I'm reading the book here https://learning.oreilly.com/library/view/learning-ray/9781098117214/ch01.html#section_data_processing and I was trying to run rllib with the provided pendulum.yml and got an error containing this line:

gym.error.Error: Attempted to look up malformed environment ID: b'Pendulumv0'. (Currently all IDs must be of the form ^(?:[\w:-]+\/)?([\w:.-]+)-v(\d+)$.)

The text mentions Pendulum-v0, but the yaml example says it's Pendulumv0, judging by the error Pendulum-v0 is correct.

Will this repo keep update?

As I start to learning ray, when I start to Chapter 4 that it's seem I can't run the gym as it's not supported.

A question about "building a distributed Ray Training" section in Chapter 3

Hi, thank you for making this book available before publishing. It is a treasure for learning ray. I have a doubt about a function in Chapter 3 at section "building a distributed Ray Training" as copied below. This function synchronously updates the policy, which is in essence a Q table, with 4 tasks each of which has an experience list. I wonder if two tasks are updating the same Q value, would this cause a conflict?

def update_policy_task(policy_ref, experiences_list):
"""Remote Ray task for updating a policy with experiences in parallel."""
[update_policy(policy_ref, ray.get(xp)) for xp in experiences_list]
return policy_ref

If the policy is a neural network, do we need to lock the weight for each update? Because the fact that each task updates the weights independently might lead to the problem that one task updates the weights on the ones that have been updated by another task, not the weights the gradients were calculated on. This should have already been dealt with in the Ray Train, but I do not know how it is handled. This problem has been haunting in my mind for a long while. Could you please share your thoughts about it? Thank you very much. I am looking forward to your reply.

Not use the updated policy

In ch_03 https://github.com/maxpumperla/learning_ray/blob/main/notebooks/ch_03_core_app.ipynb train_policy_parallel function,

def train_policy_parallel(env, num_episodes=1000, num_simulations=4):
    """Parallel policy training function."""
    policy = Policy(env)
    simulations = [SimulationActor.remote() for _ in range(num_simulations)]

    policy_ref = ray.put(policy)
    for _ in range(num_episodes):
        experiences = [sim.rollout.remote(policy_ref) for sim in simulations]

        while len(experiences) > 0:
            finished, experiences = ray.wait(experiences)
            for xp in ray.get(finished):
                update_policy(policy, xp)

    return policy

If i'm not mistaken, it appears that each episode use the initially initialized policy rather than the updated one

Example 4-11.AttributeError: 'dict' object has no attribute 'set_training'

In [267]: from ray.rllib.models.preprocessors import get_preprocessor
     ...: env = GymEnvironment()
     ...: obs_space = env.observation_space
     ...: preprocessor = get_preprocessor(obs_space)(obs_space)
     ...:
     ...: observations = env.reset()
     ...: transformed = preprocessor.transform(observations).reshape(1, -1)
     ...:
     ...: model_output, _ = model.from_batch({"obs": transformed})
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Input In [267], in <cell line: 9>()
      6 observations = env.reset()
      7 transformed = preprocessor.transform(observations).reshape(1, -1)
----> 9 model_output, _ = model.from_batch({"obs": transformed})

File ~/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/rllib/utils/deprecation.py:122, in Deprecated.<locals>._inner.<locals>._ctor(*args, **kwargs)
    115     deprecation_warning(
    116         old=old or obj.__name__,
    117         new=new,
    118         help=help,
    119         error=error,
    120     )
    121 # Call the deprecated method/function.
--> 122 return obj(*args, **kwargs)

File ~/.cache/pypoetry/virtualenvs/ray-KeD1w6JO-py3.9/lib/python3.9/site-packages/ray/rllib/models/modelv2.py:357, in ModelV2.from_batch(self, train_batch, is_training)
    350 """Convenience function that calls this model with a tensor batch.
    351
    352 All this does is unpack the tensor batch to call this model with the
    353 right input dict, state, and seq len arguments.
    354 """
    356 input_dict = train_batch.copy()
--> 357 input_dict.set_training(is_training)
    358 states = []
    359 i = 0

AttributeError: 'dict' object has no attribute 'set_training'

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.