aws / sagemaker-rl-container Goto Github PK

A set of dockerfiles that provide Reinforcement Learning solutions for use in SageMaker.

License: Apache License 2.0

HCL 4.18% C 0.29% Shell 2.29% Python 86.95% Dockerfile 2.17% Jupyter Notebook 4.12%

sagemaker-rl-container's Introduction

Amazon SageMaker RL Containers

Warning

As of April 2024, SageMaker RL containers no longer accepts new pull requests. Please follow Building Your Image to build your own RL images.

A set of Dockerfiles that enables Reinforcement Learning (RL) solutions to be used in SageMaker.

The SageMaker team uses this repository to build its official RL images. On how to use any of these images on SageMaker, see Python SDK. For end users, this repository is typically of interest if you need implementation details of the official image, or if you want to use it to build your own customized RL image.

For information on running RL jobs on SageMaker: SageMaker RLEstimators.

For notebook examples: SageMaker Notebook Examples.

Getting Started
RL Images Provided by SageMaker
Building Your Image
Running the Tests

Getting Started

Prerequisites

Make sure you have installed all of the following prerequisites on your development machine:

Docker

For Testing on GPU

Nvidia-Docker2

Terminologies

Toolkit

Toolkits are libraries that provide specific algorithms to train a Reinforcement Learning model. We currently provide Dockerfiles for these three toolkits:

Framework

Framework refers to a Deep Learning framework/library that a toolkit may need in order to train an algorithm. We use Sagemaker created framework images/prebuilt Amazon SageMaker Docker images as base images in a Toolkit's Dockerfile (whenever required). Currently we are using these two frameworks:

TensorFlow (used for Ray and Coach)
PyTorch (used for Ray)
MXNet (used for Coach)

Note: VW doesn't require a framework

RL Images Provided by SageMaker

MXNet Coach Images:

520713654638.dkr.ecr..amazonaws.com/sagemaker-rl-mxnet:coach0.11-cpu-py3
520713654638.dkr.ecr..amazonaws.com/sagemaker-rl-mxnet:coach0.11.0-cpu-py3
520713654638.dkr.ecr..amazonaws.com/sagemaker-rl-mxnet:coach0.11-gpu-py3
520713654638.dkr.ecr..amazonaws.com/sagemaker-rl-mxnet:coach0.11.0-gpu-py3

TensorFlow Coach Images:

520713654638.dkr.ecr..amazonaws.com/sagemaker-rl-tensorflow:coach0.10-cpu-py3
520713654638.dkr.ecr..amazonaws.com/sagemaker-rl-tensorflow:coach0.10.1-cpu-py3
520713654638.dkr.ecr..amazonaws.com/sagemaker-rl-tensorflow:coach0.10-gpu-py3
520713654638.dkr.ecr..amazonaws.com/sagemaker-rl-tensorflow:coach0.10.1-gpu-py3
520713654638.dkr.ecr..amazonaws.com/sagemaker-rl-tensorflow:coach0.11-cpu-py3
520713654638.dkr.ecr..amazonaws.com/sagemaker-rl-tensorflow:coach0.11.0-cpu-py3
520713654638.dkr.ecr..amazonaws.com/sagemaker-rl-tensorflow:coach0.11.1-cpu-py3
520713654638.dkr.ecr..amazonaws.com/sagemaker-rl-tensorflow:coach0.11-gpu-py3
520713654638.dkr.ecr..amazonaws.com/sagemaker-rl-tensorflow:coach0.11.0-gpu-py3
520713654638.dkr.ecr..amazonaws.com/sagemaker-rl-tensorflow:coach0.11.1-gpu-py3
462105765813.dkr.ecr..amazonaws.com/sagemaker-rl-coach-container:coach-1.0.0-tf-cpu-py3
462105765813.dkr.ecr..amazonaws.com/sagemaker-rl-coach-container:coach-1.0.0-tf-gpu-py3

TensorFlow Ray Images:

520713654638.dkr.ecr..amazonaws.com/sagemaker-rl-tensorflow:ray0.6-cpu-py3
520713654638.dkr.ecr..amazonaws.com/sagemaker-rl-tensorflow:ray0.6.5-cpu-py3
520713654638.dkr.ecr..amazonaws.com/sagemaker-rl-tensorflow:ray0.6-gpu-py3
520713654638.dkr.ecr..amazonaws.com/sagemaker-rl-tensorflow:ray0.6.5-gpu-py3
462105765813.dkr.ecr..amazonaws.com/sagemaker-rl-ray-container:ray-0.8.2-tf-cpu-py36
462105765813.dkr.ecr..amazonaws.com/sagemaker-rl-ray-container:ray-0.8.2-tf-gpu-py36
462105765813.dkr.ecr..amazonaws.com/sagemaker-rl-ray-container:ray-0.8.5-tf-cpu-py36
462105765813.dkr.ecr..amazonaws.com/sagemaker-rl-ray-container:ray-0.8.5-tf-gpu-py36
462105765813.dkr.ecr..amazonaws.com/sagemaker-rl-ray-container:ray-1.6.0-tf-cpu-py37
462105765813.dkr.ecr..amazonaws.com/sagemaker-rl-ray-container:ray-1.6.0-tf-gpu-py37

PyTorch Ray Images:

462105765813.dkr.ecr..amazonaws.com/sagemaker-rl-ray-container:ray-0.8.5-torch-cpu-py36
462105765813.dkr.ecr..amazonaws.com/sagemaker-rl-ray-container:ray-0.8.5-torch-gpu-py36
462105765813.dkr.ecr..amazonaws.com/sagemaker-rl-ray-container:ray-1.6.0-torch-cpu-py36
462105765813.dkr.ecr..amazonaws.com/sagemaker-rl-ray-container:ray-1.6.0-torch-gpu-py36

Vowpal Wabbit Images:

462105765813.dkr.ecr..amazonaws.com/sagemaker-rl-vw-container:vw-8.7.0-cpu

List of supported SageMaker regions.

Building Your Image

Amazon SageMaker utilizes Docker containers to run all training jobs and inference endpoints.

The Docker images are built from the Dockerfiles specified in this repository at:

The Dockerfiles are grouped by RL toolkit and toolkit version. Within that, they are separated by framework (if needed). For e.g., the Dockerfile for Coach v0.11.0 with MXNet framework can be found at: coach/docker/0.11.0/Dockerfile.mxnet.

For toolkits Ray and Coach, the Dockerfiles use deep learning framework images provided by SageMaker as their "base" images.

These "base" images are specified with the following naming convention:

520713654638.dkr.ecr.<region>.amazonaws.com/sagemaker-<framework>:<framework_version>-<processor>-py3

<framework> can be tensorflow-scriptmode (with <framework_version> 1.11.0 or higher depending on the toolkit requirements) or mxnet (with <framework_version> 1.3.0 or higher depending on the toolkit requirements);
<processor> can be cpu or gpu;
for valid <region> values please see `list of supported SageMaker regions <https://docs.aws.amazon.com/general/latest/gr/rande.html#sagemaker_region).

Before building images:

Pull deep learning framework "base" image, which require Docker, AWS credentials, and AWS CLI.

# Login into SageMaker ECR account
$(aws ecr get-login --no-include-email --region <region> --registry-ids 520713654638)
# Pull docker image from ECR
docker pull 520713654638.dkr.ecr.<region>.amazonaws.com/sagemaker-<framework>:<framework_version>-<processor>-py3

# Example

$(aws ecr get-login --no-include-email --region us-west-2 --registry-ids 520713654638)

# CPU TensorFlow image
docker pull 520713654638.dkr.ecr.us-west-2.amazonaws.com/sagemaker-tensorflow-scriptmode:1.11.0-cpu-py3

# GPU MXNet image
docker pull 520713654638.dkr.ecr.us-west-2.amazonaws.com/sagemaker-mxnet:1.3.0-gpu-py3

To build RL Docker image:

# All build instructions assume you're building from the root directory of the sagemaker-rl-container.

# CPU
docker build -t <image_name>:<tag> -f <rl_toolkit>docker/<rl_toolkit_version>/Dockerfile.<framework> --build-arg processor=<cpu_or_gpu> .

# GPU
docker build -t <image_name>:<tag> -f <rl_toolkit>/docker/<rl_toolkit_version>/Dockerfile.<framework> --build-arg processor=<cpu_or_gpu> .

# Example

# Ray TensorFlow CPU
docker build -t tf-ray:0.6.5-cpu-py3 -f ray/docker/0.6.5/Dockerfile.tf --build-arg processor=cpu .

# Coach TensorFlow GPU
docker build -t tf-coach:0.11.0-gpu-py3 -f coach/docker/0.11.0/Dockerfile.tf --build-arg processor=gpu .

# Coach MXNet CPU
docker build -t mxnet-coach:0.11.0-cpu-py3 -f coach/docker/0.11.0/Dockerfile.mxnet --build-arg processor=cpu .

# VW CPU
docker build -t vw:8.7.0-cpu -f vw/docker/8.7.0/Dockerfile .

Running the Tests

Running the tests requires installation of test dependencies.

git clone https://github.com/aws/sagemaker-rl-container.git
cd sagemaker-rl-container
pip install .

Tests are defined in test/ and include local integration and SageMaker integration tests.

Local Integration Tests

Running local integration tests require Docker and AWS credentials, as the local integration tests make calls to a couple of AWS services. The local integration tests and SageMaker integration tests require configurations specified within their respective conftest.py.

Local integration tests on GPU require Nvidia-Docker.

Before running local integration tests:

Build your Docker image.
Pass in the correct pytest arguments to run tests against your Docker image.

If you want to run local integration tests, then use:

# Required arguments for integration tests are found in test/conftest.py
pytest test/integration/local --toolkit <toolkit_to_run_tests_for> \
                              --docker-base-name <your_docker_image> \
                             --tag <your_docker_image_tag> \
                              --processor <cpu_or_gpu>

# Example
pytest test/integration/local --toolkit coach \
                              --docker-base-name custom-rl-coach-image \
                              --tag 1.0 \
                              --processor cpu

SageMaker Integration Tests

SageMaker integration tests require your Docker image to be within an [Amazon ECR repository](https://docs .aws.amazon.com/AmazonECS/latest/developerguide/ECS_Console_Repositories.html).

The Docker base name is your [ECR repository namespace](https://docs.aws.amazon .com/AmazonECR/latest/userguide/Repositories.html).

The instance type is your specified Amazon SageMaker Instance Type that the SageMaker integration test will run on.

Before running SageMaker integration tests:

Build your Docker image.
Push the image to your ECR repository.
Pass in the correct pytest arguments to run tests on SageMaker against the image within your ECR repository.

If you want to run a SageMaker integration end to end test on Amazon SageMaker, then use:

# Required arguments for integration tests are found in test/conftest.py
pytest test/integration/sagemaker --toolkit <toolkit_to_run_tests_for> \
                                  --aws-id <your_aws_id> \
                                  --docker-base-name <your_docker_image> \
                                  --instance-type <amazon_sagemaker_instance_type> \
                                  --tag <your_docker_image_tag> \

# Example
pytest test/integration/sagemaker --toolkit coach \
                                  --aws-id 12345678910 \
                                  --docker-base-name custom-rl-coach-image \
                                  --instance-type ml.m4.xlarge \
                                  --tag 1.0

Contributing

Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.

License

This library is licensed under the Apache 2.0 License.

Note: Specific license for Toolkits/Frameworks, if any, can be found in /docker/LICENSE or in the Framework's image

sagemaker-rl-container's People

Contributors

Stargazers

Watchers

sagemaker-rl-container's Issues

coach tf containers broken

I was trying to use
462105765813.dkr.ecr.us-east-1.amazonaws.com/sagemaker-rl-coach-container:coach-1.0.0-tf-gpu-py3
both locally and in Sagemaker Studio, and got the following error:

framework error: 
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in <module>
    from tensorflow.python.pywrap_tensorflow_internal import *
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
    _pywrap_tensorflow_internal = swig_import_helper()
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
    _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
  File "/usr/lib/python3.6/imp.py", line 243, in load_module
    return load_dynamic(name, filename, file)
  File "/usr/lib/python3.6/imp.py", line 343, in load_dynamic
    return _load(spec)
ImportError: libcuda.so.1: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/sagemaker_containers/_trainer.py", line 73, in train
    framework = importlib.import_module(framework_name)
  File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/local/lib/python3.6/dist-packages/sagemaker_tensorflow_container/training.py", line 23, in <module>
    import tensorflow as tf
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/__init__.py", line 24, in <module>
    from tensorflow.python import pywrap_tensorflow  # pylint: disable=unused-import
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/__init__.py", line 49, in <module>
    from tensorflow.python import pywrap_tensorflow
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 74, in <module>
    raise ImportError(msg)
ImportError: Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in <module>
    from tensorflow.python.pywrap_tensorflow_internal import *
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
    _pywrap_tensorflow_internal = swig_import_helper()
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
    _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
  File "/usr/lib/python3.6/imp.py", line 243, in load_module
    return load_dynamic(name, filename, file)
  File "/usr/lib/python3.6/imp.py", line 343, in load_dynamic
    return _load(spec)
ImportError: libcuda.so.1: cannot open shared object file: No such file or directory


Failed to load the native TensorFlow runtime.

See https://www.tensorflow.org/install/errors

Locally, I was able to work around it by

Error while launching a training process with Ray/RLLib 0.8.2/Gym

Container: sagemaker-rl-ray-container:ray-0.8.2-tf-*-py36

A bug was introduced in this container after an update in the latest version of pyglet. This update breaks the API contract and causes some errors when visualization is enabled (stack trace).

Solution: Downgrade pyglet to version 1.3.2 --> pyglet==1.3.2

Could you change that in the Dockerfile and also update the built images available in SageMaker, please?

Source: tensorflow/agents#163

ray.exceptions.RayTaskError(AttributeError): #33[36mray::RolloutWorker.sample()#33[39m (pid=119, ip=10.2.216.148)
File "python/ray/_raylet.pyx", line 452, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 430, in ray._raylet.execute_task.function_executor
File "/usr/local/lib/python3.6/dist-packages/ray/rllib/evaluation/rollout_worker.py", line 488, in sample
batches = [self.input_reader.next()]
File "/usr/local/lib/python3.6/dist-packages/ray/rllib/evaluation/sampler.py", line 52, in next
batches = [self.get_data()]
File "/usr/local/lib/python3.6/dist-packages/ray/rllib/evaluation/sampler.py", line 95, in get_data
item = next(self.rollout_provider)
File "/usr/local/lib/python3.6/dist-packages/ray/rllib/evaluation/sampler.py", line 301, in _env_runner
base_env.poll()
File "/usr/local/lib/python3.6/dist-packages/ray/rllib/env/base_env.py", line 308, in poll
self.new_obs = self.vector_env.vector_reset()
File "/usr/local/lib/python3.6/dist-packages/ray/rllib/env/vector_env.py", line 96, in vector_reset
return [e.reset() for e in self.envs]
File "/usr/local/lib/python3.6/dist-packages/ray/rllib/env/vector_env.py", line 96, in
return [e.reset() for e in self.envs]
File "/usr/local/lib/python3.6/dist-packages/gym/wrappers/monitor.py", line 39, in reset
self._after_reset(observation)
File "/usr/local/lib/python3.6/dist-packages/gym/wrappers/monitor.py", line 188, in _after_reset
self.reset_video_recorder()
File "/usr/local/lib/python3.6/dist-packages/gym/wrappers/monitor.py", line 209, in reset_video_recorder
self.video_recorder.capture_frame()
File "/usr/local/lib/python3.6/dist-packages/gym/wrappers/monitoring/video_recorder.py", line 101, in capture_frame
frame = self.env.render(mode=render_mode)
File "/usr/local/lib/python3.6/dist-packages/gym/core.py", line 249, in render
return self.env.render(mode, **kwargs)
File "/usr/local/lib/python3.6/dist-packages/gym/envs/classic_control/continuous_mountain_car.py", line 143, in render
return self.viewer.render(return_rgb_array = mode=='rgb_array')
File "/usr/local/lib/python3.6/dist-packages/gym/envs/classic_control/rendering.py", line 105, in render
arr = np.frombuffer(image_data.data, dtype=np.uint8)

Using redis on system memory instead of GPU memory.

Hi.

Good day.

Is it possible to not run redis on the GPU? At the moment I am getting the following error using it training deep racer:

subscribe scheduled to be closed ASAP for overcoming of output buffer limits

It seems as though the process wants to allocate gigs of memory into redis but the GPU only has about 7GB, whereas the system has free memory to use.

I'm just not sure how to get it to use the CPU. I tried creating an image myself and making the following change to start.sh:

CUDA_VISIBLE_DEVICES=-1 redis-server --bind 0.0.0.0 &

But when I run the image it doesn't use GPU at all.

Any ideas how to have redis use the systems memory and not the GPU memory? Thanks.

Regards.

does ray torch version image not supported with python3.7?

conda env:
(python3) [ec2-user@ip-172-16-92-179 sagemaker-rl-container]$ python3
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

docker build cli:
docker build -t torch-ray:cpu-py3 -f ray/docker/0.8.5/Dockerfile.torch --build-arg processor=cpu .

base iamge:
FROM 763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-training:1.5.0-cpu-py36

build error with dm-tree install:
`Attempting uninstall: protobuf
Found existing installation: protobuf 3.15.6
Uninstalling protobuf-3.15.6:
Successfully uninstalled protobuf-3.15.6
Running setup.py install for dm-tree: started
Running setup.py install for dm-tree: finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /opt/conda/bin/python3.6 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-phvm04kk/dm-tree_95eb886d3d904877b744f174998bf67d/setup.py'"'"'; file='"'"'/tmp/pip-install-phvm04kk/dm-tree_95eb886d3d904877b744f174998bf67d/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-ioqppij4/install-record.txt --single-version-externally-managed --compile --install-headers /opt/conda/include/python3.6m/dm-tree
cwd: /tmp/pip-install-phvm04kk/dm-tree_95eb886d3d904877b744f174998bf67d/
Complete output (55 lines):
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/tree
copying tree/init.py -> build/lib.linux-x86_64-3.6/tree
copying tree/sequence.py -> build/lib.linux-x86_64-3.6/tree
copying tree/tree_benchmark.py -> build/lib.linux-x86_64-3.6/tree
copying tree/tree_test.py -> build/lib.linux-x86_64-3.6/tree
warning: build_py: byte-compiling is disabled, skipping.

running build_ext
cmake version 3.5.1

CMake suite maintained and supported by Kitware (kitware.com/cmake).
Found CMake
CMake Error at CMakeLists.txt:3 (cmake_minimum_required):
  CMake 3.12 or higher is required.  You are running version 3.5.1


-- Configuring incomplete, errors occurred!
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/pip-install-phvm04kk/dm-tree_95eb886d3d904877b744f174998bf67d/setup.py", line 155, in <module>
    keywords='tree nest flatten',
  File "/opt/conda/lib/python3.6/site-packages/setuptools/__init__.py", line 163, in setup
    return distutils.core.setup(**attrs)
  File "/opt/conda/lib/python3.6/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/opt/conda/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/opt/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/opt/conda/lib/python3.6/site-packages/setuptools/command/install.py", line 61, in run
    return orig.install.run(self)
  File "/opt/conda/lib/python3.6/distutils/command/install.py", line 545, in run
    self.run_command('build')
  File "/opt/conda/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/opt/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/opt/conda/lib/python3.6/distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/opt/conda/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/opt/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/tmp/pip-install-phvm04kk/dm-tree_95eb886d3d904877b744f174998bf67d/setup.py", line 72, in run
    self.build_extension(ext)
  File "/tmp/pip-install-phvm04kk/dm-tree_95eb886d3d904877b744f174998bf67d/setup.py", line 109, in build_extension
    ['cmake', ext.source_dir] + cmake_args, cwd=self.build_temp)
  File "/opt/conda/lib/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '/tmp/pip-install-phvm04kk/dm-tree_95eb886d3d904877b744f174998bf67d/tree', '-DPython3_ROOT_DIR=/opt/conda', '-DPython3_EXECUTABLE=/opt/conda/bin/python3.6', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/tmp/pip-install-phvm04kk/dm-tree_95eb886d3d904877b744f174998bf67d/build/lib.linux-x86_64-3.6', '-DCMAKE_BUILD_TYPE=Release', '-DPython3_LIBRARY=/opt/conda/lib/python3.6', '-DPython3_INCLUDE_DIR=/opt/conda/include/python3.6m']' returned non-zero exit status 1.
----------------------------------------

ERROR: Command errored out with exit status 1: /opt/conda/bin/python3.6 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-phvm04kk/dm-tree_95eb886d3d904877b744f174998bf67d/setup.py'"'"'; file='"'"'/tmp/pip-install-phvm04kk/dm-tree_95eb886d3d904877b744f174998bf67d/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-ioqppij4/install-record.txt --single-version-externally-managed --compile --install-headers /opt/conda/include/python3.6m/dm-tree Check the logs for full command output.
The command '/bin/sh -c pip install --no-cache-dir Cython==0.29.7 tabulate tensorboardX gputil gym==0.12.1 lz4 opencv-python-headless==4.1.0.25 PyOpenGL==3.1.0 pyyaml redis==3.3.2 ray==0.8.5 ray[tune]==0.8.5 ray[rllib]==0.8.5 scipy psutil setproctitle dm-tree tensorflow-probability' returned a non-zero code: 1`

Generate release?

Is it possible to generate a release on github for this version, as set in the setup.py? Thanks!

Missing sagemaker-rl-vw-container:adf image

I was trying to follow the walkthrough bandits_movielens_testbed walkthrough because I want to build and vowpal wabbit adf model and it seems that the sagemaker-rl-vw-container:adf image has been removed and theres no documentation on how to use it.

Dockerfile is outdated

Dockerfile uses python 3.6 and some outdated packages.

Is there any plan to update the Dockerfile please?

Add stable baselines 3 containers for sagemaker

Add https://github.com/DLR-RM/stable-baselines3 and https://github.com/DLR-RM/rl-baselines3-zoo
container for sagemaker, so training rl models with stable baselines 3 will be possible.