mrrobb / gym-rs Goto Github PK

OpenAI Gym bindings for Rust

License: MIT License

Rust 100.00%

gym-rs's Introduction

👋 Hi there!

      ___           ___           ___     
     /\  \         /\  \         /\  \    
    /::\  \       /::\  \       /::\  \   
   /:/\:\  \     /:/\:\  \     /:/\:\  \  
  /::\~\:\  \   /:/  \:\  \   /::\~\:\__\ 
 /:/\:\ \:\__\ /:/__/ \:\__\ /:/\:\ \:|__|
 \/_|::\/:/  / \:\  \ /:/  / \:\~\:\/:/  /
    |:|::/  /   \:\  /:/  /   \:\ \::/  / 
    |:|\/__/     \:\/:/  /     \:\/:/  /  
    |:|  |        \::/  /       \::/  /   
     \|__|         \/__/         \/__/

About me

🔭 I’m currently working at the Centre for Genomic Regulation
🌱 Learning AWS, Lightning network protocol and RL for imperfect-information games.
📫 How to reach me: LinkedIn or [email protected].
😄 Pronouns: he / him

💻 Open source work stats

gym-rs's People

Contributors

Stargazers

Watchers

Forkers

silvancodes simon-t4 tourmark liamsig

gym-rs's Issues

Environment is private

The environment module is private, but I need access to it. Shouldnt this be made public?

Example fails.

python = 3.7.6
rust = 1.40.0
MacOS 10.13.6

Steps

git clone https://github.com/MrRobb/gym-rs.git
cd gym-rs
virtualenv -p python3 develop
source develop/bin/activate
pip install -r requirements.txt
cargo run --example basic

Errors

Finished dev [unoptimized + debuginfo] target(s) in 0.05s
Running target/debug/examples/basic
thread 'main' panicked at 'Error: import gym: PyErr { ptype: <class 'ModuleNotFoundError'>, pvalue: Some(ModuleNotFoundError("No module named 'gym'")), ptraceback: None }', src/libcore/result.rs:1165:5
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace.

cargo run --example basic fails

Fails at basic.rs::11

	let env = match client.make("CartPole-v0") {
		Ok(env) => env,
		Err(msg) => panic!("Could not make environment because of error:\n{}", msg)
	};

Should there be a CartPole-V0 file? There is not

TypeError: Object of type float32 is not JSON serializable
thread 'main' panicked at 'Could not make environment because of error:
expected value at line 1 column 1', examples/basic.rs:11:15

I am not a python hacker but looking at the python back trace it is trying to get something off the net.


Traceback (most recent call last):
  File "XXXXXXXXXXXXXXX.local/lib/python3.7/site-packages/flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "XXXXXXXXXXXXXXX.local/lib/python3.7/site-packages/flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "XXXXXXXXXXXXXXX.local/lib/python3.7/site-packages/flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "XXXXXXXXXXXXXXX.local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "XXXXXXXXXXXXXXX.local/lib/python3.7/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "XXXXXXXXXXXXXXX.local/lib/python3.7/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "server.py", line 340, in env_observation_space_info
    return jsonify(info = info)
  File "XXXXXXXXXXXXXXX.local/lib/python3.7/site-packages/flask/json/__init__.py", line 370, in jsonify
    dumps(data, indent=indent, separators=separators) + "\n",
  File "XXXXXXXXXXXXXXX.local/lib/python3.7/site-packages/flask/json/__init__.py", line 211, in dumps
    rv = _json.dumps(obj, **kwargs)
  File "/usr/lib/python3.7/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/usr/lib/python3.7/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python3.7/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "XXXXXXXXXXXXXXX.local/lib/python3.7/site-packages/flask/json/__init__.py", line 100, in default
    return _json.JSONEncoder.default(self, o)
  File "/usr/lib/python3.7/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '

Does this binding makes it feasible to run gym_env in true multi-threads?

I found this project when trying to find a way to truely run openai gym parallelly with algorithm written in rust.
Is it feasible to run gym_env in true multi-threads when coding in rust?
And even more ,could this support some gym like env, like gym_retro, gym_ple?

Unable to call 'render'

Getting the following error:

$ cargo run --example basic
thread 'main' panicked at 'Unable to call 'render': PyErr { ptype: <class 'TypeError'>, pvalue: Some(TypeError('item 1 in _argtypes_ passes a union by value, which is unsupported.')), ptraceback: Some(<traceback object at 0x7f9d9ff248c0>) }', src/libcore/result.rs:1165:5

This only happens when render is true: with rendering disabled everything works correctly.

My python gym version is 0.12.5 and python itself is Python 3.7.6. Is there a known combination of gym/python versions that work?

Calling `step` with invalid action - no error

For the CartPole-v0 example passing a invalid action does not result in a error

            let _state = match env.step(vec![0.03], false) {
                Ok(s) => {
                    eprintln!("Step succeeded.  state: '{:?}'", s);
                    if s.done {
                        break;
                    }
                },
                Err(e) => panic!("step failed.  Err: '{}'", e),
            };

The step succeeded branch is taken.

Different values for `is_done` compared to Python library after certain amount of steps

I was working with the mountain car environment and I noticed that unlike in my Python code, the Rust version would always end an episode after 200 steps due to the OpenAI gym library indicating the episode as 'done'.

Interestingly enough,

import gym
gym.make("MountainCar-v0")._max_episode_steps

returns 200, meaning the Rust library is correct in returning is_done as true after 200 steps. However I'm not observing the same result in the original Python library. Considering that this Rust library is supposed to be a frontend to the Python library, I'd argue it should mimic its results, even if incorrect.

Minimum working examples showing the difference:

Python

import gym
env = gym.make("MountainCar-v0")
env.env.reset()
for i in range(300):
    observation, reward, is_done, info = env.env.step(1)
    if is_done:
        print("is_done == true at step:", i)
        break

Rust

extern crate gym;
fn main() {
    let gym = gym::GymClient::default();
    let env = gym.make("MountainCar-v0");
    env.reset();
    for i in 0..300 {
        let gym::State {observation, reward, is_done} = env.step(&gym::Action::DISCRETE(1)).unwrap();
        if is_done {
            println!("is_done == true at step: {}", i);
            break;
        }       
    }
}

cargo test fails

On a new install fresh from githup. Tests 3,4, and 5 fail

Need to run:
pip3 install gym[atari]

Then test 4 only fails

---- tests::test_4_reset stdout ----
Response { url: "http://127.0.0.1:8003/v1/envs/d027c24a/reset/", status: 200, headers: {"content-type": "application/json", "content-length": "368264", "server": "Werkzeug/0.16.0 Python/3.7.3", "date": "Sat, 12 Oct 2019 00:32:05 GMT"} }
thread 'tests::test_4_reset' panicked at 'assertion failed: env.reset().is_ok()', src/lib.rs:353:4
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.


failures:
    tests::test_4_reset

Cannot render environments

I'm trying this on a MBP.

Maybe I'm doing something completely wrong. I modified the test_step test by adding a env.render(). The test fails with this error:

Caused by:
  process didn't exit successfully: `/Users/user/dev/gym-rs/target/debug/deps/gym-d47e830353331d9d 'tests::test_step' --nocapture` (signal: 6, SIGABRT: process abort signal)
↯ cargo test tests::test_step
   Compiling gym v2.2.1 (/Users/user/dev/gym-rs)
    Finished test [unoptimized + debuginfo] target(s) in 2.22s
     Running target/debug/deps/gym-d47e830353331d9d

running 1 test
2020-12-22 17:02:45.557 gym-d47e830353331d9d[19905:15804746] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'NSWindow drag regions should only be invalidated on the Main Thread!'
*** First throw call stack:
(
        0   CoreFoundation                      0x00007fff2d3bcb57 __exceptionPreprocess + 250
        1   libobjc.A.dylib                     0x00007fff662085bf objc_exception_throw + 48
        2   CoreFoundation                      0x00007fff2d3e534c -[NSException raise] + 9
        3   AppKit                              0x00007fff2a5df5ec -[NSWindow(NSWindow_Theme) _postWindowNeedsToResetDragMarginsUnlessPostingDisabled] + 310
        4   AppKit                              0x00007fff2a5c7052 -[NSWindow _initContent:styleMask:backing:defer:contentView:] + 1416
        5   AppKit                              0x00007fff2a5c6ac3 -[NSWindow initWithContentRect:styleMask:backing:defer:] + 42
        6   _ctypes.cpython-38-darwin.so        0x0000000110b8a177 ffi_call_unix64 + 79
        7   ???                                 0x000070000b05bdc0 0x0 + 123145487236544
)
libc++abi.dylib: terminating with uncaught exception of type NSException
error: test failed, to rerun pass '--lib'

Caused by:
  process didn't exit successfully: `/Users/user/dev/gym-rs/target/debug/deps/gym-d47e830353331d9d 'tests::test_step'` (signal: 6, SIGABRT: process abort signal)

Are all environments supporting rendering?

EDIT: I've just tried to render the VideoPinball one, it also crashes. Not sure why but FrozenLake works, probably because it only renders some characters to terminal.

migrate to gymnasium

As stated in the Gym repository, Gym is no longer maintained https://github.com/openai/gym#important-notice
Maybe we should consider migrating to Gymnasium, the new officially maintained fork.

Not all `id`s from `OpenAI` recognised

I have been playing with basic-rs replacing CartPole-v0 with envs from OpenAI. Some work some do not. The ones that fail seem to fail here:

I had errors with:

CarRacing-v0
CarRacing-v0
Both had same error

ModuleNotFoundError: No module named 'Box2D'
thread 'main' panicked at 'Could not make environment because of error:
expected value at line 1 column 1', examples/basic.rs:11:14

Ant-v2
Hopper-v2
Both had same error

[2019-10-07 09:47:18,255] Making new env: Hopper-v2
/home/worik/.local/lib/python3.7/site-packages/gym/envs/registration.py:17: PkgResourcesDeprecationWarning: Parameters to load are deprecated.  Call .resolve and .require separately.
  result = entry_point.load(False)
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', src/libcore/option.rs:378:21

Apparently working:

Acrobot-v1
MountainCar-v0

Etcetera.

What can I do to help?