Comments (22)
Thanks @miyosuda, that's definitely very helpful. Also, another person in my lab, @junhyukoh, figured out that /bazel-bin/random_agent.runfiles/org_deepmind_lab is where all the necessary files reside. Running a python interpreter from here allows one to import deepmind lab, create a lab environment, and reset it without any problems.
from lab.
@johnholl
I'm not sure whether this helps or not, but I tried UNREAL algorithm based on A3C with lab.
https://github.com/miyosuda/unreal
I put project dir inside lab
dir and put another BUILD file for it.
First I tried to call lab environment from outside, but it was complicated so I decided to do it like this.
from lab.
Hi - you would need to make your Python program depend on the module, something like this:
py_binary(
name = "foo",
srcs = ["foo.py"],
deps = [":deepmind_lab.so"],
)
Inside your Python code, import the lab module in the same way the demo code does it.
from lab.
My understanding is that after doing what you're saying, I would run the program with the command bazel run :foo
and not python foo.py
. My question is: what do I need to do to run python foo.py
, or is lab designed to not be able to run in this way. Initially after downloading and building deepmind_lab.so the command
python random_agent.py
will produce
ImportError: No module named deepmind_lab
I tried rectifying this by adding the bazel-bin directory to PYTHONPATH. This lets me import deepmind_lab, but attempting to create a lab environment ie
lab = deepmind_lab.Lab('demo_map')
produces the error
./libdmlab.so: cannot open shared object file: No such file or directory RuntimeError: Failed to connect RL API
.
This runtime error doesn't happen if the python interpreter is called inside the bazel-bin directory. Eventually, after creating symbolic links to the files in bazel-bin and creating a baselab folder containing assets, I managed to create a lab environment without error in a python interpreter. However
lab.reset()
produces a popup error message that says "VM_Create on UI failed. See "~/.deepmind_lab/baselab/crashlog.txt" for details."
I read through the crashlog but couldn't figure what was going on.
from lab.
Does the reset
error happen when you use bazel run
?
I think you have to use bazel run
in order to get everything set up right. Alternatively, you can run bazel-bin/foo
from your project root. But I don't think directly calling python
will work.
from lab.
No, there's no error when doing bazel run. I ask about running it with python foo.py
as I have an existing project (A3C implementation) which I'd like to interface with lab and it'd be much easier to do it that way, but it sounds like this might not be doable.
from lab.
No, unfortunately Bazel does not really lend itself to exporting reusable library components. This is a known issue, but I'm afraid I don't have a good solution for it right now.
from lab.
(Try ldd
on the DSO; I think it may be linked against other parts of the Bazel workspace which you would also need to copy out.)
from lab.
ldd deepmind_lab.so
outputs
linux-vdso.so.1 => (0x00007fff581cb000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fd14e116000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fd14de04000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fd14dafe000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fd14d8e7000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fd14d522000)
/lib64/ld-linux-x86-64.so.2 (0x00007fd14e528000)
from lab.
Try also the other DSO that's loaded by dmlab_so_loader.cc
.
from lab.
ldd deepmind_lab.so
produces rather more:
linux-vdso.so.1 => (0x00007ffc58bbf000) libOSMesa.so.6 => /usr/lib/x86_64-linux-gnu/libOSMesa.so.6 (0x00007fe840058000) libSDL2-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libSDL2-2.0.so.0 (0x00007fe83fd52000) liblua5.1.so.0 => /usr/lib/x86_64-linux-gnu/liblua5.1.so.0 (0x00007fe83fb25000) libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fe83f813000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fe83f50c000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fe83f2f5000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fe83ef30000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fe83ed11000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fe83eb0d000) libasound.so.2 => /usr/lib/x86_64-linux-gnu/libasound.so.2 (0x00007fe83e81d000) libpulse-simple.so.0 => /usr/lib/x86_64-linux-gnu/libpulse-simple.so.0 (0x00007fe83e618000) libpulse.so.0 => /usr/lib/x86_64-linux-gnu/libpulse.so.0 (0x00007fe83e3cf000) libX11.so.6 => /usr/lib/x86_64-linux-gnu/libX11.so.6 (0x00007fe83e09a000) libXext.so.6 => /usr/lib/x86_64-linux-gnu/libXext.so.6 (0x00007fe83de87000) libXcursor.so.1 => /usr/lib/x86_64-linux-gnu/libXcursor.so.1 (0x00007fe83dc7d000) libXinerama.so.1 => /usr/lib/x86_64-linux-gnu/libXinerama.so.1 (0x00007fe83da7a000) libXi.so.6 => /usr/lib/x86_64-linux-gnu/libXi.so.6 (0x00007fe83d869000) libXrandr.so.2 => /usr/lib/x86_64-linux-gnu/libXrandr.so.2 (0x00007fe83d65f000) libXss.so.1 => /usr/lib/x86_64-linux-gnu/libXss.so.1 (0x00007fe83d45b000) libXxf86vm.so.1 => /usr/lib/x86_64-linux-gnu/libXxf86vm.so.1 (0x00007fe83d254000) libwayland-egl.so.1 => /usr/lib/x86_64-linux-gnu/libwayland-egl.so.1 (0x00007fe83d052000) libwayland-client.so.0 => /usr/lib/x86_64-linux-gnu/libwayland-client.so.0 (0x00007fe83ce45000) libwayland-cursor.so.0 => /usr/lib/x86_64-linux-gnu/libwayland-cursor.so.0 (0x00007fe83cc3c000) libxkbcommon.so.0 => /usr/lib/x86_64-linux-gnu/libxkbcommon.so.0 (0x00007fe83ca02000) librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fe83c7fa000) /lib64/ld-linux-x86-64.so.2 (0x0000564811452000) libpulsecommon-4.0.so => /usr/lib/x86_64-linux-gnu/pulseaudio/libpulsecommon-4.0.so (0x00007fe83c592000) libjson-c.so.2 => /lib/x86_64-linux-gnu/libjson-c.so.2 (0x00007fe83c387000) libdbus-1.so.3 => /lib/x86_64-linux-gnu/libdbus-1.so.3 (0x00007fe83c141000) libxcb.so.1 => /usr/lib/x86_64-linux-gnu/libxcb.so.1 (0x00007fe83bf22000) libXrender.so.1 => /usr/lib/x86_64-linux-gnu/libXrender.so.1 (0x00007fe83bd18000) libXfixes.so.3 => /usr/lib/x86_64-linux-gnu/libXfixes.so.3 (0x00007fe83bb11000) libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007fe83b909000) libwrap.so.0 => /lib/x86_64-linux-gnu/libwrap.so.0 (0x00007fe83b6fe000) libsndfile.so.1 => /usr/lib/x86_64-linux-gnu/libsndfile.so.1 (0x00007fe83b496000) libasyncns.so.0 => /usr/lib/x86_64-linux-gnu/libasyncns.so.0 (0x00007fe83b290000) libXau.so.6 => /usr/lib/x86_64-linux-gnu/libXau.so.6 (0x00007fe83b08b000) libXdmcp.so.6 => /usr/lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007fe83ae85000) libnsl.so.1 => /lib/x86_64-linux-gnu/libnsl.so.1 (0x00007fe83ac6b000) libFLAC.so.8 => /usr/lib/x86_64-linux-gnu/libFLAC.so.8 (0x00007fe83aa39000) libvorbisenc.so.2 => /usr/lib/x86_64-linux-gnu/libvorbisenc.so.2 (0x00007fe83a56a000) libvorbis.so.0 => /usr/lib/x86_64-linux-gnu/libvorbis.so.0 (0x00007fe83a33d000) libogg.so.0 => /usr/lib/x86_64-linux-gnu/libogg.so.0 (0x00007fe83a133000) libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007fe839f18000)
Also, I was thinking it would be easier to give this up and directly run the binaries that bazel creates, however I run into the exact same issues listed above when I try to run the binaries with ./bazel-bin/foo
! So the only way I am able to run anything is using bazel run :foo
, but bazel only allows one process to be run at a time which makes running things this way not ideal.
from lab.
Ah yes, that's right: there's a wrapper script in bazel-bin/foo
, but the complete directory structure as created by the various dependencies is in bazel-bin/foo.runfiles/...
, so relative to that location all the files are where they should be.
from lab.
@tkoeppe the solution proposed by @miyosuda works perfectly fine. However I think the better way would be to let bazel handle it automatically. I am trying to get a working example which uses the bazel tools to get lab as external dependency. However I am facing some problems.
I get following error
ERROR: ~/.cache/bazel/_bazel_noone/ae6e9d00e8c783d8279db324d79ccc60/external/org_deepmind_lab/BUILD:432:1: no such package '@zlib_archive//': error loading package 'external': The repository named 'zlib_archive' could not be resolved and referenced by '@org_deepmind_lab//:bspc'.
ERROR: Analysis of target '//:random_agent' failed; build aborted.
INFO: Elapsed time: 13.452s
`
$tree
my-project
├── BUILD
├── python.BUILD
├── random_agent.py
└── WORKSPACE
0 directories, 4 files
$cat WORKSPACE
workspace(name = "maze_a3c")
new_local_repository(
name = "python_system",
build_file = "python.BUILD",
path = "/usr",
)
git_repository(
name = "org_deepmind_lab",
remote = "https://github.com/deepmind/lab.git",
commit = "9af8f87376c3418dd40ade46465d677c0b96f39e"
$cat BUILD
py_binary(
name = "random_agent",
srcs = ["random_agent.py"],
data = ["@org_deepmind_lab//:deepmind_lab.so"],
main = "random_agent.py",
`
Kindly suggest what is the correct way to add bazel repository as external dependency and also correctly build it. As this pertains more to bazel build system I have asked the question there as well. Another similar issue in bazel.
from lab.
@arpit15: I'm afraid I don't have an answer for you.
I've discussed this with the Bazel team in the past, and I think we all agree that Bazel is not very well suited at the moment to make exportable builds of reusable libraries. Bazel works well for a single-root project, where it gives you a very convenient mechanism to assemble dependencies of both compile-time code and runtime assets -- but it only works if you stay within the confines of the system.
My understanding is that the Bazel team is working on making Bazel more flexible, and I'd encourage you to keep up with the developments there. But as far as the immediate present is concerned, I am thinking that it might be useful to set up an alternative build system if you want something exportable -- but I don't know what that would look like.
from lab.
@tkoeppe , do you any updates on this issue?
from lab.
@RylanSchaeffer: I think we have some code in the works to produce a pip package. I hope we'll get to push that soon.
from lab.
@tkoeppe Awesome! Do you have a rough estimate of when "soon" will be?
from lab.
@RylanSchaeffer: No hard guarantees, but hopefully soon.
from lab.
We didn't put this into the recent release, but it should make it into HEAD very soon.
from lab.
This should be available as of 0a0638c.
from lab.
@tkoeppe , I finally returned to this issue and I ran into a problem following the instructions. Specifically,
sudo bazel build python/pip_package:build_pip_package
Extracting Bazel installation...
............
INFO: Analysed target //python/pip_package:build_pip_package (48 packages loaded).
INFO: Found 1 target...
ERROR: /localhome/lab/deepmind/lua/BUILD:140:1: C++ compilation of rule '//deepmind/lua:push_script' failed (Exit 1)
In file included from ./deepmind/lua/push_script.h:26:0,
from deepmind/lua/push_script.cc:19:
./deepmind/lua/lua.h:25:21: fatal error: lauxlib.h: No such file or directory
compilation terminated.
Target //python/pip_package:build_pip_package failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 20.730s, Critical Path: 2.02s
FAILED: Build did NOT complete successfully
Do you know what might have caused this?
from lab.
@RylanSchaeffer: On which platform is this? You may have to edit lua.BUILD
as described somewhere in https://github.com/deepmind/lab/blob/master/docs/users/build.md to match your system's library paths.
from lab.
Related Issues (20)
- Make random maze as Capture The Flag mode HOT 2
- RuntimeError: Failed to connect RL API HOT 4
- ImportError: deepmind_lab.so: undefined symbol: PyString_Type error occurred when I was running DeepMind Lab HOT 5
- RuntimeError: Failed to connect RL API
- Reset() environment to a specific state? HOT 2
- Failed to find function dmlab_connect in library! RuntimeError: Failed to connect RL API
- Adding "I-maze" map/environment HOT 5
- Access camera params (extrinsics, intrinsics, near/far plane) for pointcloud calculation HOT 1
- How to produce deterministic episode? HOT 1
- custom environment
- Bazel Build Error: no such target '@platforms//cpu:wasm32': target 'wasm32' not declared in package 'cpu' HOT 3
- Disable Velocity HOT 1
- Problem with Bazel Python3 build HOT 2
- Wheel not supported error HOT 6
- Change the speed of the agent dynamically
- Build fails with message about C++14 being required even with a C++14-capable compiler HOT 2
- Complete installation script for Ubuntu 20.04 HOT 5
- Banino et. al. Grid Cells explore_goal_locations and explore_obstructed_goals HOT 1
- MacOS installation issues HOT 5
- When I run bazel run :python_random_agent --define graphics=sdl -- \ --length=10000 --width=640 --height=480 in Ubuntu 20.04, I meet the following error. HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from lab.