Git Product home page Git Product logo

google-deepmind / lab Goto Github PK

View Code? Open in Web Editor NEW
7.0K 468.0 1.4K 449.34 MB

A customisable 3D platform for agent-based AI research

License: Other

Python 0.89% Lua 8.36% GLSL 0.95% C 81.27% Objective-C 0.02% C++ 6.47% Shell 0.04% Assembly 0.16% Makefile 0.01% HTML 0.44% Yacc 0.03% Perl 0.01% ShaderLab 0.69% Roff 0.13% Starlark 0.55%
machine-learning artificial-intelligence neural-networks deep-learning

lab's Introduction

DeepMind Lab

DeepMind Lab is a 3D learning environment based on id Software's Quake III Arena via ioquake3 and other open source software.

DeepMind Lab provides a suite of challenging 3D navigation and puzzle-solving tasks for learning agents. Its primary purpose is to act as a testbed for research in artificial intelligence, especially deep reinforcement learning.

About

Disclaimer: This is not an official Google product.

If you use DeepMind Lab in your research and would like to cite the DeepMind Lab environment, we suggest you cite the DeepMind Lab paper.

You can reach us at [email protected].

Getting started on Linux

$ git clone https://github.com/deepmind/lab
$ cd lab

For a live example of a random agent, run

lab$ bazel run :python_random_agent --define graphics=sdl -- \
               --length=10000 --width=640 --height=480

Here is some more detailed build documentation, including how to install dependencies if you don't have them.

To enable compiler optimizations, pass the flag --compilation_mode=opt, or -c opt for short, to each bazel build, bazel test and bazel run command. The flag is omitted from the examples here for brevity, but it should be used for real training and evaluation where performance matters.

Play as a human

To test the game using human input controls, run

lab$ bazel run :game -- --level_script=tests/empty_room_test --level_setting=logToStdErr=true
# or:
lab$ bazel run :game -- -l tests/empty_room_test -s logToStdErr=true

Leave the logToStdErr setting off to disable most log output.

The values of observations that the environment exposes can be printed at every step by adding a flag --observation OBSERVATION_NAME for each observation of interest.

lab$ bazel run :game -- --level_script=lt_chasm --observation VEL.TRANS --observation VEL.ROT

Train an agent

DeepMind Lab ships with an example random agent in python/random_agent.py which can be used as a starting point for implementing a learning agent. To let this agent interact with DeepMind Lab for training, run

lab$ bazel run :python_random_agent

The Python API is used for agent-environment interactions. We also provide bindings to DeepMind's "dm_env" general API for reinforcement learning, as well as a way to build a self-contained PIP package; see the separate documentation for details.

DeepMind Lab ships with different levels implementing different tasks. These tasks can be configured using Lua scripts, as described in the Lua API.


Upstream sources

DeepMind Lab is built from the ioquake3 game engine, and it uses the tools q3map2 and bspc for map creation. Bug fixes and cleanups that originate with those projects are best fixed upstream and then merged into DeepMind Lab.

  • bspc is taken from github.com/TTimo/bspc, revision d9a372db3fb6163bc49ead41c76c801a3d14cf80. There are virtually no local modifications, although we integrate this code with the main ioq3 code and do not use their copy in the deps directory. We expect this code to be stable.

  • q3map2 is taken from github.com/TTimo/GtkRadiant, revision d3d00345c542c8d7cc74e2e8a577bdf76f79c701. A few minor local modifications add synchronization. We also expect this code to be stable.

  • ioquake3 is taken from github.com/ioquake/ioq3, revision 29db64070aa0bae49953bddbedbed5e317af48ba. The code contains extensive modifications and additions. We aim to merge upstream changes occasionally.

We are very grateful to the maintainers of these repositories for all their hard work on maintaining high-quality code bases.

External dependencies, prerequisites and porting notes

DeepMind Lab currently ships as source code only. It depends on a few external software libraries, which we ship in several different ways:

  • The zlib, glib, libxml2, jpeg and png libraries are referenced as external Bazel sources, and Bazel BUILD files are provided. The dependent code itself should be fairly portable, but the BUILD rules we ship are specific to Linux on x86. To build on a different platform you will most likely have to edit those BUILD files.

  • Message digest algorithms are included in this package (in //third_party/md), taken from the reference implementations of their respective RFCs. A "generic reinforcement learning API" is included in //third_party/rl_api, which has also been created by the DeepMind Lab authors. This code is portable.

  • EGL headers are included in this package (in //third_party/GL/{EGL,KHR}), taken from the Khronos OpenGL/OpenGL ES XML API Registry at www.khronos.org/registry/EGL. The headers have been modified slightly to remove the dependency of EGL on X.

  • Several additional libraries are required but are not shipped in any form; they must be present on your system:

    • SDL 2
    • gettext (required by glib)
    • OpenGL: A hardware driver and library are needed for hardware-accelerated human play. The headless library that machine learning agents will want to use can use either hardware-accelerated rendering via EGL or GLX or software rendering via OSMesa, depending on the --define headless=... build setting.
    • Python 2.7 (other versions might work, too) with NumPy, PIL (a few tests require a NumPy version of at least 1.8), or Python 3 (at least 3.5) with NumPy and Pillow.

The build rules are using a few compiler settings that are specific to GCC. If some flags are not recognized by your compiler (typically those would be specific warning suppressions), you may have to edit those flags. The warnings should be noisy but harmless.

lab's People

Contributors

charlesbeattie avatar cool-rr avatar decster avatar pablo-davila avatar simsgautam avatar tkoeppe avatar wookayin avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

lab's Issues

Compilation Error Occured.

I ran the command on ubuntu/trusty64.
The error has occured.

$ bazel run :random_agent --define headless=false -- --length=10000 --width=640 --height=480
INFO: Found 1 target...
ERROR: /home/vagrant/lab/BUILD:980:1: C++ compilation of rule '//:deepmind_lab.so' failed: linux-sandbox failed: error executing command /home/vagrant/.cache/bazel/_bazel_vagrant/c1d0c0e8255f05e00abeeb68d70c5ac4/execroot/lab/_bin/linux-sandbox ... (remaining 48 argument(s) skipped).
python/dmlab_module.c:29:31: fatal error: numpy/arrayobject.h: No such file or directory
 #include "numpy/arrayobject.h"
                               ^
compilation terminated.
WARNING: Cannot delete sandbox directory after action execution: /home/vagrant/.cache/bazel/_bazel_vagrant/c1d0c0e8255f05e00abeeb68d70c5ac4/bazel-sandbox/867732be-eacb-4fb1-b1ab-3e24f38afc93-13 (java.io.IOException: /home/vagrant/.cache/bazel/_bazel_vagrant/c1d0c0e8255f05e00abeeb68d70c5ac4/bazel-sandbox/867732be-eacb-4fb1-b1ab-3e24f38afc93-13/execroot/lab (Device or resource busy)).
Target //:random_agent failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 48.362s, Critical Path: 4.48s
ERROR: Build failed. Not running target.

I've already installed numpy on python2.7. Please help.

Receiving depth image from the simulator, Changing the environment

Hi,

Thanks for your package.

As I noticed, it is possible to create an agent with your package, move the agent and receive the agent's view (RGB image) in every 4 steps or so.

My question is: can I receive a depth image instead of RGB image? and can I change the environment?

Regards,

I use external controls but mouse and keyboard still give actions.

Hi,
I use the C API, headless, and I set this :
if (env_c_api.setting(context, "controls", "external") != 0) {
sys_error("Failed to apply 'controls' setting.");
}

I give some actions using env_c_api.act(context, discr_actions, cont_actions);...just for tests.....I look right and start moving backwards....nothing fancy.

But if I move the mouse and press the arrows on the keyboard, they still give actions.
How do I cancel the keyboard and mouse, while I give the actions programmatically though the C API ?

Thanks,
Viorel

Process exited with status 3 in Bazel run

Hi,

I met some errors when using bazel to build the lab.

My environment information: Ubuntu 14.04. Bazel is on the stable version, installed according to bazel.io 's tutorial. All other required packages have been installed. Specific Dockerfile I used can be found in here.

When I run the command according to build.md, e.g.,

bazel build :deepmind_lab.so --define headless=glx

I can build the target :deepmind_lab.so.

Target //:deepmind_lab.so up-to-date:
  bazel-bin/deepmind_lab.so
INFO: Elapsed time: 130.453s, Critical Path: 130.11s

When it comes to bazel run, I met the error as following:

โžœ  lab git:(master) bazel run :python_module_test --define headless=glx
WARNING: Sandboxed execution is not supported on your system and thus hermeticity of actions cannot be guaranteed. See http://bazel.build/docs/bazel-user-manual.html#sandboxing for more information. You can turn off this warning via --ignore_unsupported_sandboxing.
INFO: Found 1 target...
Target //:python_module_test up-to-date:
  bazel-bin/python_module_test
INFO: Elapsed time: 0.267s, Critical Path: 0.01s

INFO: Running command line: bazel-bin/python_module_test
...ERROR: Non-zero return code '3' from command: Process exited with status 3.

also, on headless=false:

โžœ  lab git:(master) bazel run :random_agent --define headless=false
WARNING: Sandboxed execution is not supported on your system and thus hermeticity of actions cannot be guaranteed. See http://bazel.build/docs/bazel-user-manual.html#sandboxing for more information. You can turn off this warning via --ignore_unsupported_sandboxing.
INFO: Found 1 target...
Target //:random_agent up-to-date:
  bazel-bin/random_agent
INFO: Elapsed time: 3.664s, Critical Path: 2.15s

INFO: Running command line: bazel-bin/random_agent
ERROR: Non-zero return code '3' from command: Process exited with status 3.

Though with --help it can shows the list of arguments:

โžœ  lab git:(master) bazel run :random_agent -- --help
WARNING: Sandboxed execution is not supported on your system and thus hermeticity of actions cannot be guaranteed. See http://bazel.build/docs/bazel-user-manual.html#sandboxing for more information. You can turn off this warning via --ignore_unsupported_sandboxing.
INFO: Found 1 target...
Target //:random_agent up-to-date:
  bazel-bin/random_agent
INFO: Elapsed time: 3.128s, Critical Path: 2.30s

INFO: Running command line: bazel-bin/random_agent --help
usage: random_agent.py [-h] [--length LENGTH] [--width WIDTH]
                       [--height HEIGHT] [--fps FPS]
                       [--runfiles_path RUNFILES_PATH]
                       [--level_script LEVEL_SCRIPT]

Basic random agent for DeepMind Lab.

optional arguments:
  -h, --help            show this help message and exit
  --length LENGTH       Number of steps to run the agent
...

Could you please help on this?

Thanks!

Bazel --batch option causes compile errors

In the course of setting up a Dockerfile, I was getting a few compile errors. When I removed these lines from the .bazelrc file, it seems to resolve them.

startup --batch
build --spawn_strategy=standalone --genrule_strategy=standalone

The reason I had added these lines originally were because there was supposedly some issues running bazel during docker build.. but that doesn't seem to be a problem with my versions.

# Running bazel inside a `docker build` command causes trouble, cf:
#   https://github.com/bazelbuild/bazel/issues/134
# The easiest solution is to set up a bazelrc file forcing --batch.
# RUN echo "startup --batch" >>/root/.bazelrc
# Similarly, we need to workaround sandboxing issues:
#   https://github.com/bazelbuild/bazel/issues/418
# RUN echo "build --spawn_strategy=standalone --genrule_strategy=standalone" \
#    >>/root/.bazelrc
# ENV BAZELRC /root/.bazelrc

The errors:

...
external/libxml_archive/xmlIO.c:1450:52: error: 'LZMA_OK' undeclared (first use in this function)
     ret =  (__libxml2_xzclose((xzFile) context) == LZMA_OK ) ? 0 : -1;
                                                    ^
external/libxml_archive/xmlIO.c:1450:52: note: each undeclared identifier is reported only once for each function it appears in
Target //:deepmind_lab.so failed to build
INFO: Elapsed time: 23.804s, Critical Path: 10.05s
ERROR: Build failed. Not running target.

Build DeepMind Lab Failed?

Hi,
I run the command:"bazel build : deepmind_lab.so --define headless=glx", and then get the following output:

xu-jinwgei@xujinwgei-Precision-Tower-7910:~/lab-master$ bazel build :deepmind_lab.so --define headless=glx
INFO: Found 1 target...
ERROR: /home/xu-jinwgei/lab-master/BUILD:838:1: Executing genrule //:non_pk3_assets failed: linux-sandbox failed: error executing command /home/xu-jinwgei/.cache/bazel/_bazel_xu-jinwgei/7f9c394f362e8089472ee6d14cb49bf2/execroot/lab-master/_bin/linux-sandbox ... (remaining 5 argument(s) skipped).
/bin/bash: realpath: command not found
ln: missing file operand
Try 'ln --help' for more information.
Target //:deepmind_lab.so failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.571s, Critical Path: 0.27s

What should I deal with the message?Thanks!

record human demonstration and use it in RL

Hi:
I try to use human input to control agent, and I want to record this progress which includes the state of environment, action of agent, and reward. What should i do with deempind lab?

Error downloading libpng during bazel run

Hi, I see that OSX isn't supported, so I'm working on a dockerfile to make this more accessible. I've got VNC and bazil installed, but when I run the build I'm getting an error.

Step 15 : RUN bazel build :deepmind_lab.so --define headless=osmesa
 ---> Running in 93077abf42e5
INFO: Reading 'startup' options from /root/.bazelrc: --batch
Extracting Bazel installation...
WARNING: Sandboxed execution is not supported on your system and thus hermeticity of actions cannot be guaranteed. See http://bazel.build/docs/bazel-user-manual.html#sandboxing for more information. You can turn off this warning via --ignore_unsupported_sandboxing.
____Loading package:
____Loading package: @bazel_tools//tools/cpp
____Loading package: @bazel_tools//tools/jdk
____Loading package: @local_config_xcode//
____Loading package: @local_jdk//
____Loading package: @local_config_cc//
____Loading complete.  Analyzing...
____Loading package: deepmind/level_generation
____Loading package: @bazel_tools//tools/genrule
____Loading package: q3map2
____Loading package: third_party/rl_api
____Loading package: third_party/md
____Loading package: deepmind/engine
____Loading package: deepmind/include
____Loading package: @sdl_system//
____Downloading http://zlib.net/zlib-1.2.8.tar.gz: 116,930 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 42,673 bytes
____Downloading http://zlib.net/zlib-1.2.8.tar.gz: 357,298 bytes
____Failed to connect to https://github.com/glennrp/libpng/archive/v1.2.53.zip trying again in 100ms
____Downloading http://www.ijg.org/files/jpegsrc.v9a.tar.gz: 50,837 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 71,633 bytes
____Failed to connect to https://github.com/glennrp/libpng/archive/v1.2.53.zip trying again in 200ms
____Downloading http://www.ijg.org/files/jpegsrc.v9a.tar.gz: 142,817 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 122,313 bytes
____Loading package: @zlib_archive//
____Failed to connect to https://github.com/glennrp/libpng/archive/v1.2.53.zip trying again in 400ms
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 214,033 bytes
____Downloading http://ftp.gnome.org/pub/gnome/sources/glib/2.38/glib-2.38.2.tar.xz via gensho.acc.umu.se: 49,905 bytes
____Downloading http://www.ijg.org/files/jpegsrc.v9a.tar.gz: 469,857 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 261,817 bytes
____Downloading http://www.ijg.org/files/jpegsrc.v9a.tar.gz: 696,157 bytes
____Downloading http://ftp.gnome.org/pub/gnome/sources/glib/2.38/glib-2.38.2.tar.xz via gensho.acc.umu.se: 205,337 bytes
____Loading package: @jpeg_archive//
____Loading package: deepmind/lua
____Downloading http://ftp.gnome.org/pub/gnome/sources/glib/2.38/glib-2.38.2.tar.xz via gensho.acc.umu.se: 1,136,401 bytes
____Loading package: @lua_system//
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 486,257 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 535,489 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 580,377 bytes
____Downloading http://ftp.gnome.org/pub/gnome/sources/glib/2.38/glib-2.38.2.tar.xz via gensho.acc.umu.se: 3,202,697 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 632,009 bytes
____Downloading http://ftp.gnome.org/pub/gnome/sources/glib/2.38/glib-2.38.2.tar.xz via gensho.acc.umu.se: 3,742,305 bytes
____Downloading http://ftp.gnome.org/pub/gnome/sources/glib/2.38/glib-2.38.2.tar.xz via gensho.acc.umu.se: 4,545,945 bytes
____Downloading http://ftp.gnome.org/pub/gnome/sources/glib/2.38/glib-2.38.2.tar.xz via gensho.acc.umu.se: 5,240,033 bytes
____Downloading http://ftp.gnome.org/pub/gnome/sources/glib/2.38/glib-2.38.2.tar.xz via gensho.acc.umu.se: 5,847,697 bytes
____Downloading http://ftp.gnome.org/pub/gnome/sources/glib/2.38/glib-2.38.2.tar.xz via gensho.acc.umu.se: 6,492,057 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 875,769 bytes
____Failed to connect to https://github.com/glennrp/libpng/archive/v1.2.53.zip trying again in 3,200ms
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 939,481 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 998,849 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 1,050,977 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 1,095,865 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 1,145,097 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 1,194,329 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 1,256,593 bytes
____Loading package: @glib_archive//
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 1,355,057 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 1,402,841 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 1,447,729 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 1,495,513 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 1,556,329 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 1,608,457 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 1,664,929 bytes
____Failed to connect to https://github.com/glennrp/libpng/archive/v1.2.53.zip trying again in 6,400ms
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 1,792,353 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 1,835,793 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 1,879,233 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 1,955,977 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 2,005,209 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 2,027,881 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 2,100,777 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 2,151,457 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 2,203,089 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 2,277,433 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 2,317,977 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 2,354,177 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 2,383,137 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 2,428,025 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 2,455,537 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 2,487,393 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 2,540,969 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 2,577,169 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 2,622,057 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 2,652,465 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 2,713,281 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 2,775,545 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 2,832,017 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 2,868,217 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 2,918,897 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 2,969,577 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 3,014,465 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 3,070,937 bytes
____Downloading http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz: 3,130,305 bytes
ERROR: /lab/q3map2/BUILD:54:1: no such package '@png_archive//': Error downloading [http://github.com/glennrp/libpng/archive/v1.2.53.zip] to /root/.cache/bazel/_bazel_root/3267a40151b19e3e3f8d4c8a3edf34ee/external/png_archive/v1.2.53.zip: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty and referenced by '//q3map2:q3map2'.
ERROR: Analysis of target '//:deepmind_lab.so' failed; build aborted.
____Elapsed time: 19.122s
The command '/bin/sh -c bazel build :deepmind_lab.so --define headless=osmesa' returned a non-zero code: 1

bazel fetch (and build) fails on OSX with "no such package '@glib_archive//'"

Doing all this over NIPS conference wifi has been a bit of a ride, but I was able get a setup where all the external dependencies can be pulled reliably (by rehosting them locally).

Trying to just make sure I have all the prerequisites for lab, I'm running bazel fetch :game. Output:

$> bazel fetch :game
Extracting Bazel installation...
............
ERROR: /Users/a/code/deepmind-lab/q3map2/BUILD:54:1: no such package '@glib_archive//': Error creating symbolic link from /private/var/tmp/_bazel_a/160592a534b1347832993df9677151a1/external/glib_archive/BUILD to /Users/a/code/deepmind-lab/glib.BUILD: /private/var/tmp/_bazel_a/160592a534b1347832993df9677151a1/external/glib_archive/BUILD (Directory not empty) and referenced by '//q3map2:q3map2'.
ERROR: /Users/a/code/deepmind-lab/q3map2/BUILD:54:1: no such package '@glib_archive//': Error creating symbolic link from /private/var/tmp/_bazel_a/160592a534b1347832993df9677151a1/external/glib_archive/BUILD to /Users/a/code/deepmind-lab/glib.BUILD: /private/var/tmp/_bazel_a/160592a534b1347832993df9677151a1/external/glib_archive/BUILD (Directory not empty) and referenced by '//q3map2:q3map2'.
ERROR: Evaluation of query "deps((:game union @bazel_tools//tools/jdk:toolchain))" failed: errors were encountered while computing transitive closure.

Might be an issue with bazel on Mac, but want to make sure it's not something obvious I'm missing.

Thanks.

GPU benchmark?

I ran the included benchmark and received number pretty close to the CPU numbers on page 7 of the white paper

henryk@henryk-michalewski-pc:~/lab$ bazel run :python_benchmark
INFO: Found 1 target...
Target //:python_benchmark up-to-date:
  bazel-bin/python_benchmark
INFO: Elapsed time: 0.111s, Critical Path: 0.00s

INFO: Running command line: bazel-bin/python_benchmark
Map loaded: 'nav_maze_static_01'
resolution: 84 x 84, spec: RGB, steps: 3600, duration: 14.4, fps: 249.8
Map loaded: 'nav_maze_static_01'
resolution: 84 x 84, spec: RGBD, steps: 3600, duration: 15.4, fps: 233.3
Map loaded: 'nav_maze_static_01'
resolution: 160 x 120, spec: RGB, steps: 3600, duration: 39.9, fps: 90.3
Map loaded: 'nav_maze_static_01'
resolution: 160 x 120, spec: RGBD, steps: 3600, duration: 39.3, fps: 91.7
Map loaded: 'nav_maze_static_01'
resolution: 320 x 240, spec: RGB, steps: 3600, duration: 115.9, fps: 31.1
Map loaded: 'nav_maze_static_01'
resolution: 320 x 240, spec: RGBD, steps: 3600, duration: 118.4, fps: 30.4
Map loaded: 'lt_space_bounce_01'
resolution: 84 x 84, spec: RGB, steps: 3600, duration: 10.8, fps: 332.2
Map loaded: 'lt_space_bounce_01'
resolution: 84 x 84, spec: RGBD, steps: 3600, duration: 15.4, fps: 233.3
Map loaded: 'lt_space_bounce_01'
resolution: 160 x 120, spec: RGB, steps: 3600, duration: 14.0, fps: 256.8
Map loaded: 'lt_space_bounce_01'
resolution: 160 x 120, spec: RGBD, steps: 3600, duration: 15.2, fps: 237.1
Map loaded: 'lt_space_bounce_01'
resolution: 320 x 240, spec: RGB, steps: 3600, duration: 42.8, fps: 84.1
Map loaded: 'lt_space_bounce_01'
resolution: 320 x 240, spec: RGBD, steps: 3600, duration: 37.4, fps: 96.3

How can I enable/disable GPU support? My intention is to check performance on GTX 10x0 cards.

Remove Scores and Texts Display

Hi,
I get the attached RGB and depth image from the "RGBD_INTERLACED" observation, but haven't figured out how to remove the scores, texts and boxes in the bottom part of the images, can you give me some instructions? I haven't had any luck from the docs... Thanks in advance!
rgb depth

undefined symbol: _Py_ZeroStruct

Hi, when I follow the instructions from https://github.com/deepmind/lab to install lab at ubuntu 16.04, I get a "undefined symbol: _Py_ZeroStruct" error after executing the following command:

bazel run :python_module_test --define headless=glx

The full command line output is:

INFO: Found 1 target...
Target //:python_module_test up-to-date:
  bazel-bin/python_module_test
INFO: Elapsed time: 0.230s, Critical Path: 0.01s

INFO: Running command line: bazel-bin/python_module_test
Traceback (most recent call last):
  File "/home/rico/.cache/bazel/_bazel_rico/4b96fd176693dd396977e881a84a79f8/execroot/lab/bazel-out/local-fastbuild/bin/python_module_test.runfiles/org_deepmind_lab/python/dmlab_module_test.py", line 26, in <module>
    import deepmind_lab
ImportError: /home/rico/.cache/bazel/_bazel_rico/4b96fd176693dd396977e881a84a79f8/execroot/lab/bazel-out/local-fastbuild/bin/python_module_test.runfiles/org_deepmind_lab/deepmind_lab.so: undefined symbol: _Py_ZeroStruct
ERROR: Non-zero return code '1' from command: Process exited with status 1.

Imitation learning recording using python

Hi

Thanks for this great environment. I would like to ask how I can play as a human like the example provided:lab$ bazel run :game -- --level_script tests/demo_map and record the frames as well as the actions in python script? I did not find any human mode api in python.

Thanks!

Python3 Compatibility

With reference to this issue: #13
I was wondering if/when Python3 compatibility might be likely?
Having the same issue when trying to import deepmind_lab module in python3 script.

Calling C API from C++ app: no video frame

Hi,
I am running the game_main.c from a separate C++ project, without bazel, using the libdmlab.so and the files outputed by bazel in dirpath ./deepmind_lab.runfiles.

It works, meaning it starts the episode, and loops getting reward=0 most of the time, but the video frame is not opening, it's like in headless mode.
I tried setting this:
if (env_c_api.setting(context, "headless", "false") != 0) {
sys_error("Failed to apply 'headless=false' setting.");
exit(1);
}
but it's not calling exit()...so setting this param does not break the app, but doesn't have any effect either.

What setting should I put to open the video frame ?

Thanks a lot,
Viorel

Error occurs when compiling on Ubuntu 14.04

I just compiled deepmind lab according to official guidance, but error occured after I ran bazel build :deepmind_lab.so --define headless=glx --verbose_failures, at first I thought it could be a problem about privilege, but it didn't work even if I switched to root account. I used ubuntu 14.04๏ผŒcan anyone help? log below:

INFO: Found 1 target...
ERROR: /root/.cache/bazel/_bazel_root/849b6bbeac370239e55f7f9aec9e5d3e/external/libxml_archive/BUILD:1:1: Executing genrule @libxml_archive//:gen_configure failed: linux-sandbox failed: error executing command
(cd /root/.cache/bazel/_bazel_root/849b6bbeac370239e55f7f9aec9e5d3e/bazel-sandbox/a84ae3c0-6bfc-42b5-998b-9e7970b5530f-1/execroot/lab &&
exec env -
PATH=/home/everdom/.nvm/versions/node/v6.4.0/bin:/home/everdom/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
TMPDIR=/tmp/user/0
/root/.cache/bazel/_bazel_root/849b6bbeac370239e55f7f9aec9e5d3e/execroot/lab/_bin/linux-sandbox @/root/.cache/bazel/_bazel_root/849b6bbeac370239e55f7f9aec9e5d3e/bazel-sandbox/a84ae3c0-6bfc-42b5-998b-9e7970b5530f-1/linux-sandbox.params -- /bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh; ./external/libxml_archive/configure --without-lzma && cp --verbose -- config.h bazel-out/host/genfiles/external/libxml_archive/config.h && cp --verbose -- include/libxml/xmlversion.h bazel-out/host/genfiles/external/libxml_archive/include/libxml/xmlversion.h').
checking whether to enable maintainer-specific portions of Makefiles... yes
checking build system type... mkdir: cannot create directory '/tmp/user/0/cg144-10876': No such file or directory
mkdir: cannot create directory '/tmp/user/0/cg-144': No such file or directory
config.guess: cannot create a temporary directory in /tmp/user/0
configure: error: cannot guess build type; you must specify one
WARNING: Cannot delete sandbox directory after action execution: /root/.cache/bazel/_bazel_root/849b6bbeac370239e55f7f9aec9e5d3e/bazel-sandbox/a84ae3c0-6bfc-42b5-998b-9e7970b5530f-3 (java.io.IOException: /root/.cache/bazel/_bazel_root/849b6bbeac370239e55f7f9aec9e5d3e/bazel-sandbox/a84ae3c0-6bfc-42b5-998b-9e7970b5530f-3/execroot/lab (Device or resource busy)).
Target //:deepmind_lab.so failed to build
INFO: Elapsed time: 0.734s, Critical Path: 0.42s

An error occurs when running a test

the platform is ubuntu 16.04, bazel 0.32.

$bazel run :python_module_test --define headless=glx

INFO: Found 1 target...
Target //:python_module_test up-to-date:
bazel-bin/python_module_test
INFO: Elapsed time: 0.148s, Critical Path: 0.00s

INFO: Running command line: bazel-bin/python_module_test
...Failed to connect to Mir: Failed to connect to server socket: No such file or directory
Unable to init server: Could not connect: Connection refused

(zenity:28528): Gtk-WARNING **: cannot open display:
ERROR: Non-zero return code '3' from command: Process exited with status 3.

Installation failure on Ubuntu 16.04, Python 3: Non-zero return code '1' from command

I am using Ubuntu 16.04 and Python 3 with Anaconda.

I am following the instruction:

$ sudo apt-get install lua5.1 liblua5.1-0-dev libffi-dev gettext \
   freeglut3-dev libsdl2-dev libosmesa6-dev python-dev python-numpy realpath

Then I run

bazel build :deepmind_lab.so --define headless=glx

which returns

INFO: Found 1 target...
Target //:deepmind_lab.so up-to-date:
  bazel-bin/deepmind_lab.so
INFO: Elapsed time: 4.238s, Critical Path: 3.06s

Then

bazel run :python_module_test --define headless=glx

but it returns errors

INFO: Found 1 target...
Target //:python_module_test up-to-date:
  bazel-bin/python_module_test
INFO: Elapsed time: 0.201s, Critical Path: 0.01s

INFO: Running command line: bazel-bin/python_module_test
Traceback (most recent call last):
  File "/home/....../.cache/bazel/_bazel_......./7acb3f4cd7f8cdaee804168c0bd39fcc/execroot/lab/bazel-out/local-fastbuild/bin/python_module_test.runfiles/org_deepmind_lab/python/dmlab_module_test.py", line 26, in <module>
    import deepmind_lab
ImportError: /home/........./.cache/bazel/_bazel_......../7acb3f4cd7f8cdaee804168c0bd39fcc/execroot/lab/bazel-out/local-fastbuild/bin/python_module_test.runfiles/org_deepmind_lab/deepmind_lab.so: undefined symbol: _Py_ZeroStruct
ERROR: Non-zero return code '1' from command: Process exited with status 1.

RGBD frame contains strange objects in the depth channel.

Hi,
I am using the C API, and I got a frame + depth by getting the observation RGBD_INTERLACED.
But the depth is strange, because it contains 2 squares at the bottom with depth information, which I think are not useful at all, and also I don't think are part of the environment. I think only the walls and the apple should be the environment, so only they should have depth information....so I think this is a bug, but if it's not, please explain me the existence and meaning of these 2 objects in the depth information.

I attached the RGB frame, the depth frame, and the inverted depth frame.

Thanks a lot,
Viorel

frame

depth

depth_inverted

depth_inverted2

Failed with running "bazel build :deepmind_lab.so --define headless=glx"

model name : Intel(R) Core(TM) i5-5200U CPU @ 2.20GHz
ubuntu 14.04
gcc 4.8.4

========================================================================
ERROR: /home/cat/lab/BUILD:730:1: C++ compilation of rule '//:game_lib_headless_glx' failed: Process exited with status 1 [sandboxed].
engine/code/qcommon/vm_x86.c: In function 'VM_CallCompiled':
engine/code/qcommon/vm_x86.c:1803:2: error: inconsistent operand constraints in an 'asm'
asm volatile(
^
Use --strategy=CppCompile=standalone to disable sandboxing for the failing actions.
Target //:deepmind_lab.so failed to build
Use --verbose_failures to see the command lines of failed build steps.

Is it possible to render image in vnc environment?

by running ~/lab$ bazel run :random_agent --define headless=false, a graphic interface is obtained when I use physical screen, mouse and keyboard.

Is there any way to run it under vnc viewer?
I've tried but got a 'GlxBadRenderRequest'.

Building failed on Ubuntu14.04

Hey,
I'm trying to build lab on my machine by bazel build :deepmind_lab.so --define headless=glx --verbose_failures but failed so far. I came across these errors so far:

  1. "mount" permission denied
    fixed by: adding build --spawn_strategy=standalone to /etc/bazel.bazelrc
  2. python/dmlab_module.c:29:31: fatal error: numpy/arrayobject.h: No such file or directory
    fixed by: adding the following to copt in BUILD: "-I/usr/local/lib/python2.7/dist-packages/numpy/core/include",
  3. configure: error: cannot run C compiled programs.
    have no clue how to fix... below I paste the complete error message below:
INFO: Found 1 target...
ERROR: /home/zhang/.cache/bazel/_bazel_zhang/2f9f084266361be2ea70187ad5c14f49/external/glib_archive/BUILD:1:1: Executing genrule @glib_archive//:gen_configure failed: bash failed: error executing command /bin/bash -c ... (remaining 1 argument(s) skipped): com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether UID '12104' is supported by ustar format... yes
checking whether GID '1371' is supported by ustar format... yes
checking how to create a ustar tar archive... gnutar
checking whether to enable maintainer-specific portions of Makefiles... yes
checking whether make supports nested variables... (cached) yes
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... configure: error: in `/home/zhang/.cache/bazel/_bazel_zhang/2f9f084266361be2ea70187ad5c14f49/execroot/lab':
configure: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
See `config.log' for more details
Target //:deepmind_lab.so failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 3.759s, Critical Path: 3.14s

I don't have any experience with bazel so I've got really no clue as how to fix it, any help would be appreciated!
My compiler: gcc (Ubuntu 4.8.5-2ubuntu1~14.04.1) 4.8.5

Thanks in advance!

A problem when building it.

When I used "bazel build :deepmind_lab.so --define headless=glx" to build the project, it didn't work.

Here's the errors.
ERROR: /home/newman/lab/q3map2/BUILD:54:1: no such package '@libxml_archive//': Error downloading [http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz] to /home/newman/.cache/bazel/_bazel_newman/80b555ff9f115a5edca2f3451f16a6da/external/libxml_archive/libxml2-2.9.4.tar.gz: Tried to reconnect at offset 332,609 but server didn't support it and referenced by '//q3map2:q3map2'.
ERROR: Analysis of target '//:deepmind_lab.so' failed; build aborted.
INFO: Elapsed time: 136.099s

I'm a novice in Linux. I tried to download and install "libxml2-2.9.4.tar.gz" by myself, but it still didn't work.
It said "Error downloading". What does it mean? Is that some problems with my network? But I could download the package by visiting the website [http://xmlsoft.org/sources/libxml2-2.9.4.tar.gz].

I'm really a novice. Please help me. Thanks a lot.

using deepmind lab in the python repl

I try to understand the effect of each python API, so I launch the python repl and import the deepmind_lab.so. But after I could not launch the game because
lab = deepmind_lab.Lab('seekavoid_arena_01', ['RGB_INTERLACED'])
would give me error saying invalid level flag. Basically the library cannot locate the assets. But if I make a script and trying to add that script in Bazel, everything works fine.

I am curious if I could launch the game in a more interactive way?

Receiving Error after about 2248 frames in 60fps - lab.observations()

Hello and thanks for your package.

I am trying to use your package with a DQN similar agent but I receive the following error after about 2248 times that I call the observations() in order to receive the state after 2248 steps. I also receive a similar error in the following simple changed demo code:

Error:

2246
2247
2248
obs = lab.observations() # dict of Numpy arrays
RuntimeError: Environment in wrong status for call to observations()
ERROR: Non-zero return code '1' from command: Process exited with status 1.

Main code that produced the error:

import deepmind_lab
import numpy as np
import cv2
import time

lab = deepmind_lab.Lab('nav_maze_random_goal_02', ['RGBD_INTERLACED'],
config={
'fps': str(60),
'width': str(320),
'height': str(240)
}
)
lab.reset()

noobaction = np.zeros([7], dtype=np.intc)
forward_action = np.array([0, 0, 0, 1, 0, 0, 0], dtype=np.intc)
backward_action = - forward_action
look_left = np.array([20, 0, 0, 0, 0, 0, 0], dtype=np.intc) #this actually rotates right
look_right = - look_left
strafe_left = np.array([0, 0, 1, 0, 0, 0, 0], dtype=np.intc) #this actually slides righgt side
strafe_right = - strafe_left

for action_counter in range(0,10000):

if(action_counter==0):
    reward = lab.step(noobaction, num_steps=4)
elif(action_counter>1 and action_counter<90):
    reward = lab.step(look_left, num_steps=4)
elif(action_counter>95 and action_counter<100):
    reward = lab.step(forward_action, num_steps=4)
else:
    reward = lab.step(noobaction, num_steps=4)

#if not lab.is_running():
#  print('Environment stopped early')
#  lab.reset()
obs = lab.observations()  # dict of Numpy arrays

rgb_i = obs['RGBD_INTERLACED']
assert rgb_i.shape == (240, 320, 4)
#print(rgb_i.shape)
rgb_image=np.array(rgb_i[0:240,0:320,0:3])
depth_image=np.array(rgb_i[0:240,0:320,3])
depth_image=1-depth_image
#print(npx)
cv2.imshow("RGB", rgb_image)
cv2.imshow("Depth", depth_image)
print(action_counter)
cv2.waitKey(1)

Please notice that I have commented the reset() function, I am able to reset the environment similar to your 'random agent' code but I guess I need to continue the episode or continue it for enf.

Please tell me what is the problem and what did I forget to make the code works.

Regards,

Compiling Errors regarding Lua and Python versions

Hi,

It is a really interesting project. Thanks, DeepMind.

I just had some problems regarding the Lua and Python versions as well as linkage to zlib when compiling the code on Archlinux. The system default versions of Lua and Python are respectively 5.3 and 3.5, which are not support by current build files. My solution is to create a virtualenv of python27 and use the attached patch. Hopefully it will help someone. Thanks.
archlinux-2016-12-07.patch.txt

C++ compilation issue when running the demo_map

I am currently using Ubuntu 14.04.5 LTS and when I run the following command:
bazel run :game -- --level_script tests/demo_map

This is my gcc version:
gcc version 4.7.3 (Ubuntu/Linaro 4.7.3-12ubuntu1)

I got the following error:

ERROR: /home/guang/DLearning/lab/BUILD:550:1: C++ compilation of rule '//:game_lib_sdl' failed: linux-sandbox failed: error executing command /home/guang/.cache/bazel/_bazel_guang/cfaaf4f84b36caedc3226c783e7b54f4/execroot/lab/_bin/linux-sandbox ... (remaining 58 argument(s) skipped).
In file included from engine/code/renderergl1/../renderercommon/tr_common.h:27:0,
                 from engine/code/renderergl1/tr_local.h:31,
                 from engine/code/renderergl1/tr_model.c:24:
engine/code/renderergl1/../renderercommon/qgl.h:32:25: fatal error: SDL_opengl.h: No such file or directory
compilation terminated.
Target //:game failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.998s, Critical Path: 0.52s
ERROR: Build failed. Not running target.```


A visualization tool

The blog post shows an interesting visualization tool with two columns "Live play" and "Auxiliary Tasks". Are you planning to release a tool like this for the live play?

Disable map creation log printing

Is there any way to disable all the printing that occurs when creating a map? I can see that the logs are being saved (to e.g. bspc.log - any way to parametrise this too by any chance?), so there is no need to show the output from bsp2aas and similar tools.

Plus this gets quite troublesome when running multiple environments in the same binary. ๐Ÿ˜ƒ

Cannot run the example on Archlinux

I'm getting the following error:

'glib/glibconfig.h' -> 'bazel-out/local-fastbuild/genfiles/external/glib_archive/glibconfig.h'
ERROR: /home/myuser/Work/etc/lab/BUILD:50:1: Linking of rule '//:lburg' failed: process-wrapper failed: error executing command /home/myuser/.cache/bazel/_bazel_myuser/47a8e009274b17522763229b03a2ef3d/execroot/lab/_bin/process-wrapper -1 5 - - /usr/lib/hardening-wrapper/bin/gcc -o bazel-out/host/bin/lburg -Wl,-no-as-needed ... (remaining 8argument(s) skipped).
Target //:random_agent failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 52.855s, Critical Path: 50.58s
ERROR: Build failed. Not running target.

Also adding --verbose_failures did not make it verbose.

Text Map keeps getting re-compiled

Is there a way to avoid the maps getting constantly recompiled? I'd like to run some experiments on a simple, static custom environment but "compile_maps.sh" seems to run every time an episode ends?
Thanks!

Error downloading zlib

~/lab$ bazel run :random_agent --define headless=false -- \ --length=10000 --width=640 --height=480

ERROR: /home/rainer/lab/BUILD:432:1: no such package '@zlib_archive//': Error downloading [http://zlib.net/zlib-1.2.8.tar.gz] to /home/rainer/.cache/bazel/_bazel_rainer/8462f6817467f709671080a493dfc0cd/external/zlib_archive/zlib-1.2.8.tar.gz: GET returned 404 Not Found and referenced by '//:bspc'.

EDIT:

Zlib has updated their latest to 1.2.10 and just removed the 1.2.8 url. I can submit a PR for this I assume the fix for this would be to replace the old url with this:
http://zlib.net/zlib-1.2.10.tar.gz as well as replace the SHA

edit2: Looks like that fixed it. Two other issues:

INFO: Failed to connect to http://github.com/glennrp/libpng/archive/v1.2.53.zip
INFO: Failed to connect to http://bitbucket.org/eigen/eigen/get/1d454915237a.tar.gz

get extra information

What is the way to get extra information from the game like the players position for loop closure prediction as done in the paper ?

Build Failed Ubuntu 14.04

I am running ubuntu 14.04.
I have installed realpath, and checked that I have all dependencies that are listed.
I also ran bazel clean then rerun with the same results.

Thanks

nburn42@Octahedrons:lab$ bazel run :random_agent --define headless=false -- --length=10000 --width=640 --height=480 --verbose_failures
Warning: ignoring LD_PRELOAD in environment.
INFO: Found 1 target...
ERROR: /home/nburn42/Projects/deeprobotics/lab/BUILD:432:1: Linking of rule '//:bspc' failed: namespace-sandbox failed: error executing command /home/nburn42/.cache/bazel/_bazel_nburn42/8ac04bba4272ff63a5f54d73f1af83e1/lab/_bin/namespace-sandbox ... (remaining 20 argument(s) skipped).
bazel-out/local-fastbuild/bin/_objs/bspc/engine/code/botlib/be_aas_move.pic.o: In function VectorLength': be_aas_move.c:(.text+0xdf): undefined reference to sqrt'
bazel-out/local-fastbuild/bin/_objs/bspc/engine/code/botlib/be_aas_move.pic.o: In function AAS_ApplyFriction': be_aas_move.c:(.text+0x167d): undefined reference to sqrt'
bazel-out/local-fastbuild/bin/_objs/bspc/engine/code/botlib/be_aas_move.pic.o: In function AAS_HorizontalVelocityForJump': be_aas_move.c:(.text+0x467e): undefined reference to sqrt'
be_aas_move.c:(.text+0x4740): undefined reference to sqrt' bazel-out/local-fastbuild/bin/_objs/bspc/engine/code/botlib/be_aas_reach.pic.o: In function VectorLength':
be_aas_reach.c:(.text+0x67): undefined reference to sqrt' bazel-out/local-fastbuild/bin/_objs/bspc/engine/code/botlib/be_aas_reach.pic.o:be_aas_reach.c:(.text+0xd9c): more undefined references to sqrt' follow
bazel-out/local-fastbuild/bin/_objs/bspc/engine/code/botlib/be_aas_reach.pic.o: In function AAS_Reachability_Grapple': be_aas_reach.c:(.text+0xf17b): undefined reference to tan'
bazel-out/local-fastbuild/bin/_objs/bspc/engine/code/botlib/be_aas_sample.pic.o: In function VectorLength': be_aas_sample.c:(.text+0x67): undefined reference to sqrt'
bazel-out/local-fastbuild/bin/_objs/bspc/engine/code/bspc/l_math.pic.o: In function AngleVectors': l_math.c:(.text+0x4c): undefined reference to sin'
l_math.c:(.text+0x69): undefined reference to cos' l_math.c:(.text+0xad): undefined reference to sin'
l_math.c:(.text+0xca): undefined reference to cos' l_math.c:(.text+0x112): undefined reference to sin'
l_math.c:(.text+0x12f): undefined reference to cos' bazel-out/local-fastbuild/bin/_objs/bspc/engine/code/bspc/l_math.pic.o: In function VectorLength':
l_math.c:(.text+0x902): undefined reference to sqrt' bazel-out/local-fastbuild/bin/_objs/bspc/engine/code/bspc/l_math.pic.o: In function Q_rint':
l_math.c:(.text+0x9c0): undefined reference to floor' bazel-out/local-fastbuild/bin/_objs/bspc/engine/code/bspc/l_math.pic.o: In function VectorNormalize':
l_math.c:(.text+0xdf5): undefined reference to sqrt' bazel-out/local-fastbuild/bin/_objs/bspc/engine/code/bspc/l_math.pic.o: In function VectorNormalize2':
l_math.c:(.text+0xf1d): undefined reference to sqrt' bazel-out/local-fastbuild/bin/_objs/bspc/engine/code/bspc/map.pic.o: In function WriteMapBrush':
map.c:(.text+0x2a3b): undefined reference to atan2' map.c:(.text+0x2b2f): undefined reference to atan2'
bazel-out/local-fastbuild/bin/_objs/bspc/engine/code/bspc/textures.pic.o: In function TexinfoForBrushTexture': textures.c:(.text+0x793): undefined reference to sin'
textures.c:(.text+0x7b3): undefined reference to cos' bazel-out/local-fastbuild/bin/_objs/bspc/engine/code/qcommon/cm_patch.pic.o: In function VectorLength':
cm_patch.c:(.text+0x67): undefined reference to `sqrt'
collect2: error: ld returned 1 exit status
Target //:random_agent failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 1.866s, Critical Path: 1.26s
ERROR: Build failed. Not running target.

How to use lab as a python module

Hi,

I built lab on Ubuntu 14.04. The python module tests all pass. I'm a little confused about importing deepmind_lab and running experiments with it in python. I apologize if this is due to my lack of general python/bazel knowledge. My understanding is that the process of experimentation with deepmind_lab is:

  1. create a python file experiment.py. import deepmind_lab and use it in the experiment.
  2. add a py_binary entry in the BUILD file for bazel named "experiment"
  3. perform bazel run :experiment

Is this correct? And, is there any way to instead run experiments directly with python experiment.py?

Build a Docker image

I saw that OSX isn't supported, so I'm working on creating a docker file that should make it easier to experiment with it.

Error:/compile_map.sh

Extracting Bazel installation...
..........................................
INFO: Found 1 target...
ERROR: /media/Data/UbuntuFiles/Document/lab/deepmind/level_generation/BUILD:6:1: failed to create symbolic link 'deepmind/level_generation/compile_map_sh': file 'deepmind/level_generation/compile_map.sh'

Unable to build: Python include path missing

Hi

I'm on Ubuntu 16.04 and following the guidelines from https://github.com/deepmind/lab/blob/master/docs/build.md . I'm unable to run any of the examples

Here are the major library versions:

  • Bazel version: 0.4.3
  • Lua: 5.1.5
  • Python: 2.7
  • OpenGL version: 4.5.0
  • GCC: 5.4.0

Error while trying to run a random agent

lab$ bazel run :game -- --level_script tests/demo_map --verbose_failures
WARNING: Output base '/home/ml/hsatij/.cache/bazel/_bazel_hsatij/2a466b12af1ad7b6dcfd974c8d06585b' is on NFS. This may lead to surprising failures and undetermined behavior.
INFO: Found 1 target...
ERROR: /home/ml/hsatij/.cache/bazel/_bazel_hsatij/2a466b12af1ad7b6dcfd974c8d06585b/external/jpeg_archive/BUILD:74:1: Executing genrule @jpeg_archive//:configure failed: linux-sandbox failed: error executing command /home/ml/hsatij/.cache/bazel/_bazel_hsatij/2a466b12af1ad7b6dcfd974c8d06585b/execroot/lab/_bin/linux-sandbox ... (remaining 5 argument(s) skipped).
src/main/tools/linux-sandbox-pid1.cc:398: "remount(NULL, /home/ml/hsatij/.cache/bazel/_bazel_hsatij/2a466b12af1ad7b6dcfd974c8d06585b/bazel-sandbox/905e0eef-0788-42e6-8852-7b444149d38c-13/tmp/home/ndg/projects, NULL, 2101281, NULL)": No such file or directory
Target //:game failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 2.569s, Critical Path: 1.20s
ERROR: Build failed. Not running target.

On trying to build the Python interface to DeepMind Lab with OpenGL

lab$ bazel build :deepmind_lab.so --define headless=glx --verbose_failures
WARNING: Output base '/home/ml/hsatij/.cache/bazel/_bazel_hsatij/2a466b12af1ad7b6dcfd974c8d06585b' is on NFS. This may lead to surprising failures and undetermined behavior.
INFO: Found 1 target...
ERROR: /home/ml/hsatij/code/libs/lab/BUILD:972:1: C++ compilation of rule '//:dmlablib' failed: linux-sandbox failed: error executing command 
  (cd /home/ml/hsatij/.cache/bazel/_bazel_hsatij/2a466b12af1ad7b6dcfd974c8d06585b/bazel-sandbox/ff30fcd2-9759-4ca4-8fa3-b82956431988-1/execroot/lab && \
  exec env - \
  /home/ml/hsatij/.cache/bazel/_bazel_hsatij/2a466b12af1ad7b6dcfd974c8d06585b/execroot/lab/_bin/linux-sandbox @/home/ml/hsatij/.cache/bazel/_bazel_hsatij/2a466b12af1ad7b6dcfd974c8d06585b/bazel-sandbox/ff30fcd2-9759-4ca4-8fa3-b82956431988-1/linux-sandbox.params -- /usr/bin/gcc -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -Wall -Wl,-z,-relro,-z,now -B/usr/bin -B/usr/bin -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-canonical-system-headers -fno-omit-frame-pointer '-std=c++0x' -MD -MF bazel-out/local-fastbuild/bin/_objs/dmlablib/public/dmlab_so_loader.pic.d '-frandom-seed=bazel-out/local-fastbuild/bin/_objs/dmlablib/public/dmlab_so_loader.pic.o' -fPIC -iquote . -iquote bazel-out/local-fastbuild/genfiles -iquote external/bazel_tools -iquote bazel-out/local-fastbuild/genfiles/external/bazel_tools -isystem external/bazel_tools/tools/cpp/gcc3 '-DDMLAB_SO_LOCATION="libdmlab.so"' -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c public/dmlab_so_loader.cc -o bazel-out/local-fastbuild/bin/_objs/dmlablib/public/dmlab_so_loader.pic.o).
src/main/tools/linux-sandbox-pid1.cc:398: "remount(NULL, /home/ml/hsatij/.cache/bazel/_bazel_hsatij/2a466b12af1ad7b6dcfd974c8d06585b/bazel-sandbox/ff30fcd2-9759-4ca4-8fa3-b82956431988-1/tmp/home/ndg/projects, NULL, 2101281, NULL)": No such file or directory
Target //:deepmind_lab.so failed to build
INFO: Elapsed time: 2.294s, Critical Path: 1.11s

I'm new to bazel and unfortunately the error logs are too cryptic for me. Any help to resolve this will be appreciated !

glib_probes.h not found while building on Ubuntu 14.04

I installed all the dependencies as told, cleaned and re-built a few times, and I still got this message:

wangheda@wangheda-HeyStack:~/Programs/DeepMindLab/lab$ bazel build :deepmind_lab.so --define headless=glx
INFO: Found 1 target...
ERROR: /home/wangheda/.cache/bazel/_bazel_wangheda/9a2a0f97587d2d4f1381984736a5dc83/external/glib_archive/BUILD:35:1: C++ compilation of rule '@glib_archive//:glib' failed: linux-sandbox failed: error executing command /home/wangheda/.cache/bazel/_bazel_wangheda/9a2a0f97587d2d4f1381984736a5dc83/execroot/lab/_bin/linux-sandbox ... (remaining 52 argument(s) skipped).
In file included from external/glib_archive/glib/gslice.c:54:0:
external/glib_archive/glib/glib_trace.h:33:25: fatal error: glib_probes.h: No such file or directory
 #include "glib_probes.h"
                         ^
compilation terminated.
Target //:deepmind_lab.so failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.875s, Critical Path: 0.68s

Also I found that in this directory:
/home/wangheda/.cache/bazel/_bazel_wangheda/9a2a0f97587d2d4f1381984736a5dc83/external/glib_archive/glib
there is no glib_probes.h, only glib_probes.d which seems like a header file.

Any idea how to fix this?

Installation failure on Ubuntu 16.04 / Python2.7

When installing lab on Ubuntu 16.04, I have an error with Python2.7.
When running `bazel run :python_module_test --define headless=glx'

ImportError: > /home/.../deepmind_lab.so: > undefined symbol: PyCObject_Type
ERROR: Non-zero return code '1' from command: Process exited with status 1.

my python --version :

Python 2.7.12

my bazel version:

Build label: 0.4.4
Build target: bazel-out/local-fastbuild/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Wed Feb 1 18:54:21 2017 (1485975261)
Build timestamp: 1485975261
Build timestamp as int: 1485975261

Am i missing something here to build the lab?

Run random_agent in specific environment

ping humans... Hi!

All issues below have sense wrt specific IDE (PyCharm):
(everything works fine just via terminal, since that issue can be closed)

What should I do if I want to run (for example) your random_agent.py in a specific environment ?

Default script (random_agent) just tries to grub usr/bin/python or something similar.

I can set environment variables with all sugar which I want (...it's not so short string), but it isn't convenient and looks awkward.

Any other solution...?

Lab + python multiprocessing

I have implemented A3C with multiprocessing (+ pytorch) as opposed to using threads, however bazel run seems to break silently and clearly without any visible trace. This is what I do:

$ bazel run :struct_runner --define headless=false
[...]
$ echo $?  # this is the error code of the previous process
8

struct_runner.py initialises a lab environment, then creates a bunch of processes in which more envs are created. In particular, the silent crash happens when I create a process p and do p.start() - it also appears to be non-deterministic with respect to the number of processes I manage to spawn before bazel kills them and quits.

I know that @miyosuda has implemented A3C using threads here, however multiprocessing is supported very well by pytorch and it would be a shame to have to deal with threads management.

Mouse sensitivity is too high when in game mode

When running this within VNC in #24 with:

bazel run :random_agent --define headless=false -- \ --length=10000 --width=640 --height=480

Things work fine, but when I use game mode the mouse seems to be much too sensitive and ends up flipping back and forth between gimbal lock, looking straight up or down. Is anyone else experiencing that when using the following?

bazel run :game -- --level_script tests/demo_map

Normal:
screen shot 2017-02-09 at 4 59 25 am

When using mouse, it's either this or looking straight up:
screen shot 2017-02-09 at 5 00 10 am

[Please read] Requirements, supported platforms, FAQs. [update on macOS]

This issue is a placeholder until we update the documentation to make these points easier to find.

Bazel: Please use a recent version of Bazel. The continuous-integration checks use whatever up-to-date version is supplied by the system, and we try to keep the code passing those checks. A convenient way to stay up to date is to use your system vendor's package management system (see https://docs.bazel.build/versions/master/install.html), or the https://github.com/bazelbuild/bazelisk wrapper that automatically downloads the latest version.

Linux only: DeepMind Lab is currently made only for Linux. It will not build on MacOSX. Making Lab work on MacOSX should not be fundamentally impossible (some might say it should be straight-forward), but it's extra work that we have not done yet and are not currently planning to do. Support for MacOSX isn't officially provided, but the macos branch contains attempts at getting certain parts of the system to work.

Python 2.7: The Python bindings that we ship are made for Python 2.7 with experimental support for Python 3only. They don't work with Python 3 out of the box, and we have not investigated how much work it would be to make them work. Please also note that the Python bindings are more of an "example application" that a definitive model of an API; the canonical API is the C API in public/dm_lab.h and third_party/rl_api/env_c_api.h. If the Python bindings don't quite work for your use case, please do consider using the C API.

Python libraries: We also require NumPy and PIL. For a few tests, NumPy needs to be at least 1.8, but the main Python bindings work on older versions, too.

OpenGL: Except for the headless=osmesa build mode, Lab requires hardware OpenGL. This usually requires that you run the binaries directly on a machine with an OpenGL-enabled X server. It will probably not work through SSH connections onto remote machines.

Compiler: For GCC you need at least version 4.8. Recent versions of Clang should work, too. We run successful tests for Linux-x86_64 with GCC+libstdc++, Clang+libstd++, and Clang+libc++.

Other platforms: In principle MacOS on x86_64 should work, too (see the macos branch), but isn't well tested. MacOS and Linux on aarch64 (ARM64) are not currently supported, but should in principle be achievable (all the source code is there to support that CPU).

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.