llan-ml / tesp Goto Github PK

请问一定要使用ray么？如果不使用ray应该如何运行脚本？ray对整个实验加速有多快呢？

When I use tensorflow-gpu==1.11 ,I met a problem that show :
2020-03-10 23:59:04.426033: E tensorflow/stream_executor/cuda/cuda_driver.cc:300] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
But when I add code below：
from tensorflow.python.client import device_lib print (device_lib.list_local_devices())
it shows
2020-03-11 00:07:39.312175: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2020-03-11 00:07:39.628191: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:964] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2020-03-11 00:07:39.628428: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1411] Found device 0 with properties: name: GeForce GTX 1050 Ti major: 6 minor: 1 memoryClockRate(GHz): 1.62 pciBusID: 0000:01:00.0 totalMemory: 3.95GiB freeMemory: 3.89GiB 2020-03-11 00:07:39.628442: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1490] Adding visible gpu devices: 0 2020-03-11 00:07:39.910971: I tensorflow/core/common_runtime/gpu/gpu_device.cc:971] Device interconnect StreamExecutor with strength 1 edge matrix: 2020-03-11 00:07:39.910998: I tensorflow/core/common_runtime/gpu/gpu_device.cc:977] 0 2020-03-11 00:07:39.911004: I tensorflow/core/common_runtime/gpu/gpu_device.cc:990] 0: N 2020-03-11 00:07:39.911112: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1103] Created TensorFlow device (/device:GPU:0 with 3620 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1050 Ti, pci bus id: 0000:01:00.0, compute capability: 6.1) [name: "/device:CPU:0" device_type: "CPU" memory_limit: 268435456 locality { } incarnation: 308222418805500144 , name: "/device:GPU:0" device_type: "GPU" memory_limit: 3796434944 locality { bus_id: 1 links { } } incarnation: 13878747533942563053 physical_device_desc: "device: 0, name: GeForce GTX 1050 Ti, pci bus id: 0000:01:00.0, compute capability: 6.1" ]
I don't know where is the problem and if it can't run on GPU?

when I use the local mode to run the experiment, the error occurs:

File "/home/hua/tesp/tesp/maml/maml.py", line 41, in on_episode_step
"single_agent"].buffers["infos"][-1]["dist_reward"]
KeyError: 'single_agent'

Is there any wrong with the code?

When I run the main_train.py with my pc in the local mode, an error occurred as shown below:

2019-11-25 20:16:56,521	INFO resource_spec.py:205 -- Starting Ray with 5.42 GiB memory available for workers and up to 2.73 GiB for objects. You can adjust these settings with ray.init(memory=<bytes>, object_store_memory=<bytes>).
Traceback (most recent call last):
  File "/home/meta/anaconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 2862, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-230f46c907de>", line 1, in <module>
    runfile('/home/meta/code/asd/tesp2/main_train.py', wdir='/home/meta/code/asd/tesp2')
  File "/opt/pycharm-2019.2.4/helpers/pydev/_pydev_bundle/pydev_umd.py", line 197, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
  File "/opt/pycharm-2019.2.4/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/home/meta/code/asd/tesp2/main_train.py", line 75, in <module>
    register_trainable(agent_cls._agent_name, agent_cls)
  File "/home/meta/code/asd/ray1/tune/registry.py", line 37, in register_trainable
    raise TypeError("Second argument must be convertable to Trainable",trainable)
TypeError: ('Second argument must be convertable to Trainable', <class 'tesp2.tesp3.tesp.TESPAgent'>)

And the relevant code is as follows：

def register_trainable(name, trainable):
    """Register a trainable function or class.
    Args:
        name (str): Name to register.
        trainable (obj): Function or tune.Trainable class. Functions must
            take (config, status_reporter) as arguments and will be
            automatically converted into a class during registration.
    """

    from ray1.tune.trainable import Trainable, wrap_function

    if isinstance(trainable, FunctionType):
        trainable = wrap_function(trainable)
    if not issubclass(trainable, Trainable):
        **raise TypeError("Second argument must be convertable to Trainable",trainable)**
    _global_registry.register(TRAINABLE_CLASS, name, trainable)

It seems type error. For the first time, I tried to recreate the project of reinforcement learning, so I didn't quite understand it, Do you have any idea?

import ray.cloudpickle as pickle
from ray.experimental.internal_kv import _internal_kv_initialized, \ _internal_kv_get, _internal_kv_put
I found there is not the pakage of cloudpickle and experimental , any advice?

When I run python main_train.py --env wheeled --alg tesp in local mode, following error occurs:

ray.tune.error.TuneError: Insufficient cluster resources to launch trial: trial requested 22 CPUs, 0 GPUs but the cluster has only 8 CPUs, 2 GPUs. Pass queue_trials=True in ray.tune.run_experiments() or on the comm and line to queue trials until the cluster scales up

What can be done to reduce the number of CPUs used？I only have one machine and 8 CPUs.

Thank you

很欣赏您这篇文章，但是在复现的时候看这个代码真的好难懂，不知道有没有不用ray的版本，谢谢

在论文中给出的算法伪码，我理解的是，不论是task encoder的更新还是策略的更新都是直接对公式(10)进行SGD，这是一个meta-update，应该可以简单地理解成一个梯度更新。但是在源码中，我看到了ppo和a3c以及它们的loss，请问tesp需要借助ppo或者a3c的策略更新方式么？