minosworld / unreal Goto Github PK

Hi, thanks for your great work
I have one question about the weight updating protocol.
The gradients of the local network (including all auxiliary tasks) are applied in the process function in trainer.py. However, I notice that the sync function (which copies the weights of global network to the local one) is ran BEFORE the apply_gradient is ran (line 354 and 409 respectively).
Following the code behind these 2 functions, you are copying the shared weights to the local network (so basically the global variables and local variables will be the same by then), then calculating gradients on local variables, then applying the gradients to the global variables. It does not sound logic, does it?

Please correct me if I am wrong.

I trained unreal agent on the medium room goal with 13.2M steps. It costs 1day on a P5000 card. Command like this:
python3 main.py --env_type indoor --env_name roomgoal_suncg_mf, all the hyper parameters are set as default.

However, when I use display.py to display it. The performance are not satisfying. The agent tends to be turning left all the time without going forward. I am wondering how to train a baseline like this demo on 2:24, which policy is very close to optimum and action sequence output are stable.

Could you please release your baseline model or tell me how to reproduce it ?

python3 main.py --env_type indoor --env_name objectgoal_suncg_mf gave me following errors. Seems because objective_size is not defined for the objectGoal task. It works well for both pointGoal and roomGoal.

2018-01-31 16:07:15.317469: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2018-01-31 16:07:18.072400: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Found device 0 with properties:
name: GeForce GTX TITAN X major: 5 minor: 2 memoryClockRate(GHz): 1.076
pciBusID: 0000:05:00.0
totalMemory: 11.92GiB freeMemory: 11.80GiB
2018-01-31 16:07:18.072446: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX TITAN X, pci bus id: 0000:05:00.0, compute capability: 5.2)
Could not find old checkpoint
Press Ctrl+C to stop
train:87 episodes
val:32 episodes
test:34 episodes
train:87 episodes
val:32 episodes
test:34 episodes
train:87 episodes
val:32 episodes
test:34 episodes
train:87 episodes
val:32 episodes
test:34 episodes
2018-01-31 16:07:35,584 INFO {'stk_git_hash': 'e969207', 'sim_id': 'sim03', 'sim_git_hash': 'e969207', 'machine': 'vonmises.cs.umass.edu'}
2018-01-31 16:07:35,646 INFO {'stk_git_hash': 'e969207', 'sim_id': 'sim00', 'sim_git_hash': 'e969207', 'machine': 'vonmises.cs.umass.edu'}
2018-01-31 16:07:35,690 INFO {'stk_git_hash': 'e969207', 'sim_id': 'sim02', 'sim_git_hash': 'e969207', 'machine': 'vonmises.cs.umass.edu'}
2018-01-31 16:07:35,699 INFO sim03:Starting sim server at /data/hzjiang/Code/minos/server/server.js with port 45005
2018-01-31 16:07:35,699 INFO sim00:Starting sim server at /data/hzjiang/Code/minos/server/server.js with port 12407
2018-01-31 16:07:35,725 INFO sim02:Starting sim server at /data/hzjiang/Code/minos/server/server.js with port 11715
2018-01-31 16:07:35,996 INFO {'stk_git_hash': 'e969207', 'sim_id': 'sim01', 'sim_git_hash': 'e969207', 'machine': 'vonmises.cs.umass.edu'}
2018-01-31 16:07:36,027 INFO sim01:Starting sim server at /data/hzjiang/Code/minos/server/server.js with port 42946
2018-01-31 16:07:36,819 INFO Starting new HTTP connection (1): localhost
2018-01-31 16:07:36,829 INFO Starting new HTTP connection (1): localhost
2018-01-31 16:07:36,832 WARNING localhost:45005/socket.io [waiting for connection] HTTPConnectionPool(host='localhost', port=45005): Max retries exceeded with url: /socket.io/?EIO=3&t=1517432856808-0&transport=polling (Caused by <class 'ConnectionRefusedError'>: [Errno 111] Connection refused)
2018-01-31 16:07:36,854 INFO Starting new HTTP connection (1): localhost
2018-01-31 16:07:36,861 WARNING localhost:11715/socket.io [waiting for connection] HTTPConnectionPool(host='localhost', port=11715): Max retries exceeded with url: /socket.io/?EIO=3&t=1517432856853-0&transport=polling (Caused by <class 'ConnectionRefusedError'>: [Errno 111] Connection refused)
2018-01-31 16:07:37,093 INFO Starting new HTTP connection (1): localhost
2018-01-31 16:07:37,094 WARNING localhost:42946/socket.io [waiting for connection] HTTPConnectionPool(host='localhost', port=42946): Max retries exceeded with url: /socket.io/?EIO=3&t=1517432857092-0&transport=polling (Caused by <class 'ConnectionRefusedError'>: [Errno 111] Connection refused)
2018-01-31 16:07:37,834 INFO Starting new HTTP connection (2): localhost
2018-01-31 16:07:37,863 INFO Starting new HTTP connection (2): localhost
2018-01-31 16:07:38,101 INFO Starting new HTTP connection (2): localhost
2018-01-31 16:07:38,118 INFO sim00:connect
2018-01-31 16:07:38,141 INFO sim03:connect
2018-01-31 16:07:38,166 INFO sim02:connect
2018-01-31 16:07:38,306 INFO sim00:inited
2018-01-31 16:07:38,308 INFO sim02:inited
2018-01-31 16:07:38,337 INFO sim03:inited
2018-01-31 16:07:38,556 INFO sim01:connect
2018-01-31 16:07:38,683 INFO sim01:inited
Exception in thread Thread-2:
Traceback (most recent call last):
File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
self.run()
File "/usr/lib/python3.4/threading.py", line 868, in run
self._target(*self._args, **self._kwargs)
File "main.py", line 65, in train_function
self.score_input)
File "/data/hzjiang/Code/unreal/train/trainer.py", line 346, in process
self._fill_experience(sess)
File "/data/hzjiang/Code/unreal/train/trainer.py", line 123, in _fill_experience
last_action_reward)
File "/data/hzjiang/Code/unreal/model/model.py", line 403, in run_base_policy_and_value
self.base_initial_lstm_state1 : self.base_lstm_state_out[1]} )
File "/home/hzjiang/.local/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 889, in run
run_metadata_ptr)
File "/home/hzjiang/.local/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 1096, in _run
% (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (1, 8) for Tensor 'net_0/Placeholder:0', which has shape '(?, 4)'

Exception in thread Thread-3:
Traceback (most recent call last):
File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
self.run()
File "/usr/lib/python3.4/threading.py", line 868, in run
self._target(*self._args, **self._kwargs)
File "main.py", line 65, in train_function
self.score_input)
File "/data/hzjiang/Code/unreal/train/trainer.py", line 346, in process
self._fill_experience(sess)
File "/data/hzjiang/Code/unreal/train/trainer.py", line 123, in _fill_experience
last_action_reward)
File "/data/hzjiang/Code/unreal/model/model.py", line 403, in run_base_policy_and_value
self.base_initial_lstm_state1 : self.base_lstm_state_out[1]} )
File "/home/hzjiang/.local/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 889, in run
run_metadata_ptr)
File "/home/hzjiang/.local/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 1096, in _run
% (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (1, 8) for Tensor 'net_1/Placeholder:0', which has shape '(?, 4)'

Exception in thread Thread-4:
Traceback (most recent call last):
File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
self.run()
File "/usr/lib/python3.4/threading.py", line 868, in run
self._target(*self._args, **self._kwargs)
File "main.py", line 65, in train_function
self.score_input)
File "/data/hzjiang/Code/unreal/train/trainer.py", line 346, in process
self._fill_experience(sess)
File "/data/hzjiang/Code/unreal/train/trainer.py", line 123, in _fill_experience
last_action_reward)
File "/data/hzjiang/Code/unreal/model/model.py", line 403, in run_base_policy_and_value
self.base_initial_lstm_state1 : self.base_lstm_state_out[1]} )
File "/home/hzjiang/.local/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 889, in run
run_metadata_ptr)
File "/home/hzjiang/.local/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 1096, in _run
% (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (1, 8) for Tensor 'net_2/Placeholder:0', which has shape '(?, 4)'

Exception in thread Thread-5:
Traceback (most recent call last):
File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
self.run()
File "/usr/lib/python3.4/threading.py", line 868, in run
self._target(*self._args, **self._kwargs)
File "main.py", line 65, in train_function
self.score_input)
File "/data/hzjiang/Code/unreal/train/trainer.py", line 346, in process
self._fill_experience(sess)
File "/data/hzjiang/Code/unreal/train/trainer.py", line 123, in _fill_experience
last_action_reward)
File "/data/hzjiang/Code/unreal/model/model.py", line 403, in run_base_policy_and_value
self.base_initial_lstm_state1 : self.base_lstm_state_out[1]} )
File "/home/hzjiang/.local/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 889, in run
run_metadata_ptr)
File "/home/hzjiang/.local/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 1096, in _run
% (np_val.shape, subfeed_t.name, str(subfeed_t.get_shape())))
ValueError: Cannot feed value of shape (1, 8) for Tensor 'net_3/Placeholder:0', which has shape '(?, 4)'

^CYou pressed Ctrl+C!