reinforcement-learning-kr / lets-do-irl Goto Github PK
View Code? Open in Web Editor NEWInverse RL algorithms (APP, MaxEnt, GAIL, VAIL)
License: MIT License
Inverse RL algorithms (APP, MaxEnt, GAIL, VAIL)
License: MIT License
Hi,
Thanks for you providing so nice code.
But when I try to play with "lets-do-irl/mujoco/vail", It make me very confused that the code in train_model.py
:
vdb_loss = criterion(learner, torch.ones((states.shape[0], 1))) + \ criterion(expert, torch.zeros((demonstrations.shape[0], 1))) + \ beta * bottleneck_loss
I think this should be:
vdb_loss = criterion(learner, torch.zeros((states.shape[0], 1))) + \ criterion(expert, torch.ones((demonstrations.shape[0], 1))) + \ beta * bottleneck_loss
. In other words, the learner should be pushed to zeros, and expert should be pushed to ones, isn't it? Or both is fine?
By the way, the code in the same file:
beta = max(0, beta + args.alpha_beta * bottleneck_loss)
If beta equals beta + args.alpha_beta * bottleneck_loss, at the next time backward, there will report a bug about beta, which is modified by a inplace operation.
Hi sir
I finished training the robot mobile model, need help to understand the testing, i used SAC algorithm
Could we make a deal about that please
hi, how am i supposed to save expert demo in ppo main?
when i run main.py ,i met some problem:
Traceback (most recent call last):
File "main.py", line 186, in
main()
File "main.py", line 159, in main
expert_acc, learner_acc = train_vdb(vdb, memory, vdb_optim, demonstrations, 0, args)
File "/home/parallels/workspace/lets-do-irl-master/mujoco/vail/train_model.py", line 37, in train_vdb
vdb_loss.backward(retain_graph=True)
File "/home/parallels/anaconda3/envs/mujoco/lib/python3.8/site-packages/torch/_tensor.py", line 487, in backward
torch.autograd.backward(
File "/home/parallels/anaconda3/envs/mujoco/lib/python3.8/site-packages/torch/autograd/init.py", line 197, in backward
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [100, 4]], which is output 0 of AsStridedBackward0, is at version 35; expected version 34 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!
@dongminlee94 , Great work on creating a simple IRL solution - Is your work licensed under the MIT license?
Hello,
I am running the VAIL example as explained in the README.
main.py works as expected.
When running test (e.g., as written: python test.py --load_model ckpt_4000_vail.pth.tar
), there is an import error.
Specifically:
lets-do-irl/mujoco/vail/test.py
Line 8 in ad04bf1
there is no file running_state. In the main.py the running state was defined from Zfilter.
state_idx = idx_to_state(env, state)
how can I fix this?
idx_to_state
position_idx = int((state[0] - env_low[0]) / env_distance[0])
TypeError: only size-1 arrays can be converted to Python scalars
The main runs on pytorch 0.4.1 but it fails for pytorch 1.6.0.
Running the main.py for VAIL implementation I get a runtime error about a replacement in place of a variable.
I put here the stack trace.
Error detected in AddmmBackward. Traceback of forward call that caused the error:
File "/Users/lorenzoterenzi/Downloads/lets-do-irl-master/mujoco/vail/main.py", line 185, in
main()
File "/Users/lorenzoterenzi/Downloads/lets-do-irl-master/mujoco/vail/main.py", line 162, in main
train_actor_critic(actor, critic, memory, actor_optim, critic_optim, args)
File "/Users/lorenzoterenzi/Downloads/lets-do-irl-master/mujoco/vail/train_model.py", line 75, in train_actor_critic
values = critic(inputs)
File "/Users/lorenzoterenzi/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/Users/lorenzoterenzi/Downloads/lets-do-irl-master/mujoco/vail/model.py", line 36, in forward
v = self.fc3(x)
File "/Users/lorenzoterenzi/.local/lib/python3.7/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/Users/lorenzoterenzi/.local/lib/python3.7/site-packages/torch/nn/modules/linear.py", line 91, in forward
return F.linear(input, self.weight, self.bias)
File "/Users/lorenzoterenzi/.local/lib/python3.7/site-packages/torch/nn/functional.py", line 1674, in linear
ret = torch.addmm(bias, input, weight.t())
(function print_stack)
Traceback (most recent call last):
File "/Users/lorenzoterenzi/Downloads/lets-do-irl-master/mujoco/vail/main.py", line 185, in
main()
File "/Users/lorenzoterenzi/Downloads/lets-do-irl-master/mujoco/vail/main.py", line 162, in main
train_actor_critic(actor, critic, memory, actor_optim, critic_optim, args)
File "/Users/lorenzoterenzi/Downloads/lets-do-irl-master/mujoco/vail/train_model.py", line 101, in train_actor_critic
loss.backward()
File "/Users/lorenzoterenzi/.local/lib/python3.7/site-packages/torch/tensor.py", line 185, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/Users/lorenzoterenzi/.local/lib/python3.7/site-packages/torch/autograd/init.py", line 127, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [100, 1]], which is output 0 of TBackward, is at version 2; expected version 1 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.