agrimgupta92 / sgan Goto Github PK

Hello
I meet an issue when I try to train new model on wins with python 0.4.1 .

with command !python scripts/train.py --noise_dim=0

I remove all the cuda() in order to run on cpu but I meet this issue:

Traceback (most recent call last):
File "scripts/train.py", line 580, in
main(args)
File "scripts/train.py", line 245, in main
optimizer_d)
File "scripts/train.py", line 371, in discriminator_step
generator_out = generator(obs_traj, obs_traj_rel, seq_start_end)
File "C:\Users\zeya\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "C:\Users\zeya\sgan-master\scripts\sgan\models.py", line 508, in forward
final_encoder_h = self.encoder(obs_traj_rel)
File "C:\Users\zeya\Anaconda3\lib\site-packages\torch\nn\modules\module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "C:\Users\zeya\sgan-master\scripts\sgan\models.py", line 63, in forward
obs_traj_embedding = self.spatial_embedding(obs_traj.view(-1, 2))
RuntimeError: invalid argument 2: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Call .contiguous() before .view(). at c:\new-builder_3\win-wheel\pytorch\aten\src\th\generic/THTensor.cpp:237

By this paper:
The Simpler the Better: Constant Velocity for Pedestrian Motion Prediction.
https://arxiv.org/abs/1903.07933

"Because S-GAN and SoPhie were evaluated by drawing 20 samples and taking the predicted trajectory with the minimum error for evaluation, we added an extended version OUR-S of our CVM for comparability. For OUR-S we add additional angular noise to its predicted direction, which we draw from N (0, σ 2 ) with σ = 25 ◦ and evaluate its error in the same fashion."

Then you can find out the constant velocity model outperforms sgan if we draw 20 samples.

Just like @AyeshaNirma said in #8
"Yours and your colleagues, own TrajNet challenge does not allow this kind of evaluation.
Here is a quick test, take a training set, cluster 10 trajectories make a linear model (not even a regressor ), during test time, assign each individual the "best" trajectory. You will see how well Social GAN performs wrt a linear motion model. You will have your answer, if 20 is a small number or not."

I am wondering about whether it's fair or not to use this kind of evaluation.
@agrimgupta92

I noticed that the trajectories with length less than desired sequence length are filtered out in trajectories.py. Is it possible to make it work for varying trajectory lengths, with lengths being anything between 1 to seq_len?

Hi,

Is there any simple way to save the predicted trajectory (e.g., x-y coordinates) in a txt or any file format?

When I run the training commend: python scripts/train.py

I got an error message:

ValueError: could not convert string to float: '0.0\t1.0\t1.41\t-5.68'
Any idea that can help me to fix it?

(When I run 'run_traj.sh', the training runs well).

Many thanks in advance.

Hi,
I notice in your code "evaluate_model.py" that you run prediction 20 times for each test example and then use the minimum error as the final number. For other baselines like LSTM and Linear it will produce the same results every time. Do you think it would be fairer to use SGAN to predict 20 times and use the averaged result for evaluation once? Or it is a common practice? Thank you!

When I download the data, the following problems appear. How can I solve them?

dlmuth@ubuntu:~/Downloads/sgan-master$ bash scripts/download_data.sh
--2019-02-24 18:24:39-- https://www.dropbox.com/s/8n02xqv3l9q18r1/datasets.zip?dl=0
Resolving www.dropbox.com (www.dropbox.com)... 66.220.152.28
Connecting to www.dropbox.com (www.dropbox.com)|66.220.152.28|:443... failed: Connection refused.
[datasets.zip]
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
unzip: cannot find zipfile directory in one of datasets.zip or
datasets.zip.zip, and cannot find datasets.zip.ZIP, period.

Hi Agrim,

according to your training procedure we can set these two hyperparameters:

• num_iterations: Number of training iterations. Default is 10,000.

• num_epochs: Number of training iterations. Default is 200.

Can you provide a more detailed definition?
Is there any constraint they are subject to?

Thanks.

Hi Agrim!

I saw that you added noise (uniform) to the targets and labels. Can you explain why you do it and how the ranges for these uniform distributions are chosen?

From losses.py:

    y_real = torch.ones_like(scores_real) * random.uniform(0.7, 1.2)
    y_fake = torch.zeros_like(scores_fake) * random.uniform(0, 0.3)

checkpoint['g_best_state'] is corresponding to the model weights with the lowest validation error, why you just use checkpoint['g_state'], which is the final g_state saved in the training. The final g_state is not guaranteed to be the best one. Hope someone could give an explanation. Thanks!

Hi

In your paper, the speed comparison is listed in Table 2. The speed of prediction length of 8 and 12 are 0.04 seconds and 0.05 seconds. May I ask how you calculate them? Because I tested them using code in evaluate_model.py listed below and found the averaged (20 times) time is between 0.007 and 0.009 seconds which are far lower than your result...

for _ in range(num_samples):
pred_traj_fake_rel = generator(
obs_traj, obs_traj_rel, seq_start_end
)
pred_traj_fake = relative_to_abs(
pred_traj_fake_rel, obs_traj[-1]
)
ade.append(displacement_error(
pred_traj_fake, pred_traj_gt, mode='raw'
))
fde.append(final_displacement_error(
pred_traj_fake[-1], pred_traj_gt[-1], mode='raw'
))
print ("one time",toc - tic)

Thank you so much!

I just run evaluate_model.py with the trained model provided by the author, but the error of ADE and FDE are both much bigger than the results displayed in the paper. Is there anything else I have to take care when running the evaluate_model.py?

Hi I want to evaluate my trained model and check the generated predictions. I checked the evaluation function and the the test.py I am not able to get the frame and ped_id. How can I generate a file with <frame_id> <ped_id> for my test file ?
Can anyone help me out please?

abcdefg@abcdefg-virtual-machine:~$ bash sgan-master/scripts/download_data.sh--2019-05-25 09:26:42-- https://www.dropbox.com/s/8n02xqv3l9q18r1/datasets.zip?dl=0
Resolving www.dropbox.com (www.dropbox.com)... 174.37.154.236, 2620:100:6032:1::a27d:5201
Connecting to www.dropbox.com (www.dropbox.com)|174.37.154.236|:443... connected.
Unable to establish SSL connection.
[datasets.zip]
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
unzip: cannot find zipfile directory in one of datasets.zip or
datasets.zip.zip, and cannot find datasets.zip.ZIP, period.

Hi,
I was just going through the trajectories.py while testing and generated trajectories for eth dataset along with frame and ped ids. I noticed that in trajectories.py if there are pedestrians present for example 2 and 3 the because of num_peds_considered it neglects pedestrian 2 and then uses pedestrian 3 to predict the pedestrian but then it does not detect anything for pedestrian 2. Can someone explain this? @agrimgupta92

PS- Can I generate predictions for each trajectory?

unsupport multi gpus training?

I set the gpu_number as 8 and its iteration times is as the same as the gpu_number is 1.

Looking at the table 1 of your paper and under the column 1V-1, where you sample one trajectory and your numbers are actually potentially comparable to a Vanilla LSTM, since only in this case it is a fair comparison. This SOCIAL GAN work gets outperformed by a Vanilla LSTM ? Secondly, Alexandre Alahi the last author of your work and the first author of Social LSTM could not reproduce the Social LSTM results himself ? Could you please comment on these aspect ? No disrespect intended.

Could you tell me the detailed format description for the dataset example "780 1.0 8.46 3.59"?

the function gan_d_loss(scores_real, scores_fake) in the losses.py, is that a mistake we set y_fake = torch.zeros_like(scores_fake) * random.uniform(0, 0.3)? Because we always got zero there is no meaning for us to multiply a random number. Or this should be y_fake = torch.ones_like(scores_fake) * random.uniform(0, 0.3)?
full code:
`
def gan_d_loss(scores_real, scores_fake):

y_real = torch.ones_like(scores_real) * random.uniform(0.7, 1.2)
y_fake = torch.zeros_like(scores_fake) * random.uniform(0, 0.3)
loss_real = bce_loss(scores_real, y_real)
loss_fake = bce_loss(scores_fake, y_fake)
return loss_real + loss_fake

`

HI~
Excuse me, How do i to show visualization result of this model?

Hi I am training the SGAN model with all the datasets provided . After a few iterations I face the following error:

Traceback (most recent call last): File "/workspace/code/scripts/train.py", line 512, in <module> main(args) File "/workspace/code/scripts/train.py", line 191, in main optimizer_g) File "/workspace/code/scripts/train.py", line 387, in generator_step loss.backward() File "/opt/conda/lib/python3.6/site-packages/torch/tensor.py", line 96, in backward torch.autograd.backward(self, gradient, retain_graph, create_graph) File "/opt/conda/lib/python3.6/site-packages/torch/autograd/__init__.py", line 90, in backward allow_unreachable=True) # allow_unreachable flag RuntimeError: cublas runtime error : the GPU program failed to execute at /opt/pytorch/pytorch/aten/src/THC/THCBlas.cu:258

It would be great if someone can help me fix this error.
This is with num_epochs= 200
and when I tried it with num_epochs = 5 it works fine

Do you use the same hyperparameters for 1V-1? Thanks! @agrimgupta92

When I try

python scripts/evaluate_model.py \ --model_path models/sgan-models

I get the error: "unrecognized arguments: --model_path models/sgan-models"

Any idea what the error might be?

Hi, I want to know if you also have the stanford dataset as per the required format for training and testing the model? I have access to the training dataset but the test dataset is with '?' and I cannot really get the ADE and FDE with that, as it requires groundtruth for measuring the loss. It would really help if you can help me out with the stanford test set.

How did you preprocess the coordinates in each dataset? Did you do some transformation to transform the coordinates in all datasets into one global frame?

The default value for noise_dim is None but the code fails if we use that. What was the noise dimension for the results shown in the paper?

Hi Agrim,

Currently I am checking the number of collisions with the static environment (trees, benches, snow).

For the ETH dataset I have a proper transformation (Homography). The same method does not work for the UCY data. Do you have your conversion script (from pixels to meters) to generate input data?

Best

Hi, @agrimgupta92! Recently, I notice your social GAN work. When I read your code, I cannot understand "seq_start_end" in Decoder and PoolHiddenNet. I spent a lot of time and find out it has something about dataset. However, I still cannot understand the exact meaning of this parameter and why we should use it.

Could you give a gentle explanation? Thanks a lot.

Hi,

Could anyone can help me to save the 'pred_traj_fake (final prediction of this model)' tensor as the format compatible to be used in Matlab?

I am not an expert on python and torch so I am struggling at saving this variable. I would like to use it for some visualization.

Thanks in advance~!

Hello, excuse me. I trained the model directly before without modifying the parameters. There is a big gap between the running results and the results in your paper. Then I revised the corresponding parameters according to the implementation details in your paper. Because of the poor performance of my computer, I revised the batch_size to 32. As in other papers, the loss value does not converge during the training process, and when testing. The difference between the findings and those in your paper is particularly large. I'd like to ask where I didn't get it right.@agrimgupta92

Hi, I ran this program on my computer and I met some problems.

1. When I transform this program to windows version and train a new model. It will keep the same loss after several iterations. All the parameters were default and the dataset is from zara1;
2. When I run this program on Ubuntu, it will have error "Segmentation fault (core dumped)".

Would you please help me and tell me how to solve this problem?

Thank you.

I am confused with getting preprocessed data from original UCY dataset (zara01,zara02,univ), as there only gives annotations in image coordinate system. Can you provide data preprocessing code? Thanks very much!

Hi, Gupta,
The suggested dimension of hidden states in the paper are 16 and 32 for encoder and decoder. The default parameters in the code are 64 and 128. Is that mean 16 and 32 are a better value for training? I'm also wondering about the dimension of hidden states of the baseline method(LSTM and Social LSTM) used in your paper. Can you give a help?

Hi!
I set "pool_every_timestep" of the new TrajectoryGenerator (generator) in evaluate_model.py as "True" and an error occured:

Traceback (most recent call last):
File "scripts/evaluate_model.py", line 125, in
main(args)
File "scripts/evaluate_model.py", line 112, in main
generator = get_generator(checkpoint)
File "scripts/evaluate_model.py", line 41, in get_generator
generator.load_state_dict(checkpoint['g_state'])
File "....../lib/python3.5/site-packages/torch/nn/modules/module.py", line 721, in load_state_dict
self.class.name, "\n\t".join(error_msgs)))
RuntimeError: Error(s) in loading state_dict for TrajectoryGenerator:

Missing key(s) in state_dict: "decoder.mlp.0.weight", "decoder.mlp.0.bias", "decoder.mlp.2.weight", "decoder.mlp.2.bias".

When I changed the "pool_every_timestep" as "False", the error disappeared. Does this mean that I need to retrain the model for "pool_every_timestep=True" of the new TrajectoryGenerator for this evaluation setting?

Thank you!

"ValueError: could not convert string to float: '0.0\t1.0\t11.238836854\t3.746958' "

As you can see, the delimiters are set to \t and so the data becomes a string which cannot be converted to float. Could you recommend how this should be working instead?

Hi.

How do you deal with target objects that do not have 20 frames in the dataset?

Could you please give some advice about how to choose Noise_dim?
It'd be much appreciated if you could give one for reference.
Thank you!

I found your parameter in your file. Thank you!

While running training, this error happened~!

Traceback (most recent call last):
File "scripts/train.py", line 580, in
main(args)
File "scripts/train.py", line 148, in main
batch_norm=args.batch_norm)
File "/home/daeho/sgan-master/sgan/models.py", line 425, in init
if self.noise_dim[0] == 0:

Should I change noise_dim?

Hi Agrim,

I have used default parameters to train on zara01 as described in read me. The errors are horrendous so for pred len : 8 and 12 the ADE is 2.27 and 9.02 respectively. Its a but puzzling as I have done absolutely no changes to your code or model whatsoever and I have used all default params. Could you perhaps point out what potentially could be the issue.

Hi Agrim,

The description of the code is super and working with the pretrained models as well!

However, I am trying to train it myself, but the generator's and discriminator's loss do not drop. Can you please let me know if the loss changed for your setting?

Best
Tessa

Hi,

I am evaluating eth pretrained model with my own dataset. I run into this:

curr_seq[_idx, :, pad_front:pad_end] = curr_ped_seq ValueError: could not broadcast input array from shape (2,44) into shape (2,16)

error. I am not able to understand if I have to change any parameters or not while testing with my own data. Can you please help me out ?

This error occurred when I ran ‘script/train.py’

[INFO: train.py: 119]: Initializing train dataset
[INFO: train.py: 121]: Initializing val dataset
[INFO: train.py: 129]: There are 21.03125 iterations per epoch
Traceback (most recent call last):
File "scripts/train.py", line 581, in
main(args)
File "scripts/train.py", line 149, in main
batch_norm=args.batch_norm)
File "/home/xu/dengzaixu/sgan/sgan/models.py", line 425, in init
if self.noise_dim[0] == 0:
TypeError: 'NoneType' object is not subscriptable

Hi @agrimgupta92, Are you able to share with me your implementation of S-LSTM and LSTM model, that
produced the results in your paper ?

That's help a lot.
Thanks for your great work.

Hi, Agrim

I am trying to make my own dataset following the same format that you used (i.e., eth, zara, hotel, and univ).

The data files have four columns though, I couldn't clearly understand what the value of each column represents.

If you may, could you explain it?

Hi, there are some variables like loss_mask, linear_ped, non_linear_ped but I do not know their usages. I did not find them in your work of sgan. So, what your purpose of using these variables? For example, you calculated disp_error, disp_error_l, and disp_error_nl, what's the difference?

I try running the train.py script on a CPU and get the following error in the decoder's forward pass:

/Users/davidglavas/Documents/GitHub/Social-GAN/venv/bin/python /Users/davidglavas/Documents/GitHub/Social-GAN/scripts/trainCPU.py --delim tab --noise_dim=0
[INFO: trainCPU.py:  120]: Initializing train dataset
[INFO: trainCPU.py:  122]: Initializing val dataset
[INFO: trainCPU.py:  130]: There are 21.03125 iterations per epoch
[INFO: trainCPU.py:  154]: Here is the generator:
[INFO: trainCPU.py:  155]: TrajectoryGenerator(
  (encoder): Encoder(
    (encoder): LSTM(64, 64)
    (spatial_embedding): Linear(in_features=2, out_features=64, bias=True)
  )
  (decoder): Decoder(
    (decoder): LSTM(64, 128)
    (pool_net): PoolHiddenNet(
      (spatial_embedding): Linear(in_features=2, out_features=64, bias=True)
      (mlp_pre_pool): Sequential(
        (0): Linear(in_features=192, out_features=512, bias=True)
        (1): ReLU()
        (2): Linear(in_features=512, out_features=1024, bias=True)
        (3): ReLU()
      )
    )
    (mlp): Sequential(
      (0): Linear(in_features=1152, out_features=1024, bias=True)
      (1): ReLU()
      (2): Linear(in_features=1024, out_features=128, bias=True)
      (3): ReLU()
    )
    (spatial_embedding): Linear(in_features=2, out_features=64, bias=True)
    (hidden2pos): Linear(in_features=128, out_features=2, bias=True)
  )
  (pool_net): PoolHiddenNet(
    (spatial_embedding): Linear(in_features=2, out_features=64, bias=True)
    (mlp_pre_pool): Sequential(
      (0): Linear(in_features=128, out_features=512, bias=True)
      (1): ReLU()
      (2): Linear(in_features=512, out_features=1024, bias=True)
      (3): ReLU()
    )
  )
  (mlp_decoder_context): Sequential(
    (0): Linear(in_features=1088, out_features=1024, bias=True)
    (1): ReLU()
    (2): Linear(in_features=1024, out_features=128, bias=True)
    (3): ReLU()
  )
)
[INFO: trainCPU.py:  170]: Here is the discriminator:
[INFO: trainCPU.py:  171]: TrajectoryDiscriminator(
  (encoder): Encoder(
    (encoder): LSTM(64, 64)
    (spatial_embedding): Linear(in_features=2, out_features=64, bias=True)
  )
  (real_classifier): Sequential(
    (0): Linear(in_features=64, out_features=1024, bias=True)
    (1): ReLU()
    (2): Linear(in_features=1024, out_features=1, bias=True)
    (3): ReLU()
  )
)
[INFO: trainCPU.py:  234]: Starting epoch 1
Traceback (most recent call last):
  File "/Users/davidglavas/Documents/GitHub/Social-GAN/scripts/trainCPU.py", line 587, in <module>
    main(args)
  File "/Users/davidglavas/Documents/GitHub/Social-GAN/scripts/trainCPU.py", line 248, in main
    optimizer_d)
  File "/Users/davidglavas/Documents/GitHub/Social-GAN/scripts/trainCPU.py", line 376, in discriminator_step
    generator_out = generator(obs_traj, obs_traj_rel, seq_start_end)
  File "/Users/davidglavas/Documents/GitHub/Social-GAN/venv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/Users/davidglavas/Documents/GitHub/Social-GAN/sgan/models.py", line 656, in forward
    seq_start_end,
  File "/Users/davidglavas/Documents/GitHub/Social-GAN/venv/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/Users/davidglavas/Documents/GitHub/Social-GAN/sgan/models.py", line 161, in forward
    [decoder_h.view(-1, self.h_dim), pool_h], dim=1)
TypeError: expected Tensor as element 1 in argument 0, but got tuple

When i run python scripts/evaluate_model.py --model_path models/sgan-models
i got the following error

Traceback (most recent call last):
  File "scripts/evaluate_model.py", line 7, in <module>
    from sgan.data.loader import data_loader
ImportError: No module named sgan.data.loader

Hello!
First of all. thanks for your great work.
But I have a little doubt about whether the pooling really helps avoid collision. I count the number of collision times of SGAN(which doesn't use the pooling) and SGAN-P, specially, the models are downloaded from yours. The threshold distance to judge whether the collision between two person is happened is about 0.08m , which is the minimum distance between two persons in all five datasets. And as show in the following table, SGAN-P doesn't show better collision avoidence performance than SGAN and it seems V-LSTM achieves the best performance. So, really does the pooling help avoid collision? or maybe it is just our garbage imagination.

Hi, Agrim,
Have you used the value <non_linear_ped> to measure the results on nonlinear trajectories? I could not find relative part in the evaluate code.

Thanks for help!