zhenzhiwang / intercontrol Goto Github PK

Hi! i love your great job.
but i have no idea where is humanml_opt.txt during sampling in Human Interaction Generation part.
Here is my error messages.

(intercontrol) td@td:~/Project/intercontrol$ python3 -m eval.eval_interaction --model_path save/my_humanml_trans_enc_512/model000120000.pt --replication_times 10 --bfgs_times_first 5 --bfgs_times_last 10 --bfgs_interval 1 --use_posterior  --control_joint all --interaction_json './assets/all_plans.json' --multi_person
Warning: was not able to load [use_tta], using default value [False] instead.
Warning: was not able to load [concat_trans_emb], using default value [False] instead.
Warning: was not able to load [trans_emb], using default value [False] instead.
Using inpainting mask: global_joint
Will save to log file [save/my_humanml_trans_enc_512/interactive_niter_120000_all_mask1_bfgs_first5_last10_skip1_posterior_wo_mm.log]
Eval mode [wo_mm]
Logging to /tmp/openai-2023-11-29-21-43-32-799527
creating humanml loader...
Reading ././dataset/humanml_opt.txt
Traceback (most recent call last):
  File "/home/td/anaconda3/envs/intercontrol/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/home/td/anaconda3/envs/intercontrol/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/td/Project/intercontrol/eval/eval_interaction.py", line 159, in <module>
    gen_loader = get_dataset_loader(name=args.dataset, batch_size=args.batch_size, num_frames=None, split=split, load_mode='eval', size=1)  # humanml only for loading diffusion model config
  File "/home/td/Project/intercontrol/data_loaders/get_data.py", line 78, in get_dataset_loader
    dataset = get_dataset(name, num_frames, split, load_mode, batch_size, opt, short_db, cropping_sampler, size)
  File "/home/td/Project/intercontrol/data_loaders/get_data.py", line 57, in get_dataset
    dataset = DATA(split=split, num_frames=num_frames, load_mode=load_mode, size=size)
  File "/home/td/Project/intercontrol/data_loaders/humanml/data/dataset.py", line 1074, in __init__
    opt = get_opt(dataset_opt_path, device)
  File "/home/td/Project/intercontrol/data_loaders/humanml/utils/get_opt.py", line 37, in get_opt
    with open(opt_path) as f:
FileNotFoundError: [Errno 2] No such file or directory: '././dataset/humanml_opt.txt'

Also the model_path in README is wrong, so i change the path in my test case.
Thank you

Hi, I've been exploring your demo and got two motions of skeletons as shown below. Could you explain what the individual motions of the skeletons signify? Are the movements of one skeleton meant to control the other? I hope you can provide a detailed explanation of the demo results.

Thank you for making your amazing work open source.

I have a question about the evaluation results in Table 1. Are the evaluations in Table 1 the same for OmniControl? The numbers for OmniControl match, but the results for GMD and PriorMDM are different. Could you please explain these discrepancies?

OmniControl: [Table 1]

InterControl: [Table 1]

The IK guidance during generation process is clear.
But I'm confused why the training process is like below:

x_t = self.q_sample(x_start, t, noise=noise)
x_t = self.condition_mean_bfgs(x_t, num_condition)

Could you explain why you put self.condition_mean_bfgs here?
Is that a theoretical approach or empirical approach?

Wait for your reply. Thanks

Thank you for sharing the great work!

From table 1 in your paper "InterControl: Generating Human Motion Interactions by Controlling Every Joint", the evaluation result of is 0.159 for Text-to-motion evaluation on the HumanML3D. I wander how it is much better than original MDM?
as far as I understand, you used a pre-trained MDM. Or did you use the controlnet and use 0s as spatial control?
Is there a saved checkpoints so it is replicable?

Thank you!

Hey, very interesting work!

I'm curious whether I recreated the "3 people interaction sampling" correctly.
I haven't quite looked through all the code, but simply running:
python -m sample.more_people_global_joint_control --model_path save/mask0.25_bfgs5_posterior_all/model000140000.pt --multi_person --bfgs_times_first 5 --bfgs_times_last 10 --use_posterior
leads to interactions of only two people instead of three or more, as depicted in the output videos.

Is this expected behavior? Or are there additional things that need to be considered to actually create interactions of three or more people?