notmahi / dobb-e Goto Github PK

Hi, thanks for you great work!

May I ask where are the camera intrinsics and extrinsics? I failed to find them in the dataset.

Thanks!

Thanks for your awesome works! I am currently engaged in transforming the depth image into a 3D representation. However, I have not been able to locate the camera intrinsics in your dataset, HoNY. Could you provide them?

HI, I run the train.py following the imitation-in-homes README

I download all the datasets and change the path in env_vars.yaml. Other than that, all configurations are default.
but when I run the command
python train.py --config-name=finetune

The logs show:

[2024-02-06 15:48:42,360][main][INFO] - Working in ./dobb-e/imitation-in-homes
[2024-02-06 15:48:43,065][main][INFO] - Setting up dataloaders.
Filtering trajectories: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 81/81 [00:00<00:00, 132814.16it/s]
./dobb-e/imitation-in-homes/dataloaders/utils.py:283: UserWarning: No trajectories were found matching the specified criteria. Make sure you have the correct dataset root and trajectory roots, and the correct include/exclude criteria.
warnings.warn(
Filtering trajectories: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 81/81 [00:00<00:00, 155700.56it/s]
[2024-02-06 15:48:43,941][timm.models._builder][INFO] - Loading pretrained weights from Hugging Face hub (notmahi/dobb-e)
[2024-02-06 15:48:44,306][timm.models._hub][INFO] - [notmahi/dobb-e] Safe alternative available for 'pytorch_model.bin' (as 'model.safetensors'). Loading weights using safetensors.
Using static action normalization
act_mean: tensor([ 7.2950e-05, 5.2102e-03, 9.5942e-04, -1.9970e-04, -1.5271e-03,
3.3574e-04, 0.0000e+00])
act_std: tensor([0.0144, 0.0221, 0.0110, 0.0158, 0.0465, 0.0253, 1.0000])
Fatal Python error: Segmentation fault

I use faulthandler and find that the error occurred in
self.model = self.model.to(self.device)

Are the 10 real homes also used by HomeRobot challenge?

Someone is clearly driving this robot in the top right behind the curtain of Oz. So much for autonomous...

Thanks Elon

https://youtu.be/G7SBA5roNhI?t=62

Hi team,
Thank you so much for open sourcing such an amazing project! I thought this would be my perfect opportunity to learn about behavioral cloning of robots.
However, during the learning process, I encountered an error message of unknown reason. I tried to change the order of imports as mentioned in issue 4(#4). This worked, but a new error was prompted.

My error message is as follows, could you please check it for me:

$ python train.py --config-name=finetune
/home/me/dobbe/dobb-e/imitation-in-homes
checkpoints/2024-03-20/Drawer_Opening-Env1-moco-18-46-33
[2024-03-20 18:46:33,636][main][INFO] - Working in /home/me/dobbe/dobb-e/imitation-in-homes
[2024-03-20 18:46:34,283][main][INFO] - Setting up dataloaders.
Filtering trajectories: 100%|███████████████| 81/81 [00:00<00:00, 159426.85it/s]
/home/me/dobbe/dobb-e/imitation-in-homes/dataloaders/utils.py:283: UserWarning: No trajectories were found matching the specified criteria. Make sure you have the correct dataset root and trajectory roots, and the correct include/exclude criteria.
warnings.warn(
Filtering trajectories: 100%|███████████████| 81/81 [00:00<00:00, 172106.70it/s]
[2024-03-20 18:46:35,192][timm.models._builder][INFO] - Loading pretrained weights from Hugging Face hub (notmahi/dobb-e)
[2024-03-20 18:46:35,481][timm.models._hub][INFO] - [notmahi/dobb-e] Safe alternative available for 'pytorch_model.bin' (as 'model.safetensors'). Loading weights using safetensors.
Using static action normalization
act_mean: tensor([ 7.2950e-05, 5.2102e-03, 9.5942e-04, -1.9970e-04, -1.5271e-03,
3.3574e-04, 0.0000e+00])
act_std: tensor([0.0144, 0.0221, 0.0110, 0.0158, 0.0465, 0.0253, 1.0000])
0it [00:00, ?it/s] | 0/50 [00:00<?, ?it/s]
Epoch 1: 0%| | 0/50 [00:00<?, ?it/s]
Error executing job with overrides: []
Traceback (most recent call last):
File "/home/me/dobbe/dobb-e/imitation-in-homes/train.py", line 414, in main
workspace.run()
File "/home/me/dobbe/dobb-e/imitation-in-homes/train.py", line 194, in run
self._train_epoch()
File "/home/me/dobbe/dobb-e/imitation-in-homes/train.py", line 223, in _train_epoch
self._train_step(batch, overall_loss_dict, iterator)
File "/home/me/dobbe/dobb-e/imitation-in-homes/train.py", line 240, in _train_step
batch_in_device = self.to_device(batch)
File "/home/me/dobbe/dobb-e/imitation-in-homes/train.py", line 204, in to_device
manyTensors = [
TypeError: 'NoneType' object is not iterable

Besides,My env_var.yaml settings are as follows, are there any mistakes?（I did not modify the content in finetune.yaml）

home_ssl_data_root: /home/me/dobbe/dobb-e/imitation-in-homes/iphone_data # root to home ssl data folder
finetune_task_data_root: /home/me/dobbe/dobb-e/imitation-in-homes/finetune_directory # root to the exported finetuning data folder

home_ssl_data_original_root: /home/me/dobbe/dobb-e/imitation-in-homes/iphone_data # path included in the r3d_files.txt file of dataset
finetune_task_data_original_root: /home/me/dobbe/dobb-e/imitation-in-homes/finetune_directory # path included in r3d_files.txt of dataset (may be the same as finetune_task_data_root if data has not been moved nor transferred across machines)
project_root: /home/me/dobbe/dobb-e/imitation-in-homes # path to this repo

wandb:
entity: null # wandb username

Below is my file path

I would be very grateful if you could help me.