marccoru / marinedebrisdetector Goto Github PK

https://arxiv.org/abs/2307.02465

Hi

I found your paper interesting, I want to produce the inference score using the provided UNet weights.
However, I observed that there is a mismatch for the test dataset.

Could you please look into it.
here is the error:

Global seed set to 0 using checkpoint epoch=33-val_loss=0.59-auroc=0.985.ckpt Lightning automatically upgraded your loaded checkpoint from v1.8.2 to v1.9.5. To apply the upgrade to your files permanently, run python -m pytorch_lightning.utilities.upgrade_checkpoint --file checkpoints/marinedebrisdetector/a_unet1/epoch=33-val_loss=0.59-auroc=0.985.ckpt`
GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs

Dataset Composition total

train
flobs_dataset (train): 39194
shipsdataset: 29833
MARIDA (train): 2017

val
refinedflobs_val: 2451
MARIDA (val): 1185

test
flobstestdataset: 2197
maridatestdataset: 872

Dataset Composition debris/non-debris
train
flobs_dataset (train): 19587/19607
shipsdataset: 0/29833
MARIDA (train): 930/1087

val
refinedflobs_val: 868/1583
MARIDA (val): 616/569

test
flobstestdataset: 903/1294
maridatestdataset: 270/602
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [2]
/home/mbzuaiser/anaconda3/envs/changeformer_s/lib/python3.7/site-packages/lightning_fabric/loggers/csv_logs.py:184: UserWarning: Experiment logs directory ./checkpoints/marinedebrisdetector/a_unet1/test_log/version_0 exists and is not empty. Previous log files in this directory will be deleted when the new ones are saved!
f"Experiment logs directory {self.log_dir} exists and is not empty."
/home/mbzuaiser/anaconda3/envs/changeformer_s/lib/python3.7/site-packages/pytorch_lightning/trainer/connectors/data_connector.py:229: PossibleUserWarning: The dataloader, test_dataloader 0, does not have many workers which may be a bottleneck. Consider increasing the value of the num_workers argument(try 128 which is the number of cpus on this machine) in theDataLoader` init to improve performance.
category=PossibleUserWarning,
Testing DataLoader 0: 48%|████▊ | 23/48 [00:31<00:34, 1.39s/it]Traceback (most recent call last):

File "/home/mbzuaiser/anaconda3/envs/changeformer_s/lib/python3.7/site-packages/spyder_kernels/py3compat.py", line 356, in compat_exec
exec(code, globals, locals)

File "/nvme-data/change_former/Marine_debris_detection_sentinel_2/code/marinedebrisdetector-main/marinedebrisdetector/test.py", line 236, in
main(parse_args())

File "/nvme-data/change_former/Marine_debris_detection_sentinel_2/code/marinedebrisdetector-main/marinedebrisdetector/test.py", line 92, in main
trainer.test(model, marinedebris_datamodule)

File "/home/mbzuaiser/anaconda3/envs/changeformer_s/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 795, in test
self, self._test_impl, model, dataloaders, ckpt_path, verbose, datamodule

File "/home/mbzuaiser/anaconda3/envs/changeformer_s/lib/python3.7/site-packages/pytorch_lightning/trainer/call.py", line 38, in _call_and_handle_interrupt
return trainer_fn(*args, **kwargs)

File "/home/mbzuaiser/anaconda3/envs/changeformer_s/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 842, in _test_impl
results = self._run(model, ckpt_path=self.ckpt_path)

File "/home/mbzuaiser/anaconda3/envs/changeformer_s/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1112, in _run
results = self._run_stage()

File "/home/mbzuaiser/anaconda3/envs/changeformer_s/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1188, in _run_stage
return self._run_evaluate()

File "/home/mbzuaiser/anaconda3/envs/changeformer_s/lib/python3.7/site-packages/pytorch_lightning/trainer/trainer.py", line 1228, in _run_evaluate
eval_loop_results = self._evaluation_loop.run()

File "/home/mbzuaiser/anaconda3/envs/changeformer_s/lib/python3.7/site-packages/pytorch_lightning/loops/loop.py", line 199, in run
self.advance(*args, **kwargs)

File "/home/mbzuaiser/anaconda3/envs/changeformer_s/lib/python3.7/site-packages/pytorch_lightning/loops/dataloader/evaluation_loop.py", line 152, in advance
dl_outputs = self.epoch_loop.run(self._data_fetcher, dl_max_batches, kwargs)

File "/home/mbzuaiser/anaconda3/envs/changeformer_s/lib/python3.7/site-packages/pytorch_lightning/loops/loop.py", line 199, in run
self.advance(*args, **kwargs)

File "/home/mbzuaiser/anaconda3/envs/changeformer_s/lib/python3.7/site-packages/pytorch_lightning/loops/epoch/evaluation_epoch_loop.py", line 121, in advance
batch = next(data_fetcher)

File "/home/mbzuaiser/anaconda3/envs/changeformer_s/lib/python3.7/site-packages/pytorch_lightning/utilities/fetching.py", line 184, in next
return self.fetching_function()

File "/home/mbzuaiser/anaconda3/envs/changeformer_s/lib/python3.7/site-packages/pytorch_lightning/utilities/fetching.py", line 265, in fetching_function
self._fetch_next_batch(self.dataloader_iter)

File "/home/mbzuaiser/anaconda3/envs/changeformer_s/lib/python3.7/site-packages/pytorch_lightning/utilities/fetching.py", line 280, in _fetch_next_batch
batch = next(iterator)

File "/home/mbzuaiser/anaconda3/envs/changeformer_s/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 521, in next
data = self._next_data()

File "/home/mbzuaiser/anaconda3/envs/changeformer_s/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 561, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration

File "/home/mbzuaiser/anaconda3/envs/changeformer_s/lib/python3.7/site-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch
return self.collate_fn(data)

File "/home/mbzuaiser/anaconda3/envs/changeformer_s/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py", line 84, in default_collate
return [default_collate(samples) for samples in transposed]

File "/home/mbzuaiser/anaconda3/envs/changeformer_s/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py", line 84, in
return [default_collate(samples) for samples in transposed]

File "/home/mbzuaiser/anaconda3/envs/changeformer_s/lib/python3.7/site-packages/torch/utils/data/_utils/collate.py", line 56, in default_collate
return torch.stack(batch, 0, out=out)

RuntimeError: stack expects each tensor to be equal size, but got [12, 128, 128] at entry 0 and [12, 128, 127] at entry 32`