dvornikita / fewshot_ensemble Goto Github PK

Hey, thanks for the nice work.
Just reading your code, and I think there is a mistake in normalizing logits without the true label.
probs /= (all_probs.sum(dim=1, keepdim=True) + 1e-8)
should rather be
probs /= (probs.sum(dim=1, keepdim=True) + 1e-8)
in relation_loss in the ensembles.py file

And then similarly, for example for cos reg_type,
probs = probs / torch.sqrt(((all_probs ** 2).sum(dim=1, keepdim=True) + 1e-8)) # l2 normed
should be
probs = probs / torch.sqrt(((probs ** 2).sum(dim=1, keepdim=True) + 1e-8)) # l2 normed
as far as I understood your paper

Getting this error from meta_dataset.py file

chosen_class_inds = random.sample(
                all_class_inds, self.n_train + self.n_test)

I am training a single model on cub dataset. Do you know what might be causing this?

Values of self.n_train -> 15, self.n_test -> 5, len(all_class_inds) -> 30

python ensembles/train.py --model.model_name=deep_robust10 --data.dataset=mini_imagenet --model.backbone=deep --ens.num_heads=10 --ens.relation_type=robust

!!!!!! Starting ephoch 0 !!!!!!
0%| | 0/2400 [00:00<?, ?it/s]Traceback (most recent call last):
File "/xinfu/code/few_shot_learning/fewshot_ensemble/ensembles/train.py", line 178, in
main()
File "/xinfu/code/few_shot_learning/fewshot_ensemble/ensembles/train.py", line 62, in main
for i, sample in enumerate(tqdm(train_sampler)):
File "/opt/conda/lib/python3.6/site-packages/tqdm/_tqdm.py", line 931, in iter
for obj in iterable:
File "/opt/conda/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 339, in next
return self._process_next_batch(batch)
File "/opt/conda/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 360, in _process_next_batch
raise batch.exc_type(batch.exc_msg)
ValueError: Traceback (most recent call last):
File "/opt/conda/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 113, in _worker_loop
samples = collate_fn([dataset[i] for i in batch_indices])
File "/opt/conda/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 113, in
samples = collate_fn([dataset[i] for i in batch_indices])
File "/xinfu/code/few_shot_learning/fewshot_ensemble/data/datasets.py", line 60, in getitem
img = self.transform(img)
File "/opt/conda/lib/python3.6/site-packages/torchvision/transforms/transforms.py", line 49, in call
img = t(img)
File "/opt/conda/lib/python3.6/site-packages/torchvision/transforms/transforms.py", line 421, in call
i, j, h, w = self.get_params(img, self.size)
File "/opt/conda/lib/python3.6/site-packages/torchvision/transforms/transforms.py", line 399, in get_params
i = random.randint(0, h - th)
File "/opt/conda/lib/python3.6/random.py", line 221, in randint
return self.randrange(a, b+1)
File "/opt/conda/lib/python3.6/random.py", line 199, in randrange
raise ValueError("empty range for randrange() (%d,%d, %d)" % (istart, istop, width))
ValueError: empty range for randrange() (0,-139, -139)

The error occurrs while running the program. Would you please tell me the reason?

The issue as shown in the title, Looking forward to hearing from you!

My CPU RAM (not GPU) usage is increasing continuously as training progresses. When training on mini-imagenet it starts from 5 gb and continues to increase beyond 16 gb which is the max. RAM that I have hence system crashes.

Is this the expected behavior (increased RAM utilization after every iteration?). If so, how can I fix the issue?