christianmarzahl / objectdetection Goto Github PK

Some experiments with object detection in PyTorch

Python 0.14% Jupyter Notebook 99.86%

fastai-tutorials retinanet fastai object-detection

objectdetection's Introduction

ObjectDetection

Some experiments with object detection in PyTorch and FastAi. This repo is created for educational reasons and to get a deeper understanding of RetinaNet and object detection general. If you like it, please let me know, if you find any bugs or tips for improvements also.

Install

pip install object-detection-fastai

Test: Coco Colab

Image annotation

This paper describes the EXACT-Server in-depth, EXACT enables you to annotate your data and train an object detection model with this repository. Please cite if you use this tool in your research:

Marzahl et al. EXACT: A collaboration toolset for algorithm-aided annotation of almost everything

@misc{marzahl2020exact,
    title={EXACT: A collaboration toolset for algorithm-aided annotation of almost everything},
    author={Christian Marzahl and Marc Aubreville and Christof A. Bertram and Jennifer Maier and Christian Bergler and Christine Kröger and Jörn Voigt and Robert Klopfleisch and Andreas Maier},
    year={2020},
    eprint={2004.14595},
    archivePrefix={arXiv},
    primaryClass={cs.HC}
}

Update old code

# Old imports:
from helper.object_detection_helper import *
from loss.RetinaNetFocalLoss import RetinaNetFocalLoss
from models.RetinaNet import RetinaNet
from callbacks.callbacks import BBLossMetrics, BBMetrics, PascalVOCMetric

# New imports
from object_detection_fastai.helper.object_detection_helper import *
from object_detection_fastai.loss.RetinaNetFocalLoss import RetinaNetFocalLoss
from object_detection_fastai.models.RetinaNet import RetinaNet
from object_detection_fastai.callbacks.callbacks import BBLossMetrics, BBMetrics, PascalVOCMetric

The basline for this notebook was created by Sylvain Gugger from FastAi DevNotebook. Thank you very much, it was a great starting point and I'm a big fan off your work.

Publications using this code:

[x] Deep Learning-Based Quantification of Pulmonary Hemosiderophages in Cytology Slides

Examples:

Results:

Features:

[x] Coco Metric at train time via callback [x] Flexibility

# use the feature map sizes 32,18,8,4 with 32 channels and two conv layers for detection and classification
RetinaNet(encoder, n_classes=data.train_ds.c, n_anchors=18, sizes=[32,16,8,4], chs=32, final_bias=-4., n_conv=2)
'''
  (classifier): Sequential(
    (0): Sequential(
      (0): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (1): ReLU()
    )
    (1): Sequential(
      (0): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (1): ReLU()
    )
    (2): Conv2d(32, 18, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
  )

# use the feature map sizes 32 with 8 channels and three conv layers for detection and classification
RetinaNet(encoder, n_classes=data.train_ds.c, n_anchors=3, sizes=[32], chs=8, final_bias=-4., n_conv=3)

[x] Debug anchor matches for training.

On the left image we see objects that are represented by anchors. On the right objects with no corresponding anchors for training. The size of the smallest anchors should be further decreased to match the small objects on the right image.

objectdetection's People

Contributors

Stargazers

Watchers

objectdetection's Issues

ModuleNotFoundError: No module named 'BoundingBox'

Thanks for sharing this code , i'm going through your examples.

Where do i get BoundingBox and the others from ?


from BoundingBox import BoundingBox
from BoundingBoxes import BoundingBoxes
from Evaluator import *

ModuleNotFoundError: No module named 'BoundingBox'

dependencies not found

following calls from callbacks.py are not existing:
from BoundingBox import BoundingBox
from BoundingBoxes import BoundingBoxes
from Evaluator import *
from utils import *
Sorry if I miss something

In examples, incorrect anchors preview

From anchors you have format cthw, but matplotlib takes tlhw, so you should at first change datasize format.

def cthw2tlhw(boxes):
    tl = boxes[:,:2] - boxes[:,2:]/2
    return torch.cat([tl,boxes[:,2:]],1)

Index Error when using Transfer Learning

This is a great project which makes object detection in fastai much easier. Everything works pretty well until I use transfer learning for object detection. My code to build the learner is pretty much the same with the cocotiny_retina_net example in the project code:

I set size, bs = 512, 16 for the learner to train 8 rounds, and after that I use learn.data=data in which the new data has size, bs = 1024, 4 in it. Those are all the difference for transfer learning but when I train the model with image size of 1024 I always get:

IndexError                                Traceback (most recent call last)
<ipython-input-30-d81c6bd29d71> in <module>
----> 1 learn.lr_find()

D:\ProgramFile\anaconda\envs\ai\lib\site-packages\fastai\train.py in lr_find(learn, start_lr, end_lr, num_it, stop_div, wd)
     39     cb = LRFinder(learn, start_lr, end_lr, num_it, stop_div)
     40     epochs = int(np.ceil(num_it/len(learn.data.train_dl))) * (num_distrib() or 1)
---> 41     learn.fit(epochs, start_lr, callbacks=[cb], wd=wd)
     42 
     43 def to_fp16(learn:Learner, loss_scale:float=None, max_noskip:int=1000, dynamic:bool=True, clip:float=None,

D:\ProgramFile\anaconda\envs\ai\lib\site-packages\fastai\basic_train.py in fit(self, epochs, lr, wd, callbacks)
    198         else: self.opt.lr,self.opt.wd = lr,wd
    199         callbacks = [cb(self) for cb in self.callback_fns + listify(defaults.extra_callback_fns)] + listify(callbacks)
--> 200         fit(epochs, self, metrics=self.metrics, callbacks=self.callbacks+callbacks)
    201 
    202     def create_opt(self, lr:Floats, wd:Floats=0.)->None:

D:\ProgramFile\anaconda\envs\ai\lib\site-packages\fastai\basic_train.py in fit(epochs, learn, callbacks, metrics)
     99             for xb,yb in progress_bar(learn.data.train_dl, parent=pbar):
    100                 xb, yb = cb_handler.on_batch_begin(xb, yb)
--> 101                 loss = loss_batch(learn.model, xb, yb, learn.loss_func, learn.opt, cb_handler)
    102                 if cb_handler.on_batch_end(loss): break
    103 

D:\ProgramFile\anaconda\envs\ai\lib\site-packages\fastai\basic_train.py in loss_batch(model, xb, yb, loss_func, opt, cb_handler)
     28 
     29     if not loss_func: return to_detach(out), to_detach(yb[0])
---> 30     loss = loss_func(out, *yb)
     31 
     32     if opt is not None:

D:\ProgramFile\anaconda\envs\ai\lib\site-packages\torch\nn\modules\module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--> 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)

D:\ProgramFile\anaconda\envs\ai\lib\site-packages\object_detection_fastai\loss\RetinaNetFocalLoss.py in forward(self, output, bbox_tgts, clas_tgts)
     53         focal_loss = torch.tensor(0, dtype=torch.float32).to(clas_preds.device)
     54         for cp, bp, ct, bt in zip(clas_preds, bbox_preds, clas_tgts, bbox_tgts):
---> 55             bb, focal = self._one_loss(cp, bp, ct, bt)
     56 
     57             bb_loss += bb

D:\ProgramFile\anaconda\envs\ai\lib\site-packages\object_detection_fastai\loss\RetinaNetFocalLoss.py in _one_loss(self, clas_pred, bbox_pred, clas_tgt, bbox_tgt)
     32         bbox_mask = matches >= 0
     33         if bbox_mask.sum() != 0:
---> 34             bbox_pred = bbox_pred[bbox_mask]
     35             bbox_tgt = bbox_tgt[matches[bbox_mask]]
     36             bb_loss = self.reg_loss(bbox_pred, bbox_to_activ(bbox_tgt, self.anchors[bbox_mask]))

IndexError: The shape of the mask [24480] at index 0 does not match the shape of the indexed tensor [24192, 4] at index 0

I work on win10-cuda1.02-pytorch1.5.0-torchvision0.6 and everything works right besides this. Isn't it the same size in each batch when I transfer size, bs=512, 16 to size, bs=1024, 4? How could this Index error occur?

Dataset EIPH_WSI

Hi!

Where I can find the EIPH_WSI dataset? Thanks!

Any script to train the model ?

Hi @ChristianMarzahl

Thank you for your great work, I'm really excited to implement this model with my dataset.
I couldn't find any script for training the model. I have a kitti dataset, and i just converted into COCO, How can i train by using this model?

No module name BoundingBox

Validation scores

Thanks for the great RetinaNet examples. I'm training a model to recognize a single class (versus background) and validating on a held-out labeled dataset. I load in the new validation dataset and run:

learn.validate(data_test.valid_dl)

This returns [0.34860164, 0.774676853563362]

For a one-class approach, I'm assuming the first returned value is the Validation Loss and the 2nd is the class Average Precision. Is that correct?

Inference Error

I have a trained object detection model which I want to use for inference on 1 image at a time, using the following code:

learn = load_learner(Path(myPath), file='myModel.pkl')
img = open_image("myImage.jpg")
prediction = learn.predict(img)

However, I get the following error:

AttributeError: 'list' object has no attribute 'cpu'

AttributeError                            Traceback (most recent call last)
1 prediction = learn.predict(img)
~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/fastai/basic_train.py in predict(self, item, return_x, batch_first, with_dropout, **kwargs)
    372         batch = self.data.one_item(item)
    373         res = self.pred_batch(batch=batch, with_dropout=with_dropout)
--> 374         raw_pred,x = grab_idx(res,0,batch_first=batch_first),batch[0]
    375         norm = getattr(self.data,'norm',False)
    376         if norm:

~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/fastai/torch_core.py in grab_idx(x, i, batch_first)
    321 def grab_idx(x,i,batch_first:bool=True):
    322     "Grab the `i`-th batch in `x`, `batch_first` stating the batch dimension."
--> 323     if batch_first: return ([o[i].cpu() for o in x]   if is_listy(x) else x[i].cpu())
    324     else:           return ([o[:,i].cpu() for o in x] if is_listy(x) else x[:,i].cpu())
    325 

~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/fastai/torch_core.py in <listcomp>(.0)
    321 def grab_idx(x,i,batch_first:bool=True):
    322     "Grab the `i`-th batch in `x`, `batch_first` stating the batch dimension."
--> 323     if batch_first: return ([o[i].cpu() for o in x]   if is_listy(x) else x[i].cpu())
    324     else:           return ([o[:,i].cpu() for o in x] if is_listy(x) else x[:,i].cpu())
    325 

AttributeError: 'list' object has no attribute 'cpu'

Implement metrics and callbacks for a single labelled dataset

Hi!
I was wondering which changes should I perform in order to make this examples working in a 2 class problem (background vs label). Are RetinaLoss and PascalVOC ready to handle this kind of data?
Sorry if is quite of a naive question, I'm a bit new in this object detection world 😄

Anchors are totally wrong

I have analyzed make_anchors function and it is very hard and wrong. If you would try to pass other size than (32,32), for example(2,2) it would show you awfull results.

How to change number of classes for already trained retinanet model

I have trained my retinanet mode from object detection library, now i want to change the number of classes for the next dataset.
I have found a way in pytorch to change the classification head of retinanet to do the same, can anyone help me on how can i perform the same for retinanet.py from object-detection-fastai.

from torchvision.models.detection import retinanet_resnet50_fpn_v2, RetinaNet_ResNet50_FPN_V2_Weights
        from torchvision.models.detection.retinanet import RetinaNetHead, RetinaNetClassificationHead
        weights = RetinaNet_ResNet50_FPN_V2_Weights.DEFAULT
        model = retinanet_resnet50_fpn_v2(weights=weights, box_score_thresh=0.7)

        # replace classification layer
        out_channels = model.head.classification_head.conv[0].out_channels
        num_anchors = model.head.classification_head.num_anchors
        model.head.classification_head.num_classes = num_classes

        cls_logits = torch.nn.Conv2d(out_channels, num_anchors * num_classes, kernel_size=3, stride=1, padding=1)
        torch.nn.init.normal_(cls_logits.weight, std=0.01)  # as per pytorch code
        torch.nn.init.constant_(cls_logits.bias, -math.log((1 - 0.01) / 0.01))  # as per pytorcch code
        # assign cls head to model
        model.head.classification_head.cls_logits = cls_logits

Fastai V2

Hi @ChristianMarzahl , thank you for your work. It was helpful for me, especially anchors representations.

I adapted your code to fastai2, feel free to use it https://colab.research.google.com/drive/1ZA6yWj8wHwKUj3HT_LK3rsKYUPIkwnzZ

Once I get good results I'll update this repo https://github.com/manycoding/signatures-detection with the V2 code.

loss plot

I tried and get weird lr by finding

Could you help me explain why it is

any script to get prediction on single .scn file with coco .json annotations

Hey everyone,
Do we have a script or library functions where we can load the trained learner model (.pth ) and make it predict on a single aperio imagescope .scn image , and get the evaluation results on the same.
I have trained my previous model on MIDOG dataset and this is how I loaded the data there:

batch_size = 64

do_flip = True
flip_vert = True 
max_rotate = 90 
max_zoom = 1.1 
max_lighting = 0.2
max_warp = 0.2
p_affine = 0.75 
p_lighting = 0.75 

tfms = get_transforms(do_flip=do_flip,
                      flip_vert=flip_vert,
                      max_rotate=max_rotate,
                      max_zoom=max_zoom,
                      max_lighting=max_lighting,
                      max_warp=max_warp,
                      p_affine=p_affine,
                      p_lighting=p_lighting)

train, valid ,test = ObjectItemListSlide(train_images), ObjectItemListSlide(valid_images), ObjectItemListSlide(test_images)
item_list = ItemLists(".", train, test)
lls = item_list.label_from_func(lambda x: x.y, label_cls=SlideObjectCategoryList)
lls = lls.transform(tfms, tfm_y=True, size=patch_size)
data = lls.databunch(bs=batch_size, collate_fn=bb_pad_collate,num_workers=0).normalize()

Here the train images and valid images are just the list of SlideContainer objects and after the data has been populated using databunch we get data in following form:

ImageDataBunch;

Train: SlideLabelList (3000 items)
x: ObjectItemListSlide
Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256)
y: SlideObjectCategoryList
ImageBBox (256, 256),ImageBBox (256, 256),ImageBBox (256, 256),ImageBBox (256, 256),ImageBBox (256, 256)
Path: .;

Valid: SlideLabelList (1000 items)
x: ObjectItemListSlide
Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256),Image (3, 256, 256)
y: SlideObjectCategoryList
ImageBBox (256, 256),ImageBBox (256, 256),ImageBBox (256, 256),ImageBBox (256, 256),ImageBBox (256, 256)
Path: .;

Test: None

After loading it i have successfully trained my model (Retinanet with resnet 34 backbone) , now i just want to know how i can fit only single .scn image into the pipeline , i tried using the same procedure for .scn image but it gave me black and images in the batch .
Any kind of resource notebook, code snippet, etc would be really helpful.

Requesting object detection for fastai v2

Hey, Christian I would like to request you to kindly make any changes to the code so that it can be used for fastai v2.
Also, in the following version the images that are created in batches using databunch are dynamic in nature, i.e. each time i call data.show_batch() it shows different sets of images even though there are only 2 images used for creation and batch size is 2.
It would be really helpful if you can make it for a new fastai v2 datablock , instead of an older data bunch object, or if you could tell on how to create a fixed set of batches of images that doesn't keeps on changing.
Thanks and regards,
Harshit

Migrating ObjectDetection to FastAIv2

Hi Christian,

Have you given any consideration of migrating the fantastic work you did on this Object Detection library to FastAIv2? If you've done any work on that, I'd love to take a look at it.

Brian

size problem

Thanks for your work.

I managed to use the code with a custom made coco dataset which has images with much larger sizes roughly 5k*5k pixels

The code works with default size 256 but won't converge obviously since the size reduction is too much, all features are lost.
So I began tweaking the size and transform settings but as soon as i change the size variable beyond 512 either lr_find would fail or cuda would runs out of memory

RuntimeError: There were no tensor arguments to this function (e.g., you passed an empty list of Tensors), but no fallback function is registered for schema aten::_cat. This usually means that this function requires a non-empty list of Tensors. Available functions are [CUDATensorId, CPUTensorId, VariableTensorId]

Any ideas?

mAP seems to be [email protected] not mAP value

First, Thanks for you work, and it's really generous for you to open this code.
I have a question for AP.

GetPascalVOCMetrics in helper/Evaluator.py

# compute precision, recall and average precision
    acc_FP = np.cumsum(FP)  # len(FP) is length of valid bbox count
    acc_TP = np.cumsum(TP)
    rec = acc_TP / npos
    prec = np.divide(acc_TP, (acc_FP + acc_TP))
    # Depending on the method, call the right implementation
    if method == MethodAveragePrecision.EveryPointInterpolation:
        [ap, mpre, mrec, ii] = Evaluator.CalculateAveragePrecision(rec, prec)
    else:
        [ap, mpre, mrec, _] = Evaluator.ElevenPointInterpolatedAP(rec, prec)

this recall, precision value is only nms_score_thresh=0.3, iou_thresh=0.4, in VOC2007_mAP definition, nms_thresh should change between [0.05, 0.95, 0.1]. so PR curve is under different nms_thresh value. Related mAP work can be found in daveluo‘s notebook modified from Suggar’

so, I think this AP value should be [email protected], how do you think that?

Best Wishes!

Prediction Example

I am new to fastai and object detection in general. It would be helpful to have an example to make a prediction on just one image or a whole test set.

Error in NMS function

Hi all,

I'm getting an error when training the model in the NMS function, it shows up after a few epochs, its hard to debug.

  File "/lib/python3.6/site-packages/fastai/callback.py", line 241, in _call_and_update
    new = ifnone(getattr(cb, f'on_{cb_name}')(**self.state_dict, **kwargs), dict())
  File "/lib/python3.6/site-packages/object_detection_fastai/callbacks/callbacks.py", line 127, in on_batch_end
    to_keep = nms(bbox_pred, scores, self.nms_thresh)
  File "/lib/python3.6/site-packages/object_detection_fastai/helper/object_detection_helper.py", line 158, in nms
    idx_first = mask_keep.nonzero().min().item()
**RuntimeError: operation does not have an identity.**

when I checked the code it doesn't look like idx_first is being used anywhere, is this legacy, should it be removed ?

thanks,
atti

How to use fastai v2 for this library

Hey author,
I would like to request you to kindly share the fastaiv2 equivalent of this object detection library, as a lot of functionalaties are broken in fastai v1.
My issue at the moment is to save the databunch images that are created using databunch method.
I am not able to save them for using same image patches everytime.
It would be great if this library is compatible with the v2 of fastai.

How to save the image patches that are created from databunch method?

Hey author,
I was using Object detection code to create image patches using the WSI loader methods and datbunch method to wrap those images into a single fastai databunch object, but I am having issues saving those patches for reusage.
Can you please suggest how I can save the image patches I created using your code?
Here is the code that I used,

batch_size = 64

do_flip = True
flip_vert = True 
max_rotate = 90 
max_zoom = 1.1 
max_lighting = 0.2
max_warp = 0.2
p_affine = 0.75 
p_lighting = 0.75 

tfms = get_transforms(do_flip=do_flip,
                      flip_vert=flip_vert,
                      max_rotate=max_rotate,
                      max_zoom=max_zoom,
                      max_lighting=max_lighting,
                      max_warp=max_warp,
                      p_affine=p_affine,
                      p_lighting=p_lighting)
train, valid = ObjectItemListSlide(train_images) ,ObjectItemListSlide(valid_images)
item_list = ItemLists(".", train, valid)
lls = item_list.label_from_func(lambda x: x.y, label_cls=SlideObjectCategoryList)
lls = lls.transform(tfms, tfm_y=True, size=patch_size)
data = lls.databunch(bs=batch_size, collate_fn=bb_pad_collate,num_workers=0).normalize()

Now i tried saving the databunch using data.save but it throws ctype error.

The error:

ValueError                                Traceback (most recent call last)
<ipython-input-22-88cd1dd3e50b> in <module>
----> 1 data.save("test_data_bs8.pkl")

3 frames
/usr/local/lib/python3.7/dist-packages/fastai/basic_data.py in save(self, file)
    153             warn("Serializing the `DataBunch` only works when you created it using the data block API.")
    154             return
--> 155         try_save(self.label_list, self.path, file)
    156 
    157     def add_test(self, items:Iterator, label:Any=None, tfms=None, tfm_y=None)->None:

/usr/local/lib/python3.7/dist-packages/fastai/torch_core.py in try_save(state, path, file)
    414             #To avoid the warning that come from PyTorch about model not being checked
    415             warnings.simplefilter("ignore")
--> 416             torch.save(state, target)
    417     except OSError as e:
    418         raise Exception(f"{e}\n Can't write {path/file}. Pass an absolute writable pathlib obj `fname`.")

/usr/local/lib/python3.7/dist-packages/torch/serialization.py in save(obj, f, pickle_module, pickle_protocol, _use_new_zipfile_serialization)
    378         if _use_new_zipfile_serialization:
    379             with _open_zipfile_writer(opened_file) as opened_zipfile:
--> 380                 _save(obj, opened_zipfile, pickle_module, pickle_protocol)
    381                 return
    382         _legacy_save(obj, opened_file, pickle_module, pickle_protocol)

/usr/local/lib/python3.7/dist-packages/torch/serialization.py in _save(obj, zip_file, pickle_module, pickle_protocol)
    587     pickler = pickle_module.Pickler(data_buf, protocol=pickle_protocol)
    588     pickler.persistent_id = persistent_id
--> 589     pickler.dump(obj)
    590     data_value = data_buf.getvalue()
    591     zip_file.write_record('data.pkl', data_value, len(data_value))

ValueError: ctypes objects containing pointers cannot be pickled

It would be super helpful if you could look into this problem and suggest what can be done to save the image patches.
Thanks, and regards,
Harshit

Exception: It's not possible to apply those transforms to your dataset

I receive the following error when trying to build my dataset:
Exception: It's not possible to apply those transforms to your dataset: Expected object of scalar type Float but got scalar type Double for argument #3 'mat2' in call to _th_addmm_out

This problem has never occurred until a few hours ago. And I am using the exact code, directory and files as before.
Do you think it has occurred because of an update?

Lacking library

Hi,
There is no BoundingBox.
Which library I need install.

Thanks

Tensor with no elements

Im trying to run this notebook 'out of the box' and getting the following error:

RuntimeError Traceback (most recent call last)
in
----> 1 learn.lr_find()
2 learn.recorder.plot()

~/anaconda3/envs/fastai/lib/python3.7/site-packages/fastai/train.py in lr_find(learn, start_lr, end_lr, num_it, stop_div, wd)
39 cb = LRFinder(learn, start_lr, end_lr, num_it, stop_div)
40 epochs = int(np.ceil(num_it/len(learn.data.train_dl)))
---> 41 learn.fit(epochs, start_lr, callbacks=[cb], wd=wd)
42
43 def to_fp16(learn:Learner, loss_scale:float=None, max_noskip:int=1000, dynamic:bool=True, clip:float=None,

~/anaconda3/envs/fastai/lib/python3.7/site-packages/fastai/basic_train.py in fit(self, epochs, lr, wd, callbacks)
198 else: self.opt.lr,self.opt.wd = lr,wd
199 callbacks = [cb(self) for cb in self.callback_fns + listify(defaults.extra_callback_fns)] + listify(callbacks)
--> 200 fit(epochs, self, metrics=self.metrics, callbacks=self.callbacks+callbacks)
201
202 def create_opt(self, lr:Floats, wd:Floats=0.)->None:

~/anaconda3/envs/fastai/lib/python3.7/site-packages/fastai/basic_train.py in fit(epochs, learn, callbacks, metrics)
99 for xb,yb in progress_bar(learn.data.train_dl, parent=pbar):
100 xb, yb = cb_handler.on_batch_begin(xb, yb)
--> 101 loss = loss_batch(learn.model, xb, yb, learn.loss_func, learn.opt, cb_handler)
102 if cb_handler.on_batch_end(loss): break
103

~/anaconda3/envs/fastai/lib/python3.7/site-packages/fastai/basic_train.py in loss_batch(model, xb, yb, loss_func, opt, cb_handler)
28
29 if not loss_func: return to_detach(out), to_detach(yb[0])
---> 30 loss = loss_func(out, *yb)
31
32 if opt is not None:

~/anaconda3/envs/fastai/lib/python3.7/site-packages/torch/nn/modules/module.py in call(self, *input, **kwargs)
539 result = self._slow_forward(*input, **kwargs)
540 else:
--> 541 result = self.forward(*input, **kwargs)
542 for hook in self._forward_hooks.values():
543 hook_result = hook(self, input, result)

~/ObjectDetection/loss/RetinaNetFocalLoss.py in forward(self, output, bbox_tgts, clas_tgts)
53 focal_loss = torch.tensor(0, dtype=torch.float32).to(clas_preds.device)
54 for cp, bp, ct, bt in zip(clas_preds, bbox_preds, clas_tgts, bbox_tgts):
---> 55 bb, focal = self._one_loss(cp, bp, ct, bt)
56
57 bb_loss += bb

~/ObjectDetection/loss/RetinaNetFocalLoss.py in _one_loss(self, clas_pred, bbox_pred, clas_tgt, bbox_tgt)
28
29 def _one_loss(self, clas_pred, bbox_pred, clas_tgt, bbox_tgt):
---> 30 bbox_tgt, clas_tgt = self._unpad(bbox_tgt, clas_tgt)
31 matches = match_anchors(self.anchors, bbox_tgt)
32 bbox_mask = matches >= 0

~/ObjectDetection/loss/RetinaNetFocalLoss.py in _unpad(self, bbox_tgt, clas_tgt)
15
16 def _unpad(self, bbox_tgt, clas_tgt):
---> 17 i = torch.min(torch.nonzero(clas_tgt - self.pad_idx))
18 return tlbr2cthw(bbox_tgt[i:]), clas_tgt[i:] - 1 + self.pad_idx
19

RuntimeError: invalid argument 1: cannot perform reduction function min on tensor with no elements because the operation does not have an identity at /opt/conda/conda-bld/pytorch_1573049310284/work/aten/src/THC/generic/THCTensorMathReduce.cu:64

One loss Anchor selection for Class tgt

hi
below is the code for loss
in this one while building classification
clas_mask = matches>=0

Class mask will have 1 for even the bg position of anchors .
As before this we add 1 to matches tensor which has -1,-1,-1 for background, 0 for class and -2 for ignore.
that way we are taking in from Pred tensor even those anchors that are predicting the background.
Similar is the case fo bbox tensor.
Can you please let me know the rational behind taking into the account bg detecting tensor.

def _one_loss(self, clas_pred, bbox_pred, clas_tgt, bbox_tgt):
        
        bbox_tgt, clas_tgt = self._unpad(bbox_tgt, clas_tgt)
    
        matches = match_anchors(self.anchors, bbox_tgt)
    
        bbox_mask = matches>=0
        if bbox_mask.sum() != 0:
            
            bbox_pred = bbox_pred[bbox_mask]
     
            bbox_tgt = bbox_tgt[matches[bbox_mask]]
       
            bb_loss = self.reg_loss(bbox_pred, bbox_to_activ(bbox_tgt, self.anchors[bbox_mask]))
            
        else: 
            bb_loss = 0.
     
        
        matches.add_(1)
        clas_tgt = clas_tgt + 1
     
        
       
        clas_mask = matches>=0
  
        clas_pred = clas_pred[clas_mask]
    
        clas_tgt = torch.cat([clas_tgt.new_zeros(1).long(), clas_tgt]) #make one hong vector [0,1]
     
        clas_tgt = clas_tgt[matches[clas_mask]]
     
        return bb_loss , self._focal_loss(clas_pred, clas_tgt)/torch.clamp(bbox_mask.sum(), min=1.)

How can i get the predicted classes from my retinanet model detection over the images.

How to get the list of predicted classes for each image in my databunch, i want to map and show predictions of my retinanet model, I have tried using show_results_side_by_side. Still, it only shows the GT and prediction for some images in a batch only, is there any method in the evaluator.py of this repository where I can get the number of predictions for each class, e.g. if my dataset has 10 cycles so can I get the number of predictions where the cycle was detected and where it was detected as some other class, not the tp and fp from compute_class_ap as it gives the rough idea only, I want to display the number of overall correct and wrong predictions for each class.
If you could help me with how I can create such stats it would be really helpful.