Details of the Circular Convolution about snake HOT 45 CLOSED

Our paper has been accepted, and we will release the code soon.

from snake.

Circular convolution:

class CircConv(nn.Module):
    def __init__(self, state_dim, out_state_dim=None, n_adj=4):
        super(CircConv, self).__init__()

        self.n_adj = n_adj
        out_state_dim = state_dim if out_state_dim is None else out_state_dim
        self.fc = nn.Conv1d(state_dim, out_state_dim, kernel_size=self.n_adj*2+1)

    def forward(self, input, adj):
        input = torch.cat([input[..., -self.n_adj:], input, input[..., :self.n_adj]], dim=2)
        return self.fc(input)

from snake.

I want to implement it with https://github.com/kazuto1011/circular-conv-pytorch.

from snake.

Thanks a lot! I think the input shape is BxDxN, in which B, D, N are the batch size, feature dimension and the number of vertices respectively, right?

from snake.

Yes.

from snake.

Thanks!

from snake.

Hi, I have new questions here~

1.I wonder whether the first CircConvBlock has the skip connection?
2.Could you tell me the channels of every Conv layers and the Maxpool size?

from snake.

And it seems like that the output of last CirConv is sumed twice here, should I sum it once? Or the output(gary lines below) of every CirConv includes the skip connected tensor?

from snake.

Hi, I have new questions here~

1.I wonder whether the first CircConvBlock has the skip connection?
2.Could you tell me the channels of every Conv layers and the Maxpool size?

1. The first CircConvBlock does not have a skip connection.
2. The channel number is 128.
3. The maxpool aggregates the features of all vertices.

from snake.

And it seems like that the output of last CirConv is sumed twice here, should I sum it once? Or the output(gary lines below) of every CirConv includes the skip connected tensor?

The output of last CircConv is concatenated with the features of previous CircConvs.

from snake.

And it seems like that the output of last CirConv is sumed twice here, should I sum it once? Or the output(gary lines below) of every CirConv includes the skip connected tensor?

The output of last CircConv is concatenated with the features of previous CircConvs.

Thanks again, and I will have a try.

from snake.

@pengsida Hi~ I have 2 little questions here.

Do you sample N=128 vertices for every component box when handling multi-component objects, or the total number is 128? I found that there are many small boxes.
Another question, do you compute the loss separately for the component box when handling the multi-component objects?

from snake.

1. I sample N vertices for each component contour.
2. Yes, I compute the loss separately for each component.

from snake.

1. I sample N vertices for each component contour.
2. Yes, I compute the loss separately for each component.

Thanks for your promptly reply!

from snake.

1. I sample N vertices for each component contour.
2. Yes, I compute the loss separately for each component.

How do you treat the instances whose perimeter is less than 100? I want to drop them because there is no enough vertices to sample.

from snake.

I sample contour points using the code at #3

from snake.

I sample contour points using the code at #3

Yeah, I am using your code to sample the coarse poly. Although I can get enough points by it, but there are many points too 'precise', like [119.6212, 158] and [119.5212, 158], and when I try to get the correspond feature, they are the same. How about you?

from snake.

I did not notice this problem.
I use grid_sample to extract the features.

from snake.

I did not notice this problem.
I use grid_sample to extract the features.

Ok, then for each instance, do you upsample them to a fixed size? Like (224,224) or (512,512).

from snake.

No, I keep the original size.

from snake.

No, I keep the original size.
Hi~

It seems like you use the CenterNet to get the embedding of each vertex, so if it is true, do you fix the backbone(model released by CenterNet) and train only your deepsnake, or you train them all?

As for the dataloader, I noticed that the CenterNet convert the Pascal VOC to COCO format, did you do the same for SBD?

from snake.

1. I train the snake with CenterNet together.
2. Yes, I converted the SBD to COCO format.

from snake.

1. I train the snake with CenterNet together.
2. Yes, I converted the SBD to COCO format.

Thanks for your reply!

from snake.

1. I train the snake with CenterNet together.
2. Yes, I converted the SBD to COCO format.

Hi~ I noticed that you do inference with 3 iteration, and I want to know whether you train only 1 iteration when training?

from snake.

I train three iterations during training.

from snake.

I train three iterations during training.

Then how about the loss_iter when convergence? I mean the order of magnitude.

from snake.

About 2.

from snake.

About 2.

Thanks, now I can converge to below 10 in the first epoch, training only on instances having only single component.

from snake.

About 2.

Hi~
How much time for an single epoch training? It takes me about 3.5 hours with 8 batchsize and 2GPU.

from snake.

An epoch takes about 5 minutes with batchsize 80 and 4 gpus on Sbd dataset.

from snake.

An epoch takes about 5 minutes with batchsize 80 and 4 gpus on Sbd dataset.

Fine, how about CityScapes?
BTW, how many is the loss of iterative contour deformation when convergence?

from snake.

About 3 minutes on Cityscapes.
The loss is about 2.

from snake.

Our paper has been accepted, and we will release it soon.

Congratulations!

from snake.

I am so happy to see you releasing your code, since my code is too slow and I cannot wait to find out the reason.
👍

from snake.

@pengsida
Hi~ I want to know the performance you have released named 197.pth on cityscapes, since I tested it and got 36.8 on val dataset. Is it normal?

from snake.

Yes, it is normal.
The coco evaluator gives lower results than the cityscapes official evaluator.

# use coco evaluator
python run.py --type evaluate --cfg_file configs/city_rcnn_snake.yaml
# use the cityscapes offical evaluator
python run.py --type evaluate --cfg_file configs/city_rcnn_snake.yaml test.dataset CityscapesVal

from snake.

@pengsida
It would be nice if you could have some comments about
'inp', 'meta', 'act_hm', 'awh', 'act_ind', 'act_01', 'ct_01', 'cp_hm', 'cp_wh', 'cp_ind', 'cp_01', 'i_it_4py', 'c_it_4py', 'i_gt_4py', 'c_gt_4py', 'i_it_py', 'c_it_py', 'i_gt_py', 'c_gt_py'

Although I have known some words like 'cp' means component, etc.
: )

from snake.

I have updated the code 77a6b49

from snake.

I have updated the code 77a6b49

Thanks!

from snake.

I have updated the code 77a6b49

hi, there is no explanation about 'ct_01', and I want to know about it.

from snake.

It indicates if there is en element or not: https://github.com/zju3dv/snake/blob/master/lib/datasets/collate_batch.py#L24

from snake.

It indicates if there is en element or not: https://github.com/zju3dv/snake/blob/master/lib/datasets/collate_batch.py#L24

thank u very much.

from snake.

Congratulations for your wonderful work!
Could you emphasize the key details between the graph convolution layer of curve-gcn and the circular convolution layers? From what I understand the key difference is that graph convolution utilises fewer parameters for the same number of adjacent vetrices, a set of weights for the center node and a set of weights applied to all nodes in the neighbourhood, while circular convolution uses a different set of weights for each neighbour.

from snake.

Yes, you are right.

from snake.

thank you!

from snake.