Hey there!

I'm interested in using this model for my own project, but I've run into a small problem during training for both the example/Shapenet dataset and my own dataset.

Example data / ShapeNet data

So I've tried to run the example part segmentation on the ShapeNet data to see if it would even run on my system.
However, during training it returns a RunTimeError. If I remember correctly, the dimensions it returned were the following:

Traceback (most recent call last):
  File "/home/~/VNN/train_partseg.py", line 321, in <module>
    main(args)
  File "/home/~/VNN/train_partseg.py", line 205, in main
    seg_pred, trans_feat = classifier(points, to_categorical(label, num_classes))
  File "/home/~/.conda/envs/VNN/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/~/VNN/models/vn_dgcnn_partseg.py", line 86, in forward
    l = self.conv7(l)
  File "/home/~/.conda/envs/VNN/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/~/.conda/envs/VNN/lib/python3.9/site-packages/torch/nn/modules/container.py", line 141, in forward
    input = module(input)
  File "/home/~/.conda/envs/VNN/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/~/.conda/envs/VNN/lib/python3.9/site-packages/torch/nn/modules/conv.py", line 302, in forward
    return self._conv_forward(input, self.weight, self.bias)
  File "/home/~/.conda/envs/VNN/lib/python3.9/site-packages/torch/nn/modules/conv.py", line 298, in _conv_forward
    return F.conv1d(input, weight, bias, self.stride,
RuntimeError: Given groups=1, weight of size [90, 45, 3, 2], expected input[1, 15, 16, 32] to have 45 channels, but got 15 channels instead

Is this a known error for the example set?

Kind regards

Hi

The invariant layer defined in chapter 3.5 of your paper (specifically equations 14 and 15) appears to be implemented in the VNStdFeature module defined in vn_layers.py and it is used by the VN-PointNetEncoder in l.92 of vn_pointnet.py. What I do not understand, is that in equation 15 of the paper you write V_n*T_n.transpose(), however, in the code it appears to me that you instead compute [V_n, V_n.mean()]*T_n.transpose() (l.191 of vn_pointnet.py). This is because in the code, x (which is the input to the module VNStdFeature formed in l.91 of vn_pointnet.py) is the concatenated tensor [V_n, V_n.mean()] . Also, for clarification, I understand T_n (defined in equation 14 of the paper) to be the output z0 in l.170 of vn_layers.py.

An answer as to why equation 15 doesn't appear to be aligned with the code would be be greatly appreciated. Thanks

Does this architecture work on point clouds with signals (i.e., every point is defined by its location and a scalar signal sampled at that point)?

Hi,
I have a quick question about the different outputs of pointnet and DGCNN for segmentation. It looks like for pointnet you transpose the output of the network (batchsize, number of parts, number of points) -> (batchsize, number of points, number of parts) :

Thanks for your help!

Dear @FlyingGiraffe,

Thanks for your impressive work.

I have a question about the VNMaxPool layer. You used the argmax to select the idx of the pooled features. However, the idx is not differentiable, and thus the self.map_to_dir layer will not be updated anyway.

Could you explain this a little bit? Maybe I am wrong.

Thank you very much!

Hi there!

Goal:

I'm trying to figure out how to train the VN_DGCNN on my own dataset for partial segmentation. I have just one folder (instead of 16 like in the ShapeNet dataset), i.e. one object I'd like to train this model on.

Problem:

I assume I just need to alter the dataloader script (which I did; and the train-file naturally), but it's returning an error still. If I interpret the error correctly, it expects weights for 16 objects still?

If anyone can point me into a certain direction, that'd be much appreciated!

Error:

Traceback (most recent call last): File "/home/~/VNN/train_partseg_US.py", line 320, in <module> main(args) File "/home/~/VNN/train_partseg_US.py", line 204, in main seg_pred, trans_feat = classifier(points, to_categorical(label, num_classes)) File "/home/~/.conda/envs/VNN/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl return forward_call(*input, **kwargs) File "/home/~/VNN/models/vn_dgcnn_partseg.py", line 86, in forward l = self.conv7(l) File "/home/~/.conda/envs/VNN/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl return forward_call(*input, **kwargs) File "/home/~/.conda/envs/VNN/lib/python3.9/site-packages/torch/nn/modules/container.py", line 141, in forward input = module(input) File "/home/~/.conda/envs/VNN/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1110, in _call_impl return forward_call(*input, **kwargs) File "/home/~/.conda/envs/VNN/lib/python3.9/site-packages/torch/nn/modules/conv.py", line 302, in forward return self._conv_forward(input, self.weight, self.bias) File "/home/~/.conda/envs/VNN/lib/python3.9/site-packages/torch/nn/modules/conv.py", line 298, in _conv_forward return F.conv1d(input, weight, bias, self.stride, RuntimeError: Given groups=1, weight of size [64, 16, 1], expected input[16, 1, 1] to have 16 channels, but got 1 channels instead

Hi @FlyingGiraffe,
In the maxpooling implementation, at this line:

In order to get the indices idx, are we suppose to use argmax instead of max operation?

Hi

Could you please explain the purpose of get_graph_feature_cross in l.76 of vn_pointnet.py?
It doesn't seem that this function has any equivalent in the original implementation of pointnet.

Thanks!

Hi,
Thanks for your work and codes. However, when I reproduce your code with the VN_DGCNN model and default parameters, I just got 80.2% in the I/I case and 78.8% in the z/z case. The results are much less than the accuracy reported in the paper.
Are there any details I missed? Could you help us analyze the problem or give more details for discussion?
Thanks a lot!

May I ask why the accuracy rate obtained by training the model according to the commands in the readme is only about 60%?
I trained for 250 epochs.
Training command: python train_cls.py --model vn_dgcnn_cls --rot z --log_dir vn_dgcnn/aligned/

Hi

Would it be possible for you to make the code for chapter 5.3 "Neural Implicit Reconstruction" of your paper available.

Many thanks.

Hi @FlyingGiraffe ,

Thanks for your amazing work, while I found it quite confusing in vn-pointnet that for STN you concated the features rather than multiplyiny them, which is different than the original PointNet. Can you elaborate on this part?

Hi, Thanks for your work. I am very interested in your work and would like to learn more about your experimental results from the supplementary material. Where can I get the supplementary material?

Thanks for the wonderful code!

With the SGD optimizer (initial lr=0.1) and StepLR scheduler, I achieve the result 73.4% on SO3/SO3. Moreover, I find the training process rather unstable with large fluctuation, comparing to the normal DGCNN and PointNet. Is this usual? I doubt if I run the code correctly. Could you also provide the training log or the environment?

Hi! Do you have any pre-trained checkpoint for the model?
Best