The smallNORB link is dead, can you please switch the link to some other server ?

这个 地方如果是class_caps 类型，这个地方的地方是否应该修改 为

修改代码

if w_shared:
     hw = int(B / w.size(1))
     w = w.repeat(1, hw, 1, 1, 1)
else:
    w = w.repeat(b, 1, 1, 1, 1)

原代码

    def transform_view(self, x, w, C, P, w_shared=False):
        """
            For conv_caps:
                Input:     (b*H*W, K*K*B, P*P)
                Output:    (b*H*W, K*K*B, C, P*P)
            For class_caps:
                Input:     (b, H*W*B, P*P)
                Output:    (b, H*W*B, C, P*P)
        """
        b, B, psize = x.shape
        assert psize == P*P

        x = x.view(b, B, 1, P, P)
        if w_shared:
            hw = int(B / w.size(1))
            w = w.repeat(1, hw, 1, 1, 1)

        w = w.repeat(b, 1, 1, 1, 1)
        x = x.repeat(1, 1, C, 1, 1)
        v = torch.matmul(x, w)
        v = v.view(b, B, C, P*P)
        return v

I went to the test of hinton's code https://github.com/google-research/google-research/tree/master/capsule_em
The result can achieve the level of the paper
But I tried to test smallNORB with your code, but the accuracy wasn't that high, and it still fluctuated after many Epochs
Do you have any ideas about that?
thanks !

ConvCaps的m-step成员函数，第一行代码。

`r = r* a_in.

该函数的初值 r 是形状是 (b, B, C, 1), 而 a_in 的形状是 (b, C, 1). 不能广播，直接做哈达玛积会报错的。是不是需要对 a_in unsqueeze一下？

RuntimeError: cuda runtime error (2) : out of memory at /pytorch/aten/src/THC/generic/THCStorage.cu:58

Even if I set the training batch size to be 1, it still showed out of memory. And my GPU is GTX1080Ti which is enough for it.

Traceback (most recent call last):
File "/home/hongjin/PycharmProjects/Matrix-Capsules-EM-PyTorch-master/train.py", line 223, in
main()
File "/home/hongjin/PycharmProjects/Matrix-Capsules-EM-PyTorch-master/train.py", line 210, in main
best_acc = test(test_loader, model, criterion, device)
File "/home/hongjin/PycharmProjects/Matrix-Capsules-EM-PyTorch-master/train.py", line 175, in test
output = model(data)
File "/home/hongjin/anaconda3/envs/python3.6/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/hongjin/PycharmProjects/Matrix-Capsules-EM-PyTorch-master/model/capsules.py", line 338, in forward
x = self.conv_caps1(x)
File "/home/hongjin/anaconda3/envs/python3.6/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/hongjin/PycharmProjects/Matrix-Capsules-EM-PyTorch-master/model/capsules.py", line 260, in forward
p_out, a_out = self.caps_em_routing(v, a_in, self.C, self.eps)
File "/home/hongjin/PycharmProjects/Matrix-Capsules-EM-PyTorch-master/model/capsules.py", line 182, in caps_em_routing
a_out, mu, sigma_sq = self.m_step(a_in, r, v, eps, b, B, C, psize)
File "/home/hongjin/PycharmProjects/Matrix-Capsules-EM-PyTorch-master/model/capsules.py", line 124, in m_step
sigma_sq = torch.sum(coeff * (v - mu)**2, dim=1, keepdim=True) + eps
RuntimeError: CUDA error: out of memory

Process finished with exit code 1

Thanks for this great resource! I am trying to understand the original paper and am using this code as a template.

I am not sure how EM routing is a form of GMM as described in the original paper. For example, how many Gaussians are we fitting? To what data and in what ambient dimension?

It appears that the dimension of mu is 32x4x4=CxPxP. https://github.com/YuxianMeng/Matrix-Capsules-pytorch, which you cite, has mu of size CxWxWxPxP where W is the size of the convolutional kernel. Is this correct?

https://www.kaggle.com/code/tom99763/matrix-capsules-with-em-routing?scriptVersionId=124263074

我發現使用0.001的learning rate沒辦法訓練這個模型(mnist的準確度卡在0.1), 我想知道為什麼訓練這篇paper的模型learning初始要大到0.01, 是gradient的問題嗎?

還有我有發現kernel_tile的運算產生的gradient非常大, 或許是這個問題?

I think there is an issue in the way the input tensor x is reshaped in order to extract a_in and p_in.
It seems to me that the dimensions of a_in and p_in require a permutation before applying Tensor.view().

Note that I changed the training batch size to 16, also I am using A, B, C, D = 32, 32, 32, 32.

Transformation before view:

After this line:

p_in.shape
Out[2]: torch.Size([16, 3, 3, 6, 6, 512])

View:

The view is done in the following way:

To do the view in this way, p_in.shape should be:
torch.Size([16, 6, 6, 3, 3, 512])

Do you agree? I am new to Pytorch, so I might misunderstand the way Tensor.view() works.

The paper reaches a much lower Err-rate(about 1.5%) for smallNORB, but it seems that no one here can achieve the level of state-of-art. i wonder why : )

In "add_coord(self, v, b, h, w, B, C, psize):" function,

coor = 1. * torch.arange(h) / h

always ouput zeros,and I think that should be changed as

coor = torch.arange(h).float()/h
.