yl-1993 / matrix-capsules-em-pytorch Goto Github PK
View Code? Open in Web Editor NEWA PyTorch Implementation of Matrix Capsules with EM Routing
License: MIT License
A PyTorch Implementation of Matrix Capsules with EM Routing
License: MIT License
Thanks for this great resource! I am trying to understand the original paper and am using this code as a template.
I am not sure how EM routing is a form of GMM as described in the original paper. For example, how many Gaussians are we fitting? To what data and in what ambient dimension?
It appears that the dimension of mu is 32x4x4=CxPxP. https://github.com/YuxianMeng/Matrix-Capsules-pytorch, which you cite, has mu of size CxWxWxPxP where W is the size of the convolutional kernel. Is this correct?
I think there is an issue in the way the input tensor x
is reshaped in order to extract a_in
and p_in
.
It seems to me that the dimensions of a_in
and p_in
require a permutation before applying Tensor.view()
.
Note that I changed the training batch size to 16, also I am using A, B, C, D = 32, 32, 32, 32
.
After this line:
Matrix-Capsules-EM-PyTorch/model/capsules.py
Line 253 in 9cb3fc0
p_in.shape
Out[2]: torch.Size([16, 3, 3, 6, 6, 512])
The view is done in the following way:
Matrix-Capsules-EM-PyTorch/model/capsules.py
Line 255 in 9cb3fc0
To do the view in this way, p_in.shape
should be:
torch.Size([16, 6, 6, 3, 3, 512])
Do you agree? I am new to Pytorch, so I might misunderstand the way Tensor.view() works.
The paper reaches a much lower Err-rate(about 1.5%) for smallNORB, but it seems that no one here can achieve the level of state-of-art. i wonder why : )
The smallNORB link is dead, can you please switch the link to some other server ?
In "add_coord(self, v, b, h, w, B, C, psize):" function,
coor = 1. * torch.arange(h) / h
always ouput zeros,and I think that should be changed as
coor = torch.arange(h).float()/h
.
I went to the test of hinton's code https://github.com/google-research/google-research/tree/master/capsule_em
The result can achieve the level of the paper
But I tried to test smallNORB with your code, but the accuracy wasn't that high, and it still fluctuated after many Epochs
Do you have any ideas about that?
thanks !
这个 地方如果是class_caps 类型,这个地方的地方是否应该修改 为
修改代码
if w_shared:
hw = int(B / w.size(1))
w = w.repeat(1, hw, 1, 1, 1)
else:
w = w.repeat(b, 1, 1, 1, 1)
原代码
def transform_view(self, x, w, C, P, w_shared=False):
"""
For conv_caps:
Input: (b*H*W, K*K*B, P*P)
Output: (b*H*W, K*K*B, C, P*P)
For class_caps:
Input: (b, H*W*B, P*P)
Output: (b, H*W*B, C, P*P)
"""
b, B, psize = x.shape
assert psize == P*P
x = x.view(b, B, 1, P, P)
if w_shared:
hw = int(B / w.size(1))
w = w.repeat(1, hw, 1, 1, 1)
w = w.repeat(b, 1, 1, 1, 1)
x = x.repeat(1, 1, C, 1, 1)
v = torch.matmul(x, w)
v = v.view(b, B, C, P*P)
return v
ConvCaps的m-step成员函数,第一行代码。
`r = r* a_in.
该函数的初值 r 是形状是 (b, B, C, 1), 而 a_in 的形状是 (b, C, 1). 不能广播,直接做哈达玛积会报错的。是不是需要对 a_in unsqueeze一下?
Even if I set the training batch size to be 1, it still showed out of memory. And my GPU is GTX1080Ti which is enough for it.
Traceback (most recent call last):
File "/home/hongjin/PycharmProjects/Matrix-Capsules-EM-PyTorch-master/train.py", line 223, in
main()
File "/home/hongjin/PycharmProjects/Matrix-Capsules-EM-PyTorch-master/train.py", line 210, in main
best_acc = test(test_loader, model, criterion, device)
File "/home/hongjin/PycharmProjects/Matrix-Capsules-EM-PyTorch-master/train.py", line 175, in test
output = model(data)
File "/home/hongjin/anaconda3/envs/python3.6/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/hongjin/PycharmProjects/Matrix-Capsules-EM-PyTorch-master/model/capsules.py", line 338, in forward
x = self.conv_caps1(x)
File "/home/hongjin/anaconda3/envs/python3.6/lib/python3.6/site-packages/torch/nn/modules/module.py", line 477, in call
result = self.forward(*input, **kwargs)
File "/home/hongjin/PycharmProjects/Matrix-Capsules-EM-PyTorch-master/model/capsules.py", line 260, in forward
p_out, a_out = self.caps_em_routing(v, a_in, self.C, self.eps)
File "/home/hongjin/PycharmProjects/Matrix-Capsules-EM-PyTorch-master/model/capsules.py", line 182, in caps_em_routing
a_out, mu, sigma_sq = self.m_step(a_in, r, v, eps, b, B, C, psize)
File "/home/hongjin/PycharmProjects/Matrix-Capsules-EM-PyTorch-master/model/capsules.py", line 124, in m_step
sigma_sq = torch.sum(coeff * (v - mu)**2, dim=1, keepdim=True) + eps
RuntimeError: CUDA error: out of memory
Process finished with exit code 1
RuntimeError: cuda runtime error (2) : out of memory at /pytorch/aten/src/THC/generic/THCStorage.cu:58
https://www.kaggle.com/code/tom99763/matrix-capsules-with-em-routing?scriptVersionId=124263074
我發現使用0.001的learning rate沒辦法訓練這個模型(mnist的準確度卡在0.1), 我想知道為什麼訓練這篇paper的模型learning初始要大到0.01, 是gradient的問題嗎?
還有我有發現kernel_tile的運算產生的gradient非常大, 或許是這個問題?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.