visvig / forward-forward-algorithm Goto Github PK

Hi, thanks for your code in Conv for FF. However, there are several bugs in the code ff-cifar10-convnet.ipynb.

(1) goodness += [(h.pow(2).sum() / h.numel()).unsqueeze(0)] It's wrong. For a 1000*3*32*32 image, here the goodness is one value. But it should be 1000*1

Still, I am confused:

(2) x_direction = x / (x.norm(2, 1, keepdim=True) + 1e-4) . Do you think norm(2,1) is right? For example, for a 1D data 1000*1*784 (B*C*L), it should be norm(2,2) rather than norm(2,1).

(3) g_pos = self.forward(x_pos).pow(2).mean(dim=(1,2,3)).unsqueeze(0) why use unsqueeze(0) ? It seems to have no influence on the loss` if you do not use unsqueeze(0).