Thanks for the project.
By the way,
The fullcrf in your code can be training?

Hello

ConvCRF is an amazing work.

I have a question about the experiment on ConvCRF and CRFasRNN. In general, DenseCRF can increase MIoU by 3%, Which means that even without end-to-end jointly fine tuning, the CNN-DenseCRF system can achieve a better MIoU than the unary (CNN). But I notice that in this paper, MIoU of FCN-ResNet101-CRFasRNN is about 69, lower than the raw MIoU of its unary module FCN-ResNet101 (about 71).

What happends to DenseCRF that makes its accuracy declining so hard?

I try to run the newest code and always report errors.
Finally i find that there some little mistake in the code after multiple tests
1st
demo.py line97 should change to 'if args.cpu:'
2nd
convcrf.py line523 should delete the 'raise ValueError'
However，i don't know if i am right or not:) since i am a rookie and even did not fully understand your paper and all the codes

I cannot find the place you implement the transform function with the Potts model.

Hello, when I run the convcrf.py, arise error:
Traceback (most recent call last):
File "F:/googledownload/ConvCRF-master/convcrf/convcrf.py", line 620, in
tcrf = GaussCRF(conf, [10, 10], None).cuda()
File "F:/googledownload/ConvCRF-master/convcrf/convcrf.py", line 169, in init
final_softmax=conf['final_softmax'],
KeyError: 'final_softmax'

have you met such problem? how to solve it?

Hi Marvin,

Interesting work!

I would like to ask some questions.

1. It appears that there is a difference in the mean field iterations presented in your paper (section 3.1) compared to the original paper (also section 3.1). More specifically, in the original paper, the output of each iteration is softmax(- unary - message_passing) while in your paper it is softmax(unary + message_passing) (note the additive inversion).

2. In your implementation, I could not understand this part:

        if not i == num_iter - 1 or self.final_softmax:
                if self.conf['softmax']:
                    prediction = exp_and_normalize(prediction, dim=1)

According to the algorithm, prediction should be normalized at every iteration. In the code, though, it is not normalized for the last iteration, or when self.final_softmax = False, or when self.conf['softmax'] = False.

Could you please explain these two issues?

Thank you in advance!
Best regards.

Hi Marvin,

Great project! I am trying to utilize it in my own research and went through your convcrf.py code. I have a doubt around this line where if I am not wrong, if default_conf["logsoftmax"] is True you take softmax followed by log using the log_softmax function (docs) but if it's False you take the log followed by the softmax. Can you please clarify on this?

Thanks

At this line : if self.conf['softmax'] and False: .
The condition is always false, is a bug?

same input as example, but got this error:

convcrf/convcrf.py", line 598, in inference
    prediction = (self.unary_weight - self.weight) * lg_unary + \
RuntimeError: The size of tensor a (512) must match the size of tensor b (20) at non-singleton dimension 1

the input image and unary size are:

image vs unary input size: torch.Size([1, 3, 512, 1024]) vs torch.Size([1, 20, 512, 1024])

Can you someone explain why the prediction used to compute the message at the first iteration is the output from logsoftmax and for the other iterations is softmax? In the inference function.

hi~ I'm digging in your work for weeks and it's really awesome! Now I wanna ask if I set the conv size as the input size, does it mean that the convCRF is equivalent to FullyCRF in this case?

I am currently training CONV-CRF, but I have not found the corresponding formula for these parameters in your paper? I don't know their specific meaning：
'pos_feats': {
'sdims': 3,
'compat': 3,
},
'col_feats': {
'sdims': 80,
'schan': 13, # schan depend on the input scale.
# use schan = 13 for images in [0, 255]
# for normalized images in [-0.5, 0.5] try schan = 0.1
'compat': 10,
'use_bias': True

Thanks for your code. Do you have the code for end-to-end training? I wonder how to use ConvCRF in end-to-end training.

Hi Marvin,when I just run 'python demo.py data/2007_001288_0img.png data/2007_001288_5labels.png
', the error occured as following:

TypeError: avg_pool2d(): argument 'padding' must be tuple of ints, but found element of type float at pos 1
could you help me ?thanks very much.

I haven't seen a demo for grayscale images or images having higher dimensions, so I wonder if it is implemented for images with dimension not equalling to 3. Thank you!

hi,

thanks for your good work. Is it possible to modify the code to use for text sequence labelling problems such as "named entity recognition" where CRFs are commonly used in combination with NNETs. Any pointers would be great.

Thanks

Hi, @MarvinTeichmann
Thanks for your work, I have two questions about the ConvCRF.

1. I run demo.py and got a perfect result, but how the weights of Gaussian Kernels been updated?
2. I put the ConvCRF as a module into a deep network, and train the whole network by BP for semantic segmentation, No error but the loss is so unstable and the mIoU is too low. How should I choose the training protocol？

Thanks a lot.

Great work. Thanks for your code. Do you have a plan to publish the training implementation? I really want to follow your job.

Thanks for your great work. Do you have any plan to build a tensorflow implementation in the near future?

from convcrf import convcrf

config = convcrf.default_conf
shape = [3,416,416]
nclasses = 15

crf = convcrf.GaussCRF(config, shape, nclasses)
crf.cuda()

img = torch.randn(1,3,416,416).cuda()
un = torch.randn(1,15,416,416).cuda()
un = torch.softmax(un,dim=1)

pred = crf.forward(un, img)

ERROR:
Traceback (most recent call last):

File "", line 1, in
pred = crf.forward(un, img)

File "...\anaconda3\lib\site-packages\convcrf-1.0-py3.7.egg\convcrf\convcrf.py", line 202, in forward
compats, conf['merge'])

File "...\anaconda3\lib\site-packages\convcrf-1.0-py3.7.egg\convcrf\convcrf.py", line 574, in add_pairwise_energies
pyinn=self.pyinn)

File "...\anaconda3\lib\site-packages\convcrf-1.0-py3.7.egg\convcrf\convcrf.py", line 311, in init
gaussian = self._create_convolutional_filters(feats)

File "...\anaconda3\lib\site-packages\convcrf-1.0-py3.7.egg\convcrf\convcrf.py", line 353, in _create_convolutional_filters
count_include_pad=False)

RuntimeError: non-empty 3D or 4D (batch mode) tensor expected for input

Hi, Marvin:

I notice that in your paper, you post the speed of ConvCRF with different receptive field size from 3 to 13. Here is my question, why there's no more larger conv-size than size-13 ? And is the speed posted in table.1 indicate that the time cost for just one iteration?

Hi, I'm quite interested in CRF like models in segmentation and contour detection tasks. I think the ConvCRF is suitable for contour detection much, so I link it to the end of my naive CNN model, but when I set the hyperparameters as listed in the paper, the final prediction of ConvCRF would not change in 200 epochs. While I increase the learning rate to 1e-3, the prediction would generally be the same as the unary. By the way, I just apply the default_config and change the trainable mark to True. Is there any other detail I need to add? Thx for ur great work:)

I though cRF is a post process for un-pure network prediction, why using label for CRF? what's the meaning?

Does anyone know why is it used the log on the input data (or logsoftmax) instead of just softmax?

Hi @MarvinTeichmann , thanks a lot for your sharing valuable code!

I have a question about the learning curve, from the Fig. 4b, after equipped with the CRF module (epoch 100), the Train mIoU has a large jump, but seems to stay constant after that. Does this mean the CRF module learns nothing? Or maybe the network quickly converged from epoch 100 to epoch 101? Maybe I should check how the parameters of the CRF module changed from epoch 100 to epoch 250.

Thanks for your excellent work!
I noticed convcrf is run on gpu, while fullcrf is run on cpu. Is it unfair to compare two algorithms on different platforms?

@MarvinTeichmann Thank you for your great work. Could you share any idea about how to use the ConvCRF for depth estimation? Thank you in advance.