Git Product home page Git Product logo

crfasrnn_keras's People

Contributors

kautenja avatar sadeepj avatar sirotenko avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

crfasrnn_keras's Issues

NaN value Issue

Hi, I use the crf layer as an output layer to fine tune the prediction of a contour detection CNN. However, after 1000-2000 iterations, all value is changed to nan. How can I avoid the nan value issue? Thx so much for ur help.

it can't work

when I run run_demo.py, errors output

Traceback (most recent call last):
File "run_demo.py", line 46, in
main()
File "run_demo.py", line 36, in main
model = get_crfrnn_model_def()
File "/home/zhengye/crfasrnn_keras/crfrnn_model.py", line 49, in get_crfrnn_model_def
x = Conv2D(64, (3, 3), activation='relu', padding='valid', name='conv1_1')(x)
File "build/bdist.linux-x86_64/egg/keras/engine/topology.py", line 603, in call
File "build/bdist.linux-x86_64/egg/keras/layers/convolutional.py", line 164, in call
File "build/bdist.linux-x86_64/egg/keras/backend/tensorflow_backend.py", line 3183, in conv2d
AttributeError: 'module' object has no attribute 'convolution'

An illegal memory access was encounteredan illegal memory access was encountered

Tensorflow 1.11.0
Cuda 9.0
Python 2.7.15
GTX 1060, 396 driver
Happens after run python demo. CPU version works well

an illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredCuda kernel failed. Error: an illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredCuda kernel failed. Error: an illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredCuda kernel failed. Error: an illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredCuda kernel failed. Error: an illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredCuda kernel failed. Error: an illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredCuda kernel failed. Error: an illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredCuda kernel failed. Error: an illegal memory access was encounteredCuda kernel failed. Error: an illegal memory access was encounteredCuda kernel failed. Error: an illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredCuda kernel failed. Error: an illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredCuda kernel failed. Error: an illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredCuda kernel failed. Error: an illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredCuda kernel failed. Error: an illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredCuda kernel failed. Error: an illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredCuda kernel failed. Error: an illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredan illegal memory access was encounteredCuda kernel failed. Error: an illegal memory access was encounteredan illegal memory access was encountered2018-11-06 02:40:13.266582: E tensorflow/stream_executor/cuda/cuda_dnn.cc:353] Could not create cudnn handle: CUDNN_STATUS_INTERNAL_ERROR

undefined symbol: _ZTIN10tensorflow8OpKernelE

The last step, run_demo.py, I get this error:

Traceback (most recent call last):
File "run_demo.py", line 27, in
from crfrnn_model import get_crfrnn_model_def
File "./src/crfrnn_model.py", line 28, in
from crfrnn_layer import CrfRnnLayer
File "./src/crfrnn_layer.py", line 28, in
import high_dim_filter_loader
File "./src/high_dim_filter_loader.py", line 28, in
custom_module = tf.load_op_library(os.path.join(os.path.dirname(file), 'cpp', 'high_dim_filter.so'))
File "/home/fu/.local/share/virtualenvs/my_pipenv-ITNpTMim/lib/python3.5/site-packages/tensorflow/python/framework/load_library.py", line 60, in load_op_library
lib_handle = py_tf.TF_LoadLibrary(library_filename)
tensorflow.python.framework.errors_impl.NotFoundError: ./src/cpp/high_dim_filter.so: undefined symbol: _ZTIN10tensorflow8OpKernelE

What can we do?

About the Batchsize

I notice the README showing the batchsize must equal one,
But I write a simple training code about classification problem , just applying Keras API with batchsize==5.
model.fit(data, labels, epochs=10, batch_size=5)
And there is no error when running it. May there be some latent incorrectness?

Could the ‘crfrnn’ layer be used on gray image with two classes?

I am trying to use fully convolutional network on medical image segmentation these days.Our datasets are all gray images,and we need divide them into two classes. In the last layer of our network ,we use ‘sigmoid’ activation to get the final shape ‘[height,width,1]’.when we directly added the 'crfrnn' layer into our network as the new last layer,we got wrong results.The program ran without any error. It seems the value of results don't range from 0 to 1. Could the ‘crfrnn’ layer be used on gray image with two classes?
if yes,how to modify the code? Look forward to your reply.

about training

@sadeepj Hi, thx at first. I wonder is it possible to train a CRF-RNN model using the compiled "high_dim_filter.so" in Tensorflow/Keras environment? I mean is backprop part also included in "high_dim_filter.so"?Thx.

Submitting operation to Tensorflow

Hi!

Great work! More of a comment than an issue. Are you planning on submitting the operations for the CRFs to Tensorflow contrib? There is already a tensorflow.contrib.crf so perhaps this could go there?

Given CRFs are common in segmentation, I doubt you would have troubles convincing them to add the functionality.

about training code

Hi, @sadeepj ,

Thanks for releasing TF/Keras version for crfasrnn. However it seems the training code for the model is not provided, when will it be released?

Thanks!

The 'crfrnn_layer' layer is used in the last layer of the network. When the accuracy of the network reaches about 0.2, the loss function returns nan

Thank you very much for your code. But I use the 'crfrnn_layer' layer in the last layer of my network, when training my network, when the network accuracy rate of 0.2 or so, the loss function will return to nan, my data with RGB pictures or TIFF will If this happens, please help solve this problem. Looking forward to your reply, thank you very much.
The following is the training process:
81/2560 [...........] - ETA: 948s - loss: 6.5810 - binary_crossentropy: 0.5113 - accuracy: 0.1502
82/2560 [...........] - ETA: 944s - loss: 6.4899 - binary_crossentropy: 0.5118 - accuracy: 0.1719
83/2560 [...........] - ETA: 941s - loss: 6.4444 - binary_crossentropy: 0.5107 - accuracy: 0.1739
84/2560 [...........] - ETA: 940s - loss: 6.3637 - binary_crossentropy: 0.5116 - accuracy: 0.1921
85/2560 [...........] - ETA: 937s - loss: nan - binary_crossentropy: 0.5131 - accuracy: 0.1473
86/2560 [>..........] - ETA: 935s - loss: nan - binary_crossentropy: 0.5137 - accuracy: 0.1514

undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringEv

Hi,
I encountered the above notfoundError while importing the crfasrnn layer. It was throwing the error while importing the following file high_dim_filter.so.

It got fixed while I compiled the makefile using D_GLIBCXX_USE_CXX11_ABI=1 instead of zero, in the following line.

Flags required for all cases

CFLAGS := -std=c++11 -D_GLIBCXX_USE_CXX11_ABI=0 -shared -fPIC -I$(TF_INC) -O2

edit : I'm using tf 1.10 in conda.

Training code available now?

I want to use the CRF layer in my own work which class number is 2. I would appreciate if you share your training code. Thank you!

Running error

I meet the error "tensorflow.python.framwork.errors_impl.NotFoundError: libtensorflow_framework.so: cannot open shared object file : No such file or directory". I hope you have time to help solve it.

How can I add your code behind my model

Hello, I have run your model successful. 
Because I want your model to modify my classify result, could you please tell me how can I add your code behind my model? 
Such what is the shape of input and label, how can I save my own weight of model?
My input shape is (512,512, 3) , it has five classes(0~5).

Question on improvement for specific image

Hello I am working on a master project and I am interested in applying the solution into static pictures of a plant.
The pretrained model has already very good results but I am wondering what are some possible ways to improve it. I am wondering if there are some kind of techniques to improve it further. A second issue comes with the different lighting during the day that changes the plant colors.
Thank you in advance for your time.

image
labels

Adding an additional pair-wise potential

Hello Sadeep,

First all thank for your work, very helpful. I am trying to improve the semantic segmentation results by adding additional potentials in the crf energy function but I don't know if I am doing it right. I am adding these lines in the code to add a potential for the depth :

        # Weights of the bilateral kernel
        self.bilateral_ker_weights = self.add_weight(name='bilateral_ker_weights',
                                                     shape=(self.num_classes, self.num_classes),
                                                     initializer='uniform',
                                                     trainable=True)

        # Weights of the bilateral kernel_dep
        self.bilateral_ker_weights_dep = self.add_weight(name='bilateral_ker_weights_dep',
                                                     shape=(self.num_classes, self.num_classes),
                                                     initializer='uniform',
                                                     trainable=True)

        # Compatibility matrix
        self.compatibility_matrix = self.add_weight(name='compatibility_matrix',
                                                    shape=(self.num_classes, self.num_classes),
                                                    initializer='uniform',
                                                    trainable=True)

        super(CrfRnnLayer, self).build(input_shape)

    def call(self, inputs):
        unaries = tf.transpose(inputs[0][0, :, :, :], perm=(2, 0, 1))
        rgb = tf.transpose(inputs[1][0, :, :, :], perm=(2, 0, 1))

        dep = tf.transpose(inputs[2][0, :, :, :], perm=(2, 0, 1))


        c, h, w = self.num_classes, self.image_dims[0], self.image_dims[1]
        all_ones = np.ones((c, h, w), dtype=np.float32)

        # Prepare filter normalization coefficients
        spatial_norm_vals = custom_module.high_dim_filter(all_ones, rgb, bilateral=False,
                                                          theta_gamma=self.theta_gamma)

        bilateral_norm_vals = custom_module.high_dim_filter(all_ones, rgb, bilateral=True,
                                                            theta_alpha=self.theta_alpha,
                                                            theta_beta=self.theta_beta)

        bilateral_norm_vals_dep = custom_module.high_dim_filter(all_ones, dep, bilateral=True,
                                                            theta_alpha=self.theta_alpha,
                                                            theta_beta=self.theta_beta)

        q_values = unaries

        for i in range(self.num_iterations):
            softmax_out = tf.nn.softmax(q_values, dim=0)

            # Spatial filtering
            spatial_out = custom_module.high_dim_filter(softmax_out, rgb, bilateral=False,
                                                        theta_gamma=self.theta_gamma)

            spatial_out = spatial_out / spatial_norm_vals

            # Bilateral filtering
            bilateral_out = custom_module.high_dim_filter(softmax_out, rgb, bilateral=True,
                                                          theta_alpha=self.theta_alpha,
                                                          theta_beta=self.theta_beta)

            bilateral_out = bilateral_out / bilateral_norm_vals

            # Bilateral filtering_dep
            bilateral_out_dep = custom_module.high_dim_filter(softmax_out, dep, bilateral=True,
                                                          theta_alpha=self.theta_alpha,
                                                          theta_beta=self.theta_beta)

            bilateral_out_dep = bilateral_out_dep / bilateral_norm_vals_dep

            # Weighting filter outputs
            message_passing = (tf.matmul(self.spatial_ker_weights,tf.reshape(spatial_out, (c, -1))) + tf.matmul(self.bilateral_ker_weights,tf.reshape(bilateral_out, (c, -1))) + tf.matmul(self.bilateral_ker_weights_dep,tf.reshape(bilateral_out_dep, (c, -1))) )

Could you please tell me if I am doing it right ?

Regards.

nsync_cv.h No such a file or directory

I use anaconda with python 2.7 tensorflow 1.4 and when I compile the file compile.sh there is an error about

In file included from /home/zxxdls/.local/lib/python2.7/site-packages/tensorflow/include/tensorflow/core/platform/mutex.h:31:0,
from /home/zxxdls/.local/lib/python2.7/site-packages/tensorflow/include/tensorflow/core/framework/op.h:32,
from high_dim_filter.cc:25:
/home/zxxdls/.local/lib/python2.7/site-packages/tensorflow/include/tensorflow/core/platform/default/mutex.h:25:22: fatal error: nsync_cv.h: No such a file or directory
compilation terminated.
In file included from /home/zxxdls/.local/lib/python2.7/site-packages/tensorflow/include/tensorflow/core/platform/mutex.h:31:0,
from /home/zxxdls/.local/lib/python2.7/site-packages/tensorflow/include/tensorflow/core/framework/variant.h:31,
from /home/zxxdls/.local/lib/python2.7/site-packages/tensorflow/include/tensorflow/core/framework/allocator.h:26,
from /home/zxxdls/.local/lib/python2.7/site-packages/tensorflow/include/tensorflow/core/framework/tensor.h:20,
from modified_permutohedral.h:42,
from modified_permutohedral.cc:33:
/home/zxxdls/.local/lib/python2.7/site-packages/tensorflow/include/tensorflow/core/platform/default/mutex.h:25:22: fatal error: nsync_cv.h: No such a file or directory
It seems that there is an error about building a custom op in tf 1.4
I want to know how to solve it or I want to know what is the corresponding version of tf in this project
Thanks very much

Compared to pydensecrf, crfasrnn seems do not have two `compat` parameters.

For example,

d.addPairwiseGaussian(sxy=3, compat=3)
d.addPairwiseBilateral(sxy=80, srgb=13, rgbim=im, compat=10)

Here theta_alpha is 80, theta_beta is 13 and theta_gamma is 3 which corresponds to the 3 parameters in crfrnn network. However, the two compat parameters seems not available in crfrnn model.

Can you share your gpu verson?

Thanks very much for your code,its really a good job! When I use your code to train my datasets on tensorflow/keras, it's really too slow.And I realize its reason that can only run on cpu! So can you give us your GPU verson? I will appreciate it! @sadeepj @sirotenko

compile file

Hello,
in windows os , how compile file?
Thanks.

"undefined symbol" error when running the demo with a TensorFlow source installation

I tried to run the demo (run_demo.py) following the instructions provided in the README, however I incurred in the following error:

Traceback (most recent call last):
  File "run_demo.py", line 25, in <module>
    from crfrnn_model import get_crfrnn_model_def
  File "/home/massimo/repositories/crfasrnn_keras/crfrnn_model.py", line 28, in <module>
    from crfrnn_layer import CrfRnnLayer
  File "/home/massimo/repositories/crfasrnn_keras/crfrnn_layer.py", line 28, in <module>
    custom_module = tf.load_op_library('./cpp/high_dim_filter.so')
  File "/home/massimo/.local/lib/python3.5/site-packages/tensorflow/python/framework/load_library.py", line 56, in load_op_library
    lib_handle = py_tf.TF_LoadLibrary(library_filename, status)
  File "/usr/lib/python3.5/contextlib.py", line 66, in __exit__
    next(self.gen)
  File "/home/massimo/.local/lib/python3.5/site-packages/tensorflow/python/framework/errors_impl.py", line 467, in raise_exception_on_not_ok_status
    c_api.TF_GetCode(status.status))
tensorflow.python.framework.errors_impl.NotFoundError: ./cpp/high_dim_filter.so: undefined symbol: _ZTIN10tensorflow8OpKernelE

Initially (i.e. right after cloning the repository) the compilation was failing because it couldn't find neither the entire TensorFlow nor nsync_cv.h.
So I edited compile.sh as follows:

TF_INC=$(python3 -c 'import tensorflow as tf; print(tf.sysconfig.get_include())')
NSYNC_INC=$TF_INC/external/nsync/public

g++ -std=c++11 -D_GLIBCXX_USE_CXX11_ABI=0 -shared high_dim_filter.cc modified_permutohedral.cc -o high_dim_filter.so -fPIC -I $TF_INC -I $NSYNC_INC -O2

Specifically, I changed the first line to use Python3 (I have TensorFlow and everything else working under Python3) and included the nsync path.
With this modification the compilation succeeded and high_dim_filter.so was generated without any error/warning being issued.

Then, as per instructions, I downloaded the pre-trained model weights crfrnn_keras_model.h5 and finally tried to run python3 run_demo.py, which failed with the above error message.

I googled the error message, but others seemed to solve similar issues with the -D_GLIBCXX_USE_CXX11_ABI=0 option that is already in place.

Relevant system information:

  • Ubuntu 17.04
  • Python 3.5.3
  • TensorFlow 1.3.0 (compiled from source)
  • Keras 2.0.8
  • g++ 6.3.0 (I also tried with version 5.4.1, but I got the same error)
  • CUDA 9.0
  • cuDNN 7.0.2

Please let me know if I need to provide any other info to help you replicate the issue.

Thanks in advance!

TypeError: softmax() got an unexpected keyword argument 'axis'

Hello!
I am trying your tensorflow version code(crfasrnn_keras), but I met an error,the following contents are the errors:

(tensorflow) bruce@ubuntu:~/master_thesis/crfasrnn_keras$ python3 run_demo.py
/home/bruce/anaconda3/lib/python3.6/site-packages/h5py/init.py:36: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type.
from ._conv import register_converters as _register_converters
Using TensorFlow backend.
Traceback (most recent call last):
File "run_demo.py", line 48, in
main()
File "run_demo.py", line 38, in main
model = get_crfrnn_model_def()
File "./src/crfrnn_model.py", line 111, in get_crfrnn_model_def
name='crfrnn')([upscore, img_input])
File "/home/bruce/anaconda3/lib/python3.6/site-packages/keras/engine/topology.py", line 619, in call
output = self.call(inputs, **kwargs)
File "./src/crfrnn_layer.py", line 91, in call
softmax_out = tf.nn.softmax(q_values, axis=0)
TypeError: softmax() got an unexpected keyword argument 'axis'

and my computer configuration are:
ubuntu 16.04 on VMware player 14, host system is win 10,CPU core I5

Thank you very much for your answer!

Bruce

Compile Issue with high_dim_filter.so On Nvidia TX1: modified_permutohedral.cc error: 'feature' was not declared in this scope

Hi. I ran into a modified_permutohedral.cc compile issue with Nvidia TX1 (Ubuntu 16). I am running Tensorflow 1.5 and Keras 2.2.2. I am compiling the gpu_version (USE_GPU := 1) with CUDA 8.


modified_permutohedral.cc:359:22: error: 'feature' was not declared in this scope
const float * f = (feature + k * num_dimensions);
^
modified_permutohedral.cc: In member function 'void ModifiedPermutohedral::sseCompute(tensorflow::Tensor&, const tensorflow::Tensor&, int, bool, bool) const':
modified_permutohedral.cc:622:51: error: 'seqCompute_cpu' was not declared in this scope
seqCompute_cpu( out, in, value_size, reverse, add);


(cv40py35) nvidia@tegra-ubuntu:~/cviz/crfasrnn_keras/src/cpp$ make
nvcc -std=c++11 -c -o cudacode.o high_dim_filter.cu -I/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include -I/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/external/nsync/public -D_GLIBCXX_USE_CXX11_ABI=1 -x cu -Xcompiler -fPIC --expt-relaxed-constexpr -D FILTER_GPU=1
nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/stubs/atomicops_internals_arm64_gcc.h(56): warning: variable "temp" was set but never used

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/stubs/atomicops_internals_arm64_gcc.h(80): warning: variable "temp" was set but never used

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/stubs/atomicops_internals_arm64_gcc.h(100): warning: variable "temp" was set but never used

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/stubs/atomicops_internals_arm64_gcc.h(192): warning: variable "temp" was set but never used

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/stubs/atomicops_internals_arm64_gcc.h(216): warning: variable "temp" was set but never used

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/stubs/atomicops_internals_arm64_gcc.h(236): warning: variable "temp" was set but never used

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(52): warning: integer conversion resulted in a change of sign

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(147): warning: integer conversion resulted in a change of sign

nvcc -std=c++11 -c -o modified_permutohedral.o modified_permutohedral.cu -I/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include -I/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/external/nsync/public -D_GLIBCXX_USE_CXX11_ABI=1 -x cu -Xcompiler -fPIC --expt-relaxed-constexpr -D FILTER_GPU=1
nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/stubs/atomicops_internals_arm64_gcc.h(56): warning: variable "temp" was set but never used

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/stubs/atomicops_internals_arm64_gcc.h(80): warning: variable "temp" was set but never used

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/stubs/atomicops_internals_arm64_gcc.h(100): warning: variable "temp" was set but never used

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/stubs/atomicops_internals_arm64_gcc.h(192): warning: variable "temp" was set but never used

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/stubs/atomicops_internals_arm64_gcc.h(216): warning: variable "temp" was set but never used

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/stubs/atomicops_internals_arm64_gcc.h(236): warning: variable "temp" was set but never used

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(52): warning: integer conversion resulted in a change of sign

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(147): warning: integer conversion resulted in a change of sign

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/stubs/atomicops_internals_arm64_gcc.h(56): warning: variable "temp" was set but never used

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/stubs/atomicops_internals_arm64_gcc.h(80): warning: variable "temp" was set but never used

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/stubs/atomicops_internals_arm64_gcc.h(100): warning: variable "temp" was set but never used

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/stubs/atomicops_internals_arm64_gcc.h(192): warning: variable "temp" was set but never used

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/stubs/atomicops_internals_arm64_gcc.h(216): warning: variable "temp" was set but never used

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/stubs/atomicops_internals_arm64_gcc.h(236): warning: variable "temp" was set but never used

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(52): warning: integer conversion resulted in a change of sign

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(147): warning: integer conversion resulted in a change of sign

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/generated_message_reflection.h(689): warning: variable "unused" was set but never used

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/stubs/atomicops_internals_arm64_gcc.h(56): warning: variable "temp" was set but never used

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/stubs/atomicops_internals_arm64_gcc.h(80): warning: variable "temp" was set but never used

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/stubs/atomicops_internals_arm64_gcc.h(100): warning: variable "temp" was set but never used

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/stubs/atomicops_internals_arm64_gcc.h(192): warning: variable "temp" was set but never used

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/stubs/atomicops_internals_arm64_gcc.h(216): warning: variable "temp" was set but never used

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/stubs/atomicops_internals_arm64_gcc.h(236): warning: variable "temp" was set but never used

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(52): warning: integer conversion resulted in a change of sign

/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/google/protobuf/arena_impl.h(147): warning: integer conversion resulted in a change of sign

g++ -std=c++11 -D_GLIBCXX_USE_CXX11_ABI=0 -shared -fPIC -I/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include -O2 -I/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow/include/external/nsync/public -o high_dim_filter.so high_dim_filter.cc modified_permutohedral.cc cudacode.o modified_permutohedral.o -L/home/nvidia/.virtualenvs/cv40py35/lib/python3.5/site-packages/tensorflow -ltensorflow_framework -L/usr/local/cuda/lib64 -lcuda -lcudart -D FILTER_GPU=1
modified_permutohedral.cc: In member function 'void ModifiedPermutohedral::init_cpu(const float*, int, int)':
modified_permutohedral.cc:359:22: error: 'feature' was not declared in this scope
const float * f = (feature + k * num_dimensions);
^
modified_permutohedral.cc: In member function 'void ModifiedPermutohedral::sseCompute(tensorflow::Tensor&, const tensorflow::Tensor&, int, bool, bool) const':
modified_permutohedral.cc:622:51: error: 'seqCompute_cpu' was not declared in this scope
seqCompute_cpu( out, in, value_size, reverse, add);
^
Makefile:50: recipe for target 'high_dim_filter.so' failed
make: *** [high_dim_filter.so] Error 1

can't download the model weights

Hello and I really appreciate your work, but there is a problem. I clicked the "here"button wishing to download the model weights, but it didn't work, could you please solve this?

Not Found Error Is Back..

Hi there, I am using tensorflow 1.6 as 1.4 was not working so I just upgraded it.
Running demo gives me this following error... Appreciate for your help!

(ds3.6) ~/D/u/8/crf-rnn ❯❯❯ python run_demo.py                                 master ◼
/Users/lucius/miniconda3/envs/ds3.6/lib/python3.6/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
Using TensorFlow backend.
Traceback (most recent call last):
  File "run_demo.py", line 27, in <module>
    from crfrnn_model import get_crfrnn_model_def
  File "./src/crfrnn_model.py", line 28, in <module>
    from crfrnn_layer import CrfRnnLayer
  File "./src/crfrnn_layer.py", line 28, in <module>
    import high_dim_filter_loader
  File "./src/high_dim_filter_loader.py", line 28, in <module>
    custom_module = tf.load_op_library(os.path.join(os.path.dirname(__file__), 'cpp', 'high_dim_filter.so'))
  File "/Users/lucius/miniconda3/envs/ds3.6/lib/python3.6/site-packages/tensorflow/python/framework/load_library.py", line 58, in load_op_library
    lib_handle = py_tf.TF_LoadLibrary(library_filename, status)
  File "/Users/lucius/miniconda3/envs/ds3.6/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 516, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: dlopen(./src/cpp/high_dim_filter.so, 6): Symbol not found: __ZN10tensorflow15OpKernelContext21CtxFailureWithWarningENS_6StatusE
  Referenced from: ./src/cpp/high_dim_filter.so
  Expected in: /Users/lucius/miniconda3/envs/ds3.6/lib/python3.6/site-packages/tensorflow/python/../libtensorflow_framework.so
 in ./src/cpp/high_dim_filter.so

Compiling high_dim_filter.so on Raspbian

Hi all,

After successfully compiling and running the demo, I am trying to compile the custom C++ code on a Raspberry running Raspbian 9 (Stretch). I'm running the regular ./compile.sh script, however I'm getting the following error:

$ ./compile.sh
In file included from /home/pi/.local/lib/python3.5/site-packages/tensorflow/include/tensorflow/core/framework/attr_value_util.h:23:0, from /home/pi/.local/lib/python3.5/site-packages/tensorflow/include/tensorflow/core/framework/node_def_util.h:23, from /home/pi/.local/lib/python3.5/site-packages/tensorflow/include/tensorflow/core/framework/shape_inference.h:21, from high_dim_filter.cc:26: /home/pi/.local/lib/python3.5/site-packages/tensorflow/include/tensorflow/core/framework/tensor.h: In member function ‘void tensorflow::Tensor::FillDimsAndValidateCompatibleShape(Eigen::array<int, NDIMS>*, tensorflow::gtl::ArraySlice<long long int>) const [with unsigned int NDIMS = 1u]’: /home/pi/.local/lib/python3.5/site-packages/tensorflow/include/tensorflow/core/framework/tensor.h:574:6: note: parameter passing for argument of type ‘tensorflow::gtl::ArraySlice<long long int>’ will change in GCC 7.1 void Tensor::FillDimsAndValidateCompatibleShape( ^~~~~~ modified_permutohedral.cc: In member function ‘void ModifiedPermutohedral::init(const float*, int, int)’: modified_permutohedral.cc:362:22: error: ‘feature’ was not declared in this scope const float * f = (feature + k * num_dimensions);

I tried searching for solutions at the Tensorflow page you provided, tried out some their python code, but the same error popped up. Do you got any ideas on how to successfully compile the code on Raspbian? I would love to hear from you!

Best,

Boris.

Setup instructions

I ran into a problem with my numpy version (numpy was already installed):

Found existing installation: numpy 1.8.0rc1
Cannot uninstall 'numpy'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.

I fixed it with:

sudo pip install -r requirements.txt --ignore-installed numpy

demo error

Hi,I have follow all the step, and run the demo, the error is
./src/cpp/high_dim_filter.so: undefined symbol: _ZN10tensorflow8internal21CheckOpMessageBuilder9NewStringEv
I have change the makefie following this

------------------------------------------------------------------------------------------------------------i----------

Instructions:

1. Activate your Tensorflow virtualenv before running this script.

2. This script assumes gcc version >=5. If you have an older version, remove the -D_GLIBCXX_USE_CXX11_ABI=0 flag below.

3. On Mac OS X, the additional flag "-undefined dynamic_lookup" is required.

4. If this script fails, please refer to https://www.tensorflow.org/extend/adding_an_op#build_the_op_library for help.

-----------------------------------------------------------------------------------------------------------------------

TF_INC=$(python -c 'import tensorflow as tf; print(tf.sysconfig.get_include())')
TF_LIB=$(python -c 'import tensorflow as tf; print(tf.sysconfig.get_lib())')
But it still has the same problem

Relevant system information:

Ubuntu 17.10
Python 3.5
TensorFlow 1.6.0 
Keras 2.1.5

Using the CRF layer with some other segmentation network

Hi,
I wish to use the CRF layer with my own segmentation network which is an encoder-decoder network with skip connections. If I use the crf layer as the last layer what kind of things I should keep in mind?
The output of my model has number of channels equal to number of classes (2 in my case) and activation is sigmoid. I connect this layer directly with the CRF layer without any modification to CRF parameters like theta_alpha, theta_beta and theta_gamma. The final loss function that I am using is binary cross entropy. Output of the CRF is being directly fed to the loss function. At the time of inference I am taking argmax for getting the labels. But the output label for every pixel is mostly the same.
Can anyone please help me to understand that what are the things that are particular to the FCN output such that the CRF layer works with it?

Also, If you could please share the training script?

Alternative repository

Just wanted to let you know that I developed an alternative repo which is more flexible in the shape and number of channels of the images (any 2D, 3D multi-channel image should work) and that has both a CPU and GPU kernel.
Please check the link if you are interested in more info.
https://github.com/MiguelMonteiro/CRFasRNNLayer

Is there something wrong with q_values' computation ?

According to the original paper,

we use Ui(l) to denote the negative of the unary energy intoduced in the previous section

since unary energy is computed as: -log(p) , where p = softmax(FCN_OUT), so the initial q_values in crfrnn_layer.py should be log(softmax(unaries)) instead of directly using unaries ? Am I right ? hoping for your response.

3D images and batch size greater than 1

As far as I can tell this code only works for 2D images.
Would expanding this code to work with 3D images (possibly N-D) require changing the C++ code or is it just a matter of rewriting some of the python and tensorflow code?
In addition is the batch size limitation a property of the systems' design or can it fixed?

Keep up the good work!

Question about the loss and pretrained FCN-8s

Hi Sadeep,

Thanks for making this code available!

I would like to ask a few questions regarding the paper.

When you plugged the CRF layer to the pretrained FCN-8s and trained end-to-end, did the (softmax) loss actually go down? How many epochs did you train the model to see that it eventually went down? I tried training end-to-end ('lr=1e-13, momentum=0.99' as described in the paper) with the pretrained FCN-8s weights extracted from your model but the loss does not seem to go down :(

Could you please share the (Keras) pretrained FCN-8s weights that you used? Also, did you use any data augmentation?

Thank you very much in advance!
Best regards.

Value error: dimension must be 4

I have connected crfasrnn layer on top of CNN output as the code below shows.

input_shape  = (img_rows,img_cols,
output_shape = (img_rows, img_cols, NUM_CLASSES) 
input_1 = Input(shape=model.output_shape)
input_2 = Input(shape=input_shape)
output1 = CrfRnnLayer(image_dims=(img_rows, img_cols),
                         num_classes=NUM_CLASSES,
                         theta_alpha=160.,
                         theta_beta=3.,
                         theta_gamma=3.,
                         num_iterations=10,
                         name='crfrnn')([input_1, input_2])


    model       = Model(inputs=model.input, outputs=model.output, name='CNN')

    top_model   = Model(inputs=[input_1, input_2], outputs=output1, name='CRF-RNN') 

    full_model  = Model(inputs=model.input, outputs=top_model([model.output, model.input]), name='Full')

But i get the following error, CNN yields a feature map of size of [batch_size, 512,512,20] where batch size equals to 1. I can't understand the error, the tensor's dimension is 4, has anyone any idea about that error?

  File "./trainCRF.py", line 409, in <module>
    main()
  File "./trainCRF.py", line 407, in main
    crfRNN()
  File "./trainCRF.py", line 321, in crfRNN
    name='crfrnn')([input_1, input_2])
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 602, in __call__
    output = self.call(inputs, **kwargs)
  File "./crfasrnn_keras/src/crfrnn_layer.py", line 77, in call
    unaries = tf.transpose(inputs[0][0, :, :, :], perm=(2, 0, 1))
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/array_ops.py", line 1336, in transpose
    ret = gen_array_ops.transpose(a, perm, name=name)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 5694, in transpose
    "Transpose", x=x, perm=perm, name=name)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2958, in create_op
    set_shapes_for_outputs(ret)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2209, in set_shapes_for_outputs
    shapes = shape_func(op)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2159, in call_with_requiring
    return call_cpp_shape_fn(op, require_shape_fn=True)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/common_shapes.py", line 627, in call_cpp_shape_fn
    require_shape_fn)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/common_shapes.py", line 691, in _call_cpp_shape_fn_impl
    raise ValueError(err.message)
ValueError: Dimension must be 4 but is 3 for 'crfrnn/transpose' (op: 'Transpose') with input shapes: [?,512,512,20], [3].

Not found modules in trainCRF.py

Hello
Thank you for sharing your code.
There are some classes in trainCRF.py that cannot be imported. The followings are unresolved references:

import labels
from dataGenerator import DataGenerator
from weighted_categorical_crossentropy import weighted_categorical_crossentropy, weighted_loss
from BillinearUpsampling import BilinearUpSampling2D
from modelPredictor import computeMeanIou, plot_confusion_matrix
from labels import listLabels

Permutohedral Lattice implementation

Hello,

I have trying to use the permutohedral lattice code to perform bilateral filtering just to see if it works as expected.
What happens is: I get an image with a repeating artefact overlayed over the original image instead of a blur effect.
I don't know if this is how your implementation is supposed to work or if there is some kind of bug.
Would you mind attempting to perform bilateral filtering on your side to check this issue?

Also I have across a paper that might interest you since it can speed up the message passing step:

Recursive bilateral filtering - Yang, Qingxiong - European Conference on Computer Vision - 2012

Thanks.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.