sitzikbs / netvlad Goto Github PK

I'm currently creating a simple model with VGG16 as a base, cropped at the conv5 layer, like so:

cropped_VGG16 = keras.models.Model(inputs=VGG16_base.input
                                    ,outputs=VGG16_base.get_layer('block5_conv3').output)

# this is where I use the VLAD layer
vlad_layer = VLAD_pooling(inputs=cropped_VGG16.output,
                                k_centers=64,
                                scope="vlad",
                                use_xavier=True,
                                stddev=1e-3)

final_model = keras.models.Model(inputs=cropped_VGG16,
                                    outputs=vlad_layer)

However, I get the following error message at line 56:

TypeError: Failed to convert object of type <class 'list'> to Tensor. Contents: [None, 1]. Consider casting elements to a supported type.

Here is a snip of my model summary:

I directly use the output from the Keras VGG16 model as the input to the netVLAD_pooling() function. As can be seen from the screenshot above in Keras, the batch dimension (the first dimension) is set as None to allow for variable batch size. This seems to be the one that causes the issue, since your implementation of netVLAD in line 56 attempts to retrieve an int for num_batches. This also causes a problem of matmul as well

do you have any suggestions on how I can get around this?

If not, can I ask what your original implementation was like when using this netVLAD layer with a CNN?

PS
also, in line 25, you retrieve num_features from the 2nd dimension of input which according to your documentation seems to be the feature map width (BxHxWxC). is that intentional?

Hi,

First, thanks for sharing the code, get a lot of inspirations. While I am wondering whether it is possible to replace the way you compute the weights that feed into the VLAD core in the original paper with simple tf.conv2(...)? As shown in the pic, actually, the size of the filters are 1 X 1 X Dim X # of Clusters. It would be nice if you can check whether my understanding is correct. Thanks in advance!

please, you test this code?
when I apply this code in my project, (tensorflow 1.3)
I ,get this issue?

Shapes must be equal rank, but are 2 and 1 for 'siamese/opt_vlad/MatMul' (op: 'BatchMatMul') with input shapes: [5,8,8,512], [5,8,1]
in Wx_b = tf.matmul(inputs, wk) + b[k] this line.
can you have some solution?
thank you very much!

Hey @sitzikbs, was hoping to grab this code for a commercial research application. Would you be willing to add a license file?

I am using Keras+VGG and I intend to replace the pooling layer with your function, but it seems there are no place to put. Can u help me?