Hey André,

I wonder how you have done the fast weighted hamming distance with pop_count and fast corr weights...
Is there any reference?

I googled weighted hamming distance , I got this paper: LEARNING WEIGHTED HAMMING DISTANCE FOR BINARY DESCRIPTORS..

But unlickiky.. I failed to relate it to your work...

Regards ..

Good morning,

I have a question concerning the map results presented on "Large-Scale Query-by-Image Video Retrieval Using Bloom Filters" paper that uses the same code. MAP is @ 100 or @ 1? since in the "light_dataset_public.txt" ground truth results per query do not sum to 100. Same goes for "vb_light_dataset_public.txt". On the sample code you use scoring @1 from using the text files, that is clear.

On the "TEMPORAL AGGREGATION FOR LARGE-SCALE QUERY-BY-IMAGE VIDEO RETRIEVAL" paper in order to get the MAP@100, you mention "...compared to a baseline frame-based scheme that does not use asymmetric comparisons." in the end. Do you compare the raw frame descriptors (32d, 128d)? Binarized features comparison?

Thaks a lot,
Dimitris

Hey André,
Your work is awesome..
But I feel a little confused about the trained parameters ....
Are these parameters (say..PCA eigen vectors) trained from the whole I2V dataset ...
or just the videos that you have listed here?

Best regards..

Hello,

is it possible run videosearch in Windows on an easy way?

I've successfully built VLFEAT but I am facing some issues with YAEL during 'make', so I've downloaded YAEL's version for windows and compile it with Visual Studio.

Any suggestions will be appreciated!
Btw, congrats for this project!

Hi @andrefarauj, I am trying to train trained_parameters, following #10. But I get a very low mAP and P@1 on the standford600k dataset. (about 0.1 mAP, and 0.26 P@1). I want to describe some details, and hope you can give me some advice about how to improve mAP.

1. I first sampled about 20000 frame from the whole dataset, then gather all the local descriptors of the 20000 frames and shuffle them. After that sample about 150*20000 frames from all these local descriptors, and use the pca function in yael lib to reduce the 128 dim descriptor to 32 dim.
2. Next, I use the gmm function in yael to train gmm. I set niter to 60, centroid to 512. It takes about 10 hours to train the gmm.
3. For the corr_weights, I still use the old one.(sift.pre_alpha.0.50.pca.32.gmm.512.pre_alpha.0.50.corr_weights). Actually I am not quite sure whether this must also be regenerated. But since I am using the same sift descriptor, I think it is not the reason why the mAP is so low.
   Is there anything wrong in the whole process? Can you give me some advice? Thanks!

Thank you for your paper. I'm not sure not how to judge the similarity according the score.

Hi.
There are a lot of trained parameters' files in the folder.(indexer/global_descriptors/trained_parameters)
I confused about How to generate trained parameters?
(ps:I am a Chinese student. My English is not very good and may not be clearly expressed.)
Thanks. : )

usr/bin/ld: extract.o: undefined reference to symbol 'pthread_create@@GLIBC_2.2.5'
//lib/x86_64-linux-gnu/libpthread.so.0: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
Makefile:62: recipe for target 'extract' failed
make: *** [extract] Error 1

Hi @andrefaraujo, sorry to contact you in the ISSUE section, since there is no available email left on your profile. By the way,
thanks for your great work, I am a student who recently study technology of image matching. I want to do some experiments of transfer learning based on your
pre-trained delg model, but i got some bumps that the pre-trained models you have provided which are exported and inlcudes extra components
like pre-processing and post-processing module are not what i want, i am wondering pre-trained weights of the delg model. I have no resources
to training delg model from scratch, although the recipe of training you have already put out. I have tried to extract weights of the delg model downloaded from
”https://storage.googleapis.com/delf/r50delg_gldv2clean_20200914.tar.gz“ by converting it to graph_def, and found names of variables of the model
different from model written in the file "def/python/training/model/delg_model.py", Backbone of the pre-trained model seem to be implemented by slim's resnet_v1_50.
I successfully got weights of the delg model and restored them, but when I was calculating distances of descriptors for a group of pairs of images,
I found results being incorrect. Anyway, It would be grateful if you could share the weights for delf/r50delg_gldv2clean model.
You can contact me on [email protected], I am looking forward to receiving your reply.

**Hessian-Affine detector uses the netpbm image toolkit and when installed the netpdm and the depending libraries (djpeg for this particular setup), when trying to run the run_siftHesAff_extraction_test.sh i get some errors regarding the libnetpbm and undefined symbols.

any suggestions?
my error messages and library dependencies list are given below:**

_dimsam@iti-400:~/Dimitris/i2v/videosearch/indexer/local_descriptors$ ./run_siftHesAff_extraction_test.sh
time djpeg ../test_db/test_video_0_keyframes/000001.jpg | /usr/local/netpbm/bin/ppmtopgm | /usr/local/netpbm/bin/pnmnorm -bpercent=0.01 -wpercent=0.01 -maxexpand=400 | /usr/local/netpbm/bin/pamscale -pixels 786432 > ../test_db/test_video_0_keyframes/000001.pgm
/usr/local/netpbm/bin/pnmnorm: symbol lookup error: /usr/local/netpbm/bin/pnmnorm: undefined symbol: optParseOptions3
/usr/local/netpbm/bin/ppmtopgm: symbol lookup error: /usr/local/netpbm/bin/ppmtopgm: undefined symbol: ppm_allocrow
/usr/local/netpbm/bin/pamscale: symbol lookup error: /usr/local/netpbm/bin/pamscale: undefined symbol: optParseOptions3

dimsam@iti-400:/Dimitris/i2v/videosearch/indexer/local_descriptors$ ldd -r -d /usr/local/netpbm/bin/pnmnorm
linux-vdso.so.1 => (0x00007ffc0e7a9000)
libnetpbm.so.10 => /usr/lib/libnetpbm.so.10 (0x00007fa4f759c000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa4f71d7000)
/lib64/ld-linux-x86-64.so.2 (0x0000555a28cd1000)
undefined symbol: pm_tell2 (/usr/local/netpbm/bin/pnmnorm)
undefined symbol: pm_seek2 (/usr/local/netpbm/bin/pnmnorm)
undefined symbol: pnm_allocrow (/usr/local/netpbm/bin/pnmnorm)
undefined symbol: optParseOptions3 (/usr/local/netpbm/bin/pnmnorm)
dimsam@iti-400:/Dimitris/i2v/videosearch/indexer/local_descriptors$ ldd -r -d /usr/local/netpbm/bin/ppm
ppm3d ppmdist ppmlabel ppmrelief ppmtogif ppmtoneo ppmtorgb3 ppmtv
ppmbrighten ppmdither ppmmake ppmrough ppmtoicr ppmtopcx ppmtosixel ppmwheel
ppmchange ppmdmkfont ppmmix ppmshadow ppmtoilbm ppmtopgm ppmtoterm
ppmcie ppmdraw ppmnorm ppmshift ppmtojpeg ppmtopi1 ppmtotga
ppmcolormask ppmfade ppmntsc ppmspread ppmtoleaf ppmtopict ppmtouil
ppmcolors ppmflash ppmpat ppmtoacad ppmtolj ppmtopj ppmtowinicon
ppmdcfont ppmforge ppmquant ppmtoarbtxt ppmtomap ppmtopjxl ppmtoxpm
ppmddumpfont ppmglobe ppmquantall ppmtobmp ppmtomitsu ppmtoppm ppmtoyuv
ppmdim ppmhist ppmrainbow ppmtoeyuv ppmtompeg ppmtopuzz ppmtoyuvsplit
dimsam@iti-400:/Dimitris/i2v/videosearch/indexer/local_descriptors$ ldd -r -d /usr/local/netpbm/bin/ppmtopgm
linux-vdso.so.1 => (0x00007fff9cae9000)
libnetpbm.so.10 => /usr/lib/libnetpbm.so.10 (0x00007f8110d8e000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f81109c9000)
/lib64/ld-linux-x86-64.so.2 (0x000055b9eb310000)
undefined symbol: pgm_allocrow (/usr/local/netpbm/bin/ppmtopgm)
undefined symbol: ppm_allocrow (/usr/local/netpbm/bin/ppmtopgm)
dimsam@iti-400:/Dimitris/i2v/videosearch/indexer/local_descriptors$ ldd -r -d /usr/local/netpbm/bin/pamscale
linux-vdso.so.1 => (0x00007fffa23ce000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f8163366000)
libnetpbm.so.10 => /usr/lib/libnetpbm.so.10 (0x00007f8163143000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f8162d7d000)
/lib64/ld-linux-x86-64.so.2 (0x000055797d7cf000)
undefined symbol: pnm_createungammatransform (/usr/local/netpbm/bin/pamscale)
undefined symbol: pnm_unapplyopacityrown (/usr/local/netpbm/bin/pamscale)
undefined symbol: pnm_unnormalizeRow (/usr/local/netpbm/bin/pamscale)
undefined symbol: pnm_getopacity (/usr/local/netpbm/bin/pamscale)
undefined symbol: pnm_allocpamrown (/usr/local/netpbm/bin/pamscale)
undefined symbol: optParseOptions3 (/usr/local/netpbm/bin/pamscale)
undefined symbol: pnm_normalizeRow (/usr/local/netpbm/bin/pamscale)
undefined symbol: pnm_creategammatransform (/usr/local/netpbm/bin/pamscale)
undefined symbol: pnm_applyopacityrown (/usr/local/netpbm/bin/pamscale)_

In the file named sift_extractor.cc.
vl_sift_pix* data = new vl_sift_pix[number_pixels*sizeof(vl_sift_pix)];//line 61 float* this_frame = new float[4*sizeof(float)];//line 170 float* this_descr = new float[128*sizeof(float)];//line 171
The size of them is it too big? Maybe it can do like that:
vl_sift_pix* data = new vl_sift_pix[number_pixels];//line 61 float* this_frame = new float[4];//line 170 float* this_descr = new float[128];//line 171
I'm coding my graduation project which refer to your project. I'm a little confused about this problem.

Hello, first of all thank you very much for your contribution. In your code, you use vlfeat to extract sift feature. now I want to acceleration sift feature extraction process. I use siftgpu algorithm of chang chang wu, and extrac sift feature of images both in database-all and database-queries successfully. but now i find the feature doesn't work well. When I use the vlfeat code to extract sift feature, the final search accuracy is 64%. But when I uses siftgpu to extract sift feature, the finally search accuracy is just 6%。can you give some suggestion about this? thank you very much. the database I use is Stanford I2V dataset.

canceld

Hi Andre, thanks for the great work and for make the code available. This is not a code issue, but rather some questions about the last table (Table V) of your IEEE TCSVT article. In particular, the questions are about how some of the parameters were measured (I have used the 600k version of the SI2V dataset for my initial experiments):

1. For mAP, I see that the scripts available in the /scoring/ directory already display the final scores. Are those the values you used in the article?

2. For the latency, I also see some output displayed when the queries scripts are executed. Did you just take the average of all the queries? I got somewhat different results with my machine compared to the ones in Table V. For instance, results for the frame-based and scene-based approaches were, on average, 6.2s and 0.5s, respectively. For the bloom filter approach I have no idea how to measure the latency, since the retriever script works in a different manner. How did you measure the latency for the BF-PI approach?

3. Finally, I guess the memory column (Table V) refers to the file size of the joined global descriptors of each approach, am I right? For the bloom filter approach, however, I am not sure how to calculate that because there is no 'join' script. How did you measure the memory results for the BF-PI approach?

Thanks!

hi, how to train the pca and gmm parameters?

Hello, can I ask a question? I put a picture in test_query during the test. But the running result is wrong. The problem is in the file test_dataset_ground_truth_local.txt. I want to ask if this file is automatically updated with the query results. If you need to write it by yourself, it ’s not that you lose the meaning of this project

Hi andrefaraujo,

I found the SIFT result of your sift_extractor.cc is a little different from the vl_sift.c in VLfeat. The follows is from vl_sift.c

          /* Save back with MATLAB conventions. Notice tha the input
           * image was the transpose of the actual image. */
          frames [4 * nframes + 0] = k -> y + 1 ;
          frames [4 * nframes + 1] = k -> x + 1 ;
          frames [4 * nframes + 2] = k -> sigma ;
          frames [4 * nframes + 3] = VL_PI / 2 - angles [q] ;

          if (nout > 1) {
            if (! floatDescriptors) {
              for (j = 0 ; j < 128 ; ++j) {
                float x = 512.0F * rbuf [j] ;
                x = (x < 255.0F) ? x : 255.0F ;
                ((vl_uint8*)descr) [128 * nframes + j] = (vl_uint8) x ;
              }
            } else {
              for (j = 0 ; j < 128 ; ++j) {
                float x = 512.0F * rbuf [j] ;
                ((float*)descr) [128 * nframes + j] = x ;
              }
            }
          }

But your sift_extractor.cc are as follows:

                /* compute descriptor */
                vl_sift_calc_keypoint_descriptor (filt, rbuf, k, angles [q]) ;

                this_frame [0] = k -> x ;
                this_frame [1] = k -> y ;
                this_frame [2] = k -> sigma ;
                this_frame [3] = angles [q];

                frames.push_back(this_frame);

                for (j = 0 ; j < 128 ; ++j) {
                    float x;
                    if (divide_512) {
                        x = rbuf [j] ;
                    } else {
                        x = 512.0F * rbuf [j] ;
                    }
                    this_descr [j] = x ;
                }
                descr.push_back(this_descr);

I think the result of your SIFT result will the same as the VLFeat' SIFT in Matlab if it is fixed by the vl_sift.c.

Error in SIFT extraction stage:
usr/bin/ld: extract.o: undefined reference to symbol 'pthread_create@@GLIBC_2.2.5'
//lib/x86_64-linux-gnu/libpthread.so.0: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
Makefile:62: recipe for target 'extract' failed
make: *** [extract] Error 1

I Basically work with matlab programming. Completely iam unaware of python, opencv and ubuntu. With this comparitive study my final stage of PhD completes. Kindly provide a Simple algorithm for your code so that i can reprogram in matlab or else kindly provide the results of your code on my dataset and looking forward for a favorable reponse of collaborative research.

I use videosearch in LUbuntu 17.10. When I executed the step 6 by the Quick Start, it occur errors that undefined reference to symbol 'pthread_create@@GLIBC_2.2.5.

I have to add -pthread to CPPFLAGS in the makefile then it could make successfully.

step 5
main.cc:11:30: fatal error: opencv2/opencv.hpp

$ make
g++ -std=c++0x -O3 -g -Wall -fmessage-length=0 -c extract.cc
-o extract.o
g++ -std=c++0x -O3 -g -Wall -fmessage-length=0 -c sift_extractor.cc
-o sift_extractor.o
g++ -std=c++0x -O3 -g -Wall -fmessage-length=0 -c ../../common/feature_set/feature_set.cc
-o ../../common/feature_set/feature_set.o
g++ -std=c++0x -O3 -g -Wall -fmessage-length=0 -Wl,-rpath,'$ORIGIN'/../../common/vlfeat-0.9.18/bin/glnxa64 extract.o sift_extractor.o ../../common/feature_set/feature_set.o
-o extract -L../../common/vlfeat-0.9.18/bin/glnxa64/ -lvl pkg-config --cflags --libs opencv; \

/usr/bin/ld: extract.o: undefined reference to symbol 'pthread_create@@GLIBC_2.2.5'
//lib/x86_64-linux-gnu/libpthread.so.0: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
Makefile:63: recipe for target 'extract' failed
make: *** [extract] Error 1

I am getting this error in while running make in mypath = ~/videosearch/videosearch/indexer/local_descriptors$