tikroeger / of_dis Goto Github PK

Hi,

I thought I had already filed a bug about this, but evidently not, so here goes. :-)

As part of Futatabi, my realtime slow motion system, I've made a GPU reimplementation of the DIS algorithm (including variational refinement). On a recent GPU, it is more than an order of magnitude faster than the reference implementation is on a fast CPU, e.g., 1080p60 (forward flow + backward flow + interpolation) is easily realtime on my RTX 2070 at quality preset 3. In terms of EPE, it's a few percent better on the Sintel sequences—I haven't tried to track exactly why, but there are a number of smaller differences (e.g. bilinear interpolation in some places, and different edge handling). I don't know if you want to do anything with it or not, but now at least you know. :-)

Code (GPLv3), documentation, video etc. at https://nageru.sesse.net/ . Thanks for the awesome algorithm!

i used DIS optical flow algorithm in OpenCV3.2, and it is amazingly fast with good quality,but i have a problem about parameters. with many experiments, i found that parameters may be tuned according to the video. so i was wondering how to tune parameters according the size of video、or something like that

dis->setFinestScale(3);
dis->setGradientDescentIterations(8);
dis->setPatchSize(5);
dis->setPatchStride(3);
dis->setUseMeanNormalization(1);
dis->setUseSpatialPropagation(1);
dis->setVariationalRefinementAlpha(10);
dis->setVariationalRefinementDelta(5);
dis->setVariationalRefinementGamma(10);
dis->setVariationalRefinementIterations(12);

Hi Till,
I have successfully compiled and generated the 4 targets, but when I try to run the following command
~/Run/OF_DIS-master/build$ ./run_OF_RGB 0000000150.png 0000000151.png out.flo

it printed this and then crashed
TIME (Image loading ) (ms): 34.086
TIME (Pyramide+Gradients) (ms): 47.42

Can you provide some advice? Thanks a lot!

src_p1 and src_m1 is not align allocated.

sor_coupled function in solver.cpp
You never use s1[0] value in left block. Is this correct?

Hi, I congrats for your work.
I would like to know whether at inference time is it possible to use this model as part of a framework that performs the online action detection task? (i.e. let's suppose I have an input stream video, is the model able to compute optical-flow features as the frames arrive, with real-time speed ?)

Thank you!

I have had four mex files, which are run_OD_INT, run_DE_INT, run_DE_RGB and run_OF_RGB from the makefile provided. It seems that I can run the m file on matlab of linux without additional operation.

`MEX-file '/home/wxhwrdmn/Documents/zsameem-OF_DIS-9c06995/run_OF_INT.mexa64' 无效:
缺少 '/usr/lib/x86_64-linux-gnu/libopencv_shape.so.3.2->/home/wxhwrdmn/Documents/zsameem-OF_DIS-9c06995/run_OF_INT.mexa64' 需要的符号 '_ZdlPvm'

缺少'/usr/lib/x86_64-linux-gnu/libopencv_video.so.3.2->/usr/lib/x86_64-linux gnu/libopencv_shape.so.3.2->/home/wxhwrdmn/Documents/zsameem-OF_DIS 9c06995/run_OF_INT.mexa64'需要的符号 'ZdlPvm'

缺少'/usr/lib/x86_64-linux-gnu/libopencv_imgproc.so.3.2->/usr/lib/x86_64-linux gnu/libopencv_video.so.3.2->/usr/lib/x86_64-linux-gnu/libopencv_shape.so.3.2 >/home/wxhwrdmn/Documents/zsameem-OF_DIS-9c06995/run_OF_INT.mexa64'需要的符号 'ZdaPvm'...`

There are a lot of errors, indicating that

`MEX-file '/home/wxhwrdmn/Documents/zsameem-OF_DIS-9c06995/run_OF_INT.mexa64' is valid:
lack the symbol 'ZdlPvm', which is needed by '/usr/lib/x86_64-linux-gnu/libopencv_shape.so.3.2->/home/wxhwrdmn/Documents/zsameem-OF_DIS-9c06995/run_OF_INT.mexa64'

lack the symbol ''ZdlPvm'', which is needed by '/usr/lib/x86_64-linux-gnu/libopencv_video.so.3.2->/usr/lib/x86_64-linux-gnu/libopencv_shape.so.3.2->/home/wxhwrdmn/Documents/zsameem-OF_DIS-9c06995/run_OF_INT.mexa64'

lack the symbol 'ZdaPvm', which is needed by '/usr/lib/x86_64-linux-gnu/libopencv_imgproc.so.3.2->/usr/lib/x86_64-linux-gnu/libopencv_video.so.3.2->/usr/lib/x86_64-linux-gnu/libopencv_shape.so.3.2->/home/wxhwrdmn/Documents/zsameem-OF_DIS-9c06995/run_OF_INT.mexa64'`

From this line, various symbols are needed.
So do you have some other operation to avoid that?

Please see below the image, where it can be seen, that something has to be wrong with the calculation of the Maximum Motion (for auto-selection of coarsest level).

Just FYI, your licensing doesn't make sense. You released under the GPL, so you can't also stipulate non-commercial use.

It seems that the two results of run_OF_INT and run_OF_RGB are not '.flo' files, since it always indicates that:

wxhwrdmn@wxhwrdmn-PC:~/Downloads/flow-code$ ./color_flow ./other/test_OF_INT.flo output.png
ReadFlowFile: could not open ./other/test_OF_INT.flo

The two results are here:

results.zip

And the color flow code is here:
flow-code.zip

I'm not sure whether these two files are '.flo' files or not now.

In your project, how did you deal with these two files on C? It seems that matlab can not read them, either.

The DIS optical flow is so amazing, it's so fast and accurate!
And I read the paper Fast Optical Flow using Dense Inverse Search, it has done lots of experiments and analysis. But I still have a quesion, why not use bilateral weight in sparse optical flow densify stage? The optical result has smearing at edges of objects, will bilateral weight be helpful for this phenomenon？ Or change the smooth term in variational refinement stage? Allowing for discontinuies at edges?
Have you ever tried these strategies?