coupriec / videopredictioniclr2016 Goto Github PK
View Code? Open in Web Editor NEWLicense: GNU General Public License v3.0
License: GNU General Public License v3.0
Hi @MichaelMathieu , I have a small doubt: Is your generator G an autoencoder (X->z->X') or its a z-->X likelihood network. I mean generator of Adversarial Network.
Here z: sample from the distribution, and, X: generated future frame.
Is it possible to share the model fine-tuned on UCF101?
Thanks
Hi,
I tried to train network, that you provide its code (train_iclr_model.lua), on the UCF101 dataset, then I tried to evaluate the trained model using your test code but I got the following error:
/home/user/torch/install/bin/luajit: ...torch/install/share/lua/5.1/cudnn/SpatialConvolution.lua:173: attempt to call method 'isContiguous' (a nil value)
stack traceback:
...torch/install/share/lua/5.1/cudnn/SpatialConvolution.lua:173: in function 'makeContiguous'
...torch/install/share/lua/5.1/cudnn/SpatialConvolution.lua:187: in function 'func'
/home/user/torch/install/share/lua/5.1/nngraph/gmodule.lua:345: in function 'neteval'
/home/user/torch/install/share/lua/5.1/nngraph/gmodule.lua:380: in function 'func'
/home/user/torch/install/share/lua/5.1/nngraph/gmodule.lua:345: in function 'neteval'
/home/user/torch/install/share/lua/5.1/nngraph/gmodule.lua:380: in function 'forward'
test-frame-prediction-on-ucf-rec_gdl.lua:204: in main chunk
[C]: in function 'dofile'
...user/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x00405d50
I would really appreciate it if you can tell me what's going on.
When I tried th test-frame-prediction-on-ucf-rec_gdl.lua
I got the error:
gm (GraphicsMagick) binary not found, please install (see www.graphicsmagick.org) THCudaCheck FAIL file=/home/dl-box/torch/extra/cutorch/lib/THC/generic/THCStorage.cu line=65 error=2 : out of memory /home/dl-box/torch/install/bin/luajit: /home/dl-box/torch/install/share/lua/5.1/torch/File.lua:351: cuda runtime error (2) : out of memory at /home/dl-box/torch/extra/cutorch/lib/THC/generic/THCStorage.cu:65 stack traceback: [C]: in function 'read' /home/dl-box/torch/install/share/lua/5.1/torch/File.lua:351: in function </home/dl-box/torch/install/share/lua/5.1/torch/File.lua:245> [C]: in function 'read' /home/dl-box/torch/install/share/lua/5.1/torch/File.lua:351: in function 'readObject' /home/dl-box/torch/install/share/lua/5.1/torch/File.lua:369: in function 'readObject' /home/dl-box/torch/install/share/lua/5.1/nn/Module.lua:184: in function 'read' /home/dl-box/torch/install/share/lua/5.1/torch/File.lua:351: in function 'readObject' /home/dl-box/torch/install/share/lua/5.1/torch/File.lua:369: in function 'readObject' /home/dl-box/torch/install/share/lua/5.1/torch/File.lua:369: in function 'readObject' /home/dl-box/torch/install/share/lua/5.1/torch/File.lua:369: in function 'readObject' /home/dl-box/torch/install/share/lua/5.1/torch/File.lua:369: in function 'readObject' ... /home/dl-box/torch/install/share/lua/5.1/torch/File.lua:353: in function 'readObject' /home/dl-box/torch/install/share/lua/5.1/torch/File.lua:369: in function 'readObject' ...e/dl-box/torch/install/share/lua/5.1/nngraph/gmodule.lua:495: in function 'read' /home/dl-box/torch/install/share/lua/5.1/torch/File.lua:351: in function 'readObject' /home/dl-box/torch/install/share/lua/5.1/torch/File.lua:369: in function 'readObject' /home/dl-box/torch/install/share/lua/5.1/torch/File.lua:409: in function 'load' test-frame-prediction-on-ucf-rec_gdl.lua:59: in main chunk [C]: in function 'dofile' ...-box/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:145: in main chunk [C]: at 0x00406670
I think the problem is that, I am out of memory.
Now, from where I can reduce the network size?
@MichaelMathieu:
Though not a code issue - but my query is like this:
I have an existing MATLAB (say, something.m file) and I need that MATLAB file to be called from the torch file:
th train_iclr_model.lua
.
To be more specific: I need a function within train_iclr_model.lua
- that will call something.m
and will use something.m
results (like: mat file) as a part of Video Prediction.
th train_iclr_model.lua
---> within train_iclr_model.lua
I need a function `CALL_MATLAB_M_FILE()end`
May you please help me to figure out that..
Regards
Hi @MichaelMathieu: I got the error;
error loading module 'libthffmpeg' from file '/home/intern_vnb/torch/install/lib/lua/5.1/libthffmpeg.so'
The error is, when I tried:
require 'thffmpeg'
/home/intern_vnb/torch/install/share/lua/5.1/trepl/init.lua:384: /home/intern_vnb/torch/install/share/lua/5.1/trepl/init.lua:384: ...tern_vnb/torch/install/share/lua/5.1/luarocks/loader.lua:117: error loading module 'libthffmpeg' from file '/home/intern_vnb/torch/install/lib/lua/5.1/libthffmpeg.so': /home/intern_vnb/torch/install/lib/lua/5.1/libthffmpeg.so: undefined symbol: av_frame_free stack traceback: [C]: in function 'error' /home/intern_vnb/torch/install/share/lua/5.1/trepl/init.lua:384: in function 'require' [string "_RESULT={require 'thffmpeg'}"]:1: in main chunk [C]: in function 'xpcall' /home/intern_vnb/torch/install/share/lua/5.1/trepl/init.lua:652: in function 'repl' ..._vnb/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:199: in main chunk [C]: at 0x00406670
I have changed the permission of: libthffmpeg.so at that location (i.e. '/home/intern_vnb/torch/install/lib/lua/5.1/libthffmpeg.so').
Another thing: I can see libthffmpeg.so file at THFFmpeg/build location too
Hi!
I'm trying out the code, It woks fine, but I found something weird in line 144 of train_iclr_model.lua where it states that:
local w, h = opt.h, opt.w
Which seems to me is inverting the dimensions. Is that intended behavior?
Thanks in advance.
As I understand absolute values of L2 loss and GDL loss will have large difference, but in paper lambda coefficient is 1.0 for both. Is it normal?
Here I see lambda 1.0 for L2 loss and lambda 0.01 for adversarial loss:
VideoPredictionICLR2016/train_iclr_model.lua
Line 370 in 802e902
Also looks like alpha is 1 for GDL loss:
VideoPredictionICLR2016/get_model.lua
Line 298 in 802e902
Looks related:
#7 (comment)
Dear @MichaelMathieu sorry to bother you, but this time I got the following error when I was trying:
luarocks make at THFFmpeg folder:
`
Warning: unmatched variable LUALIB
cmake -E make_directory build && cd build && cmake .. -DLUALIB= -DCMAKE_BUILD_TYPE=Release -DCMAKE_PREFIX_PATH="/home/intern_vnb/torch/install/bin/.." -DCMAKE_INSTALL_PREFIX="/home/intern_vnb/torch/install/lib/luarocks/rocks/thffmpeg/1.0.alpha-0" && make
-- The C compiler identification is GNU 4.7.3
-- The CXX compiler identification is GNU 5.2.0
-- Check for working C compiler: /usr/local/bin/gcc
-- Check for working C compiler: /usr/local/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/local/bin/c++
-- Check for working CXX compiler: /usr/local/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Found Torch7 in /home/intern_vnb/torch/install
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.26")
-- checking for module 'libavcodec'
-- found libavcodec, version 57.48.101
-- checking for module 'libavformat'
-- found libavformat, version 57.41.100
-- checking for module 'libavutil'
-- found libavutil, version 55.28.100
-- Found FFMPEG or Libav: /usr/local/lib/libavcodec.a;/usr/local/lib/libavformat.a;/usr/local/lib/libavutil.a, /usr/local/include
-- Configuring done
-- Generating done
CMake Warning:
Manually-specified variables were not used by the project:
LUALIB
-- Build files have been written to: /home/intern_vnb/Arghya/VideoPredictionICLR2016/THFFmpeg/build
Scanning dependencies of target thffmpeg
[100%] Building C object CMakeFiles/thffmpeg.dir/thffmpeg.c.o
/home/intern_vnb/Arghya/VideoPredictionICLR2016/THFFmpeg/thffmpeg.c: In function ‘AV_open’:
/home/intern_vnb/Arghya/VideoPredictionICLR2016/THFFmpeg/thffmpeg.c:145:5: warning: ‘codec’ is deprecated (declared at /usr/local/include/libavformat/avformat.h:880) [-Wdeprecated-declarations]
/home/intern_vnb/Arghya/VideoPredictionICLR2016/THFFmpeg/thffmpeg.c:153:3: warning: ‘codec’ is deprecated (declared at /usr/local/include/libavformat/avformat.h:880) [-Wdeprecated-declarations]
/home/intern_vnb/Arghya/VideoPredictionICLR2016/THFFmpeg/thffmpeg.c:172:3: warning: ‘avpicture_get_size’ is deprecated (declared at /usr/local/include/libavcodec/avcodec.h:5406) [-Wdeprecated-declarations]
/home/intern_vnb/Arghya/VideoPredictionICLR2016/THFFmpeg/thffmpeg.c:172:37: error: ‘PIX_FMT_RGB24’ undeclared (first use in this function)
/home/intern_vnb/Arghya/VideoPredictionICLR2016/THFFmpeg/thffmpeg.c:172:37: note: each undeclared identifier is reported only once for each function it appears in
/home/intern_vnb/Arghya/VideoPredictionICLR2016/THFFmpeg/thffmpeg.c:190:3: warning: ‘avpicture_fill’ is deprecated (declared at /usr/local/include/libavcodec/avcodec.h:5391) [-Wdeprecated-declarations]
/home/intern_vnb/Arghya/VideoPredictionICLR2016/THFFmpeg/thffmpeg.c: In function ‘AV_read_frame’:
/home/intern_vnb/Arghya/VideoPredictionICLR2016/THFFmpeg/thffmpeg.c:203:7: warning: ‘avcodec_decode_video2’ is deprecated (declared at /usr/local/include/libavcodec/avcodec.h:4753) [-Wdeprecated-declarations]
/home/intern_vnb/Arghya/VideoPredictionICLR2016/THFFmpeg/thffmpeg.c:212:2: warning: ‘av_free_packet’ is deprecated (declared at /usr/local/include/libavcodec/avcodec.h:4414) [-Wdeprecated-declarations]
/home/intern_vnb/Arghya/VideoPredictionICLR2016/THFFmpeg/thffmpeg.c:217:5: warning: ‘av_free_packet’ is deprecated (declared at /usr/local/include/libavcodec/avcodec.h:4414) [-Wdeprecated-declarations]
/home/intern_vnb/Arghya/VideoPredictionICLR2016/THFFmpeg/thffmpeg.c: In function ‘AV_skip_frame’:
/home/intern_vnb/Arghya/VideoPredictionICLR2016/THFFmpeg/thffmpeg.c:227:7: warning: ‘avcodec_decode_video2’ is deprecated (declared at /usr/local/include/libavcodec/avcodec.h:4753) [-Wdeprecated-declarations]
/home/intern_vnb/Arghya/VideoPredictionICLR2016/THFFmpeg/thffmpeg.c:229:2: warning: ‘av_free_packet’ is deprecated (declared at /usr/local/include/libavcodec/avcodec.h:4414) [-Wdeprecated-declarations]
/home/intern_vnb/Arghya/VideoPredictionICLR2016/THFFmpeg/thffmpeg.c:233:5: warning: ‘av_free_packet’ is deprecated (declared at /usr/local/include/libavcodec/avcodec.h:4414) [-Wdeprecated-declarations]
make[2]: *** [CMakeFiles/thffmpeg.dir/thffmpeg.c.o] Error 1
make[1]: *** [CMakeFiles/thffmpeg.dir/all] Error 2
make: *** [all] Error 2
Error: Build error: Failed building.
`
I have done:
Deleted build folder and tried again "luarocks make", but no luck
My Ubuntu version is 14.04
I have cloned THFFmpeg from "https://github.com/MichaelMathieu/THFFmpeg.git"
Regards,
Arghya.
Your link does't work. Can you check it?
http://perso.esiee.fr/~coupriec/MathieuICLR16TestCode.zip
Hi,
I am trying to run test code but I get the following error:
/home/user/torch/install/bin/luajit: /home/user/torch/install/share/lua/5.1/trepl/init.lua:389: /home/user/torch/install/share/lua/5.1/trepl/init.lua:389: module 'fb.util' not found:No LuaRocks module found for fb.util
no field package.preload['fb.util']
no file '/home/user/.luarocks/share/lua/5.1/fb/util.lua'
no file '/home/user/.luarocks/share/lua/5.1/fb/util/init.lua'
no file '/home/user/torch/install/share/lua/5.1/fb/util.lua'
no file '/home/user/torch/install/share/lua/5.1/fb/util/init.lua'
no file './fb/util.lua'
no file '/home/user/torch/install/share/luajit-2.1.0-beta1/fb/util.lua'
no file '/usr/local/share/lua/5.1/fb/util.lua'
no file '/usr/local/share/lua/5.1/fb/util/init.lua'
no file '/home/user/.luarocks/lib/lua/5.1/fb/util.so'
no file '/home/user/torch/install/lib/lua/5.1/fb/util.so'
no file '/home/user/torch/install/lib/fb/util.so'
no file './fb/util.so'
no file '/usr/local/lib/lua/5.1/fb/util.so'
no file '/usr/local/lib/lua/5.1/loadall.so'
no file '/home/user/.luarocks/lib/lua/5.1/fb.so'
no file '/home/user/torch/install/lib/lua/5.1/fb.so'
no file '/home/user/torch/install/lib/fb.so'
no file './fb.so'
no file '/usr/local/lib/lua/5.1/fb.so'
no file '/usr/local/lib/lua/5.1/loadall.so'
stack traceback:
[C]: in function 'error'
/home/user/torch/install/share/lua/5.1/trepl/init.lua:389: in function 'require'
test-frame-prediction-on-ucf-rec_gdl.lua:11: in main chunk
[C]: in function 'dofile'
...user/torch/install/lib/luarocks/rocks/trepl/scm-1/bin/th:150: in main chunk
[C]: at 0x00405d50
then I tried to install fblualib by using the one-step installation script as provided in the fblualib github page here: https://github.com/facebook/fblualib/blob/master/INSTALL.md and https://github.com/facebookarchive/fblualib/blob/master/install_all.sh but it stops and shows this massage:
- echo Building fbthrift
Building fbthrift
echo
cd /tmp/fblualib-build.8taTVB/fbthrift/thrift
autoreconf -ivf
autoreconf: 'configure.ac' or 'configure.in' is required
I would really appreciate it if you can tell me what's going on.
Hi,
I'm having problems using the GDL criterion. The error stack states
../torch/install/share/lua/5.1/nn/CSubTable.lua:11: bad argument #2 to 'add' (sizes do not match at /tmp/luarocks_cutorch-scm-1-7235/cutorch/lib/THC/generated/../generic/THCTensorMathPointwise.cu:216)
stack traceback:
[C]: in function 'add'
../torch/install/share/lua/5.1/nn/CSubTable.lua:11: in function 'func'
../torch/install/share/lua/5.1/nngraph/gmodule.lua:346: in function 'neteval'
../torch/install/share/lua/5.1/nngraph/gmodule.lua:381: in function 'updateOutput'
./GDLCrit.lua:40: in function 'forward'
The size of the output does indeed change (it's smaller by one pixel). Before the target label and the output of the network is forwarded through the gdl loss they are both the same size. I haven't been able to figure out where the size change occurs. Any help would be really appreciated.
Thanks for sharing this fantastic code! I'm having trouble understanding the part of psnr calculation for motion region
in PSNR
function. The denominator for calculation of MSE used in the code is the number of all pixels in one image and shouldn't it be the number of pixels only in motion region?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.