This repository contains pytorch version implementation code for the project 'End-to-End Learning of Motion Representation for Video Understanding' (CVPR 2018) based on the original tensorflow implementation (https://github.com/LijieFan/tvnet/).

What's more, we made the u0 trainable. We also replaced the original matlib code with pure python code for visualization.

We use pytorch for our implementation.

Our current release has been tested on Ubuntu 16.04 with Python 3.6.5 and Pytorch 0.3.1.0.

https://github.com/Gasoonjia/tvnet_pytorch

The file (train_options.py) has the following options:

• --max_nscale: Max number of scales in TVNet (default: 1)
• --n_warps: Number of warppings in TVNet (default: 1)
• --n_iters: Number of iterations in TVNet (default: 50)
• --visualize: Whether save the result into image file (default: True)
• --demo: Just using original weights for demo if mark it.

Sample usages include

• Generate motion representation for frames in frame/img1.png and frame/img2.png.

python train.py --max_nscale 1 --n_warps 1 --n_iters 50 --demo

-TVNet generated results are saved in result/result.mat

-You can also find a file called result/result.png for visualize the flow as long as you set the flag --visualize in train_options.py into True.

img1.png

img2.png

The flow generated by original tensorflow code.

The flow generated by ours without any training.

if you find our code useful for your research, please cite their paper:

@inproceedings{fan2018end,
title={End-to-End Learning of Motion Representation for Video Understanding},
author={Fan, Lijie and Huang, Wenbing and Gan, Chuang and Ermon, Stefano and Gong, Boqing and Huang, Junzhou},
booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
pages={},
year={2018}
}