Git Product home page Git Product logo

automatic-video-colorization's Introduction

Hi there 👋

GitHub Stats

automatic-video-colorization's People

Contributors

chenyanglei avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

automatic-video-colorization's Issues

Error : scipy.io.matlab

In implementation of this code:

vgg_rawnet=scipy.io.loadmat('VGG_Model/imagenet-vgg-verydeep-19.mat')

I am getting the following error:

File "/home/akshat_suwalka/Fully-Automatic-Video-Colorization-with-Self-Regularization-and-Diversity-master/utils.py", line 273, in
vgg_rawnet=scipy.io.loadmat('VGG_Model/imagenet-vgg-verydeep-19.mat')

File "/home/akshat_suwalka/anaconda3/lib/python3.7/site-packages/scipy/io/matlab/mio.py", line 218, in loadmat
matfile_dict = MR.get_variables(variable_names)

File "/home/akshat_suwalka/anaconda3/lib/python3.7/site-packages/scipy/io/matlab/mio5.py", line 292, in get_variables
res = self.read_var_array(hdr, process)

File "/home/akshat_suwalka/anaconda3/lib/python3.7/site-packages/scipy/io/matlab/mio5.py", line 252, in read_var_array
return self._matrix_reader.array_from_header(header, process)

File "mio5_utils.pyx", line 673, in scipy.io.matlab.mio5_utils.VarReader5.array_from_header

File "mio5_utils.pyx", line 719, in scipy.io.matlab.mio5_utils.VarReader5.array_from_header

File "mio5_utils.pyx", line 893, in scipy.io.matlab.mio5_utils.VarReader5.read_cells

File "mio5_utils.pyx", line 671, in scipy.io.matlab.mio5_utils.VarReader5.read_mi_matrix

File "mio5_utils.pyx", line 721, in scipy.io.matlab.mio5_utils.VarReader5.array_from_header

File "mio5_utils.pyx", line 968, in scipy.io.matlab.mio5_utils.VarReader5.read_struct

File "mio5_utils.pyx", line 671, in scipy.io.matlab.mio5_utils.VarReader5.read_mi_matrix

File "mio5_utils.pyx", line 719, in scipy.io.matlab.mio5_utils.VarReader5.array_from_header

File "mio5_utils.pyx", line 893, in scipy.io.matlab.mio5_utils.VarReader5.read_cells

File "mio5_utils.pyx", line 671, in scipy.io.matlab.mio5_utils.VarReader5.read_mi_matrix

File "mio5_utils.pyx", line 703, in scipy.io.matlab.mio5_utils.VarReader5.array_from_header

File "mio5_utils.pyx", line 777, in scipy.io.matlab.mio5_utils.VarReader5.read_real_complex

File "mio5_utils.pyx", line 450, in scipy.io.matlab.mio5_utils.VarReader5.read_numeric

File "mio5_utils.pyx", line 355, in scipy.io.matlab.mio5_utils.VarReader5.read_element

File "streams.pyx", line 200, in scipy.io.matlab.streams.ZlibInputStream.read_string

File "streams.pyx", line 193, in scipy.io.matlab.streams.ZlibInputStream.read_into

OSError: could not read bytes

Please help me in it.

And if you have implemented in tensorflow 2.0 , then can you please share it??

Question about "def Bilateral_NN"

Thank you for sharing your good code:)
I have a question about function named "Bilateral_NN".

  1. What does the "division 500" mean for both color_image_X and color_image_X ??
  2. Do the bold 5s as below mean (relate) K=5??

def Bilateral_NN(color_image, neigh):
h,w = color_image.shape[:2]
color_image_X = np.tile(np.arange(w),[h,1])/500.
color_image_Y = np.tile(np.arange(h),[w,1]).T/500.
color_image_all = np.concatenate([color_image,color_image_X[:,:,np.newaxis], color_image_Y[:,:,np.newaxis]],axis=2)
neigh.fit(np.reshape(color_image_all, [-1, 5]))
idxs = neigh.kneighbors(np.reshape(color_image_all, [-1, 5]), 5, return_distance=False)
return idxs

Thank you

question in download VGG_Model/imagenet-vgg-verydeep-19.mat

Hello,thanks for your project.
I meet an error in download_models.py,line34
download_file_from_google_drive('0B_B_FOgPxgFLRjdEdE9NNTlzUWc', 'VGG_Model/imagenet-vgg-verydeep-19.mat')

FileNotFoundError: [Errno 2] No such file or directory: 'VGG_Model/imagenet-vgg-verydeep-19.mat'

I changed VGG_Model/imagenet-vgg-verydeep-19.mat with imagenet-vgg-verydeep-19.mat and download sucessful, then move it in VGG_Model. So I think if you can justify your code in download_models.py.

Download.py cant be downloaded

Download.py is not downloaded
Even the link provided by you i.e.ckpt is also not working please help
Please tell the steps to download.

Fail to download VGG_19model and ckpt-woflow.zip

I try to download the VGG_19 model by run
python download_VGG.py
And the connection error happened.
image
I can browse Google and Youtobe. Is the download link invalid? Or may I have some wrong operation?
look forward to your reply
good wish!

its not running

I treid to run your code on your example dataset but it shows me nothing coud please help me to run this
Screenshot (29)

About diversity loss function in your paper

Hi, Mr.Chenyang and Mr.Qifeng.
I have a question about in your paper "Fully-Automatic-Video-Colorization-with-Self-Regularization-and-Diversity".

In your paper, you used φ in your proposed loss function for diversity.
I wonder what this φ represents.

I checked your cited paper "Interactive_Image_Segmentation_with_Latent_Diversity", but I can't find φ.
Could you tell me, what do φ represents?

Thank you.

instructions

Hi i want to colorize my black and white video i downloaded the model and ckpt_woflow i want to ask how to use this thing. Thanks

Trainging settings

Hello, sorry to bother you! I've read your paper and source code recently. You mentioned that you train the colorization model firstly, and then the colorization model and refinement model are trained jointly. But the main_whole.py file may only include the joint training code?
If I want to reproduce the result, do I need to add a new file to train the colorization network firstly?
And in the main_whole.py file, the pre-trained model of colorization network may not be loaded. After the colorization network is trained, should I load the pre-trained model of colorization network when training the whole network?
Looking forward to your apply! Thank you!

Training

Hi!
I wanna use our code for my course project but it is not clear how data should be arrange. Is directory JPEGImages supposed to have gray frames? If so than where should i put colored (true) frames? Also it is not clear how to "generate the optical flow by PWC-Net".

Could you please clarify that?

Training speed

Hello, I've tried to train this whole model, but I found that the training speed is very slow because the KNN calculating. Is there any other choice? Thank you!

IsADirectoryError: [Errno 21] Is a directory: 'demo_imgs'

Successfully run single image colorization:

python test.py --img_path demo_imgs/ILSVRC2012_val_00040251.JPEG

however attempt to colorize the images in a folder by:

python test.py --img_path demo_imgs
throws an error:
IsADirectoryError: [Errno 21] Is a directory: 'demo_imgs'

The same error happened when running:

python test_div_video.py --use_gpu 1 --video_path demo_vid

when i use main_whole to inference,some unexpected mistake occured

Traceback (most recent call last):
File "main_whole.py", line 11, in
import flow_warp as flow_warp_op
File "/home/qianming/code/Fully-Automatic-Video-Colorization/flow_warp.py", line 4, in
tf.resource_loader.get_path_to_datafile("./lib/flow_warp.so"))
File "/home/qianming/anaconda3/envs/torch/lib/python3.6/site-packages/tensorflow/python/framework/load_library.py", line 61, in load_op_library
lib_handle = py_tf.TF_LoadLibrary(library_filename)
tensorflow.python.framework.errors_impl.NotFoundError: libcudart.so.8.0: cannot open shared object file: No such file or directory

Refinement network: g

Is it possible to have a script for the refinement network so that I can run the colourisation output through this multiple times as mentioned in your paper to gain better temporal coherence?

If is the eval code can be AVALIABLE.

Hello, your work is so interesting. I wanna know, how can i reach the eval work as you do. whats the data for eval? And how to do for eval video data?

error while converting

Namespace(model='ckpt_woflow', test_dir='test_sample0', test_img='', use_gpu=1)
'grep' is not recognized as an internal or external command,
operable program or batch file.
Traceback (most recent call last):
File "main_woflow.py", line 30, in
os.environ["CUDA_VISIBLE_DEVICES"]=str(np.argmax( [int(x.split()[2]) for x in subprocess.Popen("nvidia-smi -q -d Memory | grep -A4 GPU | grep Free", shell=True, stdout=subprocess.PIPE).stdout.readlines()]))
File "<array_function internals>", line 6, in argmax
File "E:\mi\lib\site-packages\numpy\core\fromnumeric.py", line 1153, in argmax
return _wrapfunc(a, 'argmax', axis=axis, out=out)
File "E:\mi\lib\site-packages\numpy\core\fromnumeric.py", line 58, in _wrapfunc
return _wrapit(obj, method, *args, **kwds)
File "E:\mi\lib\site-packages\numpy\core\fromnumeric.py", line 47, in _wrapit
result = getattr(asarray(obj), method)(*args, **kwds)
ValueError: attempt to get argmax of an empty sequence

(base) E:\animegan\video attack\Fully-Automatic-Video-Colorization-with-Self-Regularization-and-Diversity-master>python main_woflow.py --model ckpt_woflow --use_gpu 1 --test_dir test_sample0

how to fix it

ValueError: attempt to get argmax of an empty sequence

Trying to evaluate your code.
Unfortunately the only result I've got is:

result = getattr(asarray(obj), method)(*args, **kwds)
ValueError: attempt to get argmax of an empty sequence

I tried:
python test.py --img_path demo_imgs/ILSVRC2012_val_00040251.JPEG
python test.py --img_path demo_imgs/
Result the same.
It only creates "teat_result" folder with empty " model_imagecolor" subfolder.
Greatly appreciate for your assistance.

download

Hello, Sorry to bother you, but I really can't download the weight files "VGG_Model/ imagenet-vGG -verydeep-19.mat" and "CKpt_woflow. zip". Can you send it to my mailbox if convenient? My email address is [email protected]
thank you very much.

AttributeError: module 'scipy.misc' has no attribute 'imread'

When I run this code;

!python main_woflow.py --model ckpt_woflow --use_gpu 1 --test_img /content/frames

I'm getting this error;

File "main_woflow.py", line 129, in
im=np.float32(scipy.misc.imread(test_img, 'L'))/255.0
AttributeError: module 'scipy.misc' has no attribute 'imread'

Video colorization

I tried to run your model on video followed your instruction. However, it shows the error:

OSError: cannot identify image file 'test_video/girl_video.mp4'

I have tried with other videos, but still not run. Do I need to convert it to a specific video format?

Thank you so much.

Should I train again on my dataset?

Hello, your work is so interesting. I wanna know, if I wanna use your mehtod to compare with my method. But my training dataset and test data are different from yours, should I use my dataset to train your model again?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.