gustavla / self-supervision Goto Github PK

The paper inspires me a lot. Great work! I am wondering when the code will be released. Looking forward to it. Thanks a lot.

Hi @gustavla,

I tried running the Tensorflow code that you have uploaded using the following instructions that you mentioned:

import deepdish as dd
import selfsup.model.vgg16

data = dd.io.load('vgg16.caffemodel.h5')
x = tf.placeholder(tf.float32, shape=[1, 224, 224, 3], name='x')
phase_test = tf.placeholder(tf.bool, name='phase_test')
z = selfsup.model.vgg16.build_network(x, parameters=data, phase_test=phase_test)

The code gets stuck somewhere. I did a preliminary debug and it seems that it occurs on calls to vgg_conv() in line 302 of vgg16.py. I ran it on a titanx gpu for 4 hours and it was not able to return from build_network(). Can you please see if you can fix it?

Regards,
Abhay

Hi,

I am working with Prof. Erik Learned Miller from UMass Amherst. We are trying to use the pretrained models but can not find the testing code in the repository. Can you please upload it or point to the right direction?

Regards,
Abhay Mittal

So colorization as a proxy task in self-supervised learning just means that we pretrained the vgg16 or ResNet network for image colorization, and then transfer the weights in the pretrained model to the downstream task?
What about features that we get?

The test() function is missing. Also if you have code to finetune vgg-16 and alexnet models for semantic segmentation that will be very useful to make a fair comparison against your published results.

Hi Gustav,

I got the output from the neural net but its a (1,32) vector of h and c values whereas the input image had dimensions (514,514). I compared it with the output of the default neural net in autocolorize and it generates (1,32,512,512) output. Could you please see and provide any changes to the model needed to get a colorized output image?

Regards,
Abhay

Hi Gustav,

I downloaded the pretrained VGG-16 model from http://people.cs.uchicago.edu/~larsson/color-proxy/models/vgg16.caffemodel.h5 and tried to fine tune it for pascal classification using the script in selfsup/evaluate/__main__.py . Unfortunately, the pretrained model's conv1_1 filters are meant for single channel grayscale inputs but the model defined by voc_classification.py is for colour images. This results in the following asserting failure

(tensorflow3) aravindh@gnodeb1:~/projects/self-supervision$ CUDA_VISIBLE_DEVICES=0 python3 selfsup/evaluate/ voc2007-classification /users/aravindh/scratch/autocolorize/vgg16.caffemodel.h5 -n vgg16 --output /users/aravindh/scratch/self_supervision/gustavia/voc_vgg16/classification/ --limit 100 
Traceback (most recent call last):
  File "/usr/lib64/python3.4/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib64/python3.4/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "selfsup/evaluate/__main__.py", line 26, in <module>
    main()
  File "selfsup/evaluate/__main__.py", line 23, in main
    time_limit=args.limit, iterations=args.iterations, network_type=args.network)
  File "/users/aravindh/projects/self-supervision/selfsup/evaluate/voc_classification.py", line 476, in train_and_test
    train(*args, **kwargs)
  File "/users/aravindh/projects/self-supervision/selfsup/evaluate/voc_classification.py", line 191, in train
    network_type=network_type)
  File "/users/aravindh/projects/self-supervision/selfsup/evaluate/voc_classification.py", line 91, in build_network
    use_dropout=True)
  File "/users/aravindh/projects/self-supervision/selfsup/model/vgg16.py", line 362, in build_network
    z = conv(z, 64, name='conv1_1')
  File "/users/aravindh/projects/self-supervision/selfsup/model/vgg16.py", line 303, in conv
    return vgg_conv(z, num(ch), **kwargs)
  File "/users/aravindh/projects/self-supervision/selfsup/model/vgg16.py", line 164, in vgg_conv
    assert W_shape is None or tuple(W_shape) == tuple(shape), "Incorrect weights shape for {} (file: {}, spec: {})".format(name, W_shape, shape)
AssertionError: Incorrect weights shape for conv1_1 (file: (3, 3, 1, 64), spec: [3, 3, 3, 64])

In order to replicate the results in column 1 of table 1 in your paper (http://arxiv.org/pdf/1703.04044.pdf), please let me know what should be changed.

Best wishes,
Aravindh Mahendran