dongwu92 / autoportraitmatting Goto Github PK
View Code? Open in Web Editor NEWTensorflow implementation of Automatic Portrait Matting on paper "Automatic Portrait Segmentation for Image Stylization"
License: Apache License 2.0
Tensorflow implementation of Automatic Portrait Matting on paper "Automatic Portrait Segmentation for Image Stylization"
License: Apache License 2.0
Hi there, very interested in the project and potentially in contributing (if my limited knowledge can be of any help).
I noticed that the original paper uses as a pretrained network the pascal-fcn8 network (can be obtained here http://www.vlfeat.org/matconvnet/models/beta22/pascal-fcn8s-dag.mat) instead of the VGG19 that you use. Also they suggest using SGD as an optimizer with 1e-4 learning rate (while I believe you use Adam).
So my basic questions is: could it be that VGG19 has already learnt "too much" and is therefore not optimizable any further? Why did you use it instead of FCN8?
I am re-implementing part of the model with vgg-16 and it's not clear to me how you are feeding the train dataset which is 800x600x3 into vgg-x which takes 224x224x3 images. I tried looking at your FCN.py code but it's still a mistery to me
Hi,
Are there any codes to compute the evaluation metric like 'Gradient' and 'Connectivity'?
Thanks,
rui
Hi, @PetroWu ,thanks for your work. When running both FCN.py and FCN_plus.py , I get the same errror:
Traceback (most recent call last): File "FCN_plus.py", line 296, in <module> tf.app.run() File "/home/hjp/anaconda2/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 30, in run sys.exit(main(sys.argv)) File "FCN_plus.py", line 160, in main name="entropy"))) File "/home/hjp/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/nn_ops.py", line 265, in sparse_softmax_cross_entropy_with_logits logits, labels, name=name) File "/home/hjp/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/gen_nn_ops.py", line 962, in _sparse_softmax_cross_entropy_with_logits features=features, labels=labels, name=name) File "/home/hjp/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/op_def_library.py", line 655, in apply_op op_def=op_def) File "/home/hjp/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2156, in create_op set_shapes_for_outputs(ret) File "/home/hjp/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1612, in set_shapes_for_outputs shapes = shape_func(op) File "/home/hjp/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/nn_ops.py", line 273, in _SparseSoftmaxCrossEntropyWithLogitsShape input_shape = logits_shape.with_rank(2) File "/home/hjp/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/tensor_shape.py", line 625, in with_rank raise ValueError("Shape %s must have rank %d" % (self, rank)) ValueError: Shape (?, ?, ?, 2) must have rank 2
It seems that the error is caused by the code at line 160.Have you encounterd this error? How to hanlde it?
I'm trying to experiment with your implementation, but I found that there is no available dataset located here: http://xiaoyongshen.me/webpages/webpage_automatting/ (I wrote emails to Xiaoyong Shen but unfortunately he didn't answer :( ). May be you have a copy and you can share link to it? It will be very helpful.
crop.txt 数据代表什么??
line 26: img_data = skio.imread('data/images_data/'+name+'.jpg')
May be the x and y mask should come from "images_data_crop" folder, according to the paper.
Baidu Cloud is very slow..
I want to receive data from a dropbox
我运行代码的时候,使用python3 FCN.py
结果并没有开始训练而是得到了:
sunxinlu@DLServer2:/Templates/fcn+$ python3 FCN.py/Templates/fcn+$ python3 FCN.py -train
setting up vgg initialized conv layers ...
Setting up summary op...
Setting up Saver...
Model restored...
sunxinlu@DLServer2:
setting up vgg initialized conv layers ...
Setting up summary op...
Setting up Saver...
Model restored...
就自动跳出了,想请问一下原因。谢谢大神~
Good job!
I'm also interested in these two paper ("Automatic Portrait Segmentation for Image Stylization" and "Deep Automatic Portrait Matting"). But I'm not sure whether the method of matting proposed in these two paper are same. Could you tell me your opinion?
Generating tfrecord format, how do I assign label to each pixel in images_mask set ? eg. num_class=2;
value = 0 , label = 0; value = 255, label=1; how about value=1~254?
Hi
Thanks for providing the work. I am getting the following error
Traceback (most recent call last):
File "FCN.py", line 5, in
import read_MITSceneParsingData as scene_parsing
ImportError: No module named 'read_MITSceneParsingData'
I tried commenting it, but then in order to use data, i dont see any function from the preprocess.py file being called in FCN.py or portrait.py.
Hi there, I have successfully trained the model. However, I found that there is only a mode of 'train'. Are there any direction on how to use the trained model? i.e. how can I apply this trained model on other datasets? Thanks a lot.
When I ran data_download.m, I failed to get the images because the url http://farm6.static.flickr.com/ is unreachable. Could someone tell me how to get the data?
alldata_urls中的链接翻墙后可以打开图片,但是脚本下载图片失败,作者可以提供图片的其他下载方式吗,谢谢了!
Hi, thank you for the repository.
I'm trying to run your example with no luck.
data_download.m
and generate_data_file.m
and copied to the data/
folderimagenet-vgg-verydeep-19.mat
to Model_zoo
python FCN_plus.py
under py27/py36 I've got the same error:I tensorflow/core/common_runtime/gpu/gpu_device.cc:777] Peer access not supported between device ordinals 0 and 1
I tensorflow/core/common_runtime/gpu/gpu_device.cc:777] Peer access not supported between device ordinals 1 and 0
I tensorflow/core/common_runtime/gpu/gpu_device.cc:906] DMA: 0 1
I tensorflow/core/common_runtime/gpu/gpu_device.cc:916] 0: Y N
I tensorflow/core/common_runtime/gpu/gpu_device.cc:916] 1: N Y
I tensorflow/core/common_runtime/gpu/gpu_device.cc:975] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Graphics Device, pci bus id: 0000:02:00.0)
I tensorflow/core/common_runtime/gpu/gpu_device.cc:975] Creating TensorFlow device (/gpu:1) -> (device: 1, name: GeForce GTX 1080, pci bus id: 0000:01:00.0)
(1, 800, 600, 2)
Traceback (most recent call last):
File "FCN_plus.py", line 296, in <module>
pred()
File "FCN_plus.py", line 275, in pred
org0_im.save('res/org' + str(itr) + '.jpg')
File "/home/ubuntu/anaconda2/lib/python2.7/site-packages/PIL/Image.py", line 1695, in save
fp = builtins.open(filename, "w+b")
IOError: [Errno 2] No such file or directory: 'res/org22.jpg'
Please help me figure it out.
Hi,
Can you mention the IOU that you are getting on the dataset ? It will help us to verify whether our implementation of the code is accurate or not. Also could you find ways to improve the performance ?
Thanks.
Seems the folder is data/portraitFCN_data/ is missing, even when downloading from data link.
I train FCN.py by about 10 minutes per 100 steps, and most of the time the GPU looks idle. Is this right? Is there any way to speed up ?
Thanks!
possibly insufficient driver version: 390.87.0
F tensorflow/core/kernels/conv_ops.cc:717] Check failed: stream->parent()->GetConvolveAlgorithms( conv_parameters.ShouldIncludeWinogradNonfusedAlgo(), &algorithms)
Aborted (core dumped)
CAN ANY ONE HELP ME?
Thank you verymuch!
Tensorflow training always stops When training iterations reaches 6147. It seems the loss stills very high, So what should I do to keep model keeps training until convergence?
Can anyone share pretrained weights for FCN.py and FCN_plus.py?
我用cpu TF跑FCN.py,一天半才跑到Step2000,不知道大概要多长时间才能跑完
I'm trying to play with this, but the data doesn't seem to be available. Could you point me to a non-baidu located source?
The provided tracker mat files contain face points oriented on 800x600 image size, but in the code images don't seem to be resized.
Also I don't see pretrained tensorflow models anywhere, would you please share them?
Where to find portraitFCN_data and portraitFCN+_data. I could download all other files except these from the link mentioned.
when i using download_data.m to get the data set ,it never works, the url seems invalid.
My system crashes when I run the script to train the model.
I am using a Titan X Pascal GPU and even on the script is getting stuck.
I have changed the following parts of the script to accommodate updates on the new versions of tensorflow. (historgram_summary, scalar_sammary, sparse_softmax_cross_entropy_with_logits API's were giving me errors).
Anybody face the same issue?
This failure is this:
File "FCN_plus.py", line 295, in <module> tf.app.run() File "/home/gpu/liuruoze/Matting/venv2/local/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 43, in run sys.exit(main(sys.argv[:1] + flags_passthrough)) File "FCN_plus.py", line 181, in main train_dataset_reader = BatchDatset('data/trainlist.mat') File "/home/gpu/liuruoze/Matting/portrait_plus.py", line 24, in __init__ self.cur_imgs, self.cur_labels = self.get_variations(self.imgs[0]) File "/home/gpu/liuruoze/Matting/portrait_plus.py", line 192, in get_variations si_img = i_img.resize(resize_box, Image.ANTIALIAS) File "/home/gpu/liuruoze/Matting/venv2/local/lib/python2.7/site-packages/PIL/Image.py", line 1559, in resize return self._new(self.im.resize(size, resample)) TypeError: integer argument expected, got float
It is helpful if you could give me some advices.
For each image in the data/images folder, there is a corresponding .mat file in the data/images_tracker folder. I have two questions.
(1) What is the purpose of these .mat files?
(2) For a new image, how to generate the corresponding .mat file?
Many thanks.
Excuse me please, i am wondering how to use this repo. And i have already setup the running environment.
Hi,
Thanks for you work!
its looking awesome output.
I want to integrate your demo into android project. Is it possible to integrate model into android Project? If it possible, then How can i integrate this model into android project?
Can you please give some suggestions?
Thanks in advance.
paper作者的网页已经打不开了,能否提供一个资源下载地址?感谢
此外,你的实现Grad. Error和 Conn. Error 分别是多少,想参考下
实际做的事是segmentation,不是matting.
When I run FCN.py to get prediction result, it shows that " IOError: [Errno 2] No such file or directory: 'res/org0.jpg' ", what should I do? Before prediction, I must run training firstly?
如题
请问作者有技术交流群吗?或者联系方式?
The bottom center image of each example is the mean mask aligned with the given portrait. They're intended to be rough estimates of what the final mask should look like. The misalignment in the first example is due to using an incorrect transform.
After training I try to run pred(), but get error
setting up vgg initialized conv layers ...
Traceback (most recent call last):
File "FCN_plus.py", line 297, in
pred()
File "FCN_plus.py", line 242, in pred
sft = tf.nn.softmax(logits)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_nn_ops.py", line 1396, in softmax
result = _op_def_lib.apply_op("Softmax", logits=logits, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 703, in apply_op
op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2312, in create_op
set_shapes_for_outputs(ret)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1704, in set_shapes_for_outputs
shapes = shape_func(op)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/common_shapes.py", line 45, in _ShapeFunction
return [op.inputs[0].get_shape().with_rank(rank)]
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/tensor_shape.py", line 641, in with_rank
raise ValueError("Shape %s must have rank %d" % (self, rank))
ValueError: Shape (?, ?, ?, 2) must have rank 2
Do You have any guess what is the problem?
一定要给test annotation么,如果预测的时候只给image,需要改网络么,我现在预测结果都是0
Limit: 3406036992
InUse: 2376772608
MaxInUse: 2613990400
NumAllocs: 197
MaxAllocSize: 411041792
W tensorflow/core/common_runtime/bfc_allocator.cc:274] ________***********************_____________________
W tensorflow/core/common_runtime/bfc_allocator.cc:275] Ran out of memory trying to allocate 2.29GiB. See logs for memory state.
W tensorflow/core/framework/op_kernel.cc:993] Resource exhausted: OOM when allocating tensor with shape[20,800,600,64]
I have 4G ram, but nevertheless how much I change the batch size to , it won't work, anyone can help me?
我batch size不管怎么调,始终显示上面的错误,而且batch size越小,INuse的量反而越大
可以提供百度盤以外的數據集載點嘛 謝謝!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.