hyeonwoonoh / deconvnet Goto Github PK

View Code? Open in Web Editor NEW

325.0 325.0 151.0 116 KB

DeconvNet : Learning Deconvolution Network for Semantic Segmentation

License: Other

Shell 22.32% Python 21.56% MATLAB 56.12%

deconvnet's People

Contributors

Stargazers

Watchers

Forkers

amiltonwong mrgloom milestonesvn robinn931219 liulei2776 myungsub mlstudy vzhangmeng726 danelzh yutingzhang xzhaogit pinglmlcv ilovecv jwmneu wang-lei williford caomw lawrence444 rollingstone hyang428 daidengxin westamine nagyistge qzhao gadkins syzh1991 mrsaibot 573671712 kltsyn imclab liao1995 eriche2016 kuyun-zhangyang benjamesbabala liuzw123 evanyiweizhao zaneyang ljpadam agatabcool fuxicv tuanthng sxq2004123 maga33 bozorgtabar amds123 johannah arasharchor 10sun kilakila-heart alonsojasl jay-mahadeokar tinyjie linzhineng silasxue miltanchoudhury antran89 srinivasgowda097 ominux ml-lab huahuichen tinyloop coolhebei franklinluo rongchangzhao lusmd fanghaizhao shreyassaxena mt7go jdc08161063 fqul canbuoy marshb wang4959520 luohongcheng dreadlord1984 guker shownx nimrodshn oppa3109 ilibx poonono shichaosuper shanyuhu zzmjohn tianzq yheno jinming-su davidtranno1 lg-code-repo htt1994 yangerkun kadok dacson kingofoz sunshinnnne329 jiaxx soledad89 agataf lihua213 cwczyj

deconvnet's Issues

Data URL is down

I'm not being able to dowload the data.
Is it something from me? I even checked and the url provided is down.

Can the data be provided by other way?

DeconvNet fine tuning

Thanks for making your work available, deeply appreciated.

I'm hoping to fine tune your pre-trained DeconvNet, but I noticed that stage 003 make BN layers testable. Hence I assume that I should not load 'DeconvNet_trainval_inference.caffemodel' straightaway and starts the training, but I'm not sure. Please let me know if my concern is valid, if it does, can you please upload the caffemodel after stage 002 training? Thanks a lot.

error at cudaSuccess?

Hi,

I'm trying to train DeconvNet and run 001_start_train.sh ,but the error is:

F0907 11:37:08.948395 5462 syncedmem.cpp:51] Check failed: error == cudaSuccess (2 vs. 0) out of memory
*** Check failure stack trace: ***
@ 0x7fe9bc71da3d google::LogMessage::Fail()
@ 0x7fe9bc721ed7 google::LogMessage::SendToLog()
@ 0x7fe9bc71fd39 google::LogMessage::Flush()
@ 0x7fe9bc72003d google::LogMessageFatal::~LogMessageFatal()
@ 0x6a6f8e caffe::SyncedMemory::to_gpu()
@ 0x6a6b20 caffe::SyncedMemory::mutable_gpu_data()
@ 0x6aacfd caffe::Blob<>::mutable_gpu_data()
@ 0x6d8b7d caffe::UnpoolingLayer<>::Forward_gpu()
@ 0x577814 caffe::Layer<>::Forward()
@ 0x58e419 caffe::Net<>::ForwardFromTo()
@ 0x58e171 caffe::Net<>::ForwardPrefilled()
@ 0x58e5a4 caffe::Net<>::Forward()
@ 0x6b3962 caffe::Solver<>::Test()
@ 0x6b376d caffe::Solver<>::TestAll()
@ 0x6b2b51 caffe::Solver<>::Step()
@ 0x6b268b caffe::Solver<>::Solve()
@ 0x5738c1 train()
@ 0x575903 main
@ 0x36e9a1ed5d (unknown)
@ 0x572df9 (unknown)
./start_train.sh: line 6: 5462 Aborted GLOG_log_dir=$LOGDIR $CAFFE train -solver $SOLVER -weights $WEIGHTS -gpu 0
cp: cannot stat `./snapshot/stage_1_train_iter_20000.caffemodel': No such file or directory

I'm a new newbie in caffe , can u help me? thinks.

Detecting tiny objects?

Hello,

My issue is I want to detect tiny objects in images (e.g. eyes) and trying to do so, always the network learns to predict background. This indeed is about the 99% of the image, so the prediction is pretty accurate.

I tried the same trick as yours at stage 1, using cropped images, extracting all eyes on each image. Here the background is around 40% of the image, but at training stage learns to predict all image as background again. I also have different classes (24), all of these are little objects on the image.

Do you think your architecture is suitable for these kind of problems?

Exist any chance to penalize the misclassification of others categories except background?

MEX-file segmentation violation is detected during running demo

Hi!

I am trying to run_demo.m, but while executing MEX-file segmentation violation is detected (whole error message is displayed lower). The error appears during execution of command caffe('init', config.Path.CNN.model_proto, config.Path.CNN.model_data) from cache_FCN8s_results.m. Both variables config.Path.CNN.model_proto and config.Path.CNN.model_data contain correct path to model and its definition.

I have successfully installed (using Matlab R2012a, Ubuntu 14.04) modified version of caffe without any complications.

make all
make test
make runtest
make matcaffe

I have also set environmental variables in .bashrc and run . ~/.bashrc before executing run_demo.m.

export CAFFE_ROOT=/home/martin/DeconvNet/caffe
export CAFFE_PYTHON_PATH=/home/martin/DeconvNet/caffe/python
export PYTHONPATH=/home/martin/DeconvNet/caffe/python:$PYTHONPATH
export LD_LIBRARY_PATH=/home/martin/DeconvNet/caffe/build/lib:$LD_LIBRARY_PATH

I know that the code was tested only with Matlab 2014b, but since the compilation using older version of Matlab was successful I don't think that would be a reason for segmentation violation.

I tried to load model and its definition using python interface, but the modified version of caffe from @HyeonwooNoh seems to be a bit older, so it would need to be modified according to current master branch of caffe and in this I still haven't succeed.

Questions:

Does anybody know what could cause segmentation violation?
How to easily adapt code in order to be able to use python interface?

Thank you!

Martin

------------------------------------------------------------------------
       Segmentation violation detected at Wed Dec  2 17:13:01 2015
------------------------------------------------------------------------

Configuration:
  Crash Decoding  : Disabled
  Current Visual  : None
  Default Encoding: UTF-8
  GNU C Library   : 2.19 stable
  MATLAB Root     : /usr/local/MATLAB/R2012a
  MATLAB Version  : 7.14.0.739 (R2012a)
  Operating System: Linux 3.13.0-63-generic #103-Ubuntu SMP Fri Aug 14 21:42:59 UTC 2015 x86_64
  Processor ID    : x86 Family 6 Model 62 Stepping 4, GenuineIntel
  Virtual Machine : Java 1.6.0_17-b04 with Sun Microsystems Inc. Java HotSpot(TM) 64-Bit Server VM mixed mode
  Window System   : No active display

Fault Count: 1


Abnormal termination:
Segmentation violation

Register State (from fault):
  RAX = 0000000000000000  RBX = 0000000000000000
  RCX = 0000000000000065  RDX = 0000000000000000
  RSP = 00007f040adf8d60  RBP = 00007f034581c140
  RSI = 0000000000000000  RDI = 00007f03458d5780

   R8 = 0000000000000000   R9 = 00007f0375901740
  R10 = 00000000b2088fbc  R11 = 00000000104e428e
  R12 = 0000000000000065  R13 = 0000000100010008
  R14 = 00007f03fc8c7ee0  R15 = 00007f040adf90a0

  RIP = 00007f0363929242  EFL = 0000000000010246

   CS = 0033   FS = 0000   GS = 0000

Stack Trace (from fault):
[  0] 0x00007f041fc3092e    /usr/local/MATLAB/R2012a/bin/glnxa64/libmwfl.so+00370990 _ZN2fl4diag15stacktrace_base7captureERKNS0_14thread_contextEm+000158
[  1] 0x00007f041fc337d0    /usr/local/MATLAB/R2012a/bin/glnxa64/libmwfl.so+00382928
[  2] 0x00007f041fc33b3b    /usr/local/MATLAB/R2012a/bin/glnxa64/libmwfl.so+00383803 _ZN2fl4diag13terminate_logEPKcRKNS0_14thread_contextE+000171
[  3] 0x00007f041eb17203   /usr/local/MATLAB/R2012a/bin/glnxa64/libmwmcr.so+01253891 _ZN2fl4diag13terminate_logEPKcPK8ucontext+000067
[  4] 0x00007f041eb140fd   /usr/local/MATLAB/R2012a/bin/glnxa64/libmwmcr.so+01241341
[  5] 0x00007f041eb1579d   /usr/local/MATLAB/R2012a/bin/glnxa64/libmwmcr.so+01247133
[  6] 0x00007f041eb15925   /usr/local/MATLAB/R2012a/bin/glnxa64/libmwmcr.so+01247525
[  7] 0x00007f041eb15f01   /usr/local/MATLAB/R2012a/bin/glnxa64/libmwmcr.so+01249025
[  8] 0x00007f041eb163f5   /usr/local/MATLAB/R2012a/bin/glnxa64/libmwmcr.so+01250293
[  9] 0x00007f041d166340              /lib/x86_64-linux-gnu/libpthread.so.0+00066368
[ 10] 0x00007f0363929242             /usr/local/cuda/lib64/libcurand.so.7.5+00148034 curandCreateGenerator+000658
[ 11] 0x00007f036f22068c /home/martin/ml/DeconvNet/inference/caffe/matlab/caffe/caffe.mexa64+00931468
[ 12] 0x00007f036f1d8b25 /home/martin/ml/DeconvNet/inference/caffe/matlab/caffe/caffe.mexa64+00637733
[ 13] 0x00007f036f2ecb0f /home/martin/ml/DeconvNet/inference/caffe/matlab/caffe/caffe.mexa64+01768207
[ 14] 0x00007f036f2f52df /home/martin/ml/DeconvNet/inference/caffe/matlab/caffe/caffe.mexa64+01802975
[ 15] 0x00007f036f2f80cc /home/martin/ml/DeconvNet/inference/caffe/matlab/caffe/caffe.mexa64+01814732
[ 16] 0x00007f036f16cfa4 /home/martin/ml/DeconvNet/inference/caffe/matlab/caffe/caffe.mexa64+00196516
[ 17] 0x00007f036f16d391 /home/martin/ml/DeconvNet/inference/caffe/matlab/caffe/caffe.mexa64+00197521 mexFunction+000384
[ 18] 0x00007f04178a9cca     /usr/local/MATLAB/R2012a/bin/glnxa64/libmex.so+00109770 mexRunMexFile+000090
[ 19] 0x00007f04178a5f79     /usr/local/MATLAB/R2012a/bin/glnxa64/libmex.so+00094073
[ 20] 0x00007f04178a6de1     /usr/local/MATLAB/R2012a/bin/glnxa64/libmex.so+00097761
[ 21] 0x00007f041e7bf063 /usr/local/MATLAB/R2012a/bin/glnxa64/libmwm_dispatcher.so+00479331 _ZN8Mfh_file11dispatch_fhEiPP11mxArray_tagiS2_+000515
[ 22] 0x00007f041e085476 /usr/local/MATLAB/R2012a/bin/glnxa64/libmwm_interpreter.so+01987702
[ 23] 0x00007f041e036426 /usr/local/MATLAB/R2012a/bin/glnxa64/libmwm_interpreter.so+01664038
[ 24] 0x00007f041e03abe4 /usr/local/MATLAB/R2012a/bin/glnxa64/libmwm_interpreter.so+01682404
[ 25] 0x00007f041e037333 /usr/local/MATLAB/R2012a/bin/glnxa64/libmwm_interpreter.so+01667891
[ 26] 0x00007f041e038037 /usr/local/MATLAB/R2012a/bin/glnxa64/libmwm_interpreter.so+01671223
[ 27] 0x00007f041e0a1690 /usr/local/MATLAB/R2012a/bin/glnxa64/libmwm_interpreter.so+02102928
[ 28] 0x00007f041e7bf063 /usr/local/MATLAB/R2012a/bin/glnxa64/libmwm_dispatcher.so+00479331 _ZN8Mfh_file11dispatch_fhEiPP11mxArray_tagiS2_+000515
[ 29] 0x00007f041e085476 /usr/local/MATLAB/R2012a/bin/glnxa64/libmwm_interpreter.so+01987702
[ 30] 0x00007f041e012d24 /usr/local/MATLAB/R2012a/bin/glnxa64/libmwm_interpreter.so+01518884
[ 31] 0x00007f041e03a37e /usr/local/MATLAB/R2012a/bin/glnxa64/libmwm_interpreter.so+01680254
[ 32] 0x00007f041e037333 /usr/local/MATLAB/R2012a/bin/glnxa64/libmwm_interpreter.so+01667891
[ 33] 0x00007f041e038037 /usr/local/MATLAB/R2012a/bin/glnxa64/libmwm_interpreter.so+01671223
[ 34] 0x00007f041e0a1690 /usr/local/MATLAB/R2012a/bin/glnxa64/libmwm_interpreter.so+02102928
[ 35] 0x00007f041e7bf063 /usr/local/MATLAB/R2012a/bin/glnxa64/libmwm_dispatcher.so+00479331 _ZN8Mfh_file11dispatch_fhEiPP11mxArray_tagiS2_+000515
[ 36] 0x00007f041e085476 /usr/local/MATLAB/R2012a/bin/glnxa64/libmwm_interpreter.so+01987702
[ 37] 0x00007f041e012d24 /usr/local/MATLAB/R2012a/bin/glnxa64/libmwm_interpreter.so+01518884
[ 38] 0x00007f041e03a37e /usr/local/MATLAB/R2012a/bin/glnxa64/libmwm_interpreter.so+01680254
[ 39] 0x00007f041e037333 /usr/local/MATLAB/R2012a/bin/glnxa64/libmwm_interpreter.so+01667891
[ 40] 0x00007f041e038037 /usr/local/MATLAB/R2012a/bin/glnxa64/libmwm_interpreter.so+01671223
[ 41] 0x00007f041e0a1690 /usr/local/MATLAB/R2012a/bin/glnxa64/libmwm_interpreter.so+02102928
[ 42] 0x00007f041e7bf063 /usr/local/MATLAB/R2012a/bin/glnxa64/libmwm_dispatcher.so+00479331 _ZN8Mfh_file11dispatch_fhEiPP11mxArray_tagiS2_+000515
[ 43] 0x00007f041e07253b /usr/local/MATLAB/R2012a/bin/glnxa64/libmwm_interpreter.so+01910075
[ 44] 0x00007f041e03081c /usr/local/MATLAB/R2012a/bin/glnxa64/libmwm_interpreter.so+01640476
[ 45] 0x00007f041e02d49f /usr/local/MATLAB/R2012a/bin/glnxa64/libmwm_interpreter.so+01627295
[ 46] 0x00007f041e02dbb5 /usr/local/MATLAB/R2012a/bin/glnxa64/libmwm_interpreter.so+01629109
[ 47] 0x00007f041ed87ade /usr/local/MATLAB/R2012a/bin/glnxa64/libmwbridge.so+00146142
[ 48] 0x00007f041ed8897e /usr/local/MATLAB/R2012a/bin/glnxa64/libmwbridge.so+00149886 mnParser+000622
[ 49] 0x00007f041eafbde2   /usr/local/MATLAB/R2012a/bin/glnxa64/libmwmcr.so+01142242 _ZN11mcrInstance30mnParser_on_interpreter_threadEv+000034
[ 50] 0x00007f041eade51a   /usr/local/MATLAB/R2012a/bin/glnxa64/libmwmcr.so+01021210
[ 51] 0x00007f041eade598   /usr/local/MATLAB/R2012a/bin/glnxa64/libmwmcr.so+01021336
[ 52] 0x00007f041f310053 /usr/local/MATLAB/R2012a/bin/glnxa64/libmwservices.so+00823379
[ 53] 0x00007f041f310315 /usr/local/MATLAB/R2012a/bin/glnxa64/libmwservices.so+00824085
[ 54] 0x00007f041f30ebc1 /usr/local/MATLAB/R2012a/bin/glnxa64/libmwservices.so+00818113
[ 55] 0x00007f0413fd7605   /usr/local/MATLAB/R2012a/bin/glnxa64/libmwuix.so+00505349
[ 56] 0x00007f041f3949a1 /usr/local/MATLAB/R2012a/bin/glnxa64/libmwservices.so+01366433 _ZSt8for_eachIN9__gnu_cxx17__normal_iteratorIPN5boost8weak_ptrIN4sysq10ws_ppeHookEEESt6vectorIS6_SaIS6_EEEENS4_8during_FIS6_NS2_10shared_ptrIS5_EEEEET0_T_SH_SG_+000081
[ 57] 0x00007f041f395aab /usr/local/MATLAB/R2012a/bin/glnxa64/libmwservices.so+01370795
[ 58] 0x00007f041f3935f9 /usr/local/MATLAB/R2012a/bin/glnxa64/libmwservices.so+01361401 _Z25svWS_ProcessPendingEventsiib+000665
[ 59] 0x00007f041eadd76f   /usr/local/MATLAB/R2012a/bin/glnxa64/libmwmcr.so+01017711
[ 60] 0x00007f041eaddc3b   /usr/local/MATLAB/R2012a/bin/glnxa64/libmwmcr.so+01018939
[ 61] 0x00007f041eaddd97   /usr/local/MATLAB/R2012a/bin/glnxa64/libmwmcr.so+01019287
[ 62] 0x00007f041d15e182              /lib/x86_64-linux-gnu/libpthread.so.0+00033154
[ 63] 0x00007f041ce8b47d                    /lib/x86_64-linux-gnu/libc.so.6+01025149 clone+000109


This error was detected while a MEX-file was running. If the MEX-file
is not an official MathWorks function, please examine its source code
for errors. Please consult the External Interfaces Guide for information
on debugging MEX-files.

Using results of stage1 directly

Hi ,
Thanks for releasing your excellent work.
I would like to directly use the results of stage 1 training, without the further stages.
I attempted to get the output using the following

    prototxt  = 'DeconvNet_inference_deploy.prototxt'
    caffemodel = 'snapshot/stage_1_train_iter_6000.caffemodel'
    net = caffe.Net(prototxt,caffemodel)
    im = Image.open(imagename)
    im = im.resize(dims,Image.ANTIALIAS)
    in_ = np.array(im, dtype=np.float32)
    in_ = in_[:,:,::-1]
    in_ -= np.array((104.0,116.7,122.7))
    in_ = in_.transpose((2,0,1))
    net.blobs['data'].reshape(1, *in_.shape)
    net.blobs['data'].data[...] = in_
    # run net and take argmax for prediction
    net.forward()
    out = net.blobs['seg-score'].data[0].argmax(axis=0)
   result = Image.fromarray(out.astype(np.uint8))
    result.save(outname)

but so far get only images with all pixels = 0
Is there something else I need to do ? This script, with minor changes, works for the fcn segmentations

In generate_EDeconvNet_CRF_results.m line 115

Why is the ensemble '.*'? From your paper, I get 'mean'.

How to produce boxes_padded?

Hi!

I have run your run_demo.m successfully and got the final EDeconvNet_CRF result. I am studying your method now, and I have two questions.

Qustion 1:
Though I have downloaded the code producing edgebox object proposal and got the object proposal, I still cannot figure out how the edgebox_cached produces .
Qustion 2:
I cannot find out why there are negative coordinates in object proposals in the stage-2 training. The negative coordinates is in "stage_2_train_imgset/train.txt" or "stage_2_train_imgset/val.txt", part of them are in the following.

/VOC2012_SEG_AUG/images/2009_004507.png /VOC2012_SEG_AUG/segmentations/2009_004507.png 267 -20 517 230
/VOC2012_SEG_AUG/images/2009_002527.png /VOC2012_SEG_AUG/segmentations/2009_002527.png 88 -24 387 275
/VOC2012_SEG_AUG/images/2009_003806.png /VOC2012_SEG_AUG/segmentations/2009_003806.png 34 -116 542 392
/VOC2012_SEG_AUG/images/2007_009245.png /VOC2012_SEG_AUG/segmentations/2007_009245.png -30 -92 548 486

Thank you!

Lili Zhang

Download training set

Thank you very much for publishing helpful source code.
I am trying to reproduce this work but I can not download this dataset.
So, where can I download this dataset?

What do these four numbers stand for?

There are four numbers in each row in the /DeconvNet/data/imagesets/stage_2_train_imgset/train.txt. What do these four numbers stand for? I try to train DeconvNet in my own dataset. So how to get these numbers? Thanks.
/VOC2012_SEG_AUG/images/2010_002729.png /VOC2012_SEG_AUG/segmentations/2010_002729.png 53 188 289 424
/VOC2012_SEG_AUG/images/2010_004179.png /VOC2012_SEG_AUG/segmentations/2010_004179.png -52 -29 308 331
What do these four numbers -52 -29 308 331 stand for?

Unknown enumeration value of "BN" for field "type".

Use shell script can train net, but use python script to train ,I got this error " Unknown enumeration value of "BN" for field "type"."

Makefile:479: recipe for target '.build_release/src/caffe/layers/euclidean_loss_layer.o' failed

I use the Makefile.config copying from my machine,which compiling successfully on the initial version of caffe ,but when it goes on the modified version,it made mistakes like this：

Makefile:479: recipe for target '.build_release/src/caffe/layers/euclidean_loss_layer.o' failed

make: *** [.build_release/src/caffe/layers/euclidean_loss_layer.o] Error 1
and when I got into that directory,using "find" command

sheng@ML ~/DeconvNet/caffe $ find ~/DeconvNet/caffe/ -name euclidean_loss_layer.o
sheng@ML ~/DeconvNet/caffe $

I can not find the "euclidean_loss_layer.o",Any mistakes I make?

Out of memory error while there is enough memory

So I setup everything, and when I run the training/001_start_train.sh, I get the out of memory error:
Check failed: error == cudaSuccess (2 vs. 0) out of memory

I have 6GB on my GPU, and constantly watching its memory-usage. I even tried setting the batch-size to 1 for both training and testing. None of these helped and I still receive this error.

Since I get passed the network initialization and the initial memory size check, my guess is something is going on in the IMAGE_SEG_DATA layer.
Are you loading all the data into the GPU memory at the very beginning to reduce disk access?

Converting caffe model to Keras

I was trying to convert deconvnet caffe model to keras with tensorflow backend. I only transpose convolutional layer weights since I thought caffe and tensorflow use correlation. However I believe I've got stuck with probably batch normalization layers. I'm extracting bn weights but I don't know if they are required to be rotated or something else when using with keras batch normalization layer? Until now I've tried to use them by reshaping and setting to gamma and beta variables at keras batch norm layer.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.