Comments (11)
I have never met this problem. It seems the problem is caused by the mismatch of label and data. What's your mxnet version?
from cbst.
I have never met this problem. It seems the problem is caused by the mismatch of label and data. What's your mxnet version?
I am using mxnet 1.3.0, just follow the instructions in the README and haven't changed the code.
from cbst.
That's weird. Can you post all your environment configuration? Let might help.
It seems the label is reshaped to 1 x 26912. (928/8 * 1856/8 = 26912; 8 is the downsampling rate.) There should be something wrong in the dataloader/loss calculation part.
I just rerun the code on my computer. It is ok.
from cbst.
`2019-09-26 09:56:43,246 Host start with arguments Namespace(backward_do_mirror=False, base_lr=None, batch_images=None, cache_images=None, check_start=1, check_step=4, crop_size=None, data_root='DATA_ROOT_GTA5', data_root_tgt='/home/guangrli/Data/cityscapes/', dataset='gta', dataset_tgt='cityscapes', debug=False, from_epoch=0, gpus='0', idx_round=0, init_src_port=0.3, init_tgt_port=0.15, kc_policy='cb', kvstore='local', log_file='gta_rna-a1_cls19_s8_ep-0000_train.log', lr_steps=None, lr_type=None, mine_id_address=None, mine_id_number=3, mine_port=0.5, mine_thresh=0.001, model='cityscapes_rna-a1_cls19_s8', no_mxnet_cudnn_autotune_default=True, origin_size=2048, origin_size_tgt=2048, output='gta2city/cbst-sp', phase='val', prefetch_threads=1, prefetcher='thread', scale_rate_range='0.7,1.3', seed_int=0, split='train', split_tgt='train', stop_epoch=None, test_flipping=True, test_scales='1850', test_steps=1, to_epoch=None, weight_decay=0.0005, weights='../Data/gta_rna-a1_cls19_s8_ep-0000.params', with_prior='True')
2019-09-26 09:56:43,247 Host and model specs {'feat_stride': 8, 'dataset': 'gta', 'classes': 19, 'net_type': 'rna', 'net_name': 'a1', 'lr_params': {'base': 0.1, 'type': 'fixed', 'args': None}, 'dataset_tgt': 'cityscapes'}
Traceback (most recent call last):
File "issegm/solve_ST.py", line 1007, in
Level 0
[(64L, 3L, 224L, 224L)]
Level 1
[(64L, 64L, 224L, 224L)]
Level 2
First block on level 2, stride: 2, dilate: 1
[(64L, 128L, 112L, 112L)]
Level 3
First block on level 3, stride: 2, dilate: 1
[(64L, 256L, 56L, 56L)]
Level 4
First block on level 4, stride: 2, dilate: 1
[(64L, 512L, 28L, 28L)]
Level 5
First block on level 5, stride: 2, dilate: 1
[(64L, 1024L, 28L, 28L)]
Level 6
First block on level 6, stride: 2, dilate: 2
Level 7
[(64L, 4096L, 28L, 28L)]
Scores
[(64L, 19L, 28L, 28L)]
_val_impl(args, model_specs, logger)
File "issegm/solve_ST.py", line 806, in _val_impl
mod.bind(dataiter_tgt.provide_data, dataiter_tgt.provide_label, for_training=False, force_rebind=True)
File "/home/guangrli/anaconda/envs/py2/lib/python2.7/site-packages/mxnet/module/module.py", line 429, in bind
state_names=self._state_names)
File "/home/guangrli/anaconda/envs/py2/lib/python2.7/site-packages/mxnet/module/executor_group.py", line 279, in init
self.bind_exec(data_shapes, label_shapes, shared_group)
File "/home/guangrli/anaconda/envs/py2/lib/python2.7/site-packages/mxnet/module/executor_group.py", line 375, in bind_exec
shared_group))
File "/home/guangrli/anaconda/envs/py2/lib/python2.7/site-packages/mxnet/module/executor_group.py", line 662, in _bind_ith_exec
shared_buffer=shared_data_arrays, **input_shapes)
File "/home/guangrli/anaconda/envs/py2/lib/python2.7/site-packages/mxnet/symbol/symbol.py", line 1528, in simple_bind
raise RuntimeError(error_msg)
RuntimeError: simple_bind error. Arguments:
data: (1, 3L, 928L, 1856L)
softmax_label: (1, 26912L)
[09:56:45] src/storage/storage.cc:137: Compile with USE_CUDA=1 to enable GPU usage
Stack trace returned 10 entries:
[bt] (0) /home/guangrli/anaconda/envs/py2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x1e4d7a) [0x2b261db39d7a]
[bt] (1) /home/guangrli/anaconda/envs/py2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x1e53e1) [0x2b261db3a3e1]
[bt] (2) /home/guangrli/anaconda/envs/py2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2f70b05) [0x2b26208c5b05]
[bt] (3) /home/guangrli/anaconda/envs/py2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2f74adf) [0x2b26208c9adf]
[bt] (4) /home/guangrli/anaconda/envs/py2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2f769ff) [0x2b26208cb9ff]
[bt] (5) /home/guangrli/anaconda/envs/py2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2910c53) [0x2b2620265c53]
[bt] (6) /home/guangrli/anaconda/envs/py2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2910dcd) [0x2b2620265dcd]
[bt] (7) /home/guangrli/anaconda/envs/py2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x291e8df) [0x2b26202738df]
[bt] (8) /home/guangrli/anaconda/envs/py2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2925280) [0x2b262027a280]
[bt] (9) /home/guangrli/anaconda/envs/py2/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x293385e) [0x2b262028885e]
Traceback (most recent call last):
File "issegm/solve_AO.py", line 168, in
main()
File "issegm/solve_AO.py", line 133, in main
for path in execute(cmd_val):
File "issegm/solve_AO.py", line 165, in execute
raise subprocess.CalledProcessError(return_code, cmd)
subprocess.CalledProcessError: Command '['python', 'issegm/solve_ST.py', '--dataset', 'gta', '--dataset-tgt', 'cityscapes', '--data-root', 'DATA_ROOT_GTA5', '--data-root-tgt', '/home/guangrli/Data/cityscapes/', '--split-tgt', 'train', '--output', 'gta2city/cbst-sp', '--model', 'cityscapes_rna-a1_cls19_s8', '--phase', 'val', '--weights', '../Data/gta_rna-a1_cls19_s8_ep-0000.params', '--mine-id-number', '3', '--test-scales', '1850', '--gpus', '0', '--init-tgt-port', '0.15', '--idx-round', '0', '--no-cudnn', '--test-flipping', '--with-prior', 'True', '--kc-policy', 'cb']' returned non-zero exit status 1`
As in solver_ST.py:
Line 802 in a28d842
The label is reshaped to (1, 26912).
from cbst.
What's your hardware configurations?
from cbst.
CUDA 10.0,
GPU: NVIDIA quadro P5000,
Linux: Red Hat 7.5
from cbst.
The code runs well in my desktop. Did you compile mxnet with cuda?
from cbst.
My configuration is:
Ubuntu 16.04,
Titan Xp/X/ GTX 1080
CUDA 8.0
from cbst.
Thanks for your prompt reply, I'll try to check it.
from cbst.
Could you show me print of [ dataiter_tgt.provide_data] and [dataiter_tgt.provide_label] in this line? I want to check if the shape is correct.
Line 806 in a28d842
Plus, I have noticed the difference in version of CUDA, I will try to check it.
from cbst.
from cbst.
Related Issues (16)
- VGG16 code HOT 2
- About the basenet HOT 2
- SYNTHIA-RAND-CITYSCAPES dataset HOT 1
- Prior_array.mat file generation HOT 2
- Save confidence scores of target domain as class-wise vectors HOT 2
- File mine_id.npy and mine_id_priority.npy
- unable to download SYNTHIA-RAND-CITYSCAPES dataset
- Pytorch version? HOT 8
- pretrained file corrupted HOT 4
- About the Results HOT 1
- Training Problem HOT 9
- About the finetuning of self-training HOT 2
- About label definition of SYNTHIA HOT 1
- About label format of GTA HOT 1
- BDD-V HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cbst.