swoook / ddrnet Goto Github PK

View Code? Open in Web Editor NEW

1.0 1.0 0.0 1.43 MB

Cloned from chenjun2hao/DDRNet (https://github.com/chenjun2hao/DDRNet.pytorch).

License: Other

Python 93.61% C++ 2.53% Cuda 2.55% Jupyter Notebook 1.30%

ddrnet's People

Contributors

Stargazers

Watchers

ddrnet's Issues

ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 512, 1, 1])

Issue description

Confirmed the error messages below when trying to execute ${REPO_ROOT}/main.py with an image of arbitrary size

ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 512, 1, 1])

Code example

configuration in launch.json:

        {
            "name": "Python: Current File",
            "type": "python",
            "request": "launch",
            "program": "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/site-packages/torch/distributed/launch.py", //"${file}",
            "console": "integratedTerminal",
            "args": [
                "--nproc_per_node", "1",
                "${workspaceRoot}/main.py",
                "--mode", "train",
                "--cfg_path", "./experiments/duts/ddrnet23_slim_poolnet_train_scheme.yaml",
                ]
        },

Error messages and stack traces:

Traceback (most recent call last):
  File "/home/swook/.vscode-server/extensions/ms-python.python-2021.6.944021595/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 3293, in <module>
    main()
  File "/home/swook/.vscode-server/extensions/ms-python.python-2021.6.944021595/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 3286, in main
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/home/swook/.vscode-server/extensions/ms-python.python-2021.6.944021595/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 2360, in run
    return self._exec(is_module, entry_point_fn, module_name, file, globals, locals)
  File "/home/swook/.vscode-server/extensions/ms-python.python-2021.6.944021595/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 2367, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/home/swook/.vscode-server/extensions/ms-python.python-2021.6.944021595/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydev_imps/_pydev_execfile.py", line 25, in execfile
    exec(compile(contents + "\n", file, 'exec'), glob, loc)
  File "/data/swook/repos/chenjun2hao/ddrnet/main.py", line 115, in <module>
    main(args)
  File "/data/swook/repos/chenjun2hao/ddrnet/main.py", line 99, in main
    solver.train()
  File "/data/swook/repos/chenjun2hao/ddrnet/sod/solver.py", line 123, in train
    sal_pred = self.net(sal_image)
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/data/swook/repos/chenjun2hao/ddrnet/lib/models/ddrnet_23_slim.py", line 346, in forward
    self.spp(self.layer5(self.relu(x))),
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/data/swook/repos/chenjun2hao/ddrnet/lib/models/ddrnet_23_slim.py", line 179, in forward
    x_debug = self.scale3(x)
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/site-packages/torch/nn/modules/container.py", line 119, in forward
    input = module(input)
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/site-packages/torch/nn/modules/batchnorm.py", line 537, in forward
    return F.batch_norm(
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/site-packages/torch/nn/functional.py", line 2147, in batch_norm
    _verify_batch_size(input.size())
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/site-packages/torch/nn/functional.py", line 2114, in _verify_batch_size
    raise ValueError("Expected more than 1 value per channel when training, got input size {}".format(size))
ValueError: Expected more than 1 value per channel when training, got input size torch.Size([1, 512, 1, 1])
Killing subprocess 43746
Traceback (most recent call last):
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/swook/.vscode-server/extensions/ms-python.python-2021.6.944021595/pythonFiles/lib/python/debugpy/__main__.py", line 45, in <module>
    cli.main()
  File "/home/swook/.vscode-server/extensions/ms-python.python-2021.6.944021595/pythonFiles/lib/python/debugpy/../debugpy/server/cli.py", line 444, in main
    run()
  File "/home/swook/.vscode-server/extensions/ms-python.python-2021.6.944021595/pythonFiles/lib/python/debugpy/../debugpy/server/cli.py", line 285, in run_file
    runpy.run_path(target_as_str, run_name=compat.force_str("__main__"))
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/runpy.py", line 265, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/site-packages/torch/distributed/launch.py", line 340, in <module>
    main()
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/site-packages/torch/distributed/launch.py", line 326, in main
    sigkill_handler(signal.SIGTERM, None)  # not coming back
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/site-packages/torch/distributed/launch.py", line 301, in sigkill_handler
    raise subprocess.CalledProcessError(returncode=last_return_code, cmd=cmd)
subprocess.CalledProcessError: Command '['/data/swook/miniconda3/envs/torch18csnet/bin/python', '-u', '/data/swook/repos/chenjun2hao/ddrnet/main.py', '--local_rank=0', '--mode', 'train', '--cfg_path', './experiments/duts/ddrnet23_slim_poolnet_train_scheme.yaml']' returned non-zero exit status 1.

System Info

PyTorch version: 1.8.1
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 16.04.3 LTS (x86_64)
GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.12) 5.4.0 20160609
Clang version: Could not collect
CMake version: version 3.5.1

Python version: 3.8 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: Could not collect
GPU models and configuration: 
GPU 0: GeForce RTX 2080 Ti
GPU 1: GeForce RTX 2080 Ti

Nvidia driver version: 440.33.01
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.20.2
[pip3] torch==1.8.1
[pip3] torchvision==0.9.1
[conda] blas                      1.0                         mkl  
[conda] cudatoolkit               10.2.89              hfd86e86_1  
[conda] ffmpeg                    4.3                  hf484d3e_0    pytorch
[conda] mkl                       2021.2.0           h06a4308_296  
[conda] mkl-service               2.3.0            py38h27cfd23_1  
[conda] mkl_fft                   1.3.0            py38h42c9631_2  
[conda] mkl_random                1.2.1            py38ha9443f7_2  
[conda] numpy                     1.20.2           py38h2d18471_0  
[conda] numpy-base                1.20.2           py38hfae3a4d_0  
[conda] pytorch                   1.8.1           py3.8_cuda10.2_cudnn7.6.5_0    pytorch
[conda] torchvision               0.9.1                py38_cu102    pytorch

RuntimeError: Error(s) in loading state_dict for DualResNet: ...

Backgrounds

Faster SOD model required

10 FPS on CPU
30 FPS on 1080Ti GPU

No public detector found that satisfies the requirements
Recall the 1904.09569 (arxiv.org)
SOTA architectures for SOD follow pixel-wise prediction
Actually, most of them were proposed for segmentation tasks
U-Net is one of them which showed impressive results on segmentation tasks
And PoolNet employed it as a backbone
Maybe we can use real-time segmentation models for SOD
Refer to #6 from swoook/ucnet (github) for more details

Issue description

Confirmed the error messages below when trying to execute ${REPO_ROOT}/main.py

RuntimeError: Error(s) in loading state_dict for DualResNet:

Code example

configuration in launch.json:

        {
            "name": "Python: Current File",
            "type": "python",
            "request": "launch",
            "program": "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/site-packages/torch/distributed/launch.py", //"${file}",
            "console": "integratedTerminal",
            "args": [
                "--nproc_per_node", "1",
                "${workspaceRoot}/main.py",
                "--mode", "train",
                "--cfg_path", "./experiments/duts/ddrnet23_slim_poolnet_train_scheme.yaml",
                ]
        },

Error messages and stack traces:

Traceback (most recent call last):
  File "/home/swook/.vscode-server/extensions/ms-python.python-2021.6.944021595/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 3293, in <module>
    main()
  File "/home/swook/.vscode-server/extensions/ms-python.python-2021.6.944021595/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 3286, in main
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/home/swook/.vscode-server/extensions/ms-python.python-2021.6.944021595/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 2360, in run
    return self._exec(is_module, entry_point_fn, module_name, file, globals, locals)
  File "/home/swook/.vscode-server/extensions/ms-python.python-2021.6.944021595/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 2367, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/home/swook/.vscode-server/extensions/ms-python.python-2021.6.944021595/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydev_imps/_pydev_execfile.py", line 25, in execfile
    exec(compile(contents + "\n", file, 'exec'), glob, loc)
  File "/data/swook/repos/chenjun2hao/ddrnet/main.py", line 115, in <module>
    main(args)
  File "/data/swook/repos/chenjun2hao/ddrnet/main.py", line 98, in main
    solver = Solver(train_loader, None, config, args)
  File "/data/swook/repos/chenjun2hao/ddrnet/sod/solver.py", line 26, in __init__
    self.build_model()
  File "/data/swook/repos/chenjun2hao/ddrnet/sod/solver.py", line 74, in build_model
    self.net.load_state_dict(model_dict, strict=False)
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1223, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for DualResNet:
        size mismatch for seghead_extra.conv2.weight: copying a param with shape torch.Size([19, 64, 1, 1]) from checkpoint, the shape in current model is torch.Size([1, 64, 1, 1]).
        size mismatch for seghead_extra.conv2.bias: copying a param with shape torch.Size([19]) from checkpoint, the shape in current model is torch.Size([1]).
        size mismatch for final_layer.conv2.weight: copying a param with shape torch.Size([19, 64, 1, 1]) from checkpoint, the shape in current model is torch.Size([1, 64, 1, 1]).
        size mismatch for final_layer.conv2.bias: copying a param with shape torch.Size([19]) from checkpoint, the shape in current model is torch.Size([1]).
Killing subprocess 53003
Traceback (most recent call last):
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/swook/.vscode-server/extensions/ms-python.python-2021.6.944021595/pythonFiles/lib/python/debugpy/__main__.py", line 45, in <module>
    cli.main()
  File "/home/swook/.vscode-server/extensions/ms-python.python-2021.6.944021595/pythonFiles/lib/python/debugpy/../debugpy/server/cli.py", line 444, in main
    run()
  File "/home/swook/.vscode-server/extensions/ms-python.python-2021.6.944021595/pythonFiles/lib/python/debugpy/../debugpy/server/cli.py", line 285, in run_file
    runpy.run_path(target_as_str, run_name=compat.force_str("__main__"))
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/runpy.py", line 265, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/site-packages/torch/distributed/launch.py", line 340, in <module>
    main()
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/site-packages/torch/distributed/launch.py", line 326, in main
    sigkill_handler(signal.SIGTERM, None)  # not coming back
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/site-packages/torch/distributed/launch.py", line 301, in sigkill_handler
    raise subprocess.CalledProcessError(returncode=last_return_code, cmd=cmd)
subprocess.CalledProcessError: Command '['/data/swook/miniconda3/envs/torch18csnet/bin/python', '-u', '/data/swook/repos/chenjun2hao/ddrnet/main.py', '--local_rank=0', '--mode', 'train', '--cfg_path', './experiments/duts/ddrnet23_slim_poolnet_train_scheme.yaml']' returned non-zero exit status 1.

System Info

PyTorch version: 1.8.1
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 16.04.3 LTS (x86_64)
GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.12) 5.4.0 20160609
Clang version: Could not collect
CMake version: version 3.5.1

Python version: 3.8 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: Could not collect
GPU models and configuration: 
GPU 0: GeForce RTX 2080 Ti
GPU 1: GeForce RTX 2080 Ti

Nvidia driver version: 440.33.01
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.20.2
[pip3] torch==1.8.1
[pip3] torchvision==0.9.1
[conda] blas                      1.0                         mkl  
[conda] cudatoolkit               10.2.89              hfd86e86_1  
[conda] ffmpeg                    4.3                  hf484d3e_0    pytorch
[conda] mkl                       2021.2.0           h06a4308_296  
[conda] mkl-service               2.3.0            py38h27cfd23_1  
[conda] mkl_fft                   1.3.0            py38h42c9631_2  
[conda] mkl_random                1.2.1            py38ha9443f7_2  
[conda] numpy                     1.20.2           py38h2d18471_0  
[conda] numpy-base                1.20.2           py38hfae3a4d_0  
[conda] pytorch                   1.8.1           py3.8_cuda10.2_cudnn7.6.5_0    pytorch
[conda] torchvision               0.9.1                py38_cu102    pytorch

Study factors of unusual patterns found from results of DDRNet23-slim trained on DUTS-TR

Is your feature request related to a problem? Please describe.

Recall that we trained DDRNet23-slim on DUTS-TR

^{Figure 01. An example of input images}

^{Figure 02. Inference results of DDRNet23-slim for Figure 1}

However, we've confirmed that it creates two types of unusual patterns

diamond
rectangle

At first, we expected DAPPM might be a factor
But there's no significant improvement even after modifications on DAPPM
Refer to here for more details
We highly suspect the down-sampling rates in backbone is the reason
We'd like to change it to output feature-maps with similar size for (512, 512)
I.e. Decrease down-sampling rates to 2, 4, 8, 16 and 32

Describe the solution you'd like

Increase a resolution (r) of feature-maps in hidden layers

Describe alternatives you've considered

None

Additional context

None

Increase the resolution of training images in DUTS-TR without scaling

Is your feature request related to a problem? Please describe.

We usually scale images when we need to increase the resolution
However, upscaling can bring pixelation
It would be critical to saliency masks
And the size of the images in the DUTS-TR is distributed like:

Assume we'd like to maintain aspect ratios while increasing the resolution to 448 x 448
Then the scale factors should be small (1.12 at maximum)
If so, can we just use padding?

Describe the solution you'd like

Increase the resolution with padding, not scaling

Describe alternatives you've considered

None

Additional context

DUTS-TR is a dataset for salient object detection
We'd like to train DDRNet on this dataset for real-time salient object detection
Refer to #6 from swoook/ucnet (github) for more details

Inference feature for random data

Is your feature request related to a problem? Please describe.

We need an inference feature for deployment
Currently, chenjun2hao/DDRNet.pytorch provides a demo
However, it's based on the specific datasets
E.g. It requires a cityscapes meta-data including filename list
It's not appropriate for deployment
We need the inference feature w/o any specific datasets

Describe the solution you'd like

Implement a class based on torch.utils.data.TensorDataset
- Assume that we follow official documentation
- We have to initialize a instance every time new data arrives
Implement an inference feature w/o any torch.utils.data.Dataset

Describe alternatives you've considered

Use official repository for DDRNet
- It only provide model codes

Request a feature for training DDRNet-23-slim on datasets for salient object detection like DUTS

Is your feature request related to a problem? Please describe.

We'd like to train DDRNet23-slim on datasets for salient object detection like DUTS
Currently, chenjun2hao/DDRNet.pytorch supports training on:

Cityscapes
ADE20K
COCO stuff
...

Describe the solution you'd like

Implement a Dataset class based on lib.datsets.Cityscapes

Describe alternatives you've considered

Use CSNet

RuntimeError: The size of tensor a (38) must match the size of tensor b (37) at non-singleton dimension 2

Issue description

Confirmed the error messages below when trying to execute ${REPO_ROOT}/main.py with an image of arbitrary size

RuntimeError: The size of tensor a (34) must match the size of tensor b (33) at non-singleton dimension 3

Code example

configuration in launch.json:

        {
            "name": "Python: Current File",
            "type": "python",
            "request": "launch",
            "program": "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/site-packages/torch/distributed/launch.py", //"${file}",
            "console": "integratedTerminal",
            "args": [
                "--nproc_per_node", "1",
                "${workspaceRoot}/main.py",
                "--mode", "train",
                "--cfg_path", "./experiments/duts/ddrnet23_slim_poolnet_train_scheme.yaml",
                ]
        },

Error messages and stack traces:

Traceback (most recent call last):
  File "/home/swook/.vscode-server/extensions/ms-python.python-2021.6.944021595/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 3293, in <module>
    main()
  File "/home/swook/.vscode-server/extensions/ms-python.python-2021.6.944021595/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 3286, in main
    globals = debugger.run(setup['file'], None, None, is_module)
  File "/home/swook/.vscode-server/extensions/ms-python.python-2021.6.944021595/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 2360, in run
    return self._exec(is_module, entry_point_fn, module_name, file, globals, locals)
  File "/home/swook/.vscode-server/extensions/ms-python.python-2021.6.944021595/pythonFiles/lib/python/debugpy/_vendored/pydevd/pydevd.py", line 2367, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "/home/swook/.vscode-server/extensions/ms-python.python-2021.6.944021595/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydev_imps/_pydev_execfile.py", line 25, in execfile
    exec(compile(contents + "\n", file, 'exec'), glob, loc)
  File "/data/swook/repos/chenjun2hao/ddrnet/main.py", line 115, in <module>
    main(args)
  File "/data/swook/repos/chenjun2hao/ddrnet/main.py", line 99, in main
    solver.train()
  File "/data/swook/repos/chenjun2hao/ddrnet/sod/solver.py", line 123, in train
    sal_pred = self.net(sal_image)
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/data/swook/repos/chenjun2hao/ddrnet/lib/models/ddrnet_23_slim.py", line 324, in forward
    x_ = x_ + F.interpolate(
RuntimeError: The size of tensor a (34) must match the size of tensor b (33) at non-singleton dimension 3
Killing subprocess 6217
Traceback (most recent call last):
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/swook/.vscode-server/extensions/ms-python.python-2021.6.944021595/pythonFiles/lib/python/debugpy/__main__.py", line 45, in <module>
    cli.main()
  File "/home/swook/.vscode-server/extensions/ms-python.python-2021.6.944021595/pythonFiles/lib/python/debugpy/../debugpy/server/cli.py", line 444, in main
    run()
  File "/home/swook/.vscode-server/extensions/ms-python.python-2021.6.944021595/pythonFiles/lib/python/debugpy/../debugpy/server/cli.py", line 285, in run_file
    runpy.run_path(target_as_str, run_name=compat.force_str("__main__"))
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/runpy.py", line 265, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/runpy.py", line 97, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/site-packages/torch/distributed/launch.py", line 340, in <module>
    main()
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/site-packages/torch/distributed/launch.py", line 326, in main
    sigkill_handler(signal.SIGTERM, None)  # not coming back
  File "/data/swook/miniconda3/envs/torch18csnet/lib/python3.8/site-packages/torch/distributed/launch.py", line 301, in sigkill_handler
    raise subprocess.CalledProcessError(returncode=last_return_code, cmd=cmd)
subprocess.CalledProcessError: Command '['/data/swook/miniconda3/envs/torch18csnet/bin/python', '-u', '/data/swook/repos/chenjun2hao/ddrnet/main.py', '--local_rank=0', '--mode', 'train', '--cfg_path', './experiments/duts/ddrnet23_slim_poolnet_train_scheme.yaml']' returned non-zero exit status 1.

System Info

PyTorch version: 1.8.1
Is debug build: False
CUDA used to build PyTorch: 10.2
ROCM used to build PyTorch: N/A

OS: Ubuntu 16.04.3 LTS (x86_64)
GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.12) 5.4.0 20160609
Clang version: Could not collect
CMake version: version 3.5.1

Python version: 3.8 (64-bit runtime)
Is CUDA available: True
CUDA runtime version: Could not collect
GPU models and configuration: 
GPU 0: GeForce RTX 2080 Ti
GPU 1: GeForce RTX 2080 Ti

Nvidia driver version: 440.33.01
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A

Versions of relevant libraries:
[pip3] numpy==1.20.2
[pip3] torch==1.8.1
[pip3] torchvision==0.9.1
[conda] blas                      1.0                         mkl  
[conda] cudatoolkit               10.2.89              hfd86e86_1  
[conda] ffmpeg                    4.3                  hf484d3e_0    pytorch
[conda] mkl                       2021.2.0           h06a4308_296  
[conda] mkl-service               2.3.0            py38h27cfd23_1  
[conda] mkl_fft                   1.3.0            py38h42c9631_2  
[conda] mkl_random                1.2.1            py38ha9443f7_2  
[conda] numpy                     1.20.2           py38h2d18471_0  
[conda] numpy-base                1.20.2           py38hfae3a4d_0  
[conda] pytorch                   1.8.1           py3.8_cuda10.2_cudnn7.6.5_0    pytorch
[conda] torchvision               0.9.1                py38_cu102    pytorch

Future work for transfer learning on salient object detection

Is your feature request related to a problem? Please describe.

We trained DDRNet23-slim on DUTS-TR and it achieved 0.47 MAE on DUTS-TE
And we'd like to improve the accuracy of DDRNet23-slim on salient object detection

Describe the solution you'd like

Research BoFs and BoSs for related tasks, especially segmentation

Describe alternatives you've considered

None

Additional context

None

Modify the architecture of DAPPM considering the spatial size of the feature maps from generated by DAPPM

Is your feature request related to a problem? Please describe.

DAPPM performs large pooling kernels and exponential strides to:

enlarge effective receptive fields
fuse multi-scale context

See four different sub-branches which include pooling layer (red rectangle) from Fig. 5.

name	operation	down-sampling rate
`scale1`	`AvgPool2d(kernel_size=5, stride=2, padding=2)`	128
`scale2`	`AvgPool2d(kernel_size=9, stride=4, padding=4)`	256
`scale3`	`AvgPool2d(kernel_size=17, stride=8, padding=8)`	512
`scale4`	`AdaptiveAvgPool2d((1, 1))`	(H, W)

DAPPM down-samples an input image by factors of 128, 256 and 512
However, recall that DDRNet is trained and benchmarked on two datasets:

Cityscapes
CamVid

The resolutions of their frames are (2048, 1024) and (1024, 1024), respectively
In those cases, scale3 outputs feature-maps with spatial size of (4, 2) and (2, 2), respectively
But assume the input images are <= (512, 512)
Then, scale3 and scale4 outputs feature-maps with spatial size of (1, 1)
It means they have almost same receptive fields
I.e. Redundant maybe?
If the input images are small, wouldn't it be better to eliminate some poolings in DAPPM considering the receptive field?
Decide after seeing if the existing one is trained well

Describe the solution you'd like

If the input images are small, wouldn't it be better to eliminate some poolings in DAPPM considering the receptive field?
Decide after seeing if the existing one is trained well

Describe alternatives you've considered

None

Additional context

DUTS-TR is a dataset for salient object detection
We'd like to train DDRNet on this dataset for real-time salient object detection
Refer to #6 from swoook/ucnet (github) for more details

Train

I just need to split the two categories. How do I change it

Transfer learning for salient object detection

Is your feature request related to a problem? Please describe.

We'd like to employ the DDRNet23-slim pre-trained on the Cityscapes for salient object detection
How should we perform transfer-learning for salient object detection?

Describe the solution you'd like

Pick a strategy by referring to Transfer learning from pre-trained models (towardsdatascience) (korean)

Describe alternatives you've considered

None

Additional context

DUTS-TR is a dataset for salient object detection
We'd like to train DDRNet on this dataset for real-time salient object detection
Refer to #6 from swoook/ucnet (github) for more details

Train DDRNet on multiple famous public datasets for salient object detection

Is your feature request related to a problem? Please describe.

We trained DDRNet23-slim on DUTS-TR
However, we'd like to train it on multiple famous datasets

HKU-IS
MSRA-B
BSDS
PASCAL
...

Describe the solution you'd like

We can train jointly if we convert the formats of other datasets to the one of DUTS-TR

Describe alternatives you've considered

Train on other datasets sequentially
However, it's not optimal 😞

Additional context

None

swoook / ddrnet Goto Github PK

ddrnet's People

Contributors

Stargazers

Watchers

ddrnet's Issues

Issue description

Code example

System Info

Backgrounds

Issue description

Code example

System Info

Issue description

Code example

System Info

Recommend Projects

Recommend Topics

Recommend Org