Git Product home page Git Product logo

ms-former's Introduction

MS-Former: Memory-Supported Transformer for Weakly Supervised Change Detection with Patch-Level Annotations (under review)

This repository contains a Python implementation of our paper MS-Former.

1. Usage

  • Prepare the data:

    • Download datasets LEVIR, BCDD-BGMix, SYSU, and GVLM
    • Crop LEVIR datasets into 256x256 patches. Generate patch-level annotations for LEVIR, BCDD-BGMix, SYSU, and GVLM datasets.
    • Generate list file as ls -R ./label/* > test.txt
    • Prepare datasets into the following structure and set their path in train.py and test.py
    ├─Train
        ├─A        ...jpg/png
        ├─B        ...jpg/png
        ├─label    ...jpg/png
        └─list     ...txt
    ├─Test
        ├─A
        ├─B
        ├─label
        └─list
    
  • Prerequisites for Python:

    • Creating a virtual environment in the terminal: conda create -n MS-Former python=3.8
    • Installing necessary packages: pip install -r requirements.txt
  • Train/Test

    • sh train.sh
    • sh test.sh

2. Motivation

Performance Image

Fig. 1 Comparison of the pixel-level, image-level, and our patch-level labels for remote sensing change detection.

Performance Image

Fig. 2 Comparison of the change detection performance measured by F1 of our proposed MS-Former using patch-level labels across different patch size settings on the BCDD dataset.
Notably, as the patch size increases, the patch-level labels align more closely with image-level annotations, while decreasing patch size results in labels close to pixel-wise annotations. In this work, we observe that a slight reduction in patch size substantially enhances change detection performance. This observation suggests the potential of exploring patch-level annotations for remote sensing change detection.

3. Pipeline of the MS-Former

Performance Image

Fig. 3 A framework of the proposed MS-Former.
Initially, the bi-temporal images pass through a feature extractor to capture the temporal difference features. After that, the temporal difference features and prototypes stored in the memory bank are jointly learned by a series of bi-directional attention blocks. Finally, a patch-level supervision scheme is introduced to guide the network learning from the patch-level annotations.

4. Citation

Please cite our paper if you find the work useful:

@article{li2023ms,
    title={MS-Former: Memory-Supported Transformer for Weakly Supervised Change Detection with Patch-Level Annotations},
    author={Li, Zhenglai and Tang, Chang and Liu, Xinwang and Li, Changdong and Li, Xianju and Zhang, Wei},
    journal={arXiv preprint arXiv:2311.09726},
    year={2023}
}

ms-former's People

Contributors

guanyuezhen avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

ms-former's Issues

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.