This repository adapts AutoSAM to the segmentation of helicopter-borne Arctic thermal infrared images. The segmentation should be done into three classes: melt ponds, sea ice, and ocean.

Training and validation data are stored in data/training/ and we provide the weights of our final model in experiments/2801_2/ (lfs tracked). Test images are stored in data/prediction/preprocessed/test/.

The entire TIR dataset is in the process of publication. The dataset processing code can be found in data/temperature_preprocessing/.

This code requires python>=3.10, as well as pytorch>=1.7 and torchvision>=0.8. Install additional packages using pip install -r requirements.txt.

Segment Anything model checkpoints can be downloaded from SAM and should be placed in segment_anything_checkpoints/.

python -m scripts.infer_ponds --pref ${storage_folder_name} --weights_path 'experiments/2801_2/model_mp_145.pth' --skip_preprocessing --preprocessed_path 'data/prediction/preprocessed/val/' --val_predict

Images are then stored here data/prediction/predicted/${storage_folder_name}.

The computed melt pond fraction will be stored in metrics/melt_pond_fraction/${storage_folder_name}.

python -m scripts.infer_ponds --pref ${storage_folder_name} --weights_path 'experiments/2801_2/model_mp_145.pth' --skip_preprocessing --preprocessed_path 'data/prediction/preprocessed/test/' --val_predict

Images are then stored here data/prediction/predicted/${storage_folder_name}.

The computed melt pond fraction will be stored in metrics/melt_pond_fraction/${storage_folder_name}.

python -m scripts.main_autosam_seg --save_dir ${storage_folder_name} --pref ${wandb_pref_name_of_choice}

preprocess_training.ipynb was used to preprocess the data.

The AutoSAM implementation is based on the work of Hu, Xinrong and Xu, Xiaowei and Shi, Yiyu, licensed under the Apache 2.0 License. Modifications are listed in the respective files.

Full citation of AutoSAM:

@article{hu2023efficiently,
  title={How to Efficiently Adapt Large Segmentation Model (SAM) to Medical Images},
  author={Hu, Xinrong and Xu, Xiaowei and Shi, Yiyu},
  journal={arXiv preprint arXiv:2306.13731},
  year={2023}
}

[1] Kanzow, Thorsten (2023). The Expedition PS131 of the Research Vessel POLARSTERN to the Fram Strait in 2022. Ed. by Horst Bornemann and Susan Amir Sawadkuhi. Bremerhaven. DOI: 10.57738/BzPM_0770_2023.

[2] Reil, M., G. Spreen, M. Huntemann, L. Buth, & D. Wilson (2024). Machine Learning for the Detection of Arctic Melt Ponds from Infrared Imagery. Workshop paper at ”Tackling Climate Change with Machine Learning”, International Conference on Learning Representations (ICLR) 2024, Vienna, Austria and virtual. https://www.climatechange.ai/papers/iclr2024/56.

[3] Reil, M. (2023). Detection of Melt Ponds on Arctic Sea Ice from Infrared Images using U-net. Bachelor's thesis, University of Osnabrück in cooperation with University of Bremen. https://seaice.uni-bremen.de/fileadmin/theses_reports/Reil_Marlena-Bachelor_thesis_small.pdf.

Contact: [email protected]