This repository is the official implementation of GeoSynth [CVPRW, EarthVision, 2024]. GeoSynth is a suite of models for synthesizing satellite images with global style and image-driven layout control.
Models available in ๐ค HuggingFace diffusers:
All model ckpt
files available here - Model Zoo
- Update Gradio demo
- Release Location-Aware GeoSynth Models to ๐ค HuggingFace
- Release PyTorch
ckpt
files for all models - Release GeoSynth Models to ๐ค HuggingFace
Example inference using ๐ค HuggingFace pipeline:
from diffusers import StableDiffusionControlNetPipeline, ControlNetModel
import torch
from PIL import Image
img = Image.open("osm_tile_18_42048_101323.jpeg")
controlnet = ControlNetModel.from_pretrained("MVRL/GeoSynth-OSM")
pipe = StableDiffusionControlNetPipeline.from_pretrained("stabilityai/stable-diffusion-2-1-base", controlnet=controlnet)
pipe = pipe.to("cuda:0")
# generate image
generator = torch.manual_seed(10345340)
image = pipe(
"Satellite image features a city neighborhood",
generator=generator,
image=img,
).images[0]
image.save("generated_city.jpg")
Our model is able to synthesize based on high-level geography of a region:
Style for OSM imagery is created using MapBox. The style file can be downloaded from here. The dataset can be downloaded from here. Look at train.md for details on setting up the environment and training models on your own data.
Download GeoSynth models from the given links below:
Control | Location | Download Url |
---|---|---|
- | โ | Link |
OSM | โ | Link |
SAM | โ | Link |
Canny | โ | Link |
- | โ | Link |
OSM | โ | Link |
SAM | โ | Link |
Canny | โ | Link |
@inproceedings{sastry2024geosynth,
title={GeoSynth: Contextually-Aware High-Resolution Satellite Image Synthesis},
author={Sastry, Srikumar and Khanal, Subash and Dhakal, Aayush and Jacobs, Nathan},
booktitle={IEEE/ISPRS Workshop: Large Scale Computer Vision for Remote Sensing (EARTHVISION),
year={2024}
}
Check out our lab website for other interesting works on geospatial understanding and mapping: