Git Product home page Git Product logo

pytorch-superpixels's Introduction

โš ๏ธ The functionality provided by this repository can now be found in PyTorch Geometric in the to_superpixels() function.


PyTorch Superpixels

Why use superpixels?

Dimensionality reduction allows for the use of simpler networks or more complex objectives. A common way of doing this is to simply downsample the images so that there are fewer pixels to contend with. However, this is a lossy operation so detail (and therefore the upper bound on experimental results) is reduced.

Superpixels slightly alleviate this problem because they are able to encode information about edges within themselves. Generating superpixels is an unsupervised clustering operation. Whilst there are already clustering packages written for Python (some of which this project depends on), they all operate with NumPy arrays. This means that they cannot take advantage of GPU acceleration in the way that PyTorch tensors can.

The aim of this project is to bridge the gap between these existing packages and PyTorch so that superpixels can be readily used as an alternative to pixels in various machine learning experiments.

Example usage

Here is some example code that uses superpixels for semantic segmentation.

# Generate list of filenames from your dataset
image_list = pytorch_superpixels.list_loader.ImageList(
    'pascal-seg', './VOCdevkit/VOC2012', 'trainval')
# Use this list to create and save 100 superpixel dataset
pytorch_superpixels.preprocess.create_masks(image_list, 100)

# -----------------------------------------------
# code that sets up model, optimizer, dataloader, metrics, etc.
# -----------------------------------------------

# Iterate through images, labels and masks
# (Note that masks and labels are already in superpixel form)
for (images, labels, masks) in trainloader:
        # Set model to training mode
        model.train()
        # Send images, labels and masks to the GPU
        images = images.to('cuda:0')
        labels = labels.to('cuda:0')
        masks = masks.to('cuda:0')
        # Pass images through the model
        optimizer.zero_grad()
        outputs = model(images)
        # Convert outputs to superpixel form
        outputs_sp, sizes = pixels_to_superpixels(
            outputs, masks)
        # Calculate loss using size weighted superpixels
        loss = loss_fn(scores=outputs_sp, target=labels, size=sizes)
        # Convert back to pixels for metric evaluation
        outputs = superpixels_to_pixels(outputs_sp, masks)
        # Accumulate train metrics during train
        pred = outputs.data.max(1)[1].cpu().numpy()
        gt = labels.data.cpu().numpy()
        running_metrics_train.update(gt, pred)
        train_loss_meter.update(loss.item())
        # Backprop and optimizer step
        loss.backward()
        optimizer.step()

This project stems from a module I created for use in my master's thesis.

With some work I think it could be useful for others that wish to utilise superpixels in their Pytorch Machine learning projects.

Ideally I would like to add support for many of the more popular datasets, publish a pypi package and maybe even port this to TensorFlow.

Currently the code is fragmented and unusable, with community support I hope to change that soon.

pytorch-superpixels's People

Contributors

hmellor avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pytorch-superpixels's Issues

metrics.py is not yet ported

Describe the bug
The code in this file was directly copied from my thesis code and as such does not yet function on its own.

Suggested Solution
Dig into the file and see what needs changing so that it can be properly used by dependent experiments.

Add test scripts for evaluating code functionality

Is your feature request related to a problem? Please describe.
There is currently no agreed upon way to test the functionality of the code.

Describe the solution you'd like
By writing test scripts we can ensure that all functionality is tested when developing the package further.

Wiki Documentation

Describe the solution you'd like
A simple feature request to add documentation in the Wiki

runtime.py is not yet ported

Describe the bug
The code in this file was directly copied from my thesis code and as such does not yet function on its own.

Suggested Solution
Dig into the file and see what needs changing so that it can be properly used by dependent experiments.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.