Git Product home page Git Product logo

ulabel's Introduction

ULabel

A browser-based tool for annotating images.

Demo Gif

Interactive Demo

Usage

ULabel is an entirely "frontend" tool. It can be incorporated into any HTML page using either the unpkg cdn

<script src="https://unpkg.com/ulabel/dist/ulabel.js"></script>

Or you can use npm to install it and serve the dist/ulabel.js file from node_modules locally.

npm install ulabel
<script src="/node_modules/ulabel/dist/ulabel.js"></script>

An API spec can be found here, but as a brief overview: Once the script is included in your HTML doc, you can create a ULabel annotation session as follows.

<!DOCTYPE html>
<html>
<head>
    <script src="https://unpkg.com/ulabel/dist/ulabel.js"></script>
</head>
<body>
    <div id="ulabel-container" style="height: 800px; width: 1200px; position: absolute; top: 0; left: 0;"></div>

    <script>
        // Specify destination
        let container_id = "ulabel-container";

        // Configure the annotation session
        let classes = ["Class 1", "Class 2"];
        let allowed_modes = ["polygon", "bbox", "contour"];
        let username = "demo_user";
        let image_url = "https://tinyurl.com/y6mxeuxs";

        // Specify submit behavior
        let on_submit = async (annotations) => {
            // Download annotations as a json file
            let el = document.createElement('a');
            el.setAttribute("href", 'data:text/plain;charset=utf-8,' + encodeURIComponent(JSON.stringify(annotations, null, 2)));
            el.setAttribute("download", "annotations.json");
            el.style.display = 'none';
            document.body.appendChild(el);
            el.click();
            document.body.removeChild(el);
            return true;
        };

        let subtasks = {
            "first_task": {
                "display_name": "Task 1",
                "classes": [
                    {"name": "Class 1", "color": "blue", "id": 10},
                    {"name": "Class 2", "color": "pink", "id": 11},
                ],
                "allowed_modes": ["contour", "polygon"],
                "resume_from": null,
                "task_meta": null,
                "annotation_meta": null
            },
            "second_task": {
                "display_name": "Task 2",
                "classes": [
                    {"name": "Class 2", "color": "pink", "id": 11},
                    {"name": "Class 3", "color": "green", "id": 12},
                ],
                "allowed_modes": ["bbox", "polygon", "tbar"],
                "resume_from": null,
                "task_meta": null,
                "annotation_meta": null,
                "read_only": true
            }
        };

        // Build and initialize the toollet ulabel = new ULabel(
        let ulabel = new ULabel(container_id, image_url, username, on_submit, subtasks);
        ulabel.init(function() {
            console.log("ULabel is now ready");
        });

    </script>
</body>
</html>

Development

The recommended way to develop new features is to use the tool as if you were running the demo. For testing new API features, you can create a new HTML file in demo/. The server in demo.js runs a static server from demo/ so it will be served at localhost:8080/<new-file>.html automatically.

Requirements

Install Dependencies

npm install

Development Usage

You should now be able to run the launcher from the repository root.

pwd # Should be /path/to/ulabel
node demo.js

Build

The repository ships with the built dist/ulabel.js file. This file is not to be edited directly. Edits should be made in the src/ dir and the package is built with

npm run build

Attribution

The three demo images were downloaded from the CityScapes dataset. In particular, they are sampled from the first few frames of the leftImg8bit_demoVideo.

ulabel's People

Contributors

csolbs24 avatar dependabot[bot] avatar joshua-dean avatar neheller avatar pubkey avatar trevorburgoyne avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ulabel's Issues

Unload Handling and "Live" Submit

It would be nice if the document appeared to have a concept of the page being edited. For example, lighting up the submit button once changes have been made, and then darkening it once they have been saved. Also displaying a warning before the user leaves the page if changes were not saved.

Button to toggle initial crop

Similar to the nifty "re-center" button, it would be nice to have a button that will toggle between the initial crop settings and the full image zoom.

Add `flyTo` API

As shown here, private link, it's technically fairly simple to support a flyTo API, which can be helpful when the spatial portion of annotations is correct, but the class information is either incomplete, incorrect, or unavailable.

Filter function exposure

There's a couple of different keypoint filtering methods tied into Toolbox Items, it would be handy to be able to access these directly to use the filtering indepedently of the toolbox.

Trying to redo a brush tool created polygon after undoing it causes an error

Redoing a brush tool created polygon doesn't work. This can be recreated by creating any polygon with the brush tool. Then undoing it with the undo keybind. Then try to bring it back with the redo keybind. The first time you try to redo nothing will happen. Then the second time is when I get the error. Here is the error message JSON.parse: unexpected character at line 1 column 1 of the JSON data simplify_polygon_complex_layer

Linting / Precommit Hooks

Could use some standardization and some dev environment hook setups to keep the code structure clean

SageMaker Jobs are Erroring Out

When creating jobs with SageMaker, the window shows up blank. The console indicates

Uncaught SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at process_configuration (<anonymous>:2269:49)
    at HTMLDocument.<anonymous> (<anonymous>:2277:15)
    at e (code.jquery.com/jquery-3.5.1.min.js:2)
    at t (code.jquery.com/jquery-3.5.1.min.js:2)

Adjust the line width of an annotation / support zooming

In the current state of our pipeline, images are loaded into the tool at native resolution. This results in areas of interest that are often quite small, and it is difficult to create an accurate boundary around them:
image

There are a few ways to solve this problem I think. The tool could expose settings to alter the line width and gravity when drawing the annotation, or could support zooming (or both). Alternatively, we could simply tile the images to a more reasonable size (GSD) before passing them to the tool in the first place.

Can't use in vite + vue3

Could not resolve "../build/annotation"

node_modules/.store/[email protected]/node_modules/ulabel/src/index.js:5:33:
  5 │ import { ULabelAnnotation } from '../build/annotation';
    ╵                                  ~~~~~~~~~~~~~~~~~~~~~

X [ERROR] Could not resolve "../build/subtask"

node_modules/.store/[email protected]/node_modules/ulabel/src/index.js:6:30:
  6 │ import { ULabelSubtask } from '../build/subtask';
    ╵                               ~~~~~~~~~~~~~~~~~~

X [ERROR] Could not resolve "../build/geometric_utils"

node_modules/.store/[email protected]/node_modules/ulabel/src/index.js:7:31:
  7 │ import { GeometricUtils } from '../build/geometric_utils';
    ╵                                ~~~~~~~~~~~~~~~~~~~~~~~~~~

X [ERROR] Could not resolve "../build/annotation_operators"

node_modules/.store/[email protected]/node_modules/ulabel/src/index.js:8:42:
  8 │ import { get_annotation_confidence } from '../build/annotation_operators';
    ╵                                           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

X [ERROR] Could not resolve "../build/drawing_utilities"

node_modules/.store/[email protected]/node_modules/ulabel/src/index.js:9:31:
  9 │ import { apply_gradient } from '../build/drawing_utilities'
    ╵                                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~

X [ERROR] Could not resolve "../build/configuration"

node_modules/.store/[email protected]/node_modules/ulabel/src/index.js:10:50:
  10 │ import { Configuration, AllowedToolboxItem } from '../build/configuration';
     ╵                                                   ~~~~~~~~~~~~~~~~~~~~~~~~

X [ERROR] Could not resolve "../build/html_builder"

node_modules/.store/[email protected]/node_modules/ulabel/src/index.js:11:28:
  11 │ import { HTMLBuilder } from '../build/html_builder';
     ╵                             ~~~~~~~~~~~~~~~~~~~~~~~

dist-electron/main.js 571.00 kB │ gzip: 169.08 kB
built in 33165ms.

X [ERROR] Could not resolve "../build/annotation"

node_modules/.store/[email protected]/node_modules/ulabel/src/index.js:5:33:
  5 │ import { ULabelAnnotation } from '../build/annotation';
    ╵                                  ~~~~~~~~~~~~~~~~~~~~~

X [ERROR] Could not resolve "../build/subtask"

node_modules/.store/[email protected]/node_modules/ulabel/src/index.js:6:30:
  6 │ import { ULabelSubtask } from '../build/subtask';
    ╵                               ~~~~~~~~~~~~~~~~~~

X [ERROR] Could not resolve "../build/geometric_utils"

node_modules/.store/[email protected]/node_modules/ulabel/src/index.js:7:31:
  7 │ import { GeometricUtils } from '../build/geometric_utils';
    ╵                                ~~~~~~~~~~~~~~~~~~~~~~~~~~

X [ERROR] Could not resolve "../build/annotation_operators"

node_modules/.store/[email protected]/node_modules/ulabel/src/index.js:8:42:
  8 │ import { get_annotation_confidence } from '../build/annotation_operators';
    ╵                                           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

X [ERROR] Could not resolve "../build/drawing_utilities"

node_modules/.store/[email protected]/node_modules/ulabel/src/index.js:9:31:
  9 │ import { apply_gradient } from '../build/drawing_utilities'
    ╵                                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~

X [ERROR] Could not resolve "../build/configuration"

node_modules/.store/[email protected]/node_modules/ulabel/src/index.js:10:50:
  10 │ import { Configuration, AllowedToolboxItem } from '../build/configuration';
     ╵                                                   ~~~~~~~~~~~~~~~~~~~~~~~~

X [ERROR] Could not resolve "../build/html_builder"

Delete annotations

In the case where an annotation is created by accident, it would be nice if there was some way to delete it. Perhaps by right-clicking while hovering over it, or a dedicated menu button?

Refactor type declarations

       **General Comments**

We have a lot of overlap of types throughout this project and I think that it's getting pretty messy and confusing. I also recently learned that the way we handle custom types in this project is not considered very good practice. In general types should be declared in 3 different places. First is inside of index.d.ts. This file should be used for types that are useful for interfacing with the project when ulabel is being imported. And index.d.ts can also be imported from internally for each file that needs a given type. The second place is inside of a dedicated types.ts (or some other generic\preferred name) for types that are useful across multiple files in the project. They can then be imported as needed from that file. Finally types can should be declared at the top of a typescript file if they are useful inside of that file, but not needed in any other file in the project. This system can keep our type system more organized and reduce duplicated types.
For example we currently have two types that behave exactly the same.
type ULabelSpatialPayload = [number, number][] and
type Point2D = [number, number]
type ULabelSpatialPayload2D = Point2D[]

Originally posted by @csolbs24 in #129 (review)

Annotation color dull when low confidence

Annotation color is dull when there is a low confidence.
This behavior should be able to be modified as it is not always desired.
An option for consistent color (or a confidence-based gradient) may be useful.

Move to previously seen images in a labeling job

It would be great if there was some mechanism to return to images that were shown earlier in the labelling job, rather than always marking them as complete once the "Submit" button is selected.

"V" to vanish all ulabel subtasks

Currently pressing "v" vanishes the active subtask annotations. Adding a "V" or other keybind to vanish ALL subtasks would be very useful.

Better sanity checking of config defaults

instead of the long list of if statements checking for undefined in line ~1060 of toolbox.ts in 63bd99f, check that config values are of the expected type. could also iterate over a list of defaults instead of separate if statements

Keybind to delete annotations when hovering over it

When multiple annotations are nearly overlapping, it becomes very difficult to click on the delete icon without selecting a different annotation. Being able to press a key to delete while simply hovering over an annotation would solve this.

Document or Fix class ID sort order requirement

A common pattern is to ship only the annotations and retain the subtask config in the frontend wrapping ULabel. In doing so, I found a (usually correct) assumption about the inputs in the subtask.

If the class IDs in a subtask configuration are supplied out of order, ULabel shows incorrect coloring for the annotation.

ULabel should either enforce ordering of class IDs at parse time or reorder as needed. Order shouldn't be assumed when supplying a class ID in the array of class configs.

Dialogs are not always visible

Sometimes hovering over annotations does not cause dialogs to appear. This can be seen in demo/resume-from. If you hover over the polygon, you'll see that it works as expected. If you hover over the three keypoints on the left side of the screen, you'll see that no dialogs show up.

json2mask

Hi, thanks for providing us this convenient annotation tool
recently I am working on ureter segmentation and I have noticed that KiTS21 datasets provides the annotations of ureters which are created by 'ulabel'. But the annotations are .json not .nii. So I wonder if there are some scripts (like json2mask.py or something else) which can turn these .json into .nii automatically.

Submit Button Class Counts

add optional arg to submit button that will add a field to the callback arguments that contains info about the number of non-deprecated annotations in each class. currently users have to duplicate the counting logic internal to ulabel, when really it could be provided via some api. Speaking of which there should also be a ULabel get_class_counts() method or smth to get the counts that are displayed in the toolbox class counter item.

Recolor annotation changing wrong class

Even though the first class in the class list is shown to be selected in ulabel upon starting a job, choosing a new color in the "Recolor annotations" box will change a different class than the one shown as selected.

ie with subtasks:
row_classification: { display_name: "Row Classification", classes: [ { name: "Male", color: "blue", id: 10, }, { name: "Female", color: "white", id: 11, }, ], allowed_modes: ["polyline"], resume_from: annos, task_meta: null, annotation_meta: null, },

Class Male is shown to be selected, but Female annotations will change color when the recolor annotation box is used. This bug stops when you manually select any of the classes.

Reordering Toolbox with `KeypointSlider` errors out

Toolbox reordering is functional unless the KeypointSlider (item 6) is used, which causes an error as it has required constructor arguments.
Good behavior would be omitting toolbox items that failed to construct, as well as having the parameters in configuration.ts be defaults in the actual ToolboxItem class.

Code Structure Could Use a Refactor

Currently the vast majority of the source code lives in a single class (ULabel) within the src/index.js file. In the long run, it would be nice to break this up into various additional files and classes. Off the top of my head, the following might make for good classes

  • Geometric Utilities
    • For things like finding nearest point on a line etc.
  • HTML Builder
    • To prepare the container class when constructor is called
  • Event Listener
    • To detect and dispatch all interactions by the user
  • Subtask
    • A documented set of calls for things like
      • Beginning annotation
      • Continuing annotation
      • Ending annotation
      • Editing annotation
  • Dialog Handler
    • To manage visiblity and position of dialogs
  • Toolbox View Handler
    • A class in charge of handling the toolbox view
  • Canvas View Handler
    • A class in charge of handling the canvas (set of canvasses, really) view
    • Methods like
      • Draw annotation
  • Annotation
    • A class in charge of maintaining information about annotations, like
      • Containing box
      • Color
  • Action Stream
    • A class that maintains the set of actions within each subtask
    • Events would be placed on the action stream before/as they were dispatched?
  • ID Dialog
    • Certainly should encapsulate the handlers and view controller associated with this somehow

Of course, there's quite a bit more fleshing out to do before working on this, but in general I think it will pay off in the long run. Ultimately, what I need is a firm workflow for each interaction that includes

  • Storing the information about that interaction
  • Changing the objects' states accordingly
  • Updating the view

This is all there now in some shape or form, but it's starting to get muddled together, and the size of the file makes it difficult to debug and refactor.

Open Source Tasks

Now that the repo is public, there are several things to be done to improve its usability.

  • Adding the package to npm
  • Creating better documentation for the API
  • Differentiating "usage" and "development" on the readme
  • Writing contribution guidelines
    • Standardized tests should be a part of this
  • Adding and using a changelog

I think at the very least, testing coverage and the refactor from #10 should be done before v1.x

Configuration Option for Reload Prompt

There is a beforeunload eventListener which prompts the user to confirm navigation on unsaved changes.
There's a number of ways to implement ULabel into a project whilst allowing it to persist through several different annotation jobs, and depending on how they set the saved state this can break them.

Disabling this eventListener should be a configuration option, and additionally some reload/save utilities might be useful to streamline how different use cases manage state/save-state whilst doing soft/hard reloads.

Typescript refactor slider bar config error

Line ~20132: _ this.deprecate_annotations(current_subtask, _this.default_value);
when KeypointSliderItem.prototype.deprecate_annotations expects ulabel to be passed as the first argument, not current_subtask.

CSS Clashes

@joshua-dean noticed that ULabel style is impacted by the Bootstrap stylesheets. It would be nice to do a better job reseting/isolating the stylesheets applied to ULabel.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.