oxfordrse / electron-sight Goto Github PK

View Code? Open in Web Editor NEW

0.0 0.0 0.0 542 KB

Slide viewer for histology images

HTML 0.74% JavaScript 73.90% Python 0.60% C++ 2.58% C 21.24% CSS 0.94%

electron-sight's People

Contributors

Watchers

electron-sight's Issues

Escape to cancel current annotation drawing

Falls under #48

use redux for state

The app currently needs to store:

A classifier in the process of being built
A set of already built classifiers
An annotation in the process of being built
A set of annotations already built
A set of annotation+classifier which contain the results of a classifier used on a given annotated area

Where should this be stored?

In Menu, and passed down to other react classes via props?
Using something like redux?

I'm leaning towards 2 so I can learn how redux works....

apply watershed algorithm to predict results

to isolate individual cells, see #32 (comment)

can use opencv-js: https://docs.opencv.org/3.4/d5/d10/tutorial_js_root.html

this electron npm module might be useful as well: https://github.com/justadudewhohacks/opencv-electron

persistent classifiers / annotation

annotations saved on a per-image basis automatically, using same filename as image
classifiers saved on a user-basis automatically
default classifier included
add ability to delete annotations + classifiers

add scale information

Put scale information in lower left corner, like the osd plugin in https://github.com/usnistgov/OpenSeadragonScalebar

move onClick logic to ApplicationState

The Annotation, Classifier and Predict classes all have onClick logic that should be moved to ApplicationState

@iamleeg, was this what you had in mind for ApplicationState?

move app logic to app.jsx

at the moment most of the logic for the UI is in the Menu component. Should live in App.jsx, and then the Menu.jsx can just be stateless

test failures due to src/ApplicationState.jsx not found

npm test results in failure:

> [email protected] test /home/mrobins/git/Electron-Sight
> mocha --require babel-core/register

/home/mrobins/git/Electron-Sight/node_modules/mocha/node_modules/yargs/yargs.js:1163
      else throw err
           ^

Error: Cannot find module '../src/ApplicationState'
Require stack:
- /home/mrobins/git/Electron-Sight/test/test_application_state.js
- /home/mrobins/git/Electron-Sight/node_modules/mocha/lib/mocha.js
- /home/mrobins/git/Electron-Sight/node_modules/mocha/lib/cli/one-and-dones.js
- /home/mrobins/git/Electron-Sight/node_modules/mocha/lib/cli/options.js
- /home/mrobins/git/Electron-Sight/node_modules/mocha/bin/mocha

add slide overview view in bottom-left corner of app

Clear state when loading new file

Sight does not support multiple files at once, so we should clear state when opening a new file.

save classifier so that it can be resumed later

so that user can go back and either change classifier parameters or add new data

use cross-validation to determine predictive power of svm classifier

should be some feedback to user on cross validation score (libsvm-js has a method for calculating this)
user should be able to change parameters of svm in response to CV score

Superpixel classification does not highlight some tiles in polygon

compile Joshua superpixel feature extraction using node-gyp

There is a C library VLFeat: http://www.vlfeat.org/ (BSD licence)
This has

SLIC and quick shift superpixels
large scale SVM training

going to try and compile this as a nodejs module using node-gyp so we can use it

Use escape to cancel current mode

An example would be exiting annotation mode

create Classifier and Annotation classes

Final app will need to store multiple classifiers, annotations, and annotations+classifier results, need to create classes to store these

add annotations support

Possible candidates to use
https://github.com/Emigre/openseadragon-annotations
https://annotorious.github.io/demos/openseadragon-preview.html (can label annotations as well, but only squares supported; polygon support is beta).

create classifier functionality

Potential workflow:

User clicks on "Classifier" button
Application automatically zooms to the zoom level set/chosen (what is this? maximum zoom?)
for each user click:
- If tile has not been SLICed -> SLIC tile, add to list of SLIC tiles, set all superpixels to "not chosen"
- set superpixel clicked as "chosen"
User clicks "Finish Classifier" button

slic superpixel regions don't match up with image

for the classifier: it looks like the superpixels don't match up with features in the image, so the slic routine must be getting incorrect image data

full screen openseadragon viewer to full width/height of app

file-tree view for opening slide files

perhaps use https://github.com/mperitz/react-filetree-electron

create plotting tab

For demo in Jan, Joshua recommended a separate plotting tab that would displace summary statistics of classified regions, potential plots include:

(@joshua can you fill this out)

user paperjs for annotations

from meeting 13th Dec - @martinjrobins @jbull

Clinicians (Philip) very concerned with ease of creating annotation -

e.g. freehand drawing of region, which then is simplified to polygon, then add/edit points in polygon

paperjs has functionality for this, could integrate with app

look at using caMicroscope for the base viewer and annotations

https://github.com/camicroscope/caMicroscope

use image specific pixelsPerMeter in scalebar

Related to #4, openslide should have this information

add unit and regression testing framework

Use mocha and chai to add unit tests.

Convert slides to DeepZoom

OpenSeadragon cannot read whole slide images formats (from Aperio, Hamamatsu and so on)
But it can read DeepZoom.
So whole slide images must be first converted to Deepzoom.

The main tool for that is libvips, it relies on openslide to read whole slide images.
libvips defines a function vips_dzsave

int
vips_dzsave (VipsImage *in,
             const char *name,
             ...);

Can also be used from the command line vips utility

vips dzsave CMU-3.ndpi CMU-3.dzi

How to use libvips and openslide from javascript ?

Someone asking for node.js bindings for openslide on the
openslide repo (openslide/openslide#204)

Main libvips developer John Cupitt suggests to use sharp

sharp

sharp can create deepzoom images out of high res images.

sharp('input.tiff')
 .png()
 .tile({
   size: 512
 })
 .toFile('output.dz', function(err, info) {
   // output.dzi is the Deep Zoom XML definition
   // output_files contains 512x512 tiles grouped by zoom level
 });

since early versions sharp supports reading in whole slide images (using openslide), as well as writing DeepZoom images (lovell/sharp#146).

The documentation used to include installation instructions for installing sharp with openslide support but now no mention of openslide anymore...

I naively tried

sharp('CMU-3.ndpi')
  .png()
  .tile({
    size: 512
  })
  .toFile('output.dz', function(err, info) {
    // output.dzi is the Deep Zoom XML definition
    // output_files contains 512x512 tiles grouped by zoom level
  });

but doesn't produce any output.

In may 2017 jcupitt writes

@xmkevin, sharp can read all the formats that openslide can read,
you just need to enable openslide support. There are some notes in the README.

Somehow one may has to /enable openslide support/ and I'm not sure what that means. Potentially useful to figure it out because sharp, coupled with openslide seems to be able to do precisely what we are looking for.

node-vips

Sept. 2017: Experimental node binding for libvips
(https://github.com/libvips/node-vips)

The gitHub repo provide an example reading a whole slide image with openslide:

var vips = require('vips');

// get a rect from a level
// autocrop trims off pixels outside the image bounds
var image = vips.Image.openslideload('somefile.svs', {level: 2, autocrop: true});
console.log('level size:', image.width, 'x', image.height);
// try 'vipsheader -a somefile.svs' at the command-line to see all the metadata
// fields you can get
console.log('associated images are:', image.get('slide-associated-images'));
// crop is left, top, width, height in pixels
// images are RGBA with premultiplication taken out
image.crop(100, 100, 1000, 1000).writeToFile('x.png');

// extract an associated image
image = vips.Image.openslideload('somefile.svs', {associated: 'label'});
image.writeToFile('label.png');

That also looks like exactly what we'd need.

question What bother writing node bindings when sharp can do it ? Isn't it redundant with sharp ? How does sharp differ from libvips (in terms of functionalities)

!!! !!! Had to update libvips to the latest version.
Didn't work with the libvips-dev available in the ubuntu repo, so installed from source.

Now possible to read a slide using vips in javascript

const vips = require('vips');
var image = vips.Image.openslideload('CMU-3.ndpi')

ISSUE Cannot figure out how to use dzsave using javascript bindings

// None of these work
var test = vips.Image.dzsave('CMU-3.ndpi', 'test.dz')
image.dzsave('output.dz')
vips.Image.dzsave('CMU-3.dzi')

dzsave is only mentioned in lib/autogen.js

  vips.Image.prototype.dzsave = function (filename, options) {
    return vips.call('dzsave', this, filename, options);
  };

  vips.Image.prototype.dzsaveBuffer = function (options) {
    return vips.call('dzsave_buffer', this, options);
  };

add prediction capability

Suggested workflow:

User clicks on New Classifier, clicks to get green (positive) and red (negative) examples. User enters a name for the classifier and clicks Build, which pops it in the Classifier card on the right.
User can create multiple classifiers this way.
User selects a classifier from the card. This automatically switches them to Annotations mode with a cross hair.
Live update of predictions in annotations region.
User can switch classifier at any time, which would redraw annotation region with predictions from selected classifier.