Git Product home page Git Product logo

healthcare-api-dicom-fuse's Introduction

DICOMFuse

DICOMFuse is a file system that integrates with the Cloud Healthcare API. DICOMFuse lets you anonymize (using an anonymization tool), read, upload, and delete images in DICOM stores.
DICOMFuse is cross-platform application written in Java. You can use it in Linux, Windows or macOS.

Prerequisites

Applications

Before running DICOMFuse, ensure that you've installed the following applications:

  • Java SE Runtime Environment 8.
  • WinFsp for using FUSE in Windows (Windows 7x64, Windows 10x64).
  • OSXFuse for using FUSE in macOS (macOS 10.14 Mojave).
  • libfuse for using FUSE in Linux, install libfuse - sudo apt install libfuse2

FUSE has been supported in Linux kernel since 2.6.14 version.

Credentials

DICOMFuse will use application default credentials to authenticate access to the Cloud Healthcare API. The identity must have the permissions within the Healthcare DICOM Editor role.
Alternatively, you can specify a service account key via the --keyFile flag. The easiest way to set up credentials is to use your user credentials with the gcloud tool:

gcloud auth application-default login

Running DICOMFuse

To run DICOMFuse:

  1. Download the latest JAR from the releases tab.
  2. To start DICOMFuse, open a terminal in the DICOMFuse folder and input:
    java -jar healthcare-api-dicom-fuse-X.Y.Z.jar [options]
    In Windows - java -Dfile.encoding=UTF8 -jar healthcare-api-dicom-fuse-X.Y.Z.jar [options]
  3. To stop DICOMFuse, press CTRL+C.

ERROR and INFO logs present in the terminal. During launching DICOMFuse, logs folder will be created next to the DICOMFuse JAR, where DEBUG, INFO and ERROR logs will be saved.

Mount options

You can specify the following mount options when you start DICOMFuse:

* --datasetAddr, -a
    Web address to the Dataset.
    Example: https://healthcare.googleapis.com/v1beta1/projects/PROJECT/locations/LOCATION/datasets/DATASET
* --mountPath, -p
    Path to mount the Dataset.
    Example on Linux: /home/user/fuse-mount-folder
    Example on macOS: /Users/user/fuse-mount-folder
    Example on Windows: J:\
  --cacheTime, -t
    Each value in this option is measured in seconds.
    To optimize DICOMFuse, the following resources are cached: DICOM store folders, Study folders, Series folders,
    list of Instances (first parameter in option), and opened Instances files (second parameter).
    Opened Instance files are cached to the temporary folder in the user space on disk.
    Other objects are cached to RAM. Cached files will be deleted if you close DICOMFuse or if the cached
    files become out of date. If you delete or upload an Instance file locally, the cache will
    be invalidated, and updated if you open a folder again.
    Default: 60,300
  --cacheSize, -s
    Maximum cache size in megabytes for cached instances. The maximum file size that can be
    downloaded/uploaded is cacheSize / 4
    Default: 10000
  --enableDeletion, -d
    Some programs can delete files and deletion can also be done manually. Sometimes accidental deletions can occur.
    If you don't want to delete files, you can set `--enableDeletion=false`
    Default: true
  --keyFile, -k
    Path to the account service key
  --extraMountOptions
    You can add additional mount options for libfuse, WinFsp or OSXFuse.
    Example: --extraMountOptions allow_other,option2=VALUE,etc  Note: since DICOMFuse is specific 
    for correct behavior in different operating systems, some mount options were included initially. 
    See: https://github.com/GoogleCloudPlatform/healthcare-api-dicom-fuse/blob/develop/src/main/java/com/google/dicomwebfuse/MountOptions.java
    Default: []
  --help, -h
    Print help

Note: * are required options.
The current implementation supports 15,000 results in folders.

Description

Data model

  • Dataset: mounted folder.
  • DICOM Store: subfolder of dataset.
  • DICOM Study: subfolder of store.
  • DICOM Series: subfolder of Study.
  • DICOM Instance: a file within Series.

Operations supported

You can use a graphical interface on an OS or commands in the terminal to navigate through folders.

Operations supported in graphical interface:

  • Upload a file in the DICOM Store folder (drag and drop).
  • Upload a file in the Series folder (drag and drop).
  • Delete a *.dcm file in the Series folder.
  • Overwrite existing file.
  • Create DICOM Store folder. If you try to create a folder at a different level, you will get the error:
    Error creating directory path_to_the_directory: Operation not permitted
  • Rename empty DICOM Store (previous DICOM Store will be deleted and a new one will be created. Only supported for empty DICOM Stores, otherwise an error will be returned).
  • Replace operation (drag and drop) is not supported.

Note: to get better performance, you must upload files to DICOM Store folder, and files must have .dcm or another extension.

Operations supported in the terminal:

  • cd - change a directory. Supported navigate to unpresented results in folders if they exist on the server.
  • ls - list the objects existing in a folder.
  • cp - copy only instances files (not folders) to another DICOM store or to a local computer.
  • rm - delete instances files.
  • Create DICOM Store folder. If you try to create a folder at a different level, you will get the error:
    mkdir: cannot create directory ‘new_folder’: Operation not permitted
  • mv - rename empty DICOM Store (previous DICOM Store will be deleted and a new one will be created. Only supported for empty DICOM Stores, otherwise an error will be returned).

Any reading use cases can be used, such as de-identification or manual editing of the (binary) DICOM object.

Caching objects

To optimize DICOMFuse, the following resources are cached: DICOM store folders, Study folders, Series folders, list of Instances (first parameter in option), and opened Instances files (second parameter). Opened Instance files are cached to the temporary folder in the user space on disk. Other objects are cached to RAM. Cached files will be deleted if you close DICOMFuse or if the cached files become out of date.

  • If you delete or upload an Instance, the cache will be invalidated, and updated if you open a folder again.
  • Before opening an Instance, the size of it is 0 bytes. If you open the Instance, it will be downloaded from the server and you will see the Instance size. If you read the Instance again, it will be reading from the local cache, not from the server. After repeated requests, if the cacheTime of the Instance is out of date, the Instance will be downloaded from the server again.
  • If you create a request (e.g. cd command in a Dataset folder) and the object does not exist in the local cache, the request will be sent to the server and object will be cached if object exist on the server.

Packaging

  1. git clone https://github.com/GoogleCloudPlatform/healthcare-api-dicom-fuse.git
  2. mvn package in the healthcare-api-dicom-fuse folder in the terminal (for packaging to jar).

There is a healthcare-api-dicom-fuse-X.Y.Z.jar file in the healthcare-api-dicom-fuse/target folder.

License

This application is licensed under Apache License, Version 2.0. Full license text is available in LICENSE.

healthcare-api-dicom-fuse's People

Contributors

arvgord avatar chaopeng avatar danielbeaudreau avatar dependabot[bot] avatar dzelemba avatar msbukal avatar pavertomato avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

healthcare-api-dicom-fuse's Issues

Document and Automate Release

I'm hoping it's possible to configure Google Cloud Build to release a new pre-built JAR and add it to the GitHub releases page via the click of a button.

Output HTTP error body for STOW error

Right now the logs show:

com.google.dicomwebfuse.exception.DicomFuseException: Failed to upload - dicom/UID.dcm!HTTP/1.1 409 Conflict https://healthcare.googl
eapis.com/v1beta1/projects/<project>/locations/us-central1/datasets/<dataset>/dicomStores/<dicomStore>/dicomWeb/studies/

But the error body has more useful information about what kind of error it is (invalid DICOM file vs. already exists), so we should print that as well.

Supported sharded directories

In order to get around the fact that we can't list 100s of thousands of studies in a single folder, DICOMfuse should support a --study_dir_sharding flag that specifies what level of nesting should be used and the size of each level.

0, 5000 - current behavior
1: 5000 - 1 level of nesting with 5k files in each folder

  • So under /dicomStore it would be
    • /1/ --> studies from 1 -> 5000
    • /2/ --> studies from 5000 -> 10000
    • ....
    • 5000/ -> studies from 24,995,000 -> 25,000,000

This is just an idea for now.

Performance investigations

What transmission rate can dicomFuse handle for a single file?

How much memory/cpu does a single instance transfer require?

What transmission rate can be achieved if a user sends multiple parallel requests to DICOMfuse?

Support navigating to study even it's not in the first 5k

Right now we limit the number of studies we can handle to 5k since we can't list too many items in a single directory, but we should still be able to support someone navigating to a directory even if it doesn't appear in the first 5k.

Support passing in extra mount options

gcsfuse supports the -o flag which you can pass extra mount options through, e.g.

gcsfuse -o allow_other

Which allows other users to access this mount. We should support the same option in DICOMfuse.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.