Git Product home page Git Product logo

docker-smrtlink's Introduction

SMRT Link Dockerfile

This is a work in progress. It might not work yet.

This repository contains everything needed to get PacBio SMRT Link software running in a Docker container. The Dockerfile presented here builds a Docker image that contains SMRT Link. You don't need to download anything from here, just make sure you have a system that meets the minimum requirements and execute the commands below to build an image and run the container.

Host system

SMRTlink needs a big computer. Much bigger than laptops and most desktops. Here are the system requirements at present:

The recommended and minimum configurations are:
   Total Physical Memory:        64 GB (recommended), 32 GB (minimum)
   Total Number of Processors:   32    (recommended), 16    (minimum)
   Minimum open files limit   ('ulimit -n'):  8192  (required)
   Minimum user process limit ('ulimit -u'):  8192  (required)

Having less than the required limits will result in a warning, and by default installation will not complete if there are any warnings.

Amazon Web Services

I developed this Dockerfile on an m4.10xlarge EC2 instance, which exceeds the above recommended configuration, running Amazon Linux. I made the following changes to the default configuration:

  • Increase the storage to 100 GB

  • Open the following ports:

    • 9090-9095 (for SMRT Link services)

    • 8243 (for SMRT Link )

    • 443 (for git access via https during installation)

Once the instance is running, increase the open files limit. I took the approach presented at http://aws-labs.com/increase-open-files-limit/ Paste the following into /etc/security/limits.conf just before the line # End of file:

*         hard    nofile      500000
*         soft    nofile      500000
root      hard    nofile      500000
root      soft    nofile      500000

Reboot the amazon machine for the new settings to take effect. You can run ulimit -Sn to see if the limit was reset.

Now install docker with:

sudo yum update -y
sudo yum install -y docker
sudo yum install -y git
sudo service docker start
sudo usermod -a -G docker ec2-user

Log out and then log in again for the usermod changes to take effect.

Your Amazon machine is now ready to build and run the SMRTLink Docker container.

Build and run

To build the docker container, execute:

docker build --ulimit nofile=500000:500000 https://github.com/caseywdunn/docker-smrtlink.git#master:docker -t smrtlink:5.0.1.9585

If all goes well this will finish with the line Successfully built [id], where [id] is the image ID for the image you just built. You can then run a container based on this id with (substituting [id]):

docker run --ulimit nofile=500000:500000 [id]

To run the container interactively, eg to debug it, you can instead run:

docker run --ulimit nofile=500000:500000 -it [id] bash

Installation notes

The SMRT Link installation is done by the script /opt/pacbio/smrtlink/install/smrtlink-release_5.0.1.9585/admin/bin/installprompter. This script has a few options that can simplify installation:

  • --batch executes it in non-interactive mode, so there are no user prompts

  • --ignore-system-check will proceed with installation even if there is a warning during system check

docker-smrtlink's People

Contributors

caseywdunn avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar

Forkers

asalomatov ylock ducatimonster916

docker-smrtlink's Issues

Connection attempt failed

docker run gives the error below. The critical bit seems to be this:

Checking server status with 'pbservice status'...
  Executing: "/opt/pacbio/smrtlink/install/smrtlink-release_5.0.1.9585/admin/bin/../../bundles/smrttools/smrtcmds/bin/pbservice" status --log-level DEBUG --host "56747d2fd71e" --port "9091"
Connection attempt to 56747d2fd71e:9091 failed

It looks like networking for the container isn't configured correctly, and it can't reach the host or port.

the host is set during build, specifically:

...
Status: Downloaded newer image for ubuntu:16.04
 ---> 8b72bba4485f
Step 2/15 : MAINTAINER Casey Dunn ([email protected])
 ---> Running in 56747d2fd71e
 ---> 7c704f5cac07
Removing intermediate container 56747d2fd71e
... 
The SMRT Link installation root appears to be on a local file system,
with file system type 'overlayfs'.


Using Local File System Root (computed default): /opt/pacbio/smrtlink
Detecting DNS names...


Using dnsname (detected dnsname): 56747d2fd71e

Using GUI port (default): 9090
Using services port (computed default): 9091
Using GUI shutdown port (computed default): 9092
Using GUI ajp port (computed default): 9093
Using GUI initial memory (computed default): 8192
Using GUI maximum memory (computed default): 8192
Using services initial memory (computed default): 32768
Using services maximum memory (computed default): 32768

Using SMRT View port (computed default): 9094
Using SMRT View initial memory (computed default): 32768
Using SMRT View maximum memory (computed default): 32768
...

So the host name is from the dnsname and the dnsname is from the intermediate container in which ubuntu was instantiated.

The full run error

$ docker run 0735efbedd9a
+ set -o errexit
+ set -o posix
+ set -o pipefail
+ set -o errtrace
+ trap unexpected_error ERR
+ PATH_ORIG=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+ PATH=/usr/bin:/bin
++ basename /opt/pacbio/smrtlink/admin/bin/run-sat-services
+ g_prog=run-sat-services
++ dirname /opt/pacbio/smrtlink/admin/bin/run-sat-services
+ g_progdir=/opt/pacbio/smrtlink/admin/bin
++ readlink -f /opt/pacbio/smrtlink/admin/bin
+ g_progdir_abs=/opt/pacbio/smrtlink/install/smrtlink-release_5.0.1.9585/admin/bin
+ rootdir=/opt/pacbio/smrtlink/admin/bin/../..
+ rootdir_abs=/opt/pacbio/smrtlink/install/smrtlink-release_5.0.1.9585/admin/bin/../..
+ g_canneddata_dir=/opt/pacbio/smrtlink/install/smrtlink-release_5.0.1.9585/admin/bin/../../bundles/smrtinub/current/private/pacbio/canneddata
+ g_pbservice_exe=/opt/pacbio/smrtlink/install/smrtlink-release_5.0.1.9585/admin/bin/../../bundles/smrttools/smrtcmds/bin/pbservice
+ g_python_exe=/opt/pacbio/smrtlink/install/smrtlink-release_5.0.1.9585/admin/bin/../../bundles/smrttools/smrtcmds/bin/python
+ serverconfigfile=/opt/pacbio/smrtlink/install/smrtlink-release_5.0.1.9585/admin/bin/../../bundles/smrtlink-analysisservices-gui/current/private/pacbio/smrtlink-analysisservices-gui/smrtlink-system-config.json
+ g_root=/secondary-analysis
+ root=/secondary-analysis
+ echo 'Reading server configuration information...'
Reading server configuration information...
++ /opt/pacbio/smrtlink/install/smrtlink-release_5.0.1.9585/admin/bin/../../bundles/smrttools/smrtcmds/bin/python -c 'import json; d = json.load(open('\''/opt/pacbio/smrtlink/install/smrtlink-release_5.0.1.9585/admin/bin/../../bundles/smrtlink-analysisservices-gui/current/private/pacbio/smrtlink-analysisservices-gui/smrtlink-system-config.json'\'')); print d['\''smrtflow'\'']['\''server'\'']['\''dnsName'\'']'
+ host=56747d2fd71e
++ /opt/pacbio/smrtlink/install/smrtlink-release_5.0.1.9585/admin/bin/../../bundles/smrttools/smrtcmds/bin/python -c 'import json; d = json.load(open('\''/opt/pacbio/smrtlink/install/smrtlink-release_5.0.1.9585/admin/bin/../../bundles/smrtlink-analysisservices-gui/current/private/pacbio/smrtlink-analysisservices-gui/smrtlink-system-config.json'\'')); print d['\''smrtflow'\'']['\''server'\'']['\''port'\'']'
+ port=9091
+ /opt/pacbio/smrtlink/admin/bin/import-canneddata
Reading server configuration information...

Checking server status with 'pbservice status'...
  Executing: "/opt/pacbio/smrtlink/install/smrtlink-release_5.0.1.9585/admin/bin/../../bundles/smrttools/smrtcmds/bin/pbservice" status --log-level DEBUG --host "56747d2fd71e" --port "9091"
Connection attempt to 56747d2fd71e:9091 failed
import-canneddata: Error! Services are not running properly...
++ unexpected_error
++ local errstat=1
+++ caller
++ echo 'run-sat-services: Error! Encountered unexpected error at '\''line 63 /opt/pacbio/smrtlink/admin/bin/run-sat-services'\'', bailing out...'
run-sat-services: Error! Encountered unexpected error at 'line 63 /opt/pacbio/smrtlink/admin/bin/run-sat-services', bailing out...
++ exit 1

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.