gapfish / deep-learning-images Goto Github PK

Packer templates for Caffe and TensorFlow based EC2 GPU-enabled machine and Docker container images from RealScout.

Download the v5 cuDNN bundle from https://developer.nvidia.com/cudnn and place it in install/.

You'll also need:

• Packer
• An AWS Account to build AMIs
• Docker to build container images

Finally, clone this repo and run the commands below from the root of this project. Probably:

git clone https://github.com/RealScout/deep-learning-images.git
cd deep-learning-images

First create a base image with CUDA and cuDNN libraries installed. We need a Ubuntu 16.04 AMI to start building up from, the template includes a recent one that can be overriden by passing -var ubuntu_ami=ami-xxxx to packer.

To build your base image run: packer build base.json. You should get a message about a successful build including an AMI ID like ami-xxxxx.

==> amazon-ebs: Uploading install/cudnn-8.0-linux-x64-v5.1.tgz => /tmp/cudnn.tar.gz
==> amazon-ebs: Provisioning with shell script: install/cudnn.sh
==> amazon-ebs: Stopping the source instance...
==> amazon-ebs: Waiting for the instance to stop...
==> amazon-ebs: Creating the AMI: Base 1479906007
    amazon-ebs: AMI: ami-6908e904
==> amazon-ebs: Waiting for AMI to become ready...

Note that AMI ID, here ami-6908e904, we'll need it in the next step.

Now follow instructions for your favorite deep learning package. You could combine them and make one enormous image with all of them installed.

Each of these templates will also build Docker container images. To prevent that, pass -only=amazon-ebs to packer. Use -only=docker to only build the docker images. note: docker container images are not GPU-enabled as currently configured.

Caffe version: HEAD

To build your caffe image run (substitute the ami id from the base step): packer build -var base_ami=ami-6908e904 caffe.json.

TF Version: v0.11.0

To build your tensorflow image run (substitute the ami id from the base step): packer build -var base_ami=ami-6908e904 tensorflow.json.