Git Product home page Git Product logo

clair-singularity's Introduction

clair-singularity

CircleCI

Scan Singularity container images for security vulnerabilities using CoreOS Clair.

screenshot

The CoreOS Clair vulnerability scanner is a useful tool able to scan docker and other container formats for security vulnerabilities. It obtains up-to-date lists of vulnerabilities for various platforms (namespaces) from public databases.

We can use Clair to scan singularity containers, by exploiting the fact that an exported .tar.gz of a singularity container image is similar to a single layer docker image.

This tool:

  • Exports a singularity image to a temporary .tar.gz file (this will be under $TMPDIR)
  • Serves the .tar.gz file via an in-built http server, so the Clair service can retrieve it
  • Calls the Clair API to ingest the .tar.gz file as a layer for analysis
  • Calls the Clair API to retireve a vulnerability report for this layer
  • Displays a simple text, or full JSON format report

Based on experiments detailed in this Gist

IMPORTANT NOTES

Functionality was last tested using SingularityCE 3.9.7.

This tool should be considered proof of concept, not heavily tested. Use at your own risk.

There is no support yet for SSL client certificates to verify that we are sending API requests to a trusted Clair instance, or that only a trusted Clair instance can retrieve images from the inbuilt http server. This means that this solution is insecure except with an isolated local install of Clair.

Requirements

To use clair-singularity you will need a Linux host with:

  • Python 3.5 or greater installed
  • SingularityCE 3+ installed (tested with 3.9.7) and the singularity executable in your PATH
  • A Clair instance running somewhere, that is able to access the machine you will run clair-singularity on. It's easiest to accomplish this using docker to run a local Clair instance as below.

Starting a local Clair instance

If you have docker available on your local machine, the easiest way to start scanning your Singularity images is to fire up a Clair instance locally, with docker. The official Clair docker images are a blank slate, and do not include any vulnerability information. At startup Clair will have to download vulnerability information from the internet, which can be quite slow. Images from github user arminc are available that include pre-seeded databases:

https://github.com/arminc/clair-local-scan

To startup a Clair instance locally using these instances:

docker run -d --name db arminc/clair-db:2022-03-31
docker run -p 6060:6060 --link db:postgres -d --name clair arminc/clair-local-scan:v2.1.8_9bca9a9a7bce2fd2e84efcc98ab00c040177e258

Replace the clair-db:2022-03-31 image tag with a later date for newer vulnerabilities

Installation

Clone the git repo, or download and extract the zip then:

python setup.py install

Usage

Clair on same machine

To scan a singularity image, using a clair instance running under local docker, on port 6060:

clair-singularity myimage.sif

/If your hostname is not resolvable to a non-localhost IP of your machine, accessible to docker containers, you must specify the IP with --bind-ip/

Clair on a different machine

If clair is running on a different machine, you must use the --clair-uri option to specify the base URI to the clair instance, and the --bind-ip and/or --bind-port options to specify a public IP and port on this machine, that clair can access to retrieve images from clair-singularity.

clair-singularity \
 --clair-uri http://10.0.1.202:6060 \
 --bind-ip=10.0.1.201 \
 --bind-port=8088 myimage.img

Full JSON Reports

By default, clair-singularity gives a simplified text report on STDOUT. To obtain the full JSON report returned by Clair use the --jsoon-output option.

clair-singularity --json-output myimage.img

clair-singularity's People

Contributors

dtrudg avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

clair-singularity's Issues

Severity Filtering

Allow filtering reports by CVE severity, only returning vulns with equal or higher severity than level specificed.

SSL Client/Server certificate checking

Implement SSL certificate checks/use so that we can verify we are talking to a trusted Clair instance, and that only our trusted Clair instance is pulling the image we tell it to.

Problem with sandbox build

This very useful took worked well with a container imported from Docker.

However, I am now trying it on an image built from a Singularity definition file, and I am seeing permission problems with the sandbox build. They sound similar to other sandbox problems reported recently.
The error I see is:

(clair-singularity) [software@easybuild clair-singularity]$ clair-singularity --bind-ip 10.1.0.3 --json-output ~/src/singularity/antspynet-0.0.3.sif > antspynet-clair-scan-$(date +%F).json
Exporting image to sandbox.
INFO:    Starting build...
INFO:    Creating sandbox directory...
FATAL:   While performing build: sandbox assemble failed: exit status 1: mv: cannot move ‘/tmp/sbuild-096432949/fs’ to ‘/tmp/tmproogpums’: Permission denied
Traceback (most recent call last):
  File "/home/software/src/venv/clair-singularity/lib/python3.6/site-packages/clair_singularity-0.2.0-py3.6.egg/clair_singularity/image.py", line 33, in image_to_tgz
    subprocess.check_call(cmd)
  File "/usr/lib64/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['singularity', 'build', '-F', '--sandbox', '/tmp/tmproogpums', '/home/software/src/singularity/antspynet-0.0.3.sif']' returned non-zero exit status 255.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/software/src/venv/clair-singularity/lib/python3.6/site-packages/clair_singularity-0.2.0-py3.6.egg/clair_singularity/cli.py", line 32, in cli
    (tar_dir, tar_file) = image_to_tgz(image, quiet)
  File "/home/software/src/venv/clair-singularity/lib/python3.6/site-packages/clair_singularity-0.2.0-py3.6.egg/clair_singularity/image.py", line 35, in image_to_tgz
    raise ImageException("Error calling Singularity export to create sandbox\n%s" % e)
clair_singularity.image.ImageException: Error calling Singularity export to create sandbox
Command '['singularity', 'build', '-F', '--sandbox', '/tmp/tmproogpums', '/home/software/src/singularity/antspynet-0.0.3.sif']' returned non-zero exit status 255.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/software/src/venv/clair-singularity/bin/clair-singularity", line 11, in <module>
    load_entry_point('clair-singularity==0.2.0', 'console_scripts', 'clair-singularity')()
  File "/home/software/src/venv/clair-singularity/lib/python3.6/site-packages/Click-7.0-py3.6.egg/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/software/src/venv/clair-singularity/lib/python3.6/site-packages/Click-7.0-py3.6.egg/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/software/src/venv/clair-singularity/lib/python3.6/site-packages/Click-7.0-py3.6.egg/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/software/src/venv/clair-singularity/lib/python3.6/site-packages/Click-7.0-py3.6.egg/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/software/src/venv/clair-singularity/lib/python3.6/site-packages/clair_singularity-0.2.0-py3.6.egg/clair_singularity/cli.py", line 33, in cli
    except ImageError as e:
NameError: name 'ImageError' is not defined

Add initial test coverage

Add initial test coverage. Given the dated Clair DB docker images, should be able to get nice test verifying final output against test container images

Cannot scan centos 8 image

WIth SingularityCE 3.8.1 on PATH...

03:00 PM $ clair-singularity centos_8.sif 
tar: ./etc/shadow-: Cannot open: Permission denied
tar: ./etc/shadow: Cannot open: Permission denied
tar: ./etc/gshadow-: Cannot open: Permission denied
tar: ./etc/gshadow: Cannot open: Permission denied
tar: Exiting with failure status due to previous errors
Error calling gzip export to compress .tar file
Command '['tar', '-C', '/tmp/tmp2livbajn', '-zcf', '/tmp/tmpqxh1v6av/centos_8.sif.tar.gz', '.']' returned non-zero exit status 2.

HTML report output

Implement a simple, attractive, easy to navigate HTML report of scan results

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.