Git Product home page Git Product logo

distro2sbom's Introduction

DISTRO2SBOM

The DISTRO2SBOM generates a SBOM (Software Bill of Materials) for either an installed application or a complete system installation in a number of formats including SPDX and CycloneDX. An SBOM for an installed package will identify all of its dependent components.

It is intended to be used as part of a continuous integration system to enable accurate records of SBOMs to be maintained and also to support subsequent audit needs to determine if a particular component (and version) has been used.

Installation

To install use the following command:

pip install distro2sbom

Alternatively, just clone the repo and install dependencies using the following command:

pip install -U -r requirements.txt

The tool requires Python 3 (3.7+). It is recommended to use a virtual python environment especially if you are using different versions of python. virtualenv is a tool for setting up virtual python environments which allows you to have all the dependencies for the tool set up in a single environment, or have different environments set up for testing using different versions of Python.

Usage

usage: distro2sbom [-h] [--distro {rpm,deb,windows,auto}] [-i INPUT_FILE] [-n NAME] [-r RELEASE] [-p PACKAGE] [-s] [--root ROOT] [--distro-namespace DISTRO_NAMESPACE] [-d] [--sbom {spdx,cyclonedx}]
                   [--format {tag,json,yaml}] [-o OUTPUT_FILE] [-V]

Distro2Sbom generates a Software Bill of Materials for the specified package or distribution.

options:
  -h, --help            show this help message and exit
  -V, --version         show program's version number and exit

Input:
  --distro {rpm,deb,windows,auto}
                        type of distribution (default: auto)
  -i INPUT_FILE, --input-file INPUT_FILE
                        name of distribution file
  -n NAME, --name NAME  name of distribution
  -r RELEASE, --release RELEASE
                        release identity of distribution
  -p PACKAGE, --package PACKAGE
                        identity of package within distribution
  -s, --system          generate SBOM for installed system
  --root ROOT           location of distribution packages
  --distro-namespace DISTRO_NAMESPACE
                        namespace for distribution

Output:
  -d, --debug           add debug information
  --sbom {spdx,cyclonedx}
                        specify type of sbom to generate (default: spdx)
  --format {tag,json,yaml}
                        specify format of software bill of materials (sbom) (default: tag)
  -o OUTPUT_FILE, --output-file OUTPUT_FILE
                        output filename (default: output to stdout)

Operation

The --distro option is used to identify the type of distribution. The default option is auto which attempts to determine the type of distribution by searching for the presence of key applications required by the tool. If none of the required applications are found, the tool terminates.

The --name option and --release option is used to identify the name and release of the distribution. These options are optional but if they are specified, values for both options are required. If they are not specified, values for these options shall be obtained from system files installed on the system.

The --input-file option is used to provide a filename containing the list of packages installed on the system. The format of the file is dependent on the specified --distro option.

  • deb. The file used is the output of the following command

    dpkg -l > [filename.out]

    Sample file contents

    Desired=Unknown/Install/Remove/Purge/Hold
    | Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
    |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
    ||/ Name                                             Version                             Architecture Description
    +++-================================================-===================================-============-==================================================================================
    ii  acl                                              2.3.1-1                             amd64        access control list - utilities
    ii  adduser                                          3.129                               all          add and remove users and groups
    ii  adwaita-icon-theme                               43-1                                all          default icon theme of GNOME
    ii  alien                                            8.95.6                              all          convert and install rpm and other packages
    ii  alsa-tools                                       1.2.5-2                             amd64        Console based ALSA utilities for specific hardware
  • rpm. The file used is the output of the following command. Note that it is recommended to sort the list of files as this makes it easier to find the packages in the SBOM.

    rpm -qa | sort > [filename.out]

    Sample file contents

    accountsservice-0.6.55-10.el9.x86_64
    accountsservice-libs-0.6.55-10.el9.x86_64
    acl-2.3.1-3.el9.x86_64
    adcli-0.9.1-7.el9.x86_64
    adwaita-cursor-theme-40.1.1-3.el9.noarch
    adwaita-icon-theme-40.1.1-3.el9.noarch
  • windows. The file used is the output of the following command

    get-wmiobject -class win32_product | Out-file -filePath [filename.out]

    Sample file contents

    IdentifyingNumber : {....}
    Name              : Python 3.10.5 Utility Scripts (64-bit)
    Vendor            : Python Software Foundation
    Version           : 3.10.5150.0
    Caption           : Python 3.10.5 Utility Scripts (64-bit)
    

If the specified filename is not found, the tool will terminate.

The --package option is used to identify the name of a package or application installed on the system. If the specified package or application is not found, the tool terminates. This option is not supported if the --distro option is set to 'windows'.

The --system option is used to generate an SBOM for all the applications installed on the system. Note that this option will take some time to complete as it is dependent on the number of installed applications. This option is not supported if the --distro option is set to 'windows'.

The --root option is used to specify an alternative directory location for the installed packages. This option only applies for 'deb' distributions.

The --disto-namespace option is used to specify a namespace to be included in the generated PURL identifiers for the packages.

At least one of the --input-file, --package or --system options must be specified. If multiple options are specified, the --input-file option followed by the --system option will be assumed.

The --sbom option is used to specify the format of the generated SBOM (the default is SPDX). The --format option can be used to specify the formatting of the SBOM (the default is Tag Value format for a SPDX SBOM). JSON format is supported for both SPDX and CycloneDX SBOMs.

The --output-file option is used to control the destination of the output generated by the tool. The default is to report to the console but can be stored in a file (specified using --output-file option).

Examples

SBOM for an Installed Package

To generate an SBOM for the installed zip package.

distro2sbom --distro auto --name <distro name> --release <distro release> --package zip

This will automatically detect the type of distribution and generate an SBOM in SPDX Tag value format to the console.

SBOM for Distribution

To generate an SBOM for a system distribution.

distro2sbom --distro deb --name <distro name> --release <distro release> --input-file <distrofile> --sbom cyclonedx --output-file <distrooutfile>

This will generate an SBOM in CycloneDX JSON value for a distribution file in dpkg format (indicated by the 'deb' option)

SBOM for System

To generate an SBOM for an installed system, obtaining the name and release of the system from installed system files.

distro2sbom --distro deb --system --format json --output-file <distrooutfile>

This will generate an SBOM in SPDX JSON value for a distribution file in dpkg format (indicated by the 'deb' option)

Specific options for rpm/yum based distro

The following [optional] environment variable are available to customize rpm and yum commands used by the tool. This can be usefull for example to enable/disable some repo or to support chrooted environments.

  • DISTRO2SBOM_ROOT_PATH The path prefix where to get /etc/os-release
  • DISTRO2SBOM_RPM_OPTIONS Additional options passed to rpm commands (used by rpm -qa to list all packages and rpm -qi <pkg> to query information on a package)
  • DISTRO2SBOM_YUM_OPTIONS Additional options passed to yum commands (used by yum repoquery --deplist <pkg> to get dependencies)
export DISTRO2SBOM_ROOT_PATH=/path-to-distrib/slash
export DISTRO2SBOM_RPM_OPTIONS="--root /path-to-distrib/slash"
export DISTRO2SBOM_YUM_OPTIONS="--installroot=/path-to-distrib/slash --setopt=reposdir=/path-to-distrib/repos --setopt=install_weak_deps=False --repo=my-repo"
distro2sbom --distro rpm --system --sbom cyclonedx --format json --output-file <distrooutfile>

This will generate an SBOM in CYCLONEDX JSON value for a chrooted distribution located at /path-to-distrib/slash

Licence

Licenced under the Apache 2.0 Licence.

Limitations

This tool is meant to support software development and security audit functions. However, the usefulness of the tool is dependent on the SBOM data which is provided to the tool. Unfortunately, the tool is unable to determine the validity or completeness of such a SBOM file; users of the tool are therefore reminded that they should assert the quality of any data which is provided to the tool.

When processing and validating licenses, the application will use a set of synonyms to attempt to map some license identifiers to the correct SPDX License Identifiers. However, the user of the tool is reminded that they should assert the quality of any data which is provided by the tool particularly where the license identifier has been modified.

Dependencies between applications are only produced for the --package and --system options.

The --package option is not supported if the --distro option is set to 'windows'.

Whilst PURL and CPE references are automatically generated for components, the accuracy of such references cannot be guaranteed as they are dependent on the validity of the data associated with the component.

Feedback and Contributions

Bugs and feature requests can be made via GitHub Issues.

distro2sbom's People

Contributors

anthonyharrison avatar ericbla avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

distro2sbom's Issues

DpkgBuilder: Using input-file does not work

Example command:

distro2sbom --distro deb --name bullseye --release 11 --input-file myFile --sbom cyclonedx --format json

Output:

Traceback (most recent call last):
  File "/usr/local/bin/distro2sbom", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.9/dist-packages/distro2sbom/cli.py", line 216, in main
    sbom_build.parse_data(args["input_file"])
  File "/usr/local/lib/python3.9/dist-packages/distro2sbom/distrobuilder/dpkgbuilder.py", line 45, in parse_data
    if self.system_data.get("id") is not None:
AttributeError: 'DpkgBuilder' object has no attribute 'system_data'

Reason:

self.system_data in DpkgBuilder is only created in init if name and release were not given.

Yet, by using "--input-file" the CLI forces the user to give both name and release.

'RpmBuilder' object has no attribute 'license'

Hi.

While playing around with version 0.2.0 I got the following exception:

Traceback (most recent call last):
  File "/opt/app-root/bin/distro2sbom", line 8, in <module>
    sys.exit(main())
  File "/opt/app-root/lib64/python3.8/site-packages/distro2sbom/cli.py", line 209, in main
    sbom_build.process_distro_package(args["package"])
  File "/opt/app-root/lib64/python3.8/site-packages/distro2sbom/distrobuilder/rpmbuilder.py", line 206, in process_distro_package
    if self.process_package(module_name):
  File "/opt/app-root/lib64/python3.8/site-packages/distro2sbom/distrobuilder/rpmbuilder.py", line 149, in process_package
    license = self.license.find_license(self.get("License"))
AttributeError: 'RpmBuilder' object has no attribute 'license'

The command used to generate this exception:

distro2sbom --distro rpm --name centos --release 7 --sbom cyclonedx -o sbom.json --format json -p bash

The enviroment is a Docker container (quay.io/centos7/python-38-centos7)

I don't get the exception if I instead use -i packages.txt. However in this case there is no license information in the resulting file (even when switching the SBOM format to spdx). Which I think might be another issue.

My assumption is, that distrobuilder.py contains changes (the license attribute, along with the associated methods to find licenses), which weren't included in the commits related to version 0.2.0.

Debian Recommends dependencies are not shown in SBOM...

For example, here's the dependency information for openssh-server:

Provides: ssh-server
Depends: adduser (>= 3.9), dpkg (>= 1.9.0), libpam-modules (>= 0.72-9), libpam-runtime (>= 0.76-14), lsb-base (>= 4.1+Debian3), openssh-client (= 1:8.9p1-3ubuntu0.6), openssh-sftp-server, procps, ucf (>= 0.28), debconf (>= 0.5) | debconf-2.0, libaudit1 (>= 1:2.2.1), libc6 (>= 2.34), libcom-err2 (>= 1.43.9), libcrypt1 (>= 1:4.1.0), libgssapi-krb5-2 (>= 1.17), libkrb5-3 (>= 1.13~alpha1+dfsg), libpam0g (>= 0.99.7.1), libselinux1 (>= 3.1~), libssl3 (>= 3.0.2), libsystemd0, libwrap0 (>= 7.6-4~), zlib1g (>= 1:1.1.4)
Pre-Depends: init-system-helpers (>= 1.54~)
Recommends: default-logind | logind | libpam-systemd, ncurses-term, xauth, ssh-import-id

From the build perspective, ssh-import-id was installed based on the Recommends line above under openssh-server, and from the SBOM, ssh-import-id does show up:

    {
      "type": "library",
      "bom-ref": "447-ssh-import-id",
      "name": "ssh-import-id",
      "version": "5.11",
      "supplier": {
        "name": "Ubuntu Developers",
        "contact": [
          {
            "email": "[email protected]"
          }
        ]
      },
      "cpe": "cpe:2.3:a:ubuntu_developers:ssh-import-id:5.11-0ubuntu1:*:*:*:*:*:*:*",
      "description": "securely retrieve an SSH public key and install it locally",
      "externalReferences": [
        {
          "url": "http://launchpad.net/ssh-import-id",
          "type": "website",
          "comment": "Home page for project"
        }
      ],
      "purl": "pkg:deb/[email protected]"
    }

If I look at the SBOM dependencies for openssh-server, though, ssh-import-id doesn't show up:

    {
      "ref": "111-openssh-server",
      "dependsOn": [
        "2-adduser",
        "112-dpkg",
        "19-libpam-modules",
        "114-libpam-runtime",
        "27-lsb-base",
        "115-openssh-client",
        "122-openssh-sftp-server",
        "123-procps",
        "129-ucf",
        "12-debconf",
        "4-libaudit1",
        "6-libc6",
        "81-libcom-err2",
        "9-libcrypt1",
        "80-libgssapi-krb5-2",
        "84-libkrb5-3",
        "11-libpam0g",
        "13-libselinux1",
        "24-libssl3",
        "43-libsystemd0",
        "132-libwrap0",
        "39-zlib1g"
      ]
    }

It's confusing looking at the SBOM to see ssh-import-id show up without having traceability to the Debian package that pulls in ssh-import-id.

Enhance pkg url format with distribution name and release

Background

Across the different package types you support (e.g. DEB, RPM, etc) each has multiple distros. Each of those distros have their their own CVEs and security RSS feeds for package dependencies.

Even within the same distribution there's multiple releases across distros which have their own versioning and CVE patch channels i.e. backports.

Request

Can you make the pkg url optionally have a customizable distro type in the pkg url?

For example, instead of:

I would like to see

pkg:rpm/amazonlinux2023/[email protected]

So that security vendors can support specific operating systems and not need to infer.

Additional information

I tried getting different results with:

distro2sbom --name amazonlinux --release 2023 -p curl-minimal --distro rpm --sbom cyclonedx --format json

And it does seed some metadata about the OS but not in a way that vendors would easily be able to support it. Maybe an additional option like --release-pkg-url or something?

Why the feature

Projects are typically multi-lingual and have a stack to support. pkg url is meant to provide unique identifiers which make up a project. It is common for SBOMs to be combined into a unified SBOM.

For example, a python project with JavaScript shipping a docker container would have separate SBOMs for: Python pip, NodeJS npm packages, operating system packages. Making a unified SBOM means creating one file mixing the pkg url for all three of those in the same file.

In this scenario, you lose the extra metadata you seed and the only thing which can be relied upon is the pkg-url for full content of what that dependency provides. So it is important for the pkg url to be sufficiently unique across Linux distributions and single-distribution releases (e.g. LTS or not etc).

setup.py seems to assume repo root

You cannot seem to run from another path e.g python /test/setup.py, it look for README.md, etc. locally rather than at the location of the .py file.

Older RHEL-compatible targets seem to be unable to find dependencies

Trying this out on a few different RHEL-compatible targets seems to show issues installing the dependencies, e.g.

$ docker run --rm -it rockylinux:8 sh -c 'yum install -y python3 python3-pip;pip3 install lib4sbom'
...
Installed:
  platform-python-pip-9.0.3-22.el8.rocky.0.noarch      python3-pip-9.0.3-22.el8.rocky.0.noarch      python3-setuptools-39.2.0-6.el8.noarch      python36-3.6.8-38.module+el8.5.0+671+195e4563.x86_64     

Complete!
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Collecting lib4sbom
  Could not find a version that satisfies the requirement lib4sbom (from versions: )
No matching distribution found for lib4sbom
...

The same error is seen on:

  • centos:7
  • centos:8

However the following work fine:

  • quay.io/centos/centos:stream9
  • amazonlinux:2023
  • amazonlinux:2

The failing targets all seem to be Python 3.6 whereas the others are more recent and generally already come with Python 3.
Do I need to reconfigure pip to look at a particular index?

Additional options for SBOM generation...

As I understand it, LIB4SBOM_CYCLONEDX_VERSION and LIB4SBOM_SPDX_VERSION can control SBOM version outputs through lib4sbom. In addition, it would be nice if distro2SBOM incorporated the ability to specify the following other lib4sbom parameters:

  1. Product: "application", "framework", "library", "container", "operating-system", "device", "firmware", "file"
  2. Product Name
  3. Author

Getting information for windows systems does not work

I am trying to obtain the SBOM of a Windows 10 system and I always get the same error

[ERROR] Feature not available
[ERROR] Unable to locate package

The command line I run is:

distro2sbom --distro windows --name "Microsoft Windows 10 Pro" --release "10.0.19044" --system --sbom cyclonedx --format json --output-file "sbom.json"

The error obtained when executing in debug mode is

Distro type: windows
Input file:
Distro name: Microsoft Windows 10 Pro
Distro release: 10.0.19044
Package:
System SBOM: True
SBOM type: cyclonedx
Format: json
Output file: sbom.json
This may take some time...
[ERROR] Feature not available
[ERROR] Unable to locate package

Invalid CPE generated on Debian packages with colon ("epoch") in version number

I tried to use the tool on Ubuntu and used the zlib1g package as a guinea pig.

distro2sbom --distro deb -p zlib1g --sbom cyclonedx --format json -o zlib1g.sbom.json

The resulting JSON file cannot be parsed with Dependency Track.

024-01-03 11:07:16,676 WARN [InternalAnalysisTask] An error occurred while parsing: cpe:2.3:a:canonical:zlib1g:1:1.2.11.dfsg-2ubuntu9.2:*:*:*:*:*:*:* - The CPE is invalid and will be discarded. Invalid CPE (too many components): cpe:2.3:a:canonical:zlib1g:1:1.2.11.dfsg-2ubuntu9.2:*:*:*:*:*:*:*

This warning disappears when removing the 1: prefix from the version used in the generated CPE entry.

Issue on running utility on CentOS 7

hi,
On try to run the distro2sbom on CentOS i am getting below:

distro2sbom --distro rpm --system --format json --output-file /tmp/test.txt --sbom cyclonedx --format json
Traceback (most recent call last):
File "/usr/local/bin/distro2sbom", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.8/site-packages/distro2sbom/cli.py", line 203, in main
sbom_build = RpmBuilder(args["name"], args["release"], args["debug"])
File "/usr/local/lib/python3.8/site-packages/distro2sbom/distrobuilder/rpmbuilder.py", line 21, in init
self.system_data = self.get_system()
File "/usr/local/lib/python3.8/site-packages/distro2sbom/distrobuilder/distrobuilder.py", line 81, in get_system
metadata[data[0].lower()] = data[1].replace('"', "").strip()
IndexError: list index out of range

It seems to be related to the fact that /etc/os-release in CentOS 7 is containing empty lines like below, so most probably code to be updated to take this in account:

NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

Thanks in advance,
Oleg

Running without `--name` and `--release` throws `IndexError: list index out of range`

Running with --package but without --name and --release throws an error IndexError: list index out of range.

(test-env) [user@my ~]$ distro2sbom --distro rpm --package xyz--output-file xyz-rpm.cdx.json
Traceback (most recent call last):
  File "/home/ansible/test-env/bin/distro2sbom", line 8, in <module>
    sys.exit(main())
  File "/home/ansible/test-env/lib64/python3.9/site-packages/distro2sbom/cli.py", line 206, in main
    sbom_build = RpmBuilder(args["name"], args["release"], args["debug"])
  File "/home/ansible/test-env/lib64/python3.9/site-packages/distro2sbom/distrobuilder/rpmbuilder.py", line 22, in __init__
    self.system_data = self.get_system()
  File "/home/ansible/test-env/lib64/python3.9/site-packages/distro2sbom/distrobuilder/distrobuilder.py", line 81, in get_system
    metadata[data[0].lower()] = data[1].replace('"', "").strip()
IndexError: list index out of range

It seems adding --name xyz --release xyz fixes the command, but it does not add much additional information to the SBOM.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.