Git Product home page Git Product logo

cyclonedx-node-npm's Introduction

cyclonedx-npm

shield_npm-version shield_gh-workflow-test shield_coverage shield_ossf-best-practices shield_license
shield_website shield_slack shield_groups shield_twitter-follow


Create CycloneDX Software Bill of Materials (SBOM) from npm projects.

Based on OWASP Software Component Verification Standard for Software Bill of Materials's criteria, this tool is capable of producing SBOM documents almost passing Level-2 (only signing needs to be done externally).

The resulting SBOM documents follow official specifications and standards, and might have properties following cdx:npm Namespace Taxonomy .

Requirements

  • node >= 14
  • npm in range 6 - 10

Installation

There are multiple methods for installing this tool:

  • As a global tool ala npm:

    npm install --global @cyclonedx/cyclonedx-npm
  • As a global tool ala npx:

    npx --package @cyclonedx/cyclonedx-npm --call exit
  • As a development dependency of the current projects:

    npm install --save-dev @cyclonedx/cyclonedx-npm

Usage

Depending on the installation method, the following describes the proper usage:

  • If installed as a global tool ala npm:

    cyclonedx-npm --help
  • If installed as a global tool ala npx:
    — or —
    If installed as a development dependency of the current projects:

    npx @cyclonedx/cyclonedx-npm --help

The help page:

Usage: cyclonedx-npm [options] [--] [<package-manifest>]

Create CycloneDX Software Bill of Materials (SBOM) from Node.js NPM projects.

Arguments:
  <package-manifest>        Path to project's manifest file.
                            (default: "package.json" file in current working directory)

Options:
  --ignore-npm-errors       Whether to ignore errors of NPM.
                            This might be used, if "npm install" was run with "--force" or "--legacy-peer-deps".
                            (default: false)
  --package-lock-only       Whether to only use the lock file, ignoring "node_modules".
                            This means the output will be based only on the few details in and the tree described by the "npm-shrinkwrap.json" or "package-lock.json", rather than the contents of "node_modules" directory.
                            (default: false)
  --omit <type...>          Dependency types to omit from the installation tree.
                            (can be set multiple times)
                            (choices: "dev", "optional", "peer", default: "dev" if the NODE_ENV environment variable is set to "production", otherwise empty)
  --flatten-components      Whether to flatten the components.
                            This means the actual nesting of node packages is not represented in the SBOM result.
                            (default: false)
  --short-PURLs             Omit all qualifiers from PackageURLs.
                            This causes information loss in trade-off shorter PURLs, which might improve ingesting these strings.
                            (default: false)
  --spec-version <version>  Which version of CycloneDX spec to use.
                            (choices: "1.2", "1.3", "1.4", "1.5", "1.6", default: "1.4")
  --output-reproducible     Whether to go the extra mile and make the output reproducible.
                            This requires more resources, and might result in loss of time- and random-based-values.
                            (env: BOM_REPRODUCIBLE)
  --output-format <format>  Which output format to use.
                            (choices: "JSON", "XML", default: "JSON")
  --output-file <file>      Path to the output file.
                            Set to "-" to write to STDOUT.
                            (default: write to STDOUT)
  --validate                Validate resulting BOM before outputting.
                            Validation is skipped, if requirements not met. See the README.
  --no-validate             Disable validation of resulting BOM.
  --mc-type <type>          Type of the main component.
                            (choices: "application", "firmware", "library", default: "application")
  -v, --verbose             Increase the verbosity of messages.
                            Use multiple times to increase the verbosity even more.
  -V, --version             output the version number
  -h, --help                display help for command

Demo

For a demo of cyclonedx-npm see the demo project.

How it works

This tool utilizes npm to collect evidences of installed packages/modules. Read more in the dedicated docs.

The appropriate npm executable is detected automatically, yet can be overridden with the environment variable npm_execpath.
Autodetect: If called from npm/npx context, then the current npm executable is utilized, otherwise it is managed by SHELL and PATH.

This tool does not do artificial deduplication. Therefore, if a component is installed multiple times, it appears multiple times in the SBOM result. Read more on the topic in the dedicated docs "Component Deduplication".

Internals

This tool utilizes the CycloneDX library to generate the actual data structures, and serialize and validate them.
Validation requires transitive optional dependencies.

This tool does not expose any additional public API or classes - all code is intended to be internal and might change without any notice during version upgrades. However, the CLI is stable - you may call it programmatically like:

const { execFileSync } = require('child_process')
const { constants: { MAX_LENGTH: BUFFER_MAX_LENGTH } } = require('buffer')
const sbom = JSON.parse(execFileSync(process.execPath, [
    '../path/to/this/package/bin/cyclonedx-npm-cli.js',
    '--output-format', 'JSON',
    '--output-file', '-'
    // additional CLI args
], { stdio: ['ignore', 'pipe', 'ignore'], encoding: 'buffer', maxBuffer: BUFFER_MAX_LENGTH }))

Contributing

Feel free to open issues, bugreports or pull requests.
See the CONTRIBUTING file for details.

License

Permission to modify and redistribute is granted under the terms of the Apache 2.0 license.
See the LICENSE file for the full license.

cyclonedx-node-npm's People

Contributors

arthurlutz avatar codex- avatar dependabot[bot] avatar igord avatar jkowalleck avatar joonamo avatar xspielinbox avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

cyclonedx-node-npm's Issues

Including qualifiers in purl can cause too long purl's for tools

Hi,

As per https://github.com/package-url/purl-spec/blob/master/PURL-SPECIFICATION.rst
"Do not abuse qualifiers: it can be tempting to use many qualifier keys but their usage should be limited to the bare minimum for proper package identification to ensure that a purl stays compact and readable in most cases."

I suggest by default removing at least download_url or vcs_url qualifiers as they can create very long purl's that some tools such as Dependency-Track will not allow. Sonatype OSS index also seems to be quite picky with the length of the purl causing "500 Error" for example with secure-json-parse purl.

The purl generated with cyclonedx-npm for version 2.4.0 is:
pkg:npm/[email protected]?download_url=https://registry.npmjs.org/secure-json-parse/-/secure-json-parse-2.4.0.tgz&vcs_url=git+https://github.com/fastify/secure-json-parse.git whereas it probably should be pkg:npm/[email protected] which incidentally works with OSS Index query.

Another option would be to allow suppression of qualifiers overcoming this issue.

dev/optional component flag

in reporting you can find

  • "dev": true -- only omit with "dev"
  • "devOptional": true -- only omit with "dev" AND "optional"
  • "optional": true -- only omit with "optional"

this could be taken into the resulting SBOM.
either as properties, or as component.scope=optional -- https://cyclonedx.org/docs/1.4/json/#components_items_scope

since a property for development already exists, it this part is done.
but the "optional" part is lacking.


devOptionsl are both, optional and devDeps - at the same tme.
therefore they shxould et the property , and a sope et to "optional"

there should be an example created, that has the optionals shown.

cyclonedx-library in `bom.metadata.tools`

Is your feature request related to a problem? Please describe.

Current SBOM result contains the tool that was used to gather all data.

But it did not include the cyclonedx-library, which does data model transformations/serializations and in the end produces the SBOM result.
Therefore, it is not entirely clear, HOW a SBOM was created - in terms of reproducibility.

Describe the solution you'd like

have the cyclonedx-library in bom.metadata.tools[], with the library's name, version, and references to download/sources

Describe alternatives you've considered

none

Additional context

none

rework logging/output

current console output is done via writig to stdout in a custom console.
maybe better use a dedicated debug/log libary? like https://www.npmjs.com/package/debug

motivation:

  • have different log levels: debug,info,warn,critical,error
  • have an CLI switch to set the desired output level - like --verbose 3 or -vvv

acc crit:

  • all log/debug must go to STDERR - so stdout is free returned SBOM data

[IDEA] how it could work


utilize the NPM cli interface.
most calls have a --json option, to gather the needed information.

note

PNPM has similar capabilities. - but dfferent output formats....

reasons

a certain version of npm might have been used to setup/install the project.
therefore, this same version is installed and present on the system already.
utilizing this same version is preferred over an own/different version - cause it could lead to incompatibilities/conflicts or migration o lockfiles .... several issues even.

questions

  • are the repsonses constant? are there differences between npm versions? coudl add CI/CY runs to find out

alternatives.

  • reimplement npm logic -- not desired
  • use npm like a library -- do not want no bundle a npm, sonce the host system should have everything in place already.

discovery

check if a npm-shrinkwrap.json file or package-lock.json file exists


npm ls

output of npm ls --json -a -l --omit dev can be utilised ...
(see example output below)

interesting bits:

  • path holds the dir of the package
  • _id is a candidate for bom-ref
  • utilize extraneous -- how ?
  • integrity can be used as hash
  • how to detect bundled dependencies?

result

Click to expand command response
{
  "version": "1.0.0-alpha0",
  "name": "@cyclonedx/cyclonedx-npm",
  "description": "Create CycloneDX Software Bill of Materials (SBOM) from NPM projects. ",
  "keywords": [
    "CycloneDX",
    "SBOM",
    "BOM",
    "inventory",
    "bill-of-materials",
    "software-bill-of-materials",
    "component",
    "dependency",
    "package-url",
    "PURL",
    "spdx"
  ],
  "bugs": {
    "url": "https://github.com/CycloneDX/cyclonedx-node-npm/issues"
  },
  "repository": {
    "type": "git",
    "url": "git+https://github.com/CycloneDX/cyclonedx-node-npm.git"
  },
  "homepage": "https://github.com/CycloneDX/cyclonedx-node-npm#readme",
  "license": "Apache-2.0",
  "author": {
    "name": "Jan Kowalleck",
    "email": "[email protected]"
  },
  "contributors": [
    {
      "name": "Jan Kowalleck",
      "email": "[email protected]"
    }
  ],
  "devDependencies": {
    "@types/node": "^18.0.6",
    "eslint-plugin-jest": "^25.7.0",
    "jest": "^28.1.3",
    "jest-junit": "^14.0.0",
    "npm-run-all": "^4.1.5",
    "ts-standard": "^11.0.0",
    "typescript": "^4.7.4"
  },
  "type": "commonjs",
  "engines": {
    "node": ">=16.0.0"
  },
  "bin": {
    "cyclonedx-node-npm": "bin/cyclonedx-node-npm"
  },
  "main": "./dist/index.js",
  "exports": "./dist/index.js",
  "scripts": {
    "prepublish": "npm run build",
    "prepublishOnly": "npm run build",
    "lint": "tsc --noEmit",
    "prebuild": "node -r fs -e 'fs.rmSync(\"dist\",{recursive:true,force:true})'",
    "build": "tsc -b ./tsconfig.json",
    "cs-fix": "eslint --fix .",
    "setup-tests": "echo 'noting yet'",
    "test": "run-p --aggregate-output -lc test:*",
    "test:jest": "jest",
    "test:standard": "eslint ."
  },
  "jest-junit": {
    "suiteName": "jest tests",
    "outputDirectory": "reports/jest",
    "outputName": "tests.junit.xml"
  },
  "_id": "@cyclonedx/[email protected]",
  "extraneous": false,
  "path": "/home/flow/Documents/Coding/node/cyclonedx-node-npm",
  "_dependencies": {
    "@cyclonedx/cyclonedx-library": "^1.1.0",
    "commander": "^9.4.0",
    "xmlbuilder2": "^3.0.2"
  },
  "peerDependencies": {},
  "dependencies": {
    "@cyclonedx/cyclonedx-library": {
      "version": "1.1.0",
      "resolved": "https://registry.npmjs.org/@cyclonedx/cyclonedx-library/-/cyclonedx-library-1.1.0.tgz",
      "name": "@cyclonedx/cyclonedx-library",
      "integrity": "sha512-JJCQUwciSxUFKhnFZ/3EDqhIyw9iTJP27v5XS4YlSaAKdJY7Dmkhu/qnS9nms1R7r4ZefJVAcCdg7TDLft5wxQ==",
      "engines": {
        "node": ">=14.0.0"
      },
      "optionalDependencies": {
        "xmlbuilder2": "^3.0.2"
      },
      "_id": "@cyclonedx/[email protected]",
      "extraneous": false,
      "path": "/home/flow/Documents/Coding/node/cyclonedx-node-npm/node_modules/@cyclonedx/cyclonedx-library",
      "_dependencies": {
        "packageurl-js": ">=0.0.6 <0.0.8",
        "xmlbuilder2": "^3.0.2"
      },
      "devDependencies": {},
      "peerDependencies": {},
      "dependencies": {
        "packageurl-js": {
          "version": "0.0.7",
          "resolved": "https://registry.npmjs.org/packageurl-js/-/packageurl-js-0.0.7.tgz",
          "name": "packageurl-js",
          "integrity": "sha512-ucJzaXINlIgpqYEY6aNf0J2QGHEeMNwt9fiuhbsZsq3kZ5NRxNlnaEUe6ehB5fWjYRSp75j0/lJWpfTKmBc2oA==",
          "_id": "[email protected]",
          "extraneous": false,
          "path": "/home/flow/Documents/Coding/node/cyclonedx-node-npm/node_modules/packageurl-js",
          "_dependencies": {},
          "devDependencies": {},
          "peerDependencies": {}
        },
        "xmlbuilder2": {
          "version": "3.0.2",
          "name": "xmlbuilder2",
          "resolved": "https://registry.npmjs.org/xmlbuilder2/-/xmlbuilder2-3.0.2.tgz",
          "integrity": "sha512-h4MUawGY21CTdhV4xm3DG9dgsqyhDkZvVJBx88beqX8wJs3VgyGQgAn5VreHuae6unTQxh115aMK5InCVmOIKw==",
          "engines": {
            "node": ">=12.0"
          },
          "_id": "[email protected]",
          "extraneous": false,
          "path": "/home/flow/Documents/Coding/node/cyclonedx-node-npm/node_modules/xmlbuilder2",
          "_dependencies": {
            "@oozcitak/dom": "1.15.10",
            "@oozcitak/infra": "1.0.8",
            "@oozcitak/util": "8.3.8",
            "@types/node": "*",
            "js-yaml": "3.14.0"
          },
          "devDependencies": {},
          "peerDependencies": {}
        }
      }
    },
    "commander": {
      "version": "9.4.0",
      "resolved": "https://registry.npmjs.org/commander/-/commander-9.4.0.tgz",
      "name": "commander",
      "integrity": "sha512-sRPT+umqkz90UA8M1yqYfnHlZA7fF6nSphDtxeywPZ49ysjxDQybzk13CL+mXekDRG92skbcqCLVovuCusNmFw==",
      "engines": {
        "node": "^12.20.0 || >=14"
      },
      "_id": "[email protected]",
      "extraneous": false,
      "path": "/home/flow/Documents/Coding/node/cyclonedx-node-npm/node_modules/commander",
      "_dependencies": {},
      "devDependencies": {},
      "peerDependencies": {}
    },
    "xmlbuilder2": {
      "version": "3.0.2",
      "resolved": "https://registry.npmjs.org/xmlbuilder2/-/xmlbuilder2-3.0.2.tgz",
      "name": "xmlbuilder2",
      "integrity": "sha512-h4MUawGY21CTdhV4xm3DG9dgsqyhDkZvVJBx88beqX8wJs3VgyGQgAn5VreHuae6unTQxh115aMK5InCVmOIKw==",
      "engines": {
        "node": ">=12.0"
      },
      "_id": "[email protected]",
      "extraneous": false,
      "path": "/home/flow/Documents/Coding/node/cyclonedx-node-npm/node_modules/xmlbuilder2",
      "_dependencies": {
        "@oozcitak/dom": "1.15.10",
        "@oozcitak/infra": "1.0.8",
        "@oozcitak/util": "8.3.8",
        "@types/node": "*",
        "js-yaml": "3.14.0"
      },
      "devDependencies": {},
      "peerDependencies": {},
      "dependencies": {
        "@oozcitak/dom": {
          "version": "1.15.10",
          "resolved": "https://registry.npmjs.org/@oozcitak/dom/-/dom-1.15.10.tgz",
          "name": "@oozcitak/dom",
          "integrity": "sha512-0JT29/LaxVgRcGKvHmSrUTEvZ8BXvZhGl2LASRUgHqDTC1M5g1pLmVv56IYNyt3bG2CUjDkc67wnyZC14pbQrQ==",
          "engines": {
            "node": ">=8.0"
          },
          "_id": "@oozcitak/[email protected]",
          "extraneous": false,
          "path": "/home/flow/Documents/Coding/node/cyclonedx-node-npm/node_modules/@oozcitak/dom",
          "_dependencies": {
            "@oozcitak/infra": "1.0.8",
            "@oozcitak/url": "1.0.4",
            "@oozcitak/util": "8.3.8"
          },
          "devDependencies": {},
          "peerDependencies": {},
          "dependencies": {
            "@oozcitak/infra": {
              "version": "1.0.8",
              "name": "@oozcitak/infra",
              "resolved": "https://registry.npmjs.org/@oozcitak/infra/-/infra-1.0.8.tgz",
              "integrity": "sha512-JRAUc9VR6IGHOL7OGF+yrvs0LO8SlqGnPAMqyzOuFZPSZSXI7Xf2O9+awQPSMXgIWGtgUf/dA6Hs6X6ySEaWTg==",
              "engines": {
                "node": ">=6.0"
              },
              "_id": "@oozcitak/[email protected]",
              "extraneous": false,
              "path": "/home/flow/Documents/Coding/node/cyclonedx-node-npm/node_modules/@oozcitak/infra",
              "_dependencies": {
                "@oozcitak/util": "8.3.8"
              },
              "devDependencies": {},
              "peerDependencies": {}
            },
            "@oozcitak/url": {
              "version": "1.0.4",
              "resolved": "https://registry.npmjs.org/@oozcitak/url/-/url-1.0.4.tgz",
              "name": "@oozcitak/url",
              "integrity": "sha512-kDcD8y+y3FCSOvnBI6HJgl00viO/nGbQoCINmQ0h98OhnGITrWR3bOGfwYCthgcrV8AnTJz8MzslTQbC3SOAmw==",
              "engines": {
                "node": ">=8.0"
              },
              "_id": "@oozcitak/[email protected]",
              "extraneous": false,
              "path": "/home/flow/Documents/Coding/node/cyclonedx-node-npm/node_modules/@oozcitak/url",
              "_dependencies": {
                "@oozcitak/infra": "1.0.8",
                "@oozcitak/util": "8.3.8"
              },
              "devDependencies": {},
              "peerDependencies": {},
              "dependencies": {
                "@oozcitak/infra": {
                  "version": "1.0.8",
                  "name": "@oozcitak/infra",
                  "resolved": "https://registry.npmjs.org/@oozcitak/infra/-/infra-1.0.8.tgz",
                  "integrity": "sha512-JRAUc9VR6IGHOL7OGF+yrvs0LO8SlqGnPAMqyzOuFZPSZSXI7Xf2O9+awQPSMXgIWGtgUf/dA6Hs6X6ySEaWTg==",
                  "engines": {
                    "node": ">=6.0"
                  },
                  "_id": "@oozcitak/[email protected]",
                  "extraneous": false,
                  "path": "/home/flow/Documents/Coding/node/cyclonedx-node-npm/node_modules/@oozcitak/infra",
                  "_dependencies": {
                    "@oozcitak/util": "8.3.8"
                  },
                  "devDependencies": {},
                  "peerDependencies": {}
                },
                "@oozcitak/util": {
                  "version": "8.3.8",
                  "name": "@oozcitak/util",
                  "resolved": "https://registry.npmjs.org/@oozcitak/util/-/util-8.3.8.tgz",
                  "integrity": "sha512-T8TbSnGsxo6TDBJx/Sgv/BlVJL3tshxZP7Aq5R1mSnM5OcHY2dQaxLMu2+E8u3gN0MLOzdjurqN4ZRVuzQycOQ==",
                  "engines": {
                    "node": ">=8.0"
                  },
                  "_id": "@oozcitak/[email protected]",
                  "extraneous": false,
                  "path": "/home/flow/Documents/Coding/node/cyclonedx-node-npm/node_modules/@oozcitak/util",
                  "_dependencies": {},
                  "devDependencies": {},
                  "peerDependencies": {}
                }
              }
            },
            "@oozcitak/util": {
              "version": "8.3.8",
              "name": "@oozcitak/util",
              "resolved": "https://registry.npmjs.org/@oozcitak/util/-/util-8.3.8.tgz",
              "integrity": "sha512-T8TbSnGsxo6TDBJx/Sgv/BlVJL3tshxZP7Aq5R1mSnM5OcHY2dQaxLMu2+E8u3gN0MLOzdjurqN4ZRVuzQycOQ==",
              "engines": {
                "node": ">=8.0"
              },
              "_id": "@oozcitak/[email protected]",
              "extraneous": false,
              "path": "/home/flow/Documents/Coding/node/cyclonedx-node-npm/node_modules/@oozcitak/util",
              "_dependencies": {},
              "devDependencies": {},
              "peerDependencies": {}
            }
          }
        },
        "@oozcitak/infra": {
          "version": "1.0.8",
          "resolved": "https://registry.npmjs.org/@oozcitak/infra/-/infra-1.0.8.tgz",
          "name": "@oozcitak/infra",
          "integrity": "sha512-JRAUc9VR6IGHOL7OGF+yrvs0LO8SlqGnPAMqyzOuFZPSZSXI7Xf2O9+awQPSMXgIWGtgUf/dA6Hs6X6ySEaWTg==",
          "engines": {
            "node": ">=6.0"
          },
          "_id": "@oozcitak/[email protected]",
          "extraneous": false,
          "path": "/home/flow/Documents/Coding/node/cyclonedx-node-npm/node_modules/@oozcitak/infra",
          "_dependencies": {
            "@oozcitak/util": "8.3.8"
          },
          "devDependencies": {},
          "peerDependencies": {},
          "dependencies": {
            "@oozcitak/util": {
              "version": "8.3.8",
              "name": "@oozcitak/util",
              "resolved": "https://registry.npmjs.org/@oozcitak/util/-/util-8.3.8.tgz",
              "integrity": "sha512-T8TbSnGsxo6TDBJx/Sgv/BlVJL3tshxZP7Aq5R1mSnM5OcHY2dQaxLMu2+E8u3gN0MLOzdjurqN4ZRVuzQycOQ==",
              "engines": {
                "node": ">=8.0"
              },
              "_id": "@oozcitak/[email protected]",
              "extraneous": false,
              "path": "/home/flow/Documents/Coding/node/cyclonedx-node-npm/node_modules/@oozcitak/util",
              "_dependencies": {},
              "devDependencies": {},
              "peerDependencies": {}
            }
          }
        },
        "@oozcitak/util": {
          "version": "8.3.8",
          "resolved": "https://registry.npmjs.org/@oozcitak/util/-/util-8.3.8.tgz",
          "name": "@oozcitak/util",
          "integrity": "sha512-T8TbSnGsxo6TDBJx/Sgv/BlVJL3tshxZP7Aq5R1mSnM5OcHY2dQaxLMu2+E8u3gN0MLOzdjurqN4ZRVuzQycOQ==",
          "engines": {
            "node": ">=8.0"
          },
          "_id": "@oozcitak/[email protected]",
          "extraneous": false,
          "path": "/home/flow/Documents/Coding/node/cyclonedx-node-npm/node_modules/@oozcitak/util",
          "_dependencies": {},
          "devDependencies": {},
          "peerDependencies": {}
        },
        "@types/node": {
          "version": "18.0.6",
          "resolved": "https://registry.npmjs.org/@types/node/-/node-18.0.6.tgz",
          "name": "@types/node",
          "integrity": "sha512-/xUq6H2aQm261exT6iZTMifUySEt4GR5KX8eYyY+C4MSNPqSh9oNIP7tz2GLKTlFaiBbgZNxffoR3CVRG+cljw==",
          "_id": "@types/[email protected]",
          "extraneous": false,
          "path": "/home/flow/Documents/Coding/node/cyclonedx-node-npm/node_modules/@types/node",
          "_dependencies": {},
          "devDependencies": {},
          "peerDependencies": {}
        },
        "js-yaml": {
          "version": "3.14.0",
          "resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-3.14.0.tgz",
          "name": "js-yaml",
          "integrity": "sha512-/4IbIeHcD9VMHFqDR/gQ7EdZdLimOvW2DdcxFjdyyZ9NsbS+ccrXqVWDtab/lRl5AlUqmpBx8EhPaWR+OtY17A==",
          "bin": {
            "js-yaml": "bin/js-yaml.js"
          },
          "_id": "[email protected]",
          "extraneous": false,
          "path": "/home/flow/Documents/Coding/node/cyclonedx-node-npm/node_modules/xmlbuilder2/node_modules/js-yaml",
          "_dependencies": {
            "argparse": "^1.0.7",
            "esprima": "^4.0.0"
          },
          "devDependencies": {},
          "peerDependencies": {},
          "dependencies": {
            "argparse": {
              "version": "1.0.10",
              "resolved": "https://registry.npmjs.org/argparse/-/argparse-1.0.10.tgz",
              "name": "argparse",
              "integrity": "sha512-o5Roy6tNG4SL/FOkCAN6RzjiakZS25RLYFrcMttJqbdd8BWrnA+fGz57iN5Pb06pvBGvl5gQ0B48dJlslXvoTg==",
              "_id": "[email protected]",
              "extraneous": false,
              "path": "/home/flow/Documents/Coding/node/cyclonedx-node-npm/node_modules/argparse",
              "_dependencies": {
                "sprintf-js": "~1.0.2"
              },
              "devDependencies": {},
              "peerDependencies": {},
              "dependencies": {
                "sprintf-js": {
                  "version": "1.0.3",
                  "resolved": "https://registry.npmjs.org/sprintf-js/-/sprintf-js-1.0.3.tgz",
                  "name": "sprintf-js",
                  "integrity": "sha512-D9cPgkvLlV3t3IzL0D0YLvGA9Ahk4PcvVwUbN0dSGr1aP0Nrt4AEnTUbuGvquEC0mA64Gqt1fzirlRs5ibXx8g==",
                  "_id": "[email protected]",
                  "extraneous": false,
                  "path": "/home/flow/Documents/Coding/node/cyclonedx-node-npm/node_modules/sprintf-js",
                  "_dependencies": {},
                  "devDependencies": {},
                  "peerDependencies": {}
                }
              }
            },
            "esprima": {
              "version": "4.0.1",
              "resolved": "https://registry.npmjs.org/esprima/-/esprima-4.0.1.tgz",
              "name": "esprima",
              "integrity": "sha512-eGuFFw7Upda+g4p+QHvnW0RyTX/SVeJBDM/gCtMARO0cLuT2HcEKnTPvhjV6aGeqrCB/sbNop0Kszm0jsaWU4A==",
              "bin": {
                "esparse": "bin/esparse.js",
                "esvalidate": "bin/esvalidate.js"
              },
              "engines": {
                "node": ">=4"
              },
              "_id": "[email protected]",
              "extraneous": false,
              "path": "/home/flow/Documents/Coding/node/cyclonedx-node-npm/node_modules/esprima",
              "_dependencies": {},
              "devDependencies": {},
              "peerDependencies": {}
            }
          }
        }
      }
    }
  }
}

[BUG] When '--legacy-peer-deps' is used to install dependencies, and generate with '--ignore-npm-errors --omit peer' sbom cannot be generated.

Describe the bug

When --legacy-peer-deps is used to install dependencies, sbom cannot be generated.
Lilke:npm i @ag-grid-community/[email protected] --legacy-peer-deps
I think when I use cyclonedx-node-npm with --omit peer, the scripts should not look for peer dependencies.

To Reproduce

Install dependency:
npm i @ag-grid-community/[email protected] --legacy-peer-deps
Generate sbom:
cyclonedx-npm --ignore-npm-errors --package-lock-only --omit dev --omit peer --short-PURLs --spec-version 1.3 --mc-type library --output-file ./bom.json

Expected behavior

The SBOM is generated successfully.

Screenshots or output-paste

Some error logs:
image

Environment

  • @cyclonedx/cyclonedx-npm version: 1.12.0
  • NPM version: 8.7.0
  • Node version: 16.15.0
  • OS: windows 10

support npm v9

Is your feature request related to a problem? Please describe.

currently the project supports NPM v6 - v8.
There is no official support for NPM v9 with this project, yet.

Describe the solution you'd like

Add official support for NPM v9

Describe alternatives you've considered

no alternatives

Additional context

NPM9 was released, which should be supported
see nodejs/Release#778 (comment)
see https://www.npmjs.com/package/npm/v/9.0.0
see https://github.com/npm/cli/releases/tag/v9.0.0

"--omit dev" created a lot of "DummyComponent.InterferedDependency..." entries in the BOM

When I'm using the "--omit dev" parameter then I got a lot of entries with the prefix "DummyComponent.InterferedDependency."

I'm using "@cyclonedx/cyclonedx-npm" version 1.2.0 as dev dependency. And it seems this is also in the BOM which I didn't expected.

    {
      "ref": "DummyComponent.InterferedDependency.@cyclonedx/cyclonedx-npm",
      "dependsOn": [
        "DummyComponent.InterferedDependency.@cyclonedx/cyclonedx-library",
        "DummyComponent.InterferedDependency.@cyclonedx/cyclonedx-npm|DummyComponent.InterferedDependency.commander",
        "DummyComponent.InterferedDependency.xmlbuilder2"
      ]
    },
    {
      "ref": "DummyComponent.InterferedDependency.@cyclonedx/cyclonedx-npm|DummyComponent.InterferedDependency.commander"
    },
    {
      "ref": "DummyComponent.InterferedDependency.@cyclonedx/cyclonedx-library",
      "dependsOn": [
        "DummyComponent.InterferedDependency.packageurl-js",
        "DummyComponent.InterferedDependency.xmlbuilder2"
      ]
    },
...

To Reproduce

Snippet from my package.json with to internal dependencies removed.

"devDependencies": {
   "@cyclonedx/cyclonedx-npm": "^1.2.0",
   "css-loader": "^6.7.1",
   "cypress": "^10.10.0",
   "cypress-intellij-reporter": "^0.0.7",
   "eslint": "8.26.0",
   "eslint-plugin-cypress": "^2.12.1",
   "eslint-plugin-mocha": "^10.1.0",
   "eslint-plugin-node": "^11.1.0",
   "eslint-plugin-no-only-tests": "^3.1.0",
   "file-loader": "^6.2.0",
   "style-loader": "^3.3.1",
   "webpack": "^5.74.0",
   "webpack-cli": "^4.10.0",
   "webpack-merge": "^5.8.0"
 },
 "dependencies": {
   "@amcharts/amcharts4": "^4.10.29",
   "ace-builds": "^1.12.3",
   "ace-diff": "^3.0.3",
   "ajv": "^8.11.0",
   "ajv-formats": "^2.1.1",
   "dompurify": "^2.4.0",
   "froala-editor": "^4.0.15",
   "json-source-map": "^0.6.1",
   "vkbeautify": "^0.99.3"
 },

Expected behavior

No dummy entries and no dev dependencies.

Environment

npm - 8.5.1
node - v17.6.0

macOS 12.6

support node >= 14

currently node >= 16 is supported.

node 14 is still in active maintenance and therefore a potential runtime target.
see: https://nodejs.org/en/about/releases/

goal: support node >= 14

requirements

  • have CI/CT with node 14 passing
  • have demo/integration with node 14 passing

"--omit dev" does not omit all devDependencies

Please try this package memdown.

after I ran cyclonedx-npm --omit dev --output-file newbom.json, I got an extra devDependency(level-concat-iterator)
that is mixed with the dependencies.

{
      "ref": "[email protected]",
      "dependsOn": [
        "[email protected]",
        "DummyComponent.InterferedDependency.airtap-playwright",
        "DummyComponent.InterferedDependency.airtap-sauce",
        "DummyComponent.InterferedDependency.airtap",
        "DummyComponent.InterferedDependency.dependency-check",
        "DummyComponent.InterferedDependency.faucet",
        "[email protected]",
        "DummyComponent.InterferedDependency.hallmark",
        "[email protected]",
        "[email protected]",
        "[email protected]",
        "DummyComponent.InterferedDependency.nyc",
        "DummyComponent.InterferedDependency.standard",
        "DummyComponent.InterferedDependency.tape"
      ]
    }

Can not run without package-lock.json

Please try the package meow, which does not offer a package-lock.json file.

I got an Error after I globally installed @cyclonedx/cyclonedx-npm and ran cyclonedx-npm --omit dev --output-file newbom.json.

Error: missing package lock file or npm shrinkwrap file

feat: CLI switch to omit PURL qualifiers

some analyzers do not like long PURLs - for unknown reasons.
see #90
see #224
see #280

Therefore, a CLI switch to drop all PURL qualifiers should be introduced.

CLI switch: --short-PURLs
description: Omit all qualifiers from PackageURLs. This causes information loss in trade of shorter URLS which might improve digesting these strings

[BUG] Wrong tool name displayed in help

Describe the bug

When running cyclonedx-npm --help the usage is displayed as Usage: cyclonedx-npm-cli [options] [--] [<package-manifest>]

To Reproduce

  • Install cycloendx-npm: npm install --global @cyclonedx/cyclonedx-npm
  • Run cyclonedx-npm --help

Expected behavior

It should read Usage: cyclonedx-npm [options] [--] [<package-manifest>]

Screenshots or output-paste

image

Environment

  • @cyclonedx/cyclonedx-npm version: 1.9.1
  • NPM version: 8.13.1
  • Node version: 19.3.0
  • OS: Windows 11

Additional context

I know it's nitpicky, but i like to copy-past from the help page, I tried to find it in this repo but couldn't, so I am assuming it's in the meta-package?

chore: have check for license-text file-header

in the top of each file, there needs to be a license header, a short-version of the license that applies to it.
this license is currently set by hand and there is no test, if it was added properly.

acc crit:

  • the license must be a block-comment with exclamation mark (/*! ... */),
    so that compilers/builders/packers/transpilers and other tools detect it as a non-strippable doc-block and keep it in the code.
  • the checker must run in CI/CT
  • optional: there should be a fixer, that applies this license text when missing

FEAT: component de-duplication

based on the outcome of #304
meaning: this issue is blocked by #304.


  • have a CI switch to enable the extra processing
  • have implementation according to the outcome of #304
  • have tests according to the outcome of #304

The cyclonedx/cyclonedx-node container is completely unrelated to this repo

There are no docs for the container, also provided by cyclonedx, it does not use this code, the arguments are different and there does not seem to be a repository in CycloneDX which has the actual code form the docker container
https://hub.docker.com/r/cyclonedx/cyclonedx-node

Additionally, as far as I can tell, there is no way to specify the user of package-lock.json, the container entrypoint will only ever use node_modules which requires npm install to be run every time.

feat: generate SBOM from "global" scope

Is your feature request related to a problem? Please describe.

I try to package a nodejs + NPM installation that ships a few global tools like yarn/dart-saas as part of a larger software installer.

So basically i do the following (with some in-house BSD/Mac ports style system):

  1. Unzip a nodejs distro, e.g. node-v18.12.1-win-x64.zip and rename the folder to 'img'

  2. Provide some .tar.gz of sass & yarn, e.g. sass-1.57.0.tar.gz and yarn-v1.22.10.tar.gz in the folder 'base'

  3. Run a global npm install for the packages, resolving dependencies from the npm registry

     img\npm install --cache base/ --global [email protected]
     img\npm install --cache base/ --global [email protected]
    
  4. Try to get an SBOM for the installed global packages, either as a SBOM with multiple application components included or one SBOM per application.

Describe the solution you'd like

I would like to have an option to use this package to get SBOMs for the globally installed packages in my node_modules folder.

Describe alternatives you've considered

I used the cyclonedx-bom package before, just pointing it at the created img/node_modules to collect the actually installed global tools with dependencies. That worked mostly fine and created a useable SBOM.

With this package this does not work, as the code complains about a missing package-lock.json when i run it for the individual apps and has no option at all to just consume the img/node_modules folder.

C:\code\repro\img>.\cyclonedx-npm --output-file bom.json node_modules\yarn\package.json
DEBUG | options: {"ignoreNpmErrors":false,"packageLockOnly":false,"omit":[],"flattenComponents":false,"shortPURLs":false,"specVersion":"1.4","outputFormat":"JSON","outputFile":"bom.json","mcType":"application"}
DEBUG | packageFile: C:\code\repro\img\node_modules\yarn\package.json
INFO  | projectDir: C:\code\repro\img\node_modules\yarn
LOG   | No evidence: no package lock file nor npm shrinkwrap file
LOG   | No evidence: no node_modules dir
INFO  | ? Did you forget to run `npm install` on your project accordingly ?

Error: missing evidence

There seems to be no way to tell npm install --global to create any form of lock file. Or i could not find it.

FEAT: component's property for install path

NodeJS's module system is file-system based. It works regardless of package dependencies,
When code in module "foo" tries to use/require/access code from a different module "bar", then node will look in "foo";s own/direct "node_module" folder (depth 1). if it did not find any "bar" there, then node traverses all folders upwards and does the same lookup there, until it finds any "bar".

This file-based loading behavior happens regardless of components' "dependency graph"
To make this loader-environment visible in an SBOM, a property should reflect a modules install path.

It could even happen that an SBOM's component is installed in multiple places.
Therefore, a property should indicate all install locations. This meansthe property could appear multiple times with different values
This way it is possible to answer the question "is this component 'A' actually using component 'B1' or does it load 'B2' instead?"

An alternative would be to have the actual file tree represented as sub-components.
But component flattening or component de-duplication might change these prepared structures, which could lead to information loss.
So a property is preferred.


property value should be a representation of the install-path relative to the root directory of the project under analysis.
no absolute paths, so private data (internal file path structures or mountpoints) are not published.

property value should be posix-like path, regardless of the actual runtime nor input nor detected path-patterns.

to represent the same dir as the root dir, an empty string is expected.


remove requirement for package lock file

caused by #232

Is your feature request related to a problem? Please describe.

current implementation tells users to have a package lock file.
internally it is not really needed. instead, npm ls is utilized.

and sometimes it is just not there - see https://docs.npmjs.com/cli/v9/using-npm/config#package-lock

Describe the solution you'd like

remove the demand for a package lock file.

Additional context

see npm config for the topic: https://docs.npmjs.com/cli/v9/using-npm/config#package-lock

[IDEA] suported env

have CI tests for the following envs (matrix?)

npm

  • * npm 6.(0-x)
  • * npm 7.(0-x)
  • * npm 8.(0-x)

node

  • * node 14.(0-x) -- #70
  • * node 16.(0-x)
  • * node 18.(0-x)

os

not required for the matrix, but atleast latest version of npm & node on each of the OS

  • * Windows
  • * Linux
  • * MacOS

to be ontinued

[IDEA] feat: GitHub-Action

A pull request #472 was opened out of nowhere,
that suggested having a GitHub-Action that installs and runs this very tool.

This is an arguable topic, so let's discuss:
Why is such a GH-Action needed?
What are the actual use cases? How would you use it?
What are the pros? What are the cons?
What is the benefit of a GH-Action instead of running the installation process and the tool via a GitHub-Workflow yourself?
... and much more .

have `serialNumber` populated

Is your feature request related to a problem? Please describe.

I want to create a VEX based on the SBOM result from this tool.
Therefore, i need the SBOM result having a serial number, so i can link to it.
See https://cyclonedx.org/capabilities/vex/.

Describe the solution you'd like

SBOM results have a random serial number, unless the reproducible output switch is on.

Describe alternatives you've considered

Additional context

Allow configurable NPM registry URL mirrors to be specified

Our development environment is configured by company security policies in such a way that the public NPM registry (https://registry.npmjs.org) is not directly reachable (firewalled off). Instead, a third-party product (Sonatype Repository OSS) is used to bring in, and cache on-premises, NPM packages used by internally-developed applications -- enforcing one-way package flow to consume NPM registry packages, while blocking publishing.

Therefore, every NPM package used by internal apps is downloaded from an on-premises URL. For instance, the package @babel/helper-split-export-declaration from the NPM registry is accessible internally at https://on-premises.url/repository/npm/@babel/helper-split-export-declaration/-/helper-split-export-declaration-7.18.6.tgz

Since cyclonedx-node v4.0 is now deprecated, we have switched to cyclonedx-node-npm instead, and we started seeing failures when uploading SBOMs to Dependency-Track, specifically this:

org.datanucleus.exceptions.NucleusUserException: Attempt to store value "pkg:npm/%40babel/[email protected]?download_url=https%3A%2F%2Fon-premises.url%2Frepository%2Fnpm%2F%40babel%2Fhelper-split-export-declaration%2F-%2Fhelper-split-export-declaration-7.18.6.tgz#packages/babel-helper-split-export-declaration" in column "PURL" that has maximum length of 255. Please correct your data!
	at org.datanucleus.store.rdbms.mapping.column.CharColumnMapping.setString(CharColumnMapping.java:253)
	at org.datanucleus.store.rdbms.mapping.java.SingleFieldMapping.setString(SingleFieldMapping.java:202)
	at org.datanucleus.store.rdbms.fieldmanager.ParameterSetter.storeStringField(ParameterSetter.java:158)
	at org.datanucleus.state.StateManagerImpl.providedStringField(StateManagerImpl.java:1903)
	at org.dependencytrack.model.Component.dnProvideField(Component.java)
	at org.dependencytrack.model.Component.dnProvideFields(Component.java)
	at org.datanucleus.state.StateManagerImpl.provideFields(StateManagerImpl.java:2559)
	at org.datanucleus.store.rdbms.request.UpdateRequest.execute(UpdateRequest.java:401)
	at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.updateObjectInTable(RDBMSPersistenceHandler.java:447)
	at org.datanucleus.store.rdbms.RDBMSPersistenceHandler.updateObject(RDBMSPersistenceHandler.java:421)
	at org.datanucleus.state.StateManagerImpl.flush(StateManagerImpl.java:5890)
	at org.datanucleus.flush.FlushOrdered.execute(FlushOrdered.java:96)
	at org.datanucleus.ExecutionContextImpl.flushInternal(ExecutionContextImpl.java:3956)
	at org.datanucleus.ExecutionContextImpl.processNontransactionalAtomicChanges(ExecutionContextImpl.java:1411)
	at org.datanucleus.ExecutionContextImpl.processNontransactionalUpdate(ExecutionContextImpl.java:1372)
	at org.datanucleus.state.StateManagerImpl.setStringField(StateManagerImpl.java:2990)
	at org.dependencytrack.model.Component.dnSetpurl(Component.java)
	at org.dependencytrack.model.Component.setPurl(Component.java:540)
	at org.dependencytrack.parser.cyclonedx.util.ModelConverter.convert(ModelConverter.java:113)
	at org.dependencytrack.parser.cyclonedx.util.ModelConverter.convertComponents(ModelConverter.java:82)
	at org.dependencytrack.tasks.BomUploadProcessingTask.inform(BomUploadProcessingTask.java:106)
	at alpine.event.framework.BaseEventService.lambda$publish$0(BaseEventService.java:101)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)

This seems related to #90, CycloneDX/cyclonedx-javascript-library#204 and CycloneDX/cyclonedx-javascript-library#207, insofar as the PURL download_url qualifier is redundant in our environment, yet due to its size still causes issues with Dependency-Track.

To solve #90, it seems that https://registry.npmjs.org was special-cased here:

https://github.com/CycloneDX/cyclonedx-javascript-library/blob/e94d330f435cf1cd20d1c0208b9616b4bc140a52/src/factories/fromNodePackageJson.node.ts#L98

Could the value of the NPM registry mirror URL be made configurable via command-line arguments instead of hard-coded?

This would hopefully allow to pass a trusted mirror URL to cyclonedx-npm at runtime, reducing the size and verbosity of PURLs back to what cyclonedx-node used to provide earlier.

Here is what the expected behavior would be with NPM configured to use https://on-premises.url as the registry URL.

cyclonedx-npm sample-project would generate the PURL pkg:npm/%40babel/[email protected]?download_url=https%3A%2F%2Fon-premises.url%2Frepository%2Fnpm%2F%40babel%2Fhelper-split-export-declaration%2F-%2Fhelper-split-export-declaration-7.18.6.tgz#packages/babel-helper-split-export-declaration, as it currently does.

However, cyclonedx-npm --registry-url https://on-premises.url sample-project would match and make the download_url qualifier redundant, resulting in the bare pkg:npm/%40babel/[email protected] PURL.

FEAT: Option to add license text to BOM output

Is your feature request related to a problem? Please describe.

For legal documentation, we need the original text of the licenses of components.

Describe the solution you'd like

An option to enable integration of the license-text in the BOM file, like the old @cyclonedx/bom package had, would be great to have again here.


read https://cyclonedx.org/news/cyclonedx-v1.3-released/#copyright-and-license-evidence

Acceptance criteria

  • the feature to add license texts should be enabled by a CLI switch called --gather-license-evidence (name to be discussed)
  • the feature is disabled per default
  • only if the feature is enabled:
    • for all components, meta-components, root-components and nested components:
      regardless of SPDX license ID, SPDX license expression or named license, the deteced license texts should be added, each as an evidence
      Examples:
      {
        //...
        "evidence": { 
          "licenses": [
            {"id":"Apache-2.0", "text": {
              "contentType": "text/plain",
              "encoding": "base64",
              // base64 of content of file `LICENSE`
              "content": "bG9yZW0gaXBzdW0="
            }}
            {"name":"file: NOTICE", "text": {
              "contentType": "text/plain",
              "encoding": "base64",
              // base46 of content of file `NOTICE`
              "content": "bG9yZW0gaXBzdW0="
            }}
          ]
        },
        // ...
      }
    • if a license text is detected with the package, it would be added to Component's @.evicence.licenses
      • @.name would be 'License of : '
      • @.text would hold the test
        • the content type is to be derived from file extension
        • the content SHOULD be base64 encoded
    • license files patterns are:
      • LICEN[CS]E*
      • NOTICE* -- addendum for Apache-2.0 and others
    • if no license text is shipped with a package, no license test is added as a evidence.
      Nope, no license template is derived from package's declared SPDX license id.
      Reason: license templates (like BSD clause 3) are designed to be modified (unlike others, like Apache2, which is not a template but a complete text)

[Question] BOM creation blocked due to `npm ls` invalid package error - dependency conflict

Describe the bug

We are from a DevOps team who support BOM creation for users with npm project.
We have been using @cyclonedx/bom version 3.10.6 and are trying to move a lot of users(2000+) to cyclonedx-node-npm.

When we try to generate BOM using cyclonedx-node-npm, it internally runs npm ls with --all and it seems to be throwing invalid package error for lot of users for transitive dependencies deep down at some level.
One example would be the flowing output for one of our users who face issue for package: commander
Output for npm list commandershows following:
Screenshot 2023-06-12 at 11 58 09

This was run on a node-18 docker image ((Node.js 18.16.0, npm 9.5.1)

Other users have reported similar issues due to BOM creation being blocked due to deep down dependency conflict.
From user's perspective , it is a dependency among packages which they seem to be not directly dependent on .
[For eg., here user is only dependent on packageA and @unleash/proxy

We did see "--ignore-npm-errors" flag , but this would mean some genuine errors are also ignored.

What would you advise in this case? Would there be an option from cyclonedx-node-npm which we can use to help users out who are facing transitive dependency conflict.

Environment

  • @cyclonedx/cyclonedx-npm version: 1.11
  • docker image : node:18 (Node.js 18.16.0, npm 9.5.1)

Additional context

Add any other context about the problem here.

[BUG] buildID is cut from version

Describe the bug

If a projects version contains also build metadata (e.g "version": "1.0.0-123+456" following the semver definition) the build metadata part +456 is omitted in the purl that is put in the generated sbom. I would expect the build metadata to also be part of the version in the purl.

To Reproduce

Create a package.json file like this one:

{
  "name": "sbomtest",
  "version": "1.0.0-123+456",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "@cyclonedx/cyclonedx-npm": "^1.9.0"
  }
}

run npm pkg get version which will return "1.0.0-123+456" as expected.

Then generate the sbom npx cyclonedx-npm --output-file sbom.json
This will generate a sbom like this (note: I removed irrelevant parts)

{
  "$schema": "http://cyclonedx.org/schema/bom-1.4.schema.json",
  "bomFormat": "CycloneDX",
  "specVersion": "1.4",
  "version": 1,
  "serialNumber": "urn:uuid:b368281e-6ad1-4f66-87ef-9e3cf5ed8cc2",
  "metadata": {
    "timestamp": "2023-03-10T09:24:29.674Z",
    "component": {
      "type": "application",
      "name": "sbomtest",
      "version": "1.0.0-123",
      "bom-ref": "[email protected]+456",
      "purl": "pkg:npm/[email protected]"
    }
  }
}

As you can see for the bom-ref the version part is as expected but the purl property misses the build metadata.

Expected behavior

The purl in the sbom should be "purl": "pkg:npm/[email protected]+456"

Environment

  • @cyclonedx/cyclonedx-npm version: 1.9.0
  • NPM version: 8.19.1
  • Node version: v18.9.0
  • OS: macOS Ventura

publication checklist

  • finish all the tasks in the TODO.md file and remove it
  • run npm package and see that there is noting unnecessary in there
  • dist maps excluded from shipping
  • document global install instruction
  • document supported versions of npm
  • document supported verisons of nodejs (widen if possible)
  • merge "1.0-dev" into "main""
  • replace all the "1.0-dev" to "main""
  • write the HISTORY

to be continued

ELSPROBLEMS error when running

This is more of an FYI and how to resolve this error than any action required by the team, but posting here so others may benefit from it.

Running this for the first time may lead you to an ELSPROBLEMS error like this:

npx cyclonedx-npm
DEBUG | options: {"packageLockOnly":false,"omit":[],"flattenComponents":false,"specVersion":"1.4","outputFormat":"JSON","outputFile":"-","mcType":"application"}
DEBUG | packageFile: /Users/alexmiller/dev/app/package.json
DEBUG | projectDir: /Users/alexmiller/dev/app
DEBUG | lockFile: /Users/alexmiller/dev/app/package-lock.json
INFO  | gather dependency tree ...
DEBUG | npm-ls: run npm with ["ls","--json","--all","--long"] in /Users/alexmiller/dev/app
WARN  | npm-ls: STDERR
  npm ERR! code ELSPROBLEMS
  npm ERR! invalid: [email protected] /Users/alexmiller/dev/app/node_modules/aria-query
  npm ERR! invalid: [email protected] /Users/alexmiller/dev/app/node_modules/rxjs
  npm ERR! invalid: [email protected] /Users/alexmiller/dev/app/node_modules/pvutils
  {
    "error": {
      "code": "ELSPROBLEMS",
      "summary": "invalid: [email protected] /Users/alexmiller/dev/app/node_modules/aria-query\ninvalid: [email protected] /Users/alexmiller/dev/app/node_modules/rxjs\ninvalid: [email protected] /Users/alexmiller/dev/app/node_modules/pvutils",
      "detail": ""
    }
  }
  
  npm ERR! A complete log of this run can be found in:
  npm ERR!     /Users/alexmiller/.npm/_logs/2022-08-31T03_25_49_649Z-debug-0.log
  
ERROR | npm-ls: errors
  {}
/Users/alexmiller/dev/app/node_modules/@cyclonedx/cyclonedx-npm/dist/builders.js:80
            throw new Error(`npm-ls exited with errors: ${error.errno ?? '???'} ${error.code ?? npmLsReturns.status ?? 'noCode'} ${error.signal ?? npmLsReturns.signal ?? 'noSignal'}`);
            ^

Error: npm-ls exited with errors: ??? 1 noSignal
    at BomBuilder.fetchNpmLs (/Users/alexmiller/dev/app/node_modules/@cyclonedx/cyclonedx-npm/dist/builders.js:80:19)
    at BomBuilder.buildFromLockFile (/Users/alexmiller/dev/app/node_modules/@cyclonedx/cyclonedx-npm/dist/builders.js:43:41)
    at Object.run (/Users/alexmiller/dev/app/node_modules/@cyclonedx/cyclonedx-npm/dist/cli.js:97:19)
    at Object.<anonymous> (/Users/alexmiller/dev/app/node_modules/@cyclonedx/cyclonedx-npm/bin/cyclonedx-npm-cli.js:2:27)
    at Module._compile (node:internal/modules/cjs/loader:1103:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1157:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12)
    at node:internal/main/run_main_module:17:47

We can see it more clearly simply using the > npm ls --json -a -l command:

npm ERR! code ELSPROBLEMS
npm ERR! invalid: [email protected] /Users/alexmiller/dev/app/node_modules/aria-query
npm ERR! invalid: [email protected] /Users/alexmiller/dev/app/node_modules/rxjs
npm ERR! invalid: [email protected] /Users/alexmiller/dev/app/node_modules/pvutils
{
  "error": {
    "code": "ELSPROBLEMS",
    "summary": "invalid: [email protected] /Users/alexmiller/dev/app/node_modules/aria-query\ninvalid: [email protected] /Users/alexmiller/dev/app/node_modules/rxjs\ninvalid: [email protected] /Users/alexmiller/dev/app/node_modules/pvutils",
    "detail": ""
  }
}

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/alexmiller/.npm/_logs/2022-08-31T03_27_52_943Z-debug-0.log

These can be self-healed by npm by simply installing the packages effected then uninstalling them (returning them to the locked state of the dependency):

> npm i -D [email protected] && npm uninstall aria-query

Now when you run npm ls --json -a -l again you may meet another package that needs the same treatment, after you've resolved these you should be able to use cyclonedx-npm normally 🚀

I suppose a followup question to this is: Since we know how to resolve these simple cases, should this functionality be tied into cyclonedx-npm?

fix local install

this package ships a shrinkwrap. this is totally fine as long as downstream users install via npm i -g
if they install locally via npm i -D , they will install this package and all its dev-dependencies - which is unintended.

therefore, the shipped shrinkwrap must be stripped from dev dependencies.

it is no option to strip the feature of local installations.
because tis enables proper versioning of the tool and its dependencies via renovate, dependabot and other dependency watchers ...

[BUG] purl inconsistencies causing Dependency Track exceptions

Describe the bug

Dependency track has been raising some exceptions for some of our internal npm dependencies hosted on our github registry:

Caused by: org.datanucleus.exceptions.NucleusUserException: Attempt to store value "pkg:npm/%40company/[email protected]?checksum=sha-512%3A600c2c0e6b3521d258a58c8dc56fe8f9f3f87a0292c819277b1c2d5a4175d853568e52aef2d181074ee8d7f371f3a83176fb703689e53d2d56f92dce4163abdd&download_url=https%3A%2F%2Fnpm.pkg.github.com%2Fdownload%2F%40company%2Fpackage%2F0.0.10%2F219749b28431d4a55d9dd2ea25ad968051fe3474" in column ""PURL"" that has maximum length of 255. Please correct your data!

This appears to be coming from inconsistencies in how purl's are populated between cdxgen and @cyclonedx-npm.

purl for this package when using npx cdxgen -t nodejs -o bom.xml:

<purl>pkg:npm/company/[email protected]</purl>

purl for this package when using npx @cyclonedx/cyclonedx-npm --mc-type application --output-file bom.json:

"purl": "pkg:npm/%company/[email protected]?download_url=https%3A//npm.pkg.github.com/download/%40company/package/0.1.0/9c864f64aae09203dc92ac7d60716fcf8661fa57",

Expected behavior

I would expect the purl's to align, and likely resolve the issue we're seeing when attempting to ingest sbom's generated with @cyclonedx/cyclonedx-npm

Screenshots or output-paste

If applicable, add screenshots or past the output to help explain your problem.

Environment

  • @cyclonedx/cyclonedx-npm version: 1.11
  • NPM version: 8.19.3
  • Node version: 16.19.0
  • OS: macos

Though with our CICD we use the latest version available of node 16, and of cyclonedx-npm, running on ubuntu.

SBOM is different on package.json with same command on different systems

Describe the bug

Hello,

we ran into an issue, where the output of the generated SBOM is different if executed in the Jenkins environment compared to the results if I execute the command locally.
We use the following command to generate SBOM
npx --yes -d @cyclonedx/cyclonedx-npm@^1 --output-file "..\..\..\bom\components\portal_frontend.json" --omit dev

The major difference is that in the Jenkins environment an additional hashes property is generated for each entry. That itself isn´t an issue for us at all in general but there is one exception:
We also have included the fontawesome-pro package, which doesn´t come from npmjs but the FA npm registry. In this case the hashes property is not generated but the PURL is extended.
If I execute the mentioned command locally the tool generates
"purl": "pkg:npm/%40fortawesome/[email protected]?download_url=https://npm.fontawesome.com/@fortawesome/fontawesome-pro/-/6.2.1/fontawesome-pro-6.2.1.tgz"

In the Jenkins environment it instead generates
"purl": "pkg:npm/%40fortawesome/[email protected]?checksum=sha-512:74793b8a209fe4c0a6a14be6ad8cdf37f23781e6e9829035a2a94e7df80e4e19ec680478929690e58313764746f7d39fd619cdd01a24e91f87136f335dbdd23a&download_url=https://npm.fontawesome.com/@fortawesome/fontawesome-pro/-/6.2.1/fontawesome-pro-6.2.1.tgz"

This leads to the issue, that our Dependency Track rejects the SBOM file cause that specific purl entry exceeds the max length
image

I don´t know if that limitation comes from the specification or is an internal limitation of Dependency Track.
I was able to use the short purl options as a first workaround, however my main questions are:
Why is the output different from when I execute the command locally and is there a reason, that for the external registry, the hash is written into the PURL which isn´t the case for packages coming from npmjs?

Expected behavior

Identical output SBOM in local and jenkins environment

Screenshots or output-paste

If applicable, add screenshots or past the output to help explain your problem.

Environment

Local

  • @cyclonedx/cyclonedx-npm version: ^1
  • NPM version: 8.19.2
  • Node version: 18.12.1
  • OS: Windows 10

Jenkins

  • @cyclonedx/cyclonedx-npm version: ^1
  • NPM version: 8.8.0
  • Node version: 18.1.0
  • OS: Windows

Additional context

Some screenshot if I diff the local (left) and jenkins (right) output:
image

Large discrepancies in number of components between "cyclonedx/bom v3.10.6" and "cyclonedx/cyclonedx-npm v1.6.0"

Hello,

Environment information:

  • OS: 20.04.2 LTS (Focal Fossa)
  • npm: v6.14.14
  • node: v14.17.5

This is more of a "general knowledge question", as I am trying to conclude to which cyclonedx module/version generates the most representative results. I am sorry in advance if this is not the right place to be asking it.

My use case is that I need to generate bom.xml files for some angular projects and then send them to Dependency Track platform for analysis.

Up until version 3 of @cyclonedx/bom, I was generating my bom.xml files as follows:

    npm install -g @cyclonedx/[email protected]
    cyclonedx-bom -o bom.xml

The bom.xml that was generated in the way above, outputs 430 Components in Dependency Track.


However I understand that major version 4 release of @cyclonedx/bom introduced breaking changes.
So, I now try to utilize @cyclonedx/cyclonedx-npm, and I generate my bom.xml files like this:

   npm install --global @cyclonedx/cyclonedx-npm
   cyclonedx-npm --omit dev --ignore-npm-errors --output-format xml --output-file bom.xml

The bom.xml file that was generated in the way above, outputs 170 Components in Dependency Track.
(Without the --omit dev flag, I get >700 components)

Both tests were obviously ran for the exact same project.


So my question is, why is there such a large discrepancy in the final number of Components, for the same project, between @cyclonedx/[email protected] and @cyclonedx/[email protected] ?

Thank you in advance

feat: generate hashes for other than sha-512

Describe the bug

This tool can only generate hashes whose integrity is encrypted using sha-512.

To Reproduce

When some dependency is encrypted by sha-1,The SBOM generated by the tool does not contain hashes.

Expected behavior

When some dependency is encrypted by sha-1 or other encryption algorithm,the SBOM generated by the tool can have the correct hashes.

Screenshots or output-paste

236148684-a1e3e7f3-2f04-4c96-8fa4-2c5781ed8d07

Environment

  • @cyclonedx/cyclonedx-npm version: 1.11.0
  • NPM version: 8.5.5
  • Node version: 16.15.0
  • OS: windows 10

Unmet peer deps prevent building an SBOM

Hi -- loving the Typescript rewrite here. I'm having some pretty good results running this on my monorepo, and gathering all the package data together.

However, the build is failing due to an unmet peer dependency. npm ls causes this failure per npm/npm#17624 and https://github.com/CycloneDX/cyclonedx-node-npm/blob/1.0-dev/src/builders.ts#L141-L150.

I'm not worried about this unmet peer dependency -- I know it's just a silly library that hasn't updated its peer deps to say it supports React 18. But it causes me to be unable to generate an SBOM here. If I remove the error throwing from your package, I believe I still get an accurate SBOM, since it uses the deduped dependencies.

Is this desired behavior? Or could this be a warn in the build instead of an error that prevents building an SBOM?

DOCS: describe how component de-duplication works

target: a new file "component_deduplication" in https://github.com/CycloneDX/cyclonedx-node-npm/tree/main/docs

related docs:

goal:

  • describe the need and reasoning
  • have (alternative) processes described, how de-duplication could be achieved. which properties make components identical? {group,name,version,download-location,hashes,...}?
  • have a clear decision which process is to be used

💁 want to discuss? please use #307
-> there we have threats, votes, and everything we need

[BUG] Nested Components

Describe the bug

We noticed that nested components are being added with a | separated the parent component and the compontent.

const fill = component.bomRef.value + '|'

Expected behavior

Components should be listed individually and the relation between them should be added on the dependencies node.

[BUG] fix component's hash detection

Describe the bug

the sha512 detection in the builder is broken.
This causes, that the NPM-provided hash is not taken to the SBOM result.

To Reproduce

...

Expected behavior

the sha256 hash from NPM is added to the SBOM.

Screenshots or output-paste

Environment

  • @cyclonedx/cyclonedx-npm version: v1.4.0

Additional context

example hash in the integrity property of NPM:
sha512-EYuhVinaPQ2QMvF+SXUOxKpwmMp5rZxHQVHhuMfwhdSIjkwX+F8f+R/b0gyvZXc7eGu5qJJgOOmtR2likwgcFg==

Error running on Windows

When running on Windows, I get an error because my Node path contains a space:

C:\mypath>npx @cyclonedx/cyclonedx-npm --output-file ../artifacts/npmbom.json
DEBUG | options: {"ignoreNpmErrors":false,"packageLockOnly":false,"omit":[],"flattenComponents":false,"specVersion":"1.4","outputFormat":"JSON","outputFile":"../artifacts/npmbom.json","mcType":"application"}
DEBUG | packageFile: C:\mypath\package.json
DEBUG | projectDir: C:\mypath
DEBUG | lockFile: C:\mypath\package-lock.json
DEBUG | command: npx-cli.js usage detected, checking for npm-cli.js ...
INFO  | gather dependency tree ...
DEBUG | npm-ls: run C:\Program Files (x86)\nodejs\node.exe with ["--","C:\\Users\\myuser\\AppData\\Roaming\\npm\\node_modules\\npm\\bin\\npm-cli.js","ls","--json","--all","--long"] in C:\mypath
WARN  | npm-ls: STDERR
  'C:\Program' is not recognized as an internal or external command,
  operable program or batch file.

ERROR | npm-ls: errors
  {"error":{},"status":1,"signal":null}

Error: npm-ls exited with errors: ??? 1 noSignal

Maybe related to nodejs/node#7367 (comment)

example-results of demos

generate sboms based on the demo projects and put them in the folder example-results.
have the result explained in the readme files

[BUG]: PURL in BOM exceeds the length of 255 chars when using a private npm registry

Describe the bug

A BOM created by cyclonedx-npm contains a PURL longer than 255 chars when component is fetched from a private registry (nexus):

"purl": "pkg:npm/%40angular-builders/[email protected]?checksum=sha-512%3A14b183ac13a0d38718bf30ae75e6f5e4b598daff7576d279f807e63f4692ff44f40f4018998e2e33717165e8788c97161132ef9c75450428da9facb60adf49dc&download_url=http%3A//nexus.company.com%3A8081/repository/npm/%40angular-builders/custom-webpack/-/custom-webpack-14.1.0.tgz#packages/custom-webpack"

This leads to an error within dependency-track:

dtrack-apiserver_1  | Caused by: org.datanucleus.exceptions.NucleusUserException: Attempt to store value "pkg:npm/%40angular-builders/[email protected]?checksum=sha-512%3A14b183ac13a0d38718bf30ae75e6f5e4b598daff7576d279f807e63f4692ff44f40f4018998e2e33717165e8788c97161132ef9c75450428da9facb60adf49dc&download_url=http%3A%2F%2Fnexus.company.com%3A8081%2Frepository%2Fnpm%2F%40angular-builders%2Fcustom-webpack%2F-%2Fcustom-webpack-14.1.0.tgz#packages/custom-webpack" in column ""PURL"" that has maximum length of 255. Please correct your data!
dtrack-apiserver_1  | 	at org.datanucleus.store.rdbms.mapping.column.CharColumnMapping.setString(CharColumnMapping.java:253)
dtrack-apiserver_1  | 	at org.datanucleus.store.rdbms.mapping.java.SingleFieldMapping.setString(SingleFieldMapping.java:202)
dtrack-apiserver_1  | 	at org.datanucleus.store.rdbms.fieldmanager.ParameterSetter.storeStringField(ParameterSetter.java:158)

To Reproduce

I used the following package.json and executed npm i && npm run makeBom

{
  "name": "test",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "makeBom": "cyclonedx-npm --output-file bom.json"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "@angular-builders/custom-webpack": "^14.1.0",
    "@cyclonedx/cyclonedx-npm": "^1.9.0"
  }
}

.npmrc:

registry=http://nexus.company.com:8081/repository/npm

But without a private registry, PURL will not overflow, so reproduction needs some more work.

Expected behavior

PURL will not overflow when using a private registry.

Environment

  • @cyclonedx/cyclonedx-npm version: 1.9.0
  • NPM version: 8.19.3
  • Node version: v16.19.1
  • OS: Ubuntu 20.04.6 LTS

Additional context

Add any other context about the problem here.

Duplicate components

When creating an SBOM for a large enough package, there are inevitably cases where a single dependency is included more than once. I guess that makes sense as long as you're not using the --flatten-components option.

When using that option, however, I wouldn't expect those duplicates to appear. They each only differ in their bom-ref.
Is that behavior intentional?

Edit: Sorry, I was a little pressed for time when I hit send. Let me give an example.

Assume the following dependency tree:

Expected output

All output shortened for clarity.

Without --flatten-components:

{
  "components": [
    {
      "purl": "pkg:npm/[email protected]",
      "bom-ref": "[email protected]",
      "components": [
        {
          "purl": "pkg:npm/[email protected]",
          "bom-ref": "[email protected]|[email protected]"
        }
      ]
    },
    {
      "purl": "pkg:npm/[email protected]",
      "bom-ref": "[email protected]",
      "components": [
        {
          "purl": "pkg:npm/[email protected]",
          "bom-ref": "[email protected]|[email protected]"
        }
      ]
    }
  ]
}

With --flatten-components:

{
  "components": [
    {
      "purl": "pkg:npm/[email protected]",
      "bom-ref": "[email protected]"
    },
    {
      "purl": "pkg:npm/[email protected]",
      "bom-ref": "[email protected]"
    },
    {
      "purl": "pkg:npm/[email protected]",
      "bom-ref": "[email protected]"
    }
  ]
}

Actual output

Without --flatten-components:

Matches expected output 👍

With --flatten-components:

{
  "components": [
    {
      "purl": "pkg:npm/[email protected]",
      "bom-ref": "[email protected]"
    },
    {
      "purl": "pkg:npm/[email protected]",
      "bom-ref": "[email protected]"
    },
    {
      "purl": "pkg:npm/[email protected]",
      "bom-ref": "[email protected]|[email protected]"
    }
    {
      "purl": "pkg:npm/[email protected]",
      "bom-ref": "[email protected]|[email protected]"
    }
  ]
}

FEAT: `cdx:npm:package:bundled` when flatten components

Is your feature request related to a problem? Please describe.

This tool creates results that follow
https://github.com/CycloneDX/cyclonedx-property-taxonomy/blob/main/cdx/npm.md

when option --flatten-components=true is given, the property cdx:npm:package:bundled is missing.

flattening components and having bundled components should no longer be an issue, since #305 was done.

Describe the solution you'd like

If a NPM package was bundled, then add cdx:npm:package:bundled regardless of the CLI switch --flatten-components

Additional context

read https://github.com/CycloneDX/cyclonedx-node-npm/blob/main/docs/result.md

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.