yarn runtime vs development dependencies about dash-licenses HOT 11 OPEN

I am not a Yarn expert, but my understanding is that yarn.lock is just fixing the version of the dependency. So this is not enough to use only this file as source for the generating the report.

To know the type of dependency (build dependency vs runtime dependency) you need the package.json (and you need to traverse the complete dependency tree, using yarn.lock to know how a given dependency was resolved and then use the package.json of this dependency).

from dash-licenses.

Build time dependencies are still dependencies (i.e., "Build and Test Dependencies"). They're "works with", though, so we have considerable latitude with regard to licenses.

The worst case scenario is that a project team would maintain a dependency list (or, alternatively, an exclude list that's processed against a generated dependency list). I'm thinking that we may have no other choice for technologies that don't provide an easy means of generating a dependency list directly from the build (e.g. make files).

from dash-licenses.

I'm not a yarn expert too. 😅

But I also understand that the kind of dependencies are in package.json file. (see yarn2 doc or older but maybe clearer yarn1 doc)

{
  "dependencies": {
    "webpack": "^5.0.0"
  },
  "optionalDependencies": {
    "fsevents": "^5.0.0"
  },
  "devDependencies": {
    "webpack": "^5.0.0"
  },
  "peerDependencies": {
    "react": "*",
    "react-dom": "*"
  }
}

The yarn.lock file just fix version. (see yarn1 doc)

So ideally, I guess that @jmini is right and both files are need to get the information we want.

I don't know if there is a way to query about this kind of information with yarn command, I will search a little bit 🤔

from dash-licenses.

It seems that yarn list --prod list only "dependencies" (so I understand this is what we want to check, correct ?)
(Tested with yarn 1.22.10)
(Source : yarnpkg/yarn#4542.)

In my case, this returns 20 lines instead of 3573 (without --prod).
The result looks like :

yarn list v1.22.10
├─ @fontsource/[email protected]
├─ @mdi/[email protected]
├─ [email protected]
│  └─ follow-redirects@^1.10.0
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
│  └─ moment@^2.19.2
├─ [email protected]
├─ [email protected]
├─ [email protected]
│  └─ event-source-polyfill@^1.0.22
├─ [email protected]
├─ [email protected]
└─ [email protected]
Done in 0.99s.

If we want something more easy to parse there is something like : yarn list --prod --no-progress --json

{
  "type": "tree",
  "data": {
    "type": "list",
    "trees": [
      {
        "name": "@fontsource/[email protected]",
        "children": [],
        "hint": null,
        "color": "bold",
        "depth": 0
      },
      {
        "name": "@mdi/[email protected]",
        "children": [],
        "hint": null,
        "color": "bold",
        "depth": 0
      },
      {
        "name": "[email protected]",
        "children": [
          {
            "name": "follow-redirects@^1.10.0",
            "color": "dim",
            "shadow": true
          }
        ],
        "hint": null,
        "color": "bold",
        "depth": 0
      },
      {
        "name": "[email protected]",
        "children": [],
        "hint": null,
        "color": "bold",
        "depth": 0
      },
      {
        "name": "[email protected]",
        "children": [],
        "hint": null,
        "color": "bold",
        "depth": 0
      },
      {
        "name": "[email protected]",
        "children": [],
        "hint": null,
        "color": "bold",
        "depth": 0
      },
      {
        "name": "[email protected]",
        "children": [
          {
            "name": "moment@^2.19.2",
            "color": "dim",
            "shadow": true
          }
        ],
        "hint": null,
        "color": "bold",
        "depth": 0
      },
      {
        "name": "[email protected]",
        "children": [],
        "hint": null,
        "color": "bold",
        "depth": 0
      },
      {
        "name": "[email protected]",
        "children": [],
        "hint": null,
        "color": "bold",
        "depth": 0
      },
      {
        "name": "[email protected]",
        "children": [
          {
            "name": "event-source-polyfill@^1.0.22",
            "color": "dim",
            "shadow": true
          }
        ],
        "hint": null,
        "color": "bold",
        "depth": 0
      },
      {
        "name": "[email protected]",
        "children": [],
        "hint": null,
        "color": "bold",
        "depth": 0
      },
      {
        "name": "[email protected]",
        "children": [],
        "hint": null,
        "color": "bold",
        "depth": 0
      },
      {
        "name": "[email protected]",
        "children": [],
        "hint": null,
        "color": null,
        "depth": 0
      },
      {
        "name": "[email protected]",
        "children": [],
        "hint": null,
        "color": null,
        "depth": 0
      },
      {
        "name": "[email protected]",
        "children": [],
        "hint": null,
        "color": null,
        "depth": 0
      }
    ]
  }
}

from dash-licenses.

In case we need to write some javascript based on yarn API.
The code of the yarn list command is available here.

I also see it exists a yarn audit command which is able to query on a specific group of dependencies.
(See https://classic.yarnpkg.com/en/docs/cli/audit#toc-yarn-add)
So maybe the code of this command could be inspiring too.

from dash-licenses.

@sbernard31 - I wrote the contents of https://github.com/eclipse/dash-licenses/tree/master/yarn when I needed it almost 2 years ago, I have no doubt that there are updates and maybe better solutions now available. Please feel free to provide a PR.

(so I understand this is what we want to check, correct ?)

That is what needs checking for full license compatibility, but keep in mind as @waynebeaton said in #4 (comment) that all the hundreds to thousands of deps need handling, just that they are works-with instead. So having the tool spit out two lists would be useful. See https://www.eclipse.org/projects/handbook/#ip-third-party-test for more info.

from dash-licenses.

That is what needs checking for full license compatibility, but keep in mind as @waynebeaton said in #4 (comment) that all the hundreds to thousands of deps need handling, just that they are works-with instead

This part confused me a little because it's not clear to me what should be done exactly with "work_with".
Reading the ip-third-part-test link you provide, I understand that CQ is needed anyway (but this will be a lighter check by IP team ?)

Maybe it's not true for all projects, but for me I understand that my yarn devDependencies are pretty much like maven plugin. This is just some tools which help to build/validate the project. And I never consider to create CQ for the maven-plugin I'm using. Is it something which should be theoricaly done ?

from dash-licenses.

@paul-marechal, @marcdumais-work I see that you opened some issues about using dash-licences with yarn project.

I would be like to know your opinion about this issue ?

from dash-licenses.

Hi @sbernard31,

For the Theia project, we have an unusual number of dependencies: over 1000. When we were doing the license checks manually, using clearlydefined, we would filter-down the list to only production/runtime dependencies, to keep it somewhat manageable.

Now that dash-licenses does almost all the work (1), we no longer distinguish and let the tool deal with them all. Once in a while, some license issue is found with a dependency, then we can check how it's used, and if it's a devDependency, we can often get a "workswith" approval, to use it for build & tests.

(1): I encourage you to use this amazingly useful feature, that essentially opens the equivalent of a CQ for you for each dependency that needs it: https://github.com/eclipse/dash-licenses#automatic-ip-team-review-requests. See one of these automatically opened IP team review request, here, for example: https://gitlab.eclipse.org/eclipsefdn/emo-team/iplab/-/issues/1648

from dash-licenses.

Another option would be to use tooling to generate SBOM.
E.g. :

• https://github.com/anchore/syft
• https://aquasecurity.github.io/trivy/v0.48/docs/supply-chain/sbom/
• https://github.com/CycloneDX/cyclonedx-node-yarn (no release yet, not sure the project is still alive CycloneDX/cyclonedx-node-yarn#10, CycloneDX/cyclonedx-node-yarn#12)

Spoiler alert for now, AFAIK none of this project support to create a SBOM without devDependencies for yarn but if you think it makes sense to follow that way. I can try to open issue. (see #4 (comment))

And then give that SBOM to dash-licences : #191

from dash-licenses.

Spoiler alert for now, AFAIK none of this project support to create a SBOM without devDependencies for yarn but if you think it makes sense to follow that way. I can try to open issue.

I was wrong using trivy this is possible to generate SBOM with or without devDependencies :

Without dev dependencies :

trivy fs ./path/to/your/yarn/or/npm/project --format cyclonedx --output sbom-without-dev-dep.json

With dev dependencies :

trivy fs ./path/to/your/yarn/or/npm/project --format cyclonedx --output sbom-with-dev-dep.json --include-dev-deps

from dash-licenses.