Git Product home page Git Product logo

go-licenses's Introduction

Licenses tool

This is not an officially supported Google product.

go-licenses analyzes the dependency tree of a Go package/binary. It can output a report on the libraries used and under what license they can be used. It can also collect all of the license documents, copyright notices and source code into a directory in order to comply with license terms on redistribution.

Before you start

To use this tool, make sure:

Installation

Use the following command to download and install this tool:

go install github.com/google/go-licenses@latest

If you were using go get to install this tool, note that starting in Go 1.17, go get is deprecated for installing binaries.

Reports

$ go-licenses report github.com/google/go-licenses
W0410 06:02:57.077781   31529 library.go:86] "golang.org/x/sys/unix" contains non-Go code that can't be inspected for further dependencies:
/home/username/go/pkg/mod/golang.org/x/[email protected]/unix/asm_linux_amd64.s
W0410 06:02:59.476443   31529 library.go:86] "golang.org/x/crypto/curve25519/internal/field" contains non-Go code that can't be inspected for further dependencies:
/home/username/go/pkg/mod/golang.org/x/[email protected]/curve25519/internal/field/fe_amd64.s
W0410 06:02:59.486045   31529 library.go:86] "golang.org/x/crypto/internal/poly1305" contains non-Go code that can't be inspected for further dependencies:
/home/username/go/pkg/mod/golang.org/x/[email protected]/internal/poly1305/sum_amd64.s
W0410 06:02:59.872215   31529 library.go:253] module github.com/google/go-licenses has empty version, defaults to HEAD. The license URL may be incorrect. Please verify!
W0410 06:02:59.880621   31529 library.go:253] module github.com/google/go-licenses has empty version, defaults to HEAD. The license URL may be incorrect. Please verify!
github.com/emirpasic/gods,https://github.com/emirpasic/gods/blob/v1.12.0/LICENSE,BSD-2-Clause
github.com/golang/glog,https://github.com/golang/glog/blob/23def4e6c14b/LICENSE,Apache-2.0
github.com/golang/groupcache/lru,https://github.com/golang/groupcache/blob/41bb18bfe9da/LICENSE,Apache-2.0
github.com/google/go-licenses,https://github.com/google/go-licenses/blob/HEAD/LICENSE,Apache-2.0
github.com/google/go-licenses/internal/third_party/pkgsite,https://github.com/google/go-licenses/blob/HEAD/internal/third_party/pkgsite/LICENSE,BSD-3-Clause
github.com/google/licenseclassifier,https://github.com/google/licenseclassifier/blob/3043a050f148/LICENSE,Apache-2.0
github.com/google/licenseclassifier/stringclassifier,https://github.com/google/licenseclassifier/blob/3043a050f148/stringclassifier/LICENSE,Apache-2.0
github.com/jbenet/go-context/io,https://github.com/jbenet/go-context/blob/d14ea06fba99/LICENSE,MIT
github.com/kevinburke/ssh_config,https://github.com/kevinburke/ssh_config/blob/01f96b0aa0cd/LICENSE,MIT
github.com/mitchellh/go-homedir,https://github.com/mitchellh/go-homedir/blob/v1.1.0/LICENSE,MIT
github.com/otiai10/copy,https://github.com/otiai10/copy/blob/v1.6.0/LICENSE,MIT
github.com/sergi/go-diff/diffmatchpatch,https://github.com/sergi/go-diff/blob/v1.2.0/LICENSE,MIT
github.com/spf13/cobra,https://github.com/spf13/cobra/blob/v1.4.0/LICENSE.txt,Apache-2.0
github.com/spf13/pflag,https://github.com/spf13/pflag/blob/v1.0.5/LICENSE,BSD-3-Clause
github.com/src-d/gcfg,https://github.com/src-d/gcfg/blob/v1.4.0/LICENSE,BSD-3-Clause
github.com/xanzy/ssh-agent,https://github.com/xanzy/ssh-agent/blob/v0.2.1/LICENSE,Apache-2.0
go.opencensus.io,https://github.com/census-instrumentation/opencensus-go/blob/v0.23.0/LICENSE,Apache-2.0
golang.org/x/crypto,https://cs.opensource.google/go/x/crypto/+/5e0467b6:LICENSE,BSD-3-Clause
golang.org/x/mod/semver,https://cs.opensource.google/go/x/mod/+/9b9b3d81:LICENSE,BSD-3-Clause
golang.org/x/net,https://cs.opensource.google/go/x/net/+/69e39bad:LICENSE,BSD-3-Clause
golang.org/x/sys,https://cs.opensource.google/go/x/sys/+/5a964db0:LICENSE,BSD-3-Clause
golang.org/x/tools,https://cs.opensource.google/go/x/tools/+/v0.1.10:LICENSE,BSD-3-Clause
golang.org/x/xerrors,https://cs.opensource.google/go/x/xerrors/+/5ec99f83:LICENSE,BSD-3-Clause
gopkg.in/src-d/go-billy.v4,https://github.com/src-d/go-billy/blob/v4.3.2/LICENSE,Apache-2.0
gopkg.in/src-d/go-git.v4,https://github.com/src-d/go-git/blob/v4.13.1/LICENSE,Apache-2.0
gopkg.in/warnings.v0,https://github.com/go-warnings/warnings/blob/v0.1.2/LICENSE,BSD-2-Clause

This command prints out a comma-separated report (CSV) listing the libraries used by a binary/package, the URL where their licenses can be viewed and the type of license. A library is considered to be one or more Go packages that share a license file.

URLs are versioned based on go modules metadata.

Tip: go-licenses writes the report to stdout and info/warnings/errors logs to stderr. To save the CSV to a file licenses.csv in bash, run:

go-licenses report github.com/google/go-licenses > licenses.csv

Or, to also save error logs to an errors file, run:

go-licenses report github.com/google/go-licenses > licenses.csv 2> errors

Note: some warnings and errors may be expected, refer to Warnings and Errors for more information.

Reports with Custom Templates

go-licenses report github.com/google/go-licenses --template testdata/modules/hello01/licenses.tpl
W0822 16:56:50.696198   10200 library.go:94] "golang.org/x/sys/unix" contains non-Go code that can't be inspected for further dependencies:
/Users/willnorris/go/pkg/mod/golang.org/x/[email protected]/unix/asm_bsd_arm64.s
/Users/willnorris/go/pkg/mod/golang.org/x/[email protected]/unix/zsyscall_darwin_arm64.1_13.s
/Users/willnorris/go/pkg/mod/golang.org/x/[email protected]/unix/zsyscall_darwin_arm64.s
W0822 16:56:51.466449   10200 library.go:94] "golang.org/x/crypto/chacha20" contains non-Go code that can't be inspected for further dependencies:
/Users/willnorris/go/pkg/mod/golang.org/x/[email protected]/chacha20/chacha_arm64.s
W0822 16:56:51.475139   10200 library.go:94] "golang.org/x/crypto/curve25519/internal/field" contains non-Go code that can't be inspected for further dependencies:
/Users/willnorris/go/pkg/mod/golang.org/x/[email protected]/curve25519/internal/field/fe_arm64.s
W0822 16:56:51.602250   10200 library.go:269] module github.com/google/go-licenses has empty version, defaults to HEAD. The license URL may be incorrect. Please verify!
W0822 16:56:51.605074   10200 library.go:269] module github.com/google/go-licenses has empty version, defaults to HEAD. The license URL may be incorrect. Please verify!

 - github.com/emirpasic/gods ([BSD-2-Clause](https://github.com/emirpasic/gods/blob/v1.12.0/LICENSE))
 - github.com/golang/glog ([Apache-2.0](https://github.com/golang/glog/blob/23def4e6c14b/LICENSE))
 - github.com/golang/groupcache/lru ([Apache-2.0](https://github.com/golang/groupcache/blob/41bb18bfe9da/LICENSE))
 - github.com/google/go-licenses ([Apache-2.0](https://github.com/google/go-licenses/blob/HEAD/LICENSE))
 - github.com/google/go-licenses/internal/third_party/pkgsite ([BSD-3-Clause](https://github.com/google/go-licenses/blob/HEAD/internal/third_party/pkgsite/LICENSE))
 - github.com/google/licenseclassifier ([Apache-2.0](https://github.com/google/licenseclassifier/blob/3043a050f148/LICENSE))
 - github.com/google/licenseclassifier/licenses ([Unlicense](https://github.com/google/licenseclassifier/blob/3043a050f148/licenses/Unlicense.txt))
 - github.com/google/licenseclassifier/stringclassifier ([Apache-2.0](https://github.com/google/licenseclassifier/blob/3043a050f148/stringclassifier/LICENSE))
 - github.com/jbenet/go-context/io ([MIT](https://github.com/jbenet/go-context/blob/d14ea06fba99/LICENSE))
 - github.com/kevinburke/ssh_config ([MIT](https://github.com/kevinburke/ssh_config/blob/01f96b0aa0cd/LICENSE))
 - github.com/mitchellh/go-homedir ([MIT](https://github.com/mitchellh/go-homedir/blob/v1.1.0/LICENSE))
 - github.com/otiai10/copy ([MIT](https://github.com/otiai10/copy/blob/v1.6.0/LICENSE))
 - github.com/sergi/go-diff/diffmatchpatch ([MIT](https://github.com/sergi/go-diff/blob/v1.2.0/LICENSE))
 - github.com/spf13/cobra ([Apache-2.0](https://github.com/spf13/cobra/blob/v1.5.0/LICENSE.txt))
 - github.com/spf13/pflag ([BSD-3-Clause](https://github.com/spf13/pflag/blob/v1.0.5/LICENSE))
 - github.com/src-d/gcfg ([BSD-3-Clause](https://github.com/src-d/gcfg/blob/v1.4.0/LICENSE))
 - github.com/xanzy/ssh-agent ([Apache-2.0](https://github.com/xanzy/ssh-agent/blob/v0.2.1/LICENSE))
 - go.opencensus.io ([Apache-2.0](https://github.com/census-instrumentation/opencensus-go/blob/v0.23.0/LICENSE))
 - golang.org/x/crypto ([BSD-3-Clause](https://cs.opensource.google/go/x/crypto/+/5e0467b6:LICENSE))
 - golang.org/x/mod/semver ([BSD-3-Clause](https://cs.opensource.google/go/x/mod/+/86c51ed2:LICENSE))
 - golang.org/x/net ([BSD-3-Clause](https://cs.opensource.google/go/x/net/+/a158d28d:LICENSE))
 - golang.org/x/sys ([BSD-3-Clause](https://cs.opensource.google/go/x/sys/+/8c9f86f7:LICENSE))
 - golang.org/x/tools ([BSD-3-Clause](https://cs.opensource.google/go/x/tools/+/v0.1.12:LICENSE))
 - gopkg.in/src-d/go-billy.v4 ([Apache-2.0](https://github.com/src-d/go-billy/blob/v4.3.2/LICENSE))
 - gopkg.in/src-d/go-git.v4 ([Apache-2.0](https://github.com/src-d/go-git/blob/v4.13.1/LICENSE))
 - gopkg.in/warnings.v0 ([BSD-2-Clause](https://github.com/go-warnings/warnings/blob/v0.1.2/LICENSE))

This command executes a specified Go template file to generate a report of licenses. The template file is passed a slice of structs containing license data:

[]struct {
  Name        string
  Version     string
  LicenseURL  string
  LicenseName string
  LicensePath string
}

Each struct also has a LicenseText method which will return the text of the license stored at LicensePath if present, or an empty string if not.

Example template rendering licenses as markdown:

{{ range . }}
## {{ .Name }}

* Name: {{ .Name }}
* Version: {{ .Version }}
* License: [{{ .LicenseName }}]({{ .LicenseURL }})

```
{{ .LicenseText }}
```
{{ end }}

Save licenses, copyright notices and source code (depending on license type)

go-licenses save "github.com/google/go-licenses" --save_path="/tmp/go-licenses-cli"

This command analyzes a binary/package's dependencies and determines what needs to be redistributed alongside that binary/package in order to comply with the license terms. This typically includes the license itself and a copyright notice, but may also include the dependency's source code. All of the required artifacts will be saved in the directory indicated by --save_path.

Checking for forbidden licenses

$ go-licenses check github.com/logrusorgru/aurora
Forbidden license type WTFPL for library github.com/logrusorgru/auroraexit status 1

This command analyzes a package's dependencies and determines if any are considered forbidden by the license classifer. See github.com/google/licenseclassifier for licenses considered forbidden.

Usages

Global

Typically, specify the Go package that builds your Go binary. go-licenses expects the same package argument format as go build. For examples:

  • A rooted import path like github.com/google/go-licenses or github.com/google/go-licenses/licenses.
  • A relative path that denotes the package in that directory, like . or ./cmd/some-command.

To learn more about package argument, run go help packages.

To learn more about go-licenses usages, run go-licenses help.

Report

Report usage (default csv output):

go-licenses report <package> [package...]

Report usage (using custom template file):

go-licenses report <package> [package...] --template=<template_file>

Save

Save licenses, copyright notices and source code (depending on license type):

go-licenses save <package> [package...] --save_path=<save_path>

Check

Checking for forbidden and unknown licenses usage:

go-licenses check <package> [package...]

Tip: Usually you'll want to

  • append /... to the end of an import path prefix (e.g., your repo path) to include all packages matching that pattern
  • add --include_tests to also check packages only imported by testing code (e.g., testing libraries/frameworks)
go-licenses check --include_tests github.com/google/go-licenses/...

Checking for disallowed license types:

go-licenses check <package> [package...] --disallowed_types=<comma separated license types> 

Supported license types:

Allow only specific license names:

go-licenses check <package> [package...] --allowed_licenses=<comma separated license names> 

Build tags

To read dependencies from packages with build tags. Use the $GOFLAGS environment variable.

$ GOFLAGS="-tags=tools" go-licenses report google.golang.org/grpc/test/tools
github.com/BurntSushi/toml,https://github.com/BurntSushi/toml/blob/master/COPYING,MIT
google.golang.org/grpc/test/tools,Unknown,Apache-2.0
honnef.co/go/tools/lint,Unknown,BSD-3-Clause
golang.org/x/lint,Unknown,BSD-3-Clause
golang.org/x/tools,Unknown,BSD-3-Clause
honnef.co/go/tools,Unknown,MIT
honnef.co/go/tools/ssa,Unknown,BSD-3-Clause
github.com/client9/misspell,https://github.com/client9/misspell/blob/master/LICENSE,MIT
github.com/golang/protobuf/proto,https://github.com/golang/protobuf/blob/master/proto/LICENSE,BSD-3-Clause

Ignoring packages

Use the --ignore global flag to specify package path prefixes to be ignored. For example, to ignore your organization's internal packages under github.com/example-corporation:

$ go-licenses check \
    github.com/example-corporation/example-product \
    --ignore github.com/example-corporation

Note that dependencies from the ignored packages are still resolved and checked. This flag makes effect to check, report and save commands.

Include testing packages

Use the --include_tests global flag to include packages only imported by testing code (e.g., testing libraries/frameworks). Example command:

go-licenses check --include_tests "github.com/google/go-licenses/..."

This flag makes effect to check, report and save commands.

Warnings and errors

The tool will log warnings and errors in some scenarios. This section provides guidance on addressing them.

Dependency contains non-Go code

A warning will be logged when a dependency contains non-Go code. This is because it is not possible to check the non-Go code for further dependencies, which may conceal additional license requirements. You should investigate this code to determine whether it has dependencies and take action to comply with their license terms.

Error discovering URL

In order to determine the URL where a license file can be viewed, this tool generally performs the following steps:

  1. Locates the license file on disk.
  2. Parses go module metadata and finds the remote repo and version.
  3. Adds the license file path to this URL.

There are cases this tool finds an invalid/incorrect URL or fails to find the URL. Welcome creating an issue.

go-licenses's People

Contributors

at-wat avatar becoded avatar bobgy avatar davidhsingyuchen avatar dependabot-preview[bot] avatar dependabot[bot] avatar dprotaso avatar elenalape avatar gwatts avatar inteon avatar jarnoan avatar jdolitsky avatar jimmale avatar jingyuanliang avatar jmillerv avatar lavrd avatar marlongamez avatar melinath avatar nataliedoduc avatar otiai10 avatar slaskawi avatar thockin avatar treuherz avatar willnorris avatar wlynch avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

go-licenses's Issues

golang.org/x/sys and other packages don't seem to work

I'm walking through the dependencies in my go.mod and I find that the golang.org/x/ repositories give me trouble:

% go-licenses csv golang.org/x/sys
Error: errors for ["golang.org/x/sys"]:
golang.org/x/sys: -: module golang.org/x/sys@latest found (v0.0.0-20200420163511-1957bb5e6d1f), but does not contain package golang.org/x/sys
Usage:
  licenses csv <package> [flags]

Flags:
      --git_remote stringArray   Remote Git repositories to try (default [origin,upstream])
  -h, --help                     help for csv

Global Flags:
      --alsologtostderr                  log to standard error as well as files
      --confidence_threshold float       Minimum confidence required in order to positively identify a license. (default 0.9)
      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
      --log_dir string                   If non-empty, write log files in this directory
      --logtostderr                      log to standard error instead of files
      --stderrthreshold severity         logs at or above this threshold go to stderr (default 2)
  -v, --v Level                          log level for V logs
      --vmodule moduleSpec               comma-separated list of pattern=N settings for file-filtered logging

F0423 16:43:51.988116     851 main.go:43] errors for ["golang.org/x/sys"]:
golang.org/x/sys: -: module golang.org/x/sys@latest found (v0.0.0-20200420163511-1957bb5e6d1f), but does not contain package golang.org/x/sys

Is there a trick to this? Advice?

Detect licenses names like MIT-LICENSE.txt

https://github.com/mrjones/oauth has a MIT-LICENSE.txt in its root but it's currently not detected:

E0903 20:40:47.315979   75355 library.go:115] Failed to find license for github.com/mrjones/oauth: cannot find a known open source license for "/go/pkg/mod/github.com/mrjones/[email protected]" whose name matches regexp ^(?i)((UN)?LICEN(S|C)E|COPYING|README|NOTICE).*$ and locates up until "/go/pkg/mod/github.com/mrjones/[email protected]"

Maybe the ^ should be removed from the regex.

v1.1.0 go modules issue

I might have some special setup of go modules which does not work with v1.1.0.

Folder structure:

  • mainFolder/
    • subFolder1
    • subFolder2
    • subFodler3
    • vendor
    • go.mod
    • go.sum

vendor folder is constructed with command mainFolder/go mod vendor

go-licences csv mainFolder/subFolder1 > used_licences.csv command stopped working.

The thing is, that there is no .mod file in mainFolder/subFolder1, because it is handled in folder above, which includes also vendor.

Help Wanted: `go-licenses csv ...` command returned incorrectly formed go-package licenses URLs (inaccessible URLs)

I've been working on a bash utility script - verify-licenses.sh that verifies licenses for the go-packages used by the upstream Kubernetes Project, against the CNCF approved list of licenses.

We ran across an issue where executing the go-licenses csv ... command against Kubernetes project repo, produced a list of incorrectly formed Licenses URLs (inaccessible URLs) for a few go-packages, where in reality, those packages have valid accessible licenses present in the project.

Below is snippet from the script logs, with name/urls of the go-packages in question. (Please find the full script logs here)

go-licenses csv --git_remote "licenses" ./...

....
....
....
github.com/Azure/go-autorest/autorest                                                                :  Apache-2.0           : https://github.com/Azure/go-autorest/blob/autorest/v0.11.18/autorest/LICENSE
github.com/Azure/go-autorest/autorest/adal                                                           :  Apache-2.0           : https://github.com/Azure/go-autorest/blob/autorest/adal/v0.9.13/autorest/adal/LICENSE
github.com/Azure/go-autorest/autorest/date                                                           :  Apache-2.0           : https://github.com/Azure/go-autorest/blob/autorest/date/v0.3.0/autorest/date/LICENSE
github.com/Azure/go-autorest/autorest/mocks                                                          :  Apache-2.0           : https://github.com/Azure/go-autorest/blob/autorest/mocks/v0.4.1/autorest/mocks/LICENSE
github.com/Azure/go-autorest/autorest/to                                                             :  Apache-2.0           : https://github.com/Azure/go-autorest/blob/autorest/to/v0.4.0/autorest/to/LICENSE
github.com/Azure/go-autorest/autorest/validation                                                     :  Apache-2.0           : https://github.com/Azure/go-autorest/blob/autorest/validation/v0.1.0/autorest/validation/LICENSE
github.com/Azure/go-autorest/logger                                                                  :  Apache-2.0           : https://github.com/Azure/go-autorest/blob/logger/v0.2.1/logger/LICENSE
github.com/Azure/go-autorest/tracing                                                                 :  Apache-2.0           : https://github.com/Azure/go-autorest/blob/tracing/v0.6.0/tracing/LICENSE
github.com/blang/semver/v4                                                                           :  MIT                  : https://github.com/blang/semver/blob/v4.0.0/v4/LICENSE
....
....

For example:

For more context/information, check the WIP PR: kubernetes/kubernetes#109299


Help Wanted / Solution

We need help with implementing a fix/workaround for the 'go-licenses csv...' command so that it returns correctly formatted accessible URLs for go-packages that have valid licences present in their source code repository.

traverse vanity urls for repository location for license url

I ran against github.com/google/go-licenses and noticed that vanity urls weren't traversed for their corresponding github url.
see below for - cannot determine URL for "gopkg.in/warnings.v0" package
curl.exe https://gopkg.in/warnings.v0?go-get=1
yeilds
<meta name="go-source" content="gopkg.in/warnings.v0 _ https://github.com/go-warnings/warnings/tree/v0.1.2{/dir} https://github.com/go-warnings/warnings/blob/v0.1.2{/dir}/{file}#L{line}">

PS C:\Users\nmaliwa\go\src\github.com\google\go-licenses> .\go-licenses.exe csv .
E0706 20:48:36.424458   30424 csv.go:87] Error discovering URL for "C:\\Users\\nmaliwa\\go\\pkg\\mod\\gopkg.in\\[email protected]\\LICENSE":
- cannot determine URL for "gopkg.in/warnings.v0" package
E0706 20:48:36.576080   30424 csv.go:87] Error discovering URL for "C:\\Users\\nmaliwa\\go\\pkg\\mod\\golang.org\\x\\[email protected]\\LICENSE":
- unsupported package host "golang.org" for "golang.org/x/tools"
E0706 20:48:36.601091   30424 csv.go:87] Error discovering URL for "C:\\Users\\nmaliwa\\go\\pkg\\mod\\gopkg.in\\src-d\\[email protected]\\LICENSE":
- unsupported package host "gopkg.in" for "gopkg.in/src-d/go-git.v4"
E0706 20:48:36.728275   30424 csv.go:87] Error discovering URL for "C:\\Users\\nmaliwa\\go\\pkg\\mod\\golang.org\\x\\[email protected]\\LICENSE":
- unsupported package host "golang.org" for "golang.org/x/crypto"
E0706 20:48:36.741540   30424 csv.go:87] Error discovering URL for "C:\\Users\\nmaliwa\\go\\pkg\\mod\\gopkg.in\\src-d\\[email protected]\\LICENSE":
- unsupported package host "gopkg.in" for "gopkg.in/src-d/go-billy.v4"
E0706 20:48:36.813157   30424 csv.go:87] Error discovering URL for "C:\\Users\\nmaliwa\\go\\pkg\\mod\\golang.org\\x\\[email protected]\\LICENSE":
- unsupported package host "golang.org" for "golang.org/x/sys/windows"
E0706 20:48:36.827165   30424 csv.go:87] Error discovering URL for "C:\\Users\\nmaliwa\\go\\pkg\\mod\\golang.org\\x\\[email protected]\\LICENSE":
- unsupported package host "golang.org" for "golang.org/x/net"
github.com/google/licenseclassifier/stringclassifier,https://github.com/google/licenseclassifier/blob/master/stringclassifier/LICENSE,Apache-2.0        
github.com/sergi/go-diff/diffmatchpatch,https://github.com/sergi/go-diff/blob/master/diffmatchpatch/LICENSE,MIT
gopkg.in/warnings.v0,Unknown,BSD-2-Clause
github.com/otiai10/copy,https://github.com/otiai10/copy/blob/master/LICENSE,MIT
github.com/golang/glog,https://github.com/golang/glog/blob/master/LICENSE,Apache-2.0
golang.org/x/tools,Unknown,BSD-3-Clause
gopkg.in/src-d/go-git.v4,Unknown,Apache-2.0
golang.org/x/crypto,Unknown,BSD-3-Clause
gopkg.in/src-d/go-billy.v4,Unknown,Apache-2.0
golang.org/x/sys/windows,Unknown,BSD-3-Clause
golang.org/x/net,Unknown,BSD-3-Clause
github.com/mitchellh/go-homedir,https://github.com/mitchellh/go-homedir/blob/master/LICENSE,MIT
github.com/spf13/cobra,https://github.com/spf13/cobra/blob/master/LICENSE.txt,Apache-2.0
github.com/inconshreveable/mousetrap,https://github.com/inconshreveable/mousetrap/blob/master/LICENSE,Apache-2.0
github.com/google/go-licenses,https://github.com/google/go-licenses/blob/master/LICENSE,Apache-2.0
github.com/google/licenseclassifier,https://github.com/google/licenseclassifier/blob/master/LICENSE,Apache-2.0
github.com/spf13/pflag,https://github.com/spf13/pflag/blob/master/LICENSE,BSD-3-Clause
github.com/src-d/gcfg,https://github.com/src-d/gcfg/blob/master/LICENSE,BSD-3-Clause
github.com/jbenet/go-context/io,https://github.com/jbenet/go-context/blob/master/io/LICENSE,MIT
github.com/emirpasic/gods,https://github.com/emirpasic/gods/blob/master/LICENSE,BSD-2-Clause
github.com/kevinburke/ssh_config,https://github.com/kevinburke/ssh_config/blob/master/LICENSE,MIT
github.com/xanzy/ssh-agent,https://github.com/xanzy/ssh-agent/blob/master/LICENSE,Apache-2.0

Export local license paths

Hey

as mentioned in another ticket I'm creating a licenses page in my web app. I would like to link the licenses I dumped with the save command instead of relying on the online link.

For example I basically need to link to
webroot/licenses_backend/github.com/cpuguy83/go-md2man/v2/md2man/LICENSE.md
for
github.com/cpuguy83/go-md2man/v2/md2man,https://github.com/cpuguy83/go-md2man/blob/v2.0.2/LICENSE.md,MIT

I could now try to do some string magic and extract LICENSE.md from the URL and concatenate it with the package url, but that seems like a very brittle solution.
Could you make github.com/cpuguy83/go-md2man/v2/md2man/LICENSE.md to the templates?

Repository url paths should be versioned

Rather than always give a url to master, give to the appropriate tree.

Example in project go-licenses I ran PS C:\Users\nmaliwa\go\src\github.com\google\go-licenses> .\go-licenses.exe csv .

we get output
github.com/spf13/cobra,https://github.com/spf13/cobra/blob/master/LICENSE.txt,Apache-2.0
we also know that it is tag v0.0.5
I would expect
github.com/spf13/cobra,https://github.com/spf13/cobra/blob/v0.0.5/LICENSE.txt,Apache-2.0

Dependabot can't resolve your Go dependency files

Dependabot can't resolve your Go dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

github.com/google/go-licenses/licenses: cannot find module providing package github.com/google/go-licenses/licenses

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Reduce doc and CLI usage maintenance cost

Goal: users should be able to find full documentation in both CLI usage and from github documentation.

Problem: doc for both are duplicated, problems:

  1. some information is only available in one place
  2. duplicated efforts needed to maintain docs must be applied to both places

Proposal

  1. Include all the info into CLI usage docstring.
  2. Generate markdown CLI reference using https://github.com/spf13/cobra/blob/master/doc/md_docs.md.
  3. Remove non-quickstart necessary information from README.md and just link to the generated reference doc.

Context: this is an idea generated when updating docs for v1.1 #111.

Support for repositories that are dual-licensed

When i run go-licenses csv ./... against github.com/kubernetes/kubernetes:

the csv contains:

github.com/heketi/heketi,https://github.com/kubernetes/kubernetes/blob/master/vendor/github.com/heketi/heketi/COPYING-GPLV2,GPL-2.0

because https://github.com/heketi/heketi has multiple licenses (GPL 2.0 and Apache 2). Should the tool print both licenses?

version 1.1.0: Segmentation fault panic

go-licenses check main.go
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x38 pc=0x92f859]

goroutine 1 [running]:
github.com/google/go-licenses/licenses.Libraries.func1(0xc009f34140)
/home/ubuntu/work/pkg/mod/github.com/google/[email protected]/licenses/library.go:100 +0x259
golang.org/x/tools/go/packages.Visit.func1(0xc009f34140)
/home/ubuntu/work/pkg/mod/golang.org/x/[email protected]/go/packages/visit.go:26 +0xaa
golang.org/x/tools/go/packages.Visit({0xc003ea5b50, 0x1, 0x1?}, 0xc009d1b850, 0x0)
/home/ubuntu/work/pkg/mod/golang.org/x/[email protected]/go/packages/visit.go:43 +0x10e
github.com/google/go-licenses/licenses.Libraries({0x134b2a8?, 0xc00011c000?}, {0x1344960, 0xc003ea55a8}, {0xc0002b0ce0?, 0xc00035fd68?, 0xc00035fd90?})
/home/ubuntu/work/pkg/mod/github.com/google/[email protected]/licenses/library.go:76 +0x22c
main.checkMain(0x16d44e0?, {0xc0002b0ce0, 0x1, 0x1})
/home/ubuntu/work/pkg/mod/github.com/google/[email protected]/check.go:47 +0x8e
github.com/spf13/cobra.(*Command).execute(0x16d44e0, {0xc0002b0cc0, 0x1, 0x1})
/home/ubuntu/work/pkg/mod/github.com/spf13/[email protected]/command.go:856 +0x67c
github.com/spf13/cobra.(*Command).ExecuteC(0x16d4260)
/home/ubuntu/work/pkg/mod/github.com/spf13/[email protected]/command.go:974 +0x3b4
github.com/spf13/cobra.(*Command).Execute(...)
/home/ubuntu/work/pkg/mod/github.com/spf13/[email protected]/command.go:902
main.main()
/home/ubuntu/work/pkg/mod/github.com/google/[email protected]/main.go:72 +0x85

Provide binaries and releases

Hi, I was wondering if it would be possible to provide proper release versions and binaries along with them?
Today we hit a weird issue on our CI as github.com/otiai10/copy introduced a breaking change on their latest release and the way we were installing go-licenses was triggering this problem, our CI is a bit weird but so far we managed to solve it with some extra conf but having a binary we can just download and run would solve this kind of issue (not to mention that is also usually faster to do it like that)

So leaving this here for consideration

Transitive dependencies scan

Hello there!

Didn't find information about possibility to scan transitive dependencies.
For example I have project with several dependencies, this dependencies has their own dependencies, and their dependencies has dependencies ... etc.

Can I get list of all dependencies of dependencies with their licenses?
Or maybe you solve this problem by another way?

Check returns exit code 0 when LICENSE is not found

Expected behavior

go-licenses check returns non-zero exit code when it cannot find a valid license so that tools/scripts can report error.

Actual behavior

Exit code 0 is returned.

$ go-licenses check github.com/daaku/go.zipexe
E0701 18:49:16.281709   29087 library.go:108] Failed to find license for github.com/daaku/go.zipexe: no file/directory matching regexp "^(LICEN(S|C)E|COPYING|README|NOTICE)(\\..+)?$" found for /Users/wlynch/go/pkg/mod/github.com/daaku/[email protected]
$ echo $?
0

The actual cause of the missing license is because go-licenses doesn't consider casing. It sounds like #30 may fix this.

Bring back support for non go modules projects to v1.1+

In v1.1, we made a breaking change of no longer supporting non go modules managed projects per discussions in #70. However, #125 seems to show that some users are still using GOPATH.

Creating this issue to track users who are still using GOPATH mode.
Please thumbs up if this affected you or comment about your scenario.

Go modules support?

UPDATE: I sent go-licenses/v2 proposal that takes go modules as first class citizen #70

Is there any plan for go modules support?

I'd like to get licenses for certain version of a library, because transitive dependency can be different between versions.

I believe this should address #33.

Add Bazel support

Is there a possibility to support Bazel and not only the native Go tooling?

Libraries are ignored when goroot and gopath share the same prefix

If your go-root is (for example) /home/ubuntu/go and your gopath is /home/ubuntu/gopath, go-licenses will treat all dependencies as stdlib and therefore ignore them.

The issue lies here: https://github.com/google/go-licenses/blob/master/licenses/library.go#L204
When packages are stored inside the gopath, their path will start with /home/ubuntu/gopath, and as /home/ubuntu/go is a prefix of this, go-licenses thinks these packages belong to the stdlib.

The solution would be to append a path-seperator to build.Default.GOROOT (if it does not already end in one) before performing the prefix-check.

Rerunning go-licenses now breaks when using --force and already have an output dir

This PR has introduced a bug where when you run the tool on an existing repo with the output directory in the project being scanned, https://github.com/google/go-licenses/pull/90/files#diff-63d2514e96da76ad8bd5c2460f4ea67c0a6c6ba1055a8890b7ec0c686bcc183fR70 finds all the previous license, which then are not locatable after https://github.com/google/go-licenses/pull/90/files#diff-63d2514e96da76ad8bd5c2460f4ea67c0a6c6ba1055a8890b7ec0c686bcc183fR76 if you use the tool with the the --force flag in play.

Originally posted by @n3wscott in #90 (comment)

.: -: no Go files

Steps to reproduce:

  1. Check out [email protected]:mongodb/mongodb-atlas-kubernetes.git
  2. install and run go-licenses csv . from the directory

Output:
F0225 13:03:44.868040 99714 main.go:43] errors for ["."]:
.: -: no Go files in /pathtorepo

bug: Odd lack of dependency licenses for moby/buildkit

~/go/bin/go-licenses csv github.com/moby/buildkit
prints only:

github.com/moby/buildkit,https://github.com/moby/buildkit/blob/master/LICENSE,Apache-2.0

Build buildkit’s go.mod looks like this: https://github.com/moby/buildkit/blob/master/go.mod

go.mod
module github.com/moby/buildkit

go 1.13

require (
    github.com/BurntSushi/toml v0.3.1
    github.com/Microsoft/go-winio v0.4.17
    github.com/Microsoft/hcsshim v0.8.16
    github.com/codahale/hdrhistogram v0.0.0-20160425231609-f8ad88b59a58 // indirect
    github.com/containerd/console v1.0.2
    github.com/containerd/containerd v1.5.0
    github.com/containerd/continuity v0.1.0
    github.com/containerd/fuse-overlayfs-snapshotter v1.0.2
    github.com/containerd/go-cni v1.0.2
    github.com/containerd/go-runc v1.0.0
    github.com/containerd/stargz-snapshotter v0.5.0
    github.com/containerd/typeurl v1.0.2
    github.com/coreos/go-systemd/v22 v22.1.0
    github.com/docker/cli v20.10.5+incompatible
    github.com/docker/distribution v2.7.1+incompatible
    github.com/docker/docker v20.10.5+incompatible
    github.com/docker/go-connections v0.4.0
    github.com/docker/libnetwork v0.8.0-dev.2.0.20201215162534-fa125a3512ee
    github.com/gofrs/flock v0.7.3
    github.com/gogo/googleapis v1.4.0
    github.com/gogo/protobuf v1.3.2
    // protobuf: the actual version is replaced in replace()
    github.com/golang/protobuf v1.4.3
    github.com/google/go-cmp v0.5.4
    github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510
    github.com/gorilla/mux v1.8.0 // indirect
    github.com/grpc-ecosystem/go-grpc-middleware v1.2.0
    github.com/grpc-ecosystem/grpc-opentracing v0.0.0-20180507213350-8e809c8a8645
    github.com/hashicorp/go-immutable-radix v1.0.0
    github.com/hashicorp/golang-lru v0.5.3
    github.com/hashicorp/uuid v0.0.0-20160311170451-ebb0a03e909c // indirect
    github.com/ishidawataru/sctp v0.0.0-20191218070446-00ab2ac2db07 // indirect
    github.com/jaguilar/vt100 v0.0.0-20150826170717-2703a27b14ea
    github.com/mitchellh/hashstructure v1.0.0
    github.com/moby/locker v1.0.1
    github.com/moby/sys/mount v0.2.0 // indirect; force more current version of sys/mount than go mod selects automatically
    github.com/moby/term v0.0.0-20201110203204-bea5bbe245bf // indirect
    github.com/morikuni/aec v1.0.0
    github.com/opencontainers/go-digest v1.0.0
    github.com/opencontainers/image-spec v1.0.1
    github.com/opencontainers/runc v1.0.0-rc93
    github.com/opencontainers/runtime-spec v1.0.3-0.20200929063507-e6143ca7d51d
    github.com/opencontainers/selinux v1.8.0
    github.com/opentracing-contrib/go-stdlib v1.0.0
    github.com/opentracing/opentracing-go v1.2.0
    github.com/pkg/errors v0.9.1
    github.com/pkg/profile v1.5.0
    github.com/serialx/hashring v0.0.0-20190422032157-8b2912629002
    github.com/sirupsen/logrus v1.7.0
    github.com/stretchr/testify v1.7.0
    github.com/tonistiigi/fsutil v0.0.0-20201103201449-0834f99b7b85
    github.com/tonistiigi/units v0.0.0-20180711220420-6950e57a87ea
    github.com/uber/jaeger-client-go v2.25.0+incompatible
    github.com/uber/jaeger-lib v2.2.0+incompatible // indirect
    github.com/urfave/cli v1.22.2
    go.etcd.io/bbolt v1.3.5
    golang.org/x/crypto v0.0.0-20210322153248-0c34fe9e7dc2
    golang.org/x/net v0.0.0-20210226172049-e18ecbb05110
    golang.org/x/sync v0.0.0-20201207232520-09787c993a3a
    golang.org/x/sys v0.0.0-20210324051608-47abb6519492
    golang.org/x/time v0.0.0-20200630173020-3af7569d3a1e
    // genproto: the actual version is replaced in replace()
    google.golang.org/genproto v0.0.0-20201110150050-8816d57aaa9a
    // grpc: the actual version is replaced in replace()
    google.golang.org/grpc v1.35.0
)

replace (
    // protobuf: corresponds to containerd
    github.com/golang/protobuf => github.com/golang/protobuf v1.3.5
    github.com/hashicorp/go-immutable-radix => github.com/tonistiigi/go-immutable-radix v0.0.0-20170803185627-826af9ccf0fe
    github.com/jaguilar/vt100 => github.com/tonistiigi/vt100 v0.0.0-20190402012908-ad4c4a574305
    // genproto: corresponds to containerd
    google.golang.org/genproto => google.golang.org/genproto v0.0.0-20200224152610-e50cd9704f63
    // grpc: corresponds to protobuf
    google.golang.org/grpc => google.golang.org/grpc v1.30.0
)

Libraries function returns non-licensed vendored package as the package with LICENSE from root project

Steps to reproduce:

  1. Create Go project with some external package as dependency and run go mod vendor to place this package inside vendor folder
  2. Delete LICENSE file from this package inside vendor folder
  3. Run go-licenses report or go-licenses check on your project. The command lists external package as package without a license (as it should be)
  4. Now add some default LICENSE in the root folder of your project. Project folder looks like this: go.mod go.sum LICENSE main.go vendor/
  5. Run go-licenses report or go-licenses check on your project again. Now commands consider external package as having LICENSE from the root project folder: go-licenses check doesn't fire an error and go-licenses report doesn't list external package at all (considering this as the part of root project).

I guess it's because when Libraries function calls Find function, it passes p.Module.Dir as rootDir (which is nil for vendored packages for some reason) and then Find does rootDir, err = filepath.Abs(rootDir), nil turns into working directory (root package folder) and findUpwards looks for license file until it finds it in root package folder.

native go modules support proposal (originally named v2 proposal)

Updates

2022-03-12: #70 (comment) This will be released as v1.1 instead, because the breaking change might not affect anyone now (TL;DR there's only a regression for people still using GOPATH). Also renamed this issue to make the purpose clearer.

2021-1-11: #70 (comment) I sent out a bunch of PRs for merging changes back to go-licenses repo. The final PRs I'm sending now try to change as minimal as possible. There are no substantial command/arg changes and there are no significant behavior changes. The only major theme is using go modules information to get license URLs more stably than the v1 go-licenses tool.

Original Proposal

Hi everyone, I'm building a rewrite of go-licenses that I propose to be go-licenses/v2.
Latest version on my dev branch: https://github.com/Bobgy/go-licenses/blob/main/v2. No longer latest.
Built binary releases: https://github.com/Bobgy/go-licenses/releases.

I initially built it as a new tool, but after chatting with @wlynch, we agreed that it would better be v2 of this tool rather than a new brand.

The major changes are:

  • only supports go modules, it can get license URL for modules without a need for you to vendor your dependencies.
  • do not assume each package has a single license, v2 will scan all the files for each module to find licenses. Deferred for future considerations.

For more details, refer to the README.

The new tool already addresses many open issues:

  • #32 vanity urls are curled and extracted to find the repo
  • #33 urls are now versioned according to go modules info
  • #38 now we use modules info, no custom parsing
  • #53 proper go modules support
  • #64 is a subproblem of proper go modules support

Deprioritized features

  • #39 we scan all files for licenses, so we can find UNLICENSE files too

and might help the following (but needs confirmation):

Roadmap

  • #101
  • CSV command
    • Get modules/packages list
    • #94
    • #110
    • Config file customization & overrides
  • Documentation
  • Smaller Issues

Dependabot can't resolve your Go dependency files

Dependabot can't resolve your Go dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

github.com/google/go-licenses/licenses: cannot find module providing package github.com/google/go-licenses/licenses

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Include version

Can you include the version as found in go list -m all to the output?

For OSS clearings, this is important, as licenses can change in later versions

improve robustness of linking to license on hosting website

In v2, I implemented some utils to get github repo from go-import=1 and use it to generate public & versioned links to detected licenses' hosting website (for now, only github).

I noticed some harder problems:

  1. distinguishing "major branch" and "major subdirectory" conventions

There is one problem: for a major version greater than 1, the templates for “major branch” and “major subdirectory” conventions differ (See https://research.swtch.com/vgo-module for a discussion of these conventions.) To determine the right template, make a HEAD request for the go.mod file using each template, and select the one that succeeds. For example, for module github.com/a/b/v2 at version v2.3.4, probe both github.com/a/b/blob/v2.3.4/go.mod (the location of the go.mod file using the “major branch” convention) and github.com/a/b/blob/v2.3.4/v2/go.mod (its location using “major subdirectory”).

  1. support modules not at root of a repo, example https://github.com/Azure/go-autorest/tree/autorest/v0.9.0. Note that tags are also different, a tag "autorest/v0.9.0" means v0.9.0 version of the module ROOT/autorest. https://github.com/googleapis/google-cloud-go/tree/master/storage is another example, tags for it has "storage/` prefix.
  2. support other source hosting websites

Potential Solution

@wlynch pointed out the following references, there's an internal source package built for pkgsite that exactly provides a package that can figure out repo hosting website of a go import path and get a public link to source code. However, the package is internal, so we cannot directly import it.

I'll ask if they are ready to make it public, or I have to vendor it in some way.

EDIT: the reply is that we need to vendor it: golang/go#40477 (comment).

References

Error when 'import' or 'go get'

$ go get github.com/google/go-license
/src/github.com/google/go-licenses/save.go:121:22: cannot use func literal (type func(string) bool) as type func(string) (bool, error) in field value

Reciprocal modules copies read-only directories on save. Prevents multiple runs.

Problem

The copy function used for save preserves all permissions, which means when files are copied from the module cache for Reciprocal/Restricted licenses all the files and directories are readonly as well. This means that multiple runs of save won't work because the directories containing

Example

package example

import (
        _ "github.com/hashicorp/errwrap"
)
[2019-11-22 14:52:03 example ]$ go-licenses save . --save_path=third_party --force 
[2019-11-22 14:52:10 example ]$ go-licenses save . --save_path=third_party --force
Error: unlinkat ./third_party/github.com/hashicorp/errwrap/go.mod: permission denied
Usage:
  licenses save <package> [flags]

Flags:
      --force              Delete the destination directory if it already exists.
  -h, --help               help for save
      --save_path string   Directory into which files should be saved that are required by license terms

Global Flags:
      --alsologtostderr                  log to standard error as well as files
      --confidence_threshold float       Minimum confidence required in order to positively identify a license. (default 0.9)
      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
      --log_dir string                   If non-empty, write log files in this directory
      --logtostderr                      log to standard error instead of files
      --stderrthreshold severity         logs at or above this threshold go to stderr (default 2)
  -v, --v Level                          log level for V logs
      --vmodule moduleSpec               comma-separated list of pattern=N settings for file-filtered logging

F1122 14:52:12.566725  140815 main.go:43] unlinkat ./third_party/github.com/hashicorp/errwrap/go.mod: permission denied
[2019-11-22 14:52:12 example ]$ ls -lR third_party 
third_party:
total 4
drwxr-x--- 3 wlynch primarygroup 4096 Nov 22 14:52 github.com

third_party/github.com:
total 4
drwxr-x--- 3 wlynch primarygroup 4096 Nov 22 14:52 hashicorp

third_party/github.com/hashicorp:
total 4
dr-x------ 2 wlynch primarygroup 4096 Nov 22 14:52 errwrap

third_party/github.com/hashicorp/errwrap:
total 36
-r--r----- 1 wlynch primarygroup  4377 Nov 22 14:52 errwrap.go
-r--r----- 1 wlynch primarygroup  1292 Nov 22 14:52 errwrap_test.go
-r--r----- 1 wlynch primarygroup    36 Nov 22 14:52 go.mod
-r--r----- 1 wlynch primarygroup 15977 Nov 22 14:52 LICENSE
-r--r----- 1 wlynch primarygroup  2484 Nov 22 14:52 README.md

Solution

We should run the equivalent of chmod +w $(find third_party -type d) when copying over source in order to make sure directories can be deleted properly on multiple runs of the tool.

cannot register licenses from archive

I tried with various approaches, but always end up with cannot register licenses from archive

e.g. with the docker container:

docker run --rm -it reslocal/go-licenses-container check github.com/logrusorgru/aurora

gives:

Error: cannot register licenses from archive: open /go/pkg/mod/github.com/google/[email protected]/licenses/licenses.db: no such file or directory
Usage:
  licenses check <package> [flags]

Flags:
  -h, --help   help for check

Global Flags:
      --alsologtostderr                  log to standard error as well as files
      --confidence_threshold float       Minimum confidence required in order to positively identify a license. (default 0.9)
      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
      --log_dir string                   If non-empty, write log files in this directory
      --logtostderr                      log to standard error instead of files
      --stderrthreshold severity         logs at or above this threshold go to stderr (default 2)
  -v, --v Level                          log level for V logs
      --vmodule moduleSpec               comma-separated list of pattern=N settings for file-filtered logging

F0319 14:31:29.784486       1 main.go:43] cannot register licenses from archive: open /go/pkg/mod/github.com/google/[email protected]/licenses/licenses.db: no such file or directory

or

sh-4.4# pwd
/workdir/build/imx6ul-phytec-segin-virt-telegraf-wic-master/tmp/work/armv7at2hf-vfp-resy-linux-gnueabi/github.com-influxdata-telegraf/1.18.0-r0/github.com-influxdata-telegraf-1.18.0/src/github.com/influxdata/telegraf
sh-4.4# go-licenses csv .
Error: cannot register licenses from archive: open github.com/google/[email protected]/licenses/licenses.db: no such file or directory
Usage:
  licenses csv <package> [flags]

Flags:
      --git_remote stringArray   Remote Git repositories to try (default [origin,upstream])
  -h, --help                     help for csv

Global Flags:
      --alsologtostderr                  log to standard error as well as files
      --confidence_threshold float       Minimum confidence required in order to positively identify a license. (default 0.9)
      --log_backtrace_at traceLocation   when logging hits line file:N, emit a stack trace (default :0)
      --log_dir string                   If non-empty, write log files in this directory
      --logtostderr                      log to standard error instead of files
      --stderrthreshold severity         logs at or above this threshold go to stderr (default 2)
  -v, --v Level                          log level for V logs
      --vmodule moduleSpec               comma-separated list of pattern=N settings for file-filtered logging

F0319 14:34:30.064271  354052 main.go:43] cannot register licenses from archive: open github.com/google/[email protected]/licenses/licenses.db: no such file or directory
sh-4.4# 

I am wondering what I am missing here.

Ability to ignore unlicensed packages

Hi, we currently have azure sdk dependencies that have direct dependencies with internal packages which doesn't contain license. As it's a indirect dependency on our side, we can't just bump to a version that contains a license file, we need to have the ability to avoid checking license of those dependencies. In other terms, we want to avoid having errors for packages on which we don't have any control.

Examples

$ go-licenses save . --save_path=./acknowledgements \
  --filter github.com/Azure/azure-sdk-for-go/sdk/internal/log \
  --filter github.com/Azure/azure-sdk-for-go/sdk/internal/diag \
  --filter github.com/Azure/azure-sdk-for-go/sdk/internal/errorinfo \
  --filter github.com/Azure/azure-sdk-for-go/sdk/internal/uuid

$ go-licenses save . --save_path=./acknowledgements --filter github.com/Azure/azure-sdk-for-go/sdk/internal/*

# Another option would be to provide an option to ignore all packages that doesn't contain a license file.
$ go-licenses save . --save_path=./acknowledgements --ignore-unlicensed
# or
$ go-licenses save . --save_path=./acknowledgements --skip-missing

Dependabot can't resolve your Go dependency files

Dependabot can't resolve your Go dependency files.

As a result, Dependabot couldn't update your dependencies.

The error Dependabot encountered was:

github.com/google/go-licenses/licenses: cannot find module providing package github.com/google/go-licenses/licenses

If you think the above is an error on Dependabot's side please don't hesitate to get in touch - we'll do whatever we can to fix it.

View the update logs.

Projects licensed MPL-2.0 includes go code

TL;DR: I'm not convinced MPL-2.0 actually requires a copy of source code when used as a library. It just being publicly available where you say it is looks like enough


MPL-2.0 license FAQ

Q7: I want to distribute (outside my organization) complete and unchanged executable programs built from MPL-licensed software by someone other than me. What do I have to do?

As long as the people who distributed the program to you have complied with the MPL, typically nothing. To check and see if the people who distributed the program to you have complied with the MPL, look for the notice that tells you where the software is available in Source Code form (i.e., check that it complies with Section 3.2(a)), and then check that the Source Code is available in that place, including a notice that informs you that the Source Code is available under the terms of the MPL (i.e., check that it complies with Section 3.1).

Also not a lawyer but that just sounds like if you have used some-library and you say it's from github.com/org/some-library (either in go.mod/go.sum or vendor or third_party/VENDOR-LICENSE), as long as the code is there (github.com/org/some-library) with a MPL license it meets the requirements

https://www.npmjs.com/package/next-mdx-enhanced
next-mdx-enhanced (GitHub, NPM) is MPL-2.0 but I doubt that every project that depends on it is now required to commit their node_modules directory so the source is distributed with their code.
That's 2.6k uses in repos commited to GitHub alone

Fossa are probably the most famous License Compliance company/service and their https://github.com/fossas/fossa-cli is apparently 13% MPL-2.0 + 25% MPL-2.0-no-copyleft-exception here, but they don't have any external source-code outside of their vendor dir. In fact they have deleted their vendor dir after https://github.com/fossas/fossa-cli/tree/v1.1.4 in https://github.com/fossas/fossa-cli/tree/v1.1.5

Even assuming MPL-2.0-no-copyleft-exception doesn't have the same source code requirement, thats 13% of their dependencies which would require the source code be included if that was the case


Related tektoncd/cli#1307

Warnings from assembly files in golang.org/x/*

When run on projects that refer to x/sys or x/crypto, go-licenses prints warnings like:

W0329 16:23:18.204626  221077 library.go:86] "[golang.org/x/sys/unix](http://golang.org/x/sys/unix)" contains non-Go code that can't be inspected for further dependencies:
/home/drigz/go/pkg/mod/[golang.org/x/[email protected]/unix/asm_linux_amd64.s](http://golang.org/x/[email protected]/unix/asm_linux_amd64.s)

It might print 10-20 warnings per run, which can be confusing the first time you see them, but the tool is operating as expected as these files don't introduce external dependencies (as far as I'm aware - I'm not sure how go mod would handle an external dependency from a non-go file).

This doesn't block use of the tool but it's nice to avoid printing warnings in the normal case, as it teaches people to ignore warnings.

I can imagine the following alternatives although I'm not sure how to choose between them:

  • Completely ignore OtherFiles: go mod will ignore these files so they are unable to pull in external dependencies. Instead, those projects should vendor the external dependencies and record this in their LICENSE file.
  • Ignore .[sS] files in OtherFiles, as assembly doesn't have a way of modelling external dependencies, whereas plausibly other files might.
  • Ignore a hardcoded list of .[sS] files from golang.org/x/*, after manually reviewing them to check that they don't have external dependencies.
  • Allow the user to ignore individual files with a similar approach to the --ignore flag in #46. This would allow users to record in a script which files they have manually reviewed. This would be useful if we expect users to manually review these files, rather than ignoring the warnings.

I do not know how I would review the files listed in the "non-Go code" warning for external dependencies that should be listed. For example, asm_linux_amd64.s has no reference to an external repository (that I can see), but includes textflag.h, which has the same license header. Is this sufficient to conclude that the warning can be ignored? What is the user expected to do when they see this warning? Can it be automated?

`go get github.com/google/go-licenses` is broken

24e2421 seems to break. Running go get github.com/google/go-licenses encounters the following error:

go: github.com/google/go-licenses upgrade => v0.0.0-20210727195037-24e2421190d8
/go/pkg/mod/github.com/google/[email protected]/licenses/embed.go:4:2: package embed is not in GOROOT (/usr/local/go/src/embed)
/go/pkg/mod/github.com/google/[email protected]/licenses/embed.go:5:2: package io/fs is not in GOROOT (/usr/local/go/src/io/fs)

Also check forbidden/unknown licenses for save, csv commands

I think it doesn't make sense to successfully terminate save/csv commands when there are forbidden licenses.

Also, it'd be time wasting if we run the check command and save/csv command together, because with the current implementation, we will need to re-scan for all licenses again.

Proposal

Two options:

  • breaking change: let save/csv commands check license type as well and emit errors if any problems are found
  • backwards compatible: add a new flag like --exit_status or --check, and also do the check logic in other commands as well

How to get licenses of just built package

Hi everybody,

I have a question about extracting the license of a freshly built package.

Background is, that I would like to add this tool to the conda-forge docs to make it easier for maintainers to correctly include go licenses.

The collection of licenses would therefore be right after a package is built with the ideal process being the following:

  1. Conda-build builds the package

  2. go-licenses downloads all licenses

  3. conda-build includes the output folder in the package and finishes packaging it.

Step 1. and 3. is taken care of by our infrastructure but 2. fails with

Error: errors for ["github.com/mozilla/sops"]:
github.com/mozilla/sops: -: no required module provides package github.com/mozilla/sops; to add it:
	go get github.com/mozilla/sops

Logs: https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=479803&view=logs&jobId=656edd35-690f-5c53-9ba3-09c10d0bea97&j=656edd35-690f-5c53-9ba3-09c10d0bea97&t=e5c8ab1d-8ff9-5cae-b332-e15ae582ed2d

The script for this is:

script:
     - pushd {{ pkg_src }}
     - make install
     - go-licenses save "github.com/mozilla/sops" --save_path="go_licenses"

Ref: https://github.com/conda-forge/go-sops-feedstock/pull/9/files

And the PR where I tried this is conda-forge/go-sops-feedstock#9

Therefore, my question would be:

  1. How do I need to include go-licenses in the workflow so that everything is captured?

  2. And additionally, the docs include / before the folder where the output is stored. Is that OS-specific, or is that correctly translated for Win? Ref: https://github.com/google/go-licenses#complying-with-license-terms

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.