uw-labs / lichen Goto Github PK

If a binary is built with a development build of Go, the go version -m ... output cannot be parsed. I observed this when trying go-licenses v2: https://github.com/Bobgy/go-licenses/blob/initial/v2/README.md

$ ~/git/go-licenses/v2/go-licenses csv
ListModulesInGoBinary(Path='my-binary'): unrecognised version line: my-binary: devel +b7a85e0003
$ go version -m my-binary    
my-binary: devel +b7a85e0003
        path    my-binary/cmd/my-binary
        mod     my-binary       (devel)
        dep     gopkg.in/yaml.v2        v2.4.0  h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY=
$ go version
go version devel +b7a85e0003 linux/amd64

This doesn't occur when analyzing a binary built with a normal release of Go.

It appears to be failing on blank lines in the output of go version -m. This is the complete output:

2022/03/15 12:49:04 failed to evaluate licenses: unrecognised line: 	

Hi - I am working on replacing https://github.com/mitchellh/golicense with lichen for Project Antrea, since golicense is no longer actively maintained and lichen offers me support for Windows binaries.

I am running into one issue during module fetching:

2021/01/29 00:26:16 failed to evaluate licenses: failed to fetch: exit status 1 (output: go: cannot use relative path ../../@ to specify module

This is because Antrea uses a replace directive pointing to a local directory in one of our go.mod files: https://github.com/vmware-tanzu/antrea/blob/57dcaec0c561875dac6dee9bac72a66b9aefbab6/plugins/octant/go.mod#L14. This is a "trick" that enables us to ensure that plugins are built using the correct version of the "parent" Go module. And it works great for building.

When using golicense, I was able to provide an override ("../../": "Apache-2.0") in the config to handle that case. I'd like to be able to do something similar with lichen, but lichen applies unresolvableLicense config overrides after fetching modules and will error out early if a module cannot be fetched.

Would it be possible to provide a new exception type (e.g. moduleUnfetchable) in the config that would apply when a module cannot be fetched, instead of returning right away. Thanks!

Would it make sense to also include any NOTICE found in packages? This is something that go-licenses can do https://github.com/google/go-licenses/blob/5b654af5dcd3ef8090baaceae6009c20d75a87e8/save.go#L109

Would it be possible to create a new release tag? Now that support for Windows binaries was fixed (1d6c981). Thanks!

Hello and thank you for your work with lichen!

I’m currently in need of running lichen in environments where a substantial amount of "unresolvable licence" errors can happen (running it as part of a tool for other developers to use at my company). In my case returning an empty licence is a much better default than returning an error. And there might be others with similar needs so I suggest we add a config for this?

I have two different suggestions for additional configs:

A:

ignoreUnresolvableLicense: true
Would result in ´func ignoreUnresolvable()´ always returning true.

Or B:

unresolvableLicense:
 - regex: true 
 - path: "github.com/test/.*"

regex: true would result in unresolvableLicense's path beeing a regular expression.

Which of these suggestions would be best for lichen?

I can create a PR with this if you wan't to test it in practice, the additional code will be short.

Is there a way for this to support resolving modules from a vendor directory?

We have a private gitlab where we host our own go repos, and it seems, even with overrides or exceptions, I can't prevent the sum.golang.org lookup, which causes a failure -- thus preventing us from using this in CI. To be clear, our gitlab instance isn't accessible except on internal networks, so this lookup fails. See snippet for more details (I anonymized the URL, but I think you'll get the point).

$ (lichen -c lichen.yaml ./${PROJECT_NAME} | tee license_report.txt) || true
2021/07/22 15:04:44 failed to evaluate licenses: failed to fetch: exit status 1 (output: 
{
	"Path": "github.com/hashicorp/hcl",
	"Version": "v1.0.0",
	"Info": "/go/pkg/mod/cache/download/github.com/hashicorp/hcl/@v/v1.0.0.info",
	"GoMod": "/go/pkg/mod/cache/download/github.com/hashicorp/hcl/@v/v1.0.0.mod",
	"Zip": "/go/pkg/mod/cache/download/github.com/hashicorp/hcl/@v/v1.0.0.zip",
	"Dir": "/go/pkg/mod/github.com/hashicorp/[email protected]",
	"Sum": "h1:0Anlzjpi4vEasTeNFn2mLJgTSwt0+6sfsiTG8qcWGx4=",
	"GoModSum": "h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ="
}
{
	"Path": "github.com/spf13/afero",
	"Version": "v1.3.4",
	"Info": "/go/pkg/mod/cache/download/github.com/spf13/afero/@v/v1.3.4.info",
	"GoMod": "/go/pkg/mod/cache/download/github.com/spf13/afero/@v/v1.3.4.mod",
	"Zip": "/go/pkg/mod/cache/download/github.com/spf13/afero/@v/v1.3.4.zip",
	"Dir": "/go/pkg/mod/github.com/spf13/[email protected]",
	"Sum": "h1:8q6vk3hthlpb2SouZcnBVKboxWQWMDNF38bwholZrJc=",
	"GoModSum": "h1:Ai8FlHk4v/PARR026UzYexafAt9roJ7LcLMAmO6Z93I="
}
# ...
# truncated ...
# ...
{
	"Path": "gitlab.ourcompany.com/software/whatever/thing",
	"Version": "v0.0.0-20210610205813-eb9a997ab73f",
	"Error": "gitlab.ourcompany.com/software/whatever/[email protected]: verifying go.mod: gitlab.ourcompany.com/software/whatever/[email protected]/go.mod: reading https://sum.golang.org/lookup/gitlab.ourcompany.com/software/whatever/[email protected]: 410 Gone\n\tserver response: not found: gitlab.ourcompany.com/software/whatever/[email protected]: unrecognized import path \"gitlab.ourcompany.com/software/whatever/thing\": https fetch: Get \"https://gitlab.ourcompany.com/software/whatever/thing?go-get=1\": dial tcp: lookup gitlab.everactive.com on 8.8.8.8:53: no such host",
	"Info": "/go/pkg/mod/cache/download/gitlab.ourcompany.com/software/whatever/thing/@v/v0.0.0-20210610205813-eb9a997ab73f.info"
}

Not sure if this is really a bug or a feature request, so I'll leave that up to you, but I would imagine maybe adding some sort of config option to ignore modules that start with a certain URL?

ignoreURL: gitlab.ourcompany.com

Or making it so that override directives don't query sum.golang.org at all.

Lichen returns an unexpected error when there are no modules used in the project.

To reproduce it:

1. Create a simple main.go file:

package main

import "fmt"

func main() {
 fmt.Println("Hi!")
}

2. Create a simple go.mod file:

module my-module

go 1.15

3. Build it:

go build

4. Run lichen:

lichen ./my-module

Expected behavior:

• success status code 0
• no error messages

Current behavior:

• error status code 1
• error message: failed to evaluate licenses: failed to fetch: exit status 1 (output: go mod download: no modules specified (see 'go help mod download')

I get this error when scanning binaries built with Go 1.18 beta2.

2022/02/16 14:46:01 failed to evaluate licenses: unrecognised line: 	build	-compiler=gc

At least in some circumstances, go version -m with Go 1.18 on a binary built with Go 1.18 outputs the current module as a dep line instead of the previous mod line, i.e.

        dep     github.com/submariner-io/submariner     (devel) 

instead of

        mod     github.com/submariner-io/submariner     (devel) 

lichen then fails with

{
        "Path": "github.com/submariner-io/submariner",
        "Version": "(devel)",
        "Error": "github.com/submariner-io/submariner@(devel): invalid version: reading https://proxy.golang.org/github.com/submariner-io/submariner/@v/%28devel%29.info: 410 Gone\n\ts
erver response: not found: github.com/submariner-io/submariner@(devel): invalid version: unknown revision (devel)"
}

Hi,

Would it be possible to tag a new release? Installing lichen on a system with Go 1.18 now results in a binary which fails in certain circumstances (#15). It’s possible to build a binary from master of course but being able to install directly would be much more convenient.

Regards,

Stephen

I don't know why it's happening on fan2go

lichen /var/tmp/portage/sys-power/fan2go-0.6.2/image/usr/bin/fan2go
2022/05/07 05:08:29 failed to evaluate licenses: unrecognised line:     build   -compiler=gc