kamilchm / go2nix Goto Github PK

View Code? Open in Web Editor NEW

92.0 92.0 17.0 117 KB

Reproducible builds and development environment for Go

License: MIT License

Go 63.14% Nix 19.02% Shell 11.00% Roff 6.84%

go golang golang-tools nix nixpkgs

go2nix's Introduction

go2nix's People

Contributors

Stargazers

Watchers

Forkers

roblabla cmfwyp nixcloud chris-martin profpatsch fossabot nlewo cstrahan duog dingxiangfei2009 rowhit alyssais turbomack ehmry milahu isgasho

go2nix's Issues

How do you use this?

With the Haskell infrastructure, I check out a project and run cabal2nix . > projectname.nix.

So for Go, I tried the same thing. I cloned btcd, and then

$ nix-shell -p go2nix --command 'go2nix save'
2016/04/23 10:38:23 No GOPATH set, can't find dependencies

Okay, so it doesn't look in the present working directory by default?

$ env GOPATH=$PWD nix-shell -p go2nix --command 'go2nix save'
2016/04/23 10:39:19 Current dir /home/chris/git/btcd is outside of GOPATH(/home/chris/git/btcd). Can't get current package name

Is that a bug? It says a directory is outside of itself.

(I'm gussing this would be clear if I were already more familiar with building Go projects, but I haven't used Go before.)

Missing dependencies on Siad

When packaging Siad (https://github.com/NebulousLabs/Sia) I'm getting missing dependencies when I nix-build the result.

default.nix: https://github.com/NixOS/nixpkgs/compare/master...grahamc:sia?expand=1#diff-2ca016fb77cf0742c3396dc243dd0c6cR1 (modified to add subPackages = [ "siad" ];)
deps.nix: https://github.com/NixOS/nixpkgs/compare/master...grahamc:sia?expand=1#diff-5549d2234c56be4ae44c96a69f071d87R1
how I made these files: https://github.com/NixOS/nixpkgs/compare/master...grahamc:sia?expand=1#diff-1f8b3ed888bf7049648315e826cb5856R11

Interestingly they have a makefile: https://github.com/NebulousLabs/Sia/blob/master/Makefile but after the go get (without calling make) the resulting binary works fine. However, the nix package isn't correct and does not compile due to missing dependencies.

... is outside of GOPATH

i'm not really sure how to use the go2nix. i made it work with leaps some time ago but can't remember how i managed it. can you please advice?

preparations

in my project dir, as you describe in the 'brief' manual, i do:
export GOPATH=`pwd`

go build is working btw but i used go get github.com/fatih/structs to get the dependency

go2nix

[nix-shell:~/Desktop/projects/fablab/robin/github.com/qknight/selenium_crawler]$ go2nix save
2016/10/25 10:05:02 Current dir /home/joachim/Desktop/projects/fablab/robin/github.com/qknight/selenium_crawler is outside of GOPATH(/home/joachim/Desktop/projects/fablab/robin/github.com/qknight/selenium_crawler). Can't get current package name

glide support?

I'm currently building the Cerana tools with what is probably a bit of a terrible hack:
https://github.com/cerana/nixpkgs/blob/ceranaos/pkgs/os-specific/linux/cerana/default.nix#L21

We use glide to manage the dependencies, and I'm wondering if go2nix either already supports software doing vendoring with glide, if it's something that might come in the future, or if I should plan to stick with that hack for a while.

Thank you for all of your hard work making working with Go software in Nix as pleasant as possible!

Does the package need to be a git repository?

The readme seems to imply that the code that you're packaging up needs to be a git repository? Can it be brought in otherways? I may have just a directory under src, and that is not a git repository.

Fix circle CI builds

Circle builds are based on current stable where there's no buildGoPackage.goDeps and it's failing on master.

Error building

go get github.com/kamilchm/go2nix gives:

$GOPATH/src/github.com/kamilchm/go2nix/nix.go:109: undefined: Asset

edit: seems to build fine with nix though. as in I add pkgs.go2nix to environment.systemPackages

Cannot find branch of package

go2nix save

2019/08/22 02:07:41 cannot find package "github.com/bakape/thumbnailer/v2" in any of:
        /nix/store/q6b1ywhnvmrfd84azzqnrb4mchkihsvb-go-1.12.7/share/go/src/github.com/bakape/thumbnailer/v2 (from $GOROOT)
        /run/user/1000/tmp.pY8OMnWUi5/src/github.com/bakape/thumbnailer/v2 (from $GOPATH)

I'm not really familiar with how this works, so if more information is needed, I'll try to provide it.

Unclear what `go2nix save` is looking for

I'm trying to package the AWS SSM agent with go2nix and am running into some trouble. I'm following the instructions in the go2nix readme and have wrangled the project into building for me, but when I type go2nix save, I get

$ go2nix save
2017/02/06 16:18:25 no buildable Go source files in /Users/copumpkin/Sandbox/amazon-ssm-agent/vendor/src/github.com/aws/amazon-ssm-agent

Which is presumably because the Go sources are in a subdirectory. I cd into agent, which contains all the go sources, and in there go2nix save works just fine but produces an empty deps.nix. How do I convince go2nix to find my source files and my dependencies?

Cannot work out how to package rabbitmq-explorer

➜  work mkdir rabbitmq_explorer                                                                                                                                        ~/work
➜  work cd rabbitmq_explorer                                                                                                                                           ~/work
➜  rabbitmq_explorer nix-shell -p go dep                                                                                                             ~/work/rabbitmq_explorer

[nix-shell:~/work/rabbitmq_explorer]$ go get -v github.com/kbudde/rabbitmq_exporter
github.com/kbudde/rabbitmq_exporter (download)
github.com/Sirupsen/logrus (download)
Fetching https://golang.org/x/crypto/ssh/terminal?go-get=1
Parsing meta tags from https://golang.org/x/crypto/ssh/terminal?go-get=1 (status code 200)
get "golang.org/x/crypto/ssh/terminal": found meta tag get.metaImport{Prefix:"golang.org/x/crypto", VCS:"git", RepoRoot:"https://go.googlesource.com/crypto"} at https://golang.org/x/crypto/ssh/terminal?go-get=1
get "golang.org/x/crypto/ssh/terminal": verifying non-authoritative meta tag
Fetching https://golang.org/x/crypto?go-get=1
Parsing meta tags from https://golang.org/x/crypto?go-get=1 (status code 200)
golang.org/x/crypto (download)
Fetching https://golang.org/x/sys/unix?go-get=1
Parsing meta tags from https://golang.org/x/sys/unix?go-get=1 (status code 200)
get "golang.org/x/sys/unix": found meta tag get.metaImport{Prefix:"golang.org/x/sys", VCS:"git", RepoRoot:"https://go.googlesource.com/sys"} at https://golang.org/x/sys/unix?go-get=1
get "golang.org/x/sys/unix": verifying non-authoritative meta tag
Fetching https://golang.org/x/sys?go-get=1
Parsing meta tags from https://golang.org/x/sys?go-get=1 (status code 200)
golang.org/x/sys (download)
github.com/kbudde/gobert (download)
github.com/prometheus/client_golang (download)
github.com/beorn7/perks (download)
github.com/golang/protobuf (download)
github.com/prometheus/client_model (download)
github.com/prometheus/common (download)
github.com/matttproud/golang_protobuf_extensions (download)
github.com/prometheus/procfs (download)
golang.org/x/sys/unix
github.com/kbudde/gobert
github.com/beorn7/perks/quantile
github.com/golang/protobuf/proto
github.com/prometheus/common/internal/bitbucket.org/ww/goautoneg
github.com/prometheus/procfs/xfs
github.com/prometheus/common/model
github.com/prometheus/procfs
golang.org/x/crypto/ssh/terminal
github.com/Sirupsen/logrus
github.com/prometheus/client_model/go
github.com/matttproud/golang_protobuf_extensions/pbutil
github.com/prometheus/common/expfmt
github.com/prometheus/client_golang/prometheus
github.com/kbudde/rabbitmq_exporter

[nix-shell:~/work/rabbitmq_explorer]$ cd src/g
github.com/ golang.org/ 

[nix-shell:~/work/rabbitmq_explorer]$ cd src/github.com/kbudde/
gobert/            rabbitmq_exporter/ 

[nix-shell:~/work/rabbitmq_explorer]$ cd src/github.com/kbudde/rabbitmq_exporter/

[nix-shell:~/work/rabbitmq_explorer/src/github.com/kbudde/rabbitmq_exporter]$ go get

[nix-shell:~/work/rabbitmq_explorer/src/github.com/kbudde/rabbitmq_exporter]$ dep ensure -v
Gopkg.lock was already in sync with imports and Gopkg.toml
(1/18) Wrote github.com/beorn7/perks@master
(2/18) Wrote github.com/matttproud/[email protected]
(3/18) Wrote github.com/pkg/[email protected]
(4/18) Wrote github.com/cenk/[email protected]
(5/18) Wrote github.com/kylelemons/godebug@master
(6/18) Wrote github.com/kbudde/gobert@master
(7/18) Wrote golang.org/x/crypto@master
(8/18) Wrote github.com/prometheus/client_model@master
(9/18) Wrote golang.org/x/sys@master
(10/18) Wrote github.com/prometheus/procfs@master
(11/18) Wrote github.com/prometheus/common@master
(12/18) Wrote github.com/Sirupsen/logrus@master
(13/18) Wrote gopkg.in/ory-am/[email protected]
(14/18) Wrote golang.org/x/net@master
(15/18) Wrote github.com/prometheus/[email protected]
(16/18) Wrote github.com/streadway/amqp@master
(17/18) Wrote github.com/fsouza/[email protected]/go-1.4
(18/18) Wrote github.com/golang/protobuf@master

[nix-shell:~/work/rabbitmq_explorer/src/github.com/kbudde/rabbitmq_exporter]$ go2nix save

[nix-shell:~/work/rabbitmq_explorer/src/github.com/kbudde/rabbitmq_exporter]$ nix-build -E 'with import <nixpkgs> {}; callPackage ./. {}'
these derivations will be built:
  /nix/store/b4ri3na2mf0qhp41q0s2l77jfba2dqww-rabbitmq_exporter-unstable-2017-12-02.drv
these paths will be fetched (1.96 MiB download, 5.23 MiB unpacked):
  /nix/store/dk0hjq9hxxlwi301i5sql42fscmw2bs6-parallel-20170722
  /nix/store/qnw30iqd5zs2wld9nz0c46nrrpalfn1n-govers-20150109-3b5f175
  /nix/store/wp3k5x5iw9s18pabi5s4kkiy0c3pkvbf-govers-20150109-3b5f175-bin
fetching path ‘/nix/store/wp3k5x5iw9s18pabi5s4kkiy0c3pkvbf-govers-20150109-3b5f175-bin’...
fetching path ‘/nix/store/dk0hjq9hxxlwi301i5sql42fscmw2bs6-parallel-20170722’...

*** Downloading ‘http://cache.nixos.org/nar/0wmgswq2bqc7bp790h5vwc6qkhhwzmrggq78s3j3bnwk6nb6k418.nar.xz’ to ‘/nix/store/dk0hjq9hxxlwi301i5sql42fscmw2bs6-parallel-20170722’...

*** Downloading ‘http://cache.nixos.org/nar/1xsdl4hkj36rhspycsxb2y39hnicnwnlm56r3805yvqbqi2q3jbi.nar.xz’ to ‘/nix/store/wp3k5x5iw9s18pabi5s4kkiy0c3pkvbf-govers-20150109-3b5f175-bin’...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  780k  100  780k    0     0   780k      0  0:00:01  0:00:01 --:--:--  600k

fetching path ‘/nix/store/qnw30iqd5zs2wld9nz0c46nrrpalfn1n-govers-20150109-3b5f175’...
 87 1225k   87 1068k    0     0  1068k      0  0:00:01  0:00:01 --:--:--  762k
*** Downloading ‘http://cache.nixos.org/nar/1h42w323z56d6wwgxx2vriw7jnycnkndh7prg0101569bmb885zb.nar.xz’ to ‘/nix/store/qnw30iqd5zs2wld9nz0c46nrrpalfn1n-govers-20150109-3b5f175’...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1225k  100 1225k    0     0  1225k      0  0:00:01  0:00:01 --:--:--  808k

100  4080  100  4080    0     0   4080      0  0:00:01 --:--:--  0:00:01 28531

building path(s) ‘/nix/store/3wd6asnixh00lgm7iw6im8pkynlc9619-rabbitmq_exporter-unstable-2017-12-02-bin’, ‘/nix/store/8f1x0aknqkj8g0apzz0jgxvjjxbx08fh-rabbitmq_exporter-unstable-2017-12-02’
unpacking sources
unpacking source archive /nix/store/m05bz2bqfja0g5mdcnr9y9rbr0y22s5z-rabbitmq_exporter-8238c5d
source root is rabbitmq_exporter-8238c5d
patching sources
configuring
unpacking source archive /nix/store/0bx688c40a4jh06ng79g6si6c4hgx9gd-logrus-d682213
unpacking source archive /nix/store/vig8636xz89jj36725rdgi5jjw7n8n31-perks-4c0e845
unpacking source archive /nix/store/n3ma80dcyqqzli3b22lq8d6iq3w8zxd8-protobuf-1e59b77
unpacking source archive /nix/store/hw16868wxgj2ygnv1sff4r91l9ai4bv5-gobert-a6daecb
unpacking source archive /nix/store/3z30spv34fzy2dckaw65rzshhbdgn6cm-golang_protobuf_extensions-c12348c
unpacking source archive /nix/store/0rby1zkhs2bwxh1axvz851cy9jxaxlfh-client_golang-661e31b
unpacking source archive /nix/store/7lfwbx42rmjrl9n6m94xnf13mn62rx61-client_model-99fa1f4
unpacking source archive /nix/store/ifvvihp3s1pv1ik38q74lbk5vv4pj9py-common-2e54d0b
unpacking source archive /nix/store/6325fw09bz5j0a1ss8ydbaa13bq3243x-procfs-f98634e
unpacking source archive /nix/store/f03mmim3hqfbyxxm456wkfw42m7lrs3d-crypto-d585fd2
unpacking source archive /nix/store/wx7zzfdkiib6m4x2aj40944aaynyac0f-sys-571f7bb
building
go/src/github.com/kbudde/rabbitmq_exporter/testenv/rabbit.go:8:2: cannot find package "github.com/streadway/amqp" in any of:
	/nix/store/120g15mjvimcvlvz0525rz9cqbpkyjhz-go-1.9.2/share/go/src/github.com/streadway/amqp (from $GOROOT)
	/tmp/nix-build-rabbitmq_exporter-unstable-2017-12-02.drv-0/go/src/github.com/streadway/amqp (from $GOPATH)
	/nix/store/qnw30iqd5zs2wld9nz0c46nrrpalfn1n-govers-20150109-3b5f175/share/go/src/github.com/streadway/amqp
go/src/github.com/kbudde/rabbitmq_exporter/testenv/testenv.go:18:2: cannot find package "gopkg.in/ory-am/dockertest.v3" in any of:
	/nix/store/120g15mjvimcvlvz0525rz9cqbpkyjhz-go-1.9.2/share/go/src/gopkg.in/ory-am/dockertest.v3 (from $GOROOT)
	/tmp/nix-build-rabbitmq_exporter-unstable-2017-12-02.drv-0/go/src/gopkg.in/ory-am/dockertest.v3 (from $GOPATH)
	/nix/store/qnw30iqd5zs2wld9nz0c46nrrpalfn1n-govers-20150109-3b5f175/share/go/src/gopkg.in/ory-am/dockertest.v3
golang.org/x/sys/unix
github.com/kbudde/gobert
github.com/beorn7/perks/quantile
github.com/golang/protobuf/proto
github.com/prometheus/common/internal/bitbucket.org/ww/goautoneg
github.com/prometheus/common/model
github.com/prometheus/procfs/xfs
github.com/prometheus/procfs
golang.org/x/crypto/ssh/terminal
github.com/Sirupsen/logrus
github.com/prometheus/client_model/go
github.com/matttproud/golang_protobuf_extensions/pbutil
github.com/prometheus/common/expfmt
github.com/prometheus/client_golang/prometheus
github.com/kbudde/rabbitmq_exporter
builder for ‘/nix/store/b4ri3na2mf0qhp41q0s2l77jfba2dqww-rabbitmq_exporter-unstable-2017-12-02.drv’ failed with exit code 1
error: build of ‘/nix/store/b4ri3na2mf0qhp41q0s2l77jfba2dqww-rabbitmq_exporter-unstable-2017-12-02.drv’ failed

leaps packaging fail

i've tried multipe hours to get a deps.nix from leaps which i once even got working.

can you please advice how to approach the problem?

https://github.com/Jeffail/leaps

concept

nix-shell -p go2nix git go
mkdir /tmp/l
cd /tmp/l
export GOPATH=`pwd`:`pwd`/vendor/
go get github.com/jeffail/leaps/cmd/...
cd src/github.com/jeffail/leaps/cmd/leaps
go2nix save

deps.nix

# This file was generated by go2nix.
[
]

New release?

There have been bug fixes since the 1.1.1 release.

Crash with apparent JSON parse error

I get the following output when I try to run go2nix save on the go2nix repository, with the latest master:

2016/10/17 14:59:35 invalid character 'u' looking for beginning of object key string

I tried this on OS X and Ubuntu. On OS X, I installed from source (version is 1.2.0-devel). On Ubuntu, I installed from the 16.09 channel using Nix.

When I try this on NixOS, it does work.

Write go2nix version to generated deps file

CC: @grahamc
NixOS/nixpkgs#19341 (comment)

Add support for sub-packages

If you have a number of sub-packages that share a common repo, it can be very inconvenient to have to run go2nix save in each package separately.

Support for gdm Godeps files

See https://github.com/zimbatm/nixpkgs/blob/8a1ebe98f3cbf2229db13d6cc6100c0a89753e42/pkgs/servers/nosql/influxdb/gdm2nix.rb

Only imported used VCS's

At this moment, all VC's are imported in the default.nix. It would be nicer to only import the VCS that is actually used.

Help generating a package for Leanote

I'm unfamiliar with Go development but would like to create a Nix package for leanote.

From what I gather it's a web application using the Revel framework. Running the application involves installing the revel binary and then executing it, providing it the path to leanote:

revel run github.com/leanote/leanote

In this case, I'm not really sure if go2nix can generate a package for me. Or perhaps I need to generate two packages, one for revel and one for leanote. Either way, I'm confused how to create a package that would allow someone to ultimately run the command above (or some wrapped variation of it).

Any help would be greatly appreciated. Thank you.

Support --version

There is currently no way to determine the version of the command line tool. Supporting the --version option would fix this.

selector ‘go2nix’ matches no derivations

The command in the README isn't working for me:

code/nixpkgs > nix-env -i go2nix
error: selector ‘go2nix’ matches no derivations

code/nixpkgs > nixos-version
17.03.git.3d80ba4 (Gorilla)

Support for gb manifest files

For gb projects, we should be able to produce deps.nix directly from the manifest, right?

A question, though: How do you express in deps.nix packages that aren't based in a repo's root? For example: https://github.com/stellar/horizon/blob/94a9f35d93f33e365e3a154d6dce08b0e15df608/vendor/manifest#L78

nix-prefetch-git fails without NIX_REMOTE being set (nix 2.0)

When go2nix invokes nix-prefetch-git, it fails because nix can’t access the nix-daemon:

→ go2nix save
2018/04/10 17:56:06 Initialized empty Git repository in /tmp/git-checkout-tmp-2emNcsso/payment-processors/.git/
From file:///home/philip/.go/src/gitlab.techcultivation.org/sangha/payment-processors
 * branch            HEAD       -> FETCH_HEAD
Switched to a new branch 'fetchgit'
removing `.git'...
error: Nix database directory ‘/nix/var/nix/db’ is not writable: Permission denied
2018/04/10 17:56:06 Command /nix/store/sgzjh8gbvx2r9lsbzjycmi1n9gfgbn8a-nix-prefetch-git/bin/nix-prefetch-git [nix-prefetch-git --fetch-submodules file:///home/philip/.go/src/gitlab.techcultivation.org/sangha/payment-processors] failed: exit status 1

When I set NIX_REMOTE, it works:

→ env NIX_REMOTE=daemon go2nix save
→

This is quite strange, because nix 2.0 shouldn’t need a set NIX_REMOTE to work properly. Even stranger, when I run the failing nix-prefetch-git command directly in my shell, it works:

→ env | grep NIX
NIX_PROFILES=/run/current-system/sw /nix/var/nix/profiles/default /home/philip/.nix-profile /etc/profiles/per-user/philip
NIX_PATH=vuizvui=/home/philip/vuizvui:nixpkgs=/home/philip/nixpkgs:nixos-config=/nix/store/6dl6avhcagic7hyq8pd2a33xg153a7q1-katara-configuration.nix
NIX_USER_PROFILE_DIR=/nix/var/nix/profiles/per-user/philip
NIXPKGS_CONFIG=/etc/nix/nixpkgs-config.nix
NIX_GHC=/nix/store/7ki3psspnd8a5azy2yxliglsklgp322b-ghc-8.2.2-with-packages/bin/ghc
→ nix-prefetch-git --fetch-submodules file:///home/philip/.go/src/gitlab.techcultivation.org/sangha/payment-processors
Initialized empty Git repository in /tmp/git-checkout-tmp-iHjVchBE/payment-processors/.git/
… git stuff …
{
  "url": "file:///home/philip/.go/src/gitlab.techcultivation.org/sangha/payment-processors",
  "rev": "7f0adc67638e9b369519d85b9a59c776bd89ccc4",
  "date": "2018-03-12T10:58:19+01:00",
  "sha256": "149nqp1299llhb1xy0ljz6r3h77gxa9jyqzhx1hgkrghjlij1hzk",
  "fetchSubmodules": true
}

cc @aszlig

Gopath detection broken

☆  katara in ~/code/work/nixcloud/nixcloud-backend/src/nixcloud/backend
○ → go2nix save
2017/03/10 14:12:23 Current dir /home/philip/code/work/nixcloud/nixcloud-backend/src/nixcloud/backend is outside of GOPATH(/home/philip/code/work/nixcloud/nixcloud-backend:/run/user/1000/tmp.3UWHIm33Uo-go1.6-nixcloud-0.0.1:/nix/store/j0r6j7s2hzi1hb0fj3kwy4gbgym34w2k-go-1.6.2/share/go:/nix/store/f06y099h32rvx9blgh84nlkp8qslchfw-go1.6-govers-20150109-3b5f175/share/go:/nix/store/zjqn56sq9vhc8s991932qk4g2jakkdng-go1.6-captcha-9e95214/share/go:/nix/store/pb12mhbbcs1wfv4mdrpgbn0q414wd16p-go1.6-gomailv2-5ceb8e6/share/go:/nix/store/2xql0m3ssmb3yxy23x77whyk9mwmqhga-go1.6-gocraft-web-054fd62/share/go:/nix/store/xp30akdnmbs7lrb9b7r3i69mqnay18cz-go1.6-gorm-84c6b46/share/go:/nix/store/374c3vnh1i1fvv5lmwmvl3yfs6k6qwyz-go1.6-inflection-3272df6/share/go:/nix/store/mw4wrf42nl9cqj0c5sckdvf2jy79q5dr-go1.6-pq-11fc39a/share/go:/nix/store/wjm0mp3n41va0by73p4jg3ip4yyqndlp-go1.6-crypto-7b85b09/share/go:/nix/store/2m1lkyb745rahxgv28hyq2qpnvlz1gg5-go1.6-go2nix-0.1.0/share/go:/nix/store/ldp0rdpr0ixskwkz81846pmqhacz2m0f-leaps). Can't get current package name

> Current dir /home/philip/code/work/nixcloud/nixcloud-backend/src/nixcloud/backend

> outside of GOPATH(/home/philip/code/work/nixcloud/nixcloud-backend:

I call bullshit.

go2nix save takes too long?

Doing go2nix save for github.com/jeffail/leaps/cmd/leaps like described in #42 (comment) takes:

real	6m3.184s
user	0m14.674s
sys	0m2.782s

Is there something unwanted happens in go2nix code?

Undeclared dependency on nix-prefetch-git

I tried the go2nix packaged in the current unstable nixpkgs channel.

It did not terminate, and straceing the invocation showed that it tried to run nix-prefetch-get, which is not in my $PATH by default.

Could you add nix-prefetch-get as a runtime dependency to go2nix?

problem with packaging

i've been using this go2nix and created a deps.nix which i converted into a deps.json manually.

the package

default.nix

{ stdenv, buildGoPackage, fetchgit, fetchhg, fetchbzr, fetchsvn }:

buildGoPackage rec {
  name = "leaps-${version}";
  version = "20160626-${stdenv.lib.strings.substring 0 7 rev}";
  rev = "5cf7328a8c498041d2a887e89f22f138498f4621";

  goPackagePath = "github.com/jeffail/leaps";

  src = fetchgit {
    inherit rev;
    url = "https://github.com/jeffail/leaps";
    sha256 = "1qbgz48x9yi0w9yz39zsnnhx5nx2xmrns9v8hx28jah2bvag6sq7";
    fetchSubmodules = false;  
  };

  goDeps = ./deps.json;

  preConfigure = ''
    rm -Rf src
  '';

  # TODO: add metadata https://nixos.org/nixpkgs/manual/#sec-standard-meta-attributes
  meta = {
  };
}

deps.json

[
  {
    "goPackagePath" : "github.com/amir/raidman",
    "fetch": {
      "type": "git",
      "url": "https://github.com/amir/raidman",
      "rev":  "91c20f3f475cab75bb40ad7951d9bbdde357ade7",
      "sha256": "0pkqy5hzjkk04wj1ljq8jsyla358ilxi4lkmvkk73b3dh2wcqvpp"
    }
  },
  {
    "goPackagePath" : "github.com/elazarl/go-bindata-assetfs",
    "fetch" : {
      "type": "git",
      "url": "https://github.com/elazarl/go-bindata-assetfs",
      "rev": "57eb5e1fc594ad4b0b1dbea7b286d299e0cb43c2",
      "sha256": "1za29pa15y2xsa1lza97jlkax9qj93ks4a2j58xzmay6rczfkb9i"
    }
  },
  {
   "goPackagePath": "github.com/garyburd/redigo",
    "fetch": {
      "type": "git",
       "url": "https://github.com/garyburd/redigo",
       "rev": "8873b2f1995f59d4bcdd2b0dc9858e2cb9bf0c13",
       "sha256": "1lzhb99pcwwf5ddcs0bw00fwf9m1d0k7b92fqz2a01jlij4pm5l2"
    }
  },
  {
   "goPackagePath": "github.com/go-sql-driver/mysql",
    "fetch": {
      "type": "git",
       "url": "https://github.com/go-sql-driver/mysql",
       "rev": "7ebe0a500653eeb1859664bed5e48dec1e164e73",
       "sha256": "1gyan3lyn2j00di9haq7zm3zcwckn922iigx3fvml6s2bsp6ljas"
    }
  },
  {
   "goPackagePath": "github.com/golang/protobuf",
    "fetch": {
      "type": "git",
       "url": "https://github.com/golang/protobuf",
       "rev": "bf531ff1a004f24ee53329dfd5ce0b41bfdc17df",
       "sha256": "10lnvmq28jp2wk1xc32mdk4745lal2bmdvbjirckb9wlv07zzzf0"
    }
  },
  {
   "goPackagePath": "github.com/jeffail/gabs",
    "fetch": {
      "type": "git",
       "url": "https://github.com/jeffail/gabs",
       "rev": "ee1575a53249b51d636e62464ca43a13030afdb5",
       "sha256": "0svv57193n8m86r7v7n0y9lny0p6nzr7xvz98va87h00mg146351"
    }
  },
  {
   "goPackagePath": "github.com/jeffail/util",
    "fetch": {
      "type": "git",
       "url": "https://github.com/jeffail/util",
       "rev": "48ada8ff9fcae546b5986f066720daa9033ad523",
       "sha256": "0k8zz7gdv4hb691fdyb5mhlixppcq8x4ny84fanflypnv258a3i0"
    }
  },
  {
   "goPackagePath": "github.com/lib/pq",
    "fetch": {
      "type": "git",
       "url": "https://github.com/lib/pq",
       "rev": "3cd0097429be7d611bb644ef85b42bfb102ceea4",
       "sha256": "1q7qfzyfgjk6rvid548r43fi4jhvsh4dhfvfjbp2pz4xqsvpsm7a"
    }
  },
  {
   "goPackagePath": "github.com/satori/go.uuid",
    "fetch": {
      "type": "git",
       "url": "https://github.com/satori/go.uuid",
       "rev": "f9ab0dce87d815821e221626b772e3475a0d2749",
       "sha256": "0z18j6zxq9kw4lgcpmhh3k7jrb9gy1lx252xz5qhs4ywi9w77xwi"
    }
  },
  {
   "goPackagePath": "github.com/jeffail/leaps",
    "fetch": {
      "type": "git",
       "url": "https://github.com/jeffail/leaps",
       "rev": "5cf7328a8c498041d2a887e89f22f138498f4621",
       "sha256": "1qbgz48x9yi0w9yz39zsnnhx5nx2xmrns9v8hx28jah2bvag6sq7"
    }
  },
  {
   "goPackagePath": "golang.org/x/net",
    "fetch": {
      "type": "git",
       "url": "https://go.googlesource.com/net",
       "rev": "07b51741c1d6423d4a6abab1c49940ec09cb1aaf",
       "sha256": "12lvdj0k2gww4hw5f79qb9yswqpy4i3bgv1likmf3mllgdxfm20w"
    }
  }
]

the error

nix-env -f /home/joachim/Desktop/projects/nixos/nixpkgs/default.nix -I nixpkgs=/home/joachim/Desktop/projects/nixos/nixpkgs -iA leaps
installing ‘go1.6-leaps-20160626-5cf7328’
these derivations will be built:
  /nix/store/yishlw4wwp937zkb0vzd39qqmw7l10w5-go1.6-leaps-20160626-5cf7328.drv
building path(s) ‘/nix/store/fc1dn5fv2jiix62zfhm7d0n4wibmv8q8-go1.6-leaps-20160626-5cf7328’, ‘/nix/store/lwrndn6hyi9bl2n900v1s671na2bcwz9-go1.6-leaps-20160626-5cf7328-bin’
unpacking sources
unpacking source archive /nix/store/27vc3vfkb7cjpjjnbdni6bmajkbbnf1a-leaps-5cf7328
source root is leaps-5cf7328
patching sources
configuring
grep: Invalid range end
unpacking source archive /nix/store/kwf6af210nhpipxki0hffz52zsc4rj4v-raidman-91c20f3
unpacking source archive /nix/store/8i5kw5i82r10psg9fni49sdh736h7ag7-go-bindata-assetfs-57eb5e1
unpacking source archive /nix/store/l63kp458n182fymdc8w5wl3s41hw9bpc-redigo-8873b2f
unpacking source archive /nix/store/wcw7vfm7abz3bbmsyrwr392vkg6p6p7p-mysql-7ebe0a5
unpacking source archive /nix/store/dmnbwirpnbkaxb1gxr4z2j27iix70sq2-protobuf-bf531ff
unpacking source archive /nix/store/j0ipypgkbkfm0p9mri71w449jhjf2cvx-gabs-ee1575a
unpacking source archive /nix/store/cpk6ssqvb44jx6kg5vcp231kc0fkj89k-util-48ada8f
unpacking source archive /nix/store/0g6yrisq8dsid473ygkpgw7ywlvgzd3g-pq-3cd0097
unpacking source archive /nix/store/mmzrswsnnsg65i26m86395nrling058g-go.uuid-f9ab0dc
unpacking source archive /nix/store/27vc3vfkb7cjpjjnbdni6bmajkbbnf1a-leaps-5cf7328
unpacking source archive /nix/store/1ki1728win67jg7jj3rqiww9nxqv8gkb-net-07b5174
building
github.com/jeffail/leaps/vendor/github.com/elazarl/go-bindata-assetfs
github.com/jeffail/leaps/vendor/github.com/garyburd/redigo/internal
github.com/jeffail/leaps/vendor/github.com/jeffail/util/log
github.com/jeffail/leaps/vendor/github.com/go-sql-driver/mysql
github.com/jeffail/leaps/vendor/github.com/garyburd/redigo/redis
github.com/jeffail/leaps/vendor/github.com/satori/go.uuid
github.com/jeffail/leaps/vendor/github.com/lib/pq/oid
github.com/jeffail/leaps/vendor/github.com/lib/pq
github.com/jeffail/leaps/lib/util
github.com/jeffail/leaps/vendor/github.com/golang/protobuf/proto
github.com/jeffail/leaps/lib/acl
golang.org/x/net/proxy
github.com/jeffail/leaps/vendor/github.com/jeffail/gabs
golang.org/x/net/websocket
github.com/jeffail/leaps/lib/store
github.com/jeffail/leaps/vendor/github.com/amir/raidman/proto
github.com/jeffail/leaps/vendor/github.com/amir/raidman
github.com/jeffail/leaps/vendor/github.com/jeffail/util/metrics
github.com/jeffail/leaps/lib/binder
github.com/jeffail/leaps/lib/curator
github.com/jeffail/leaps/lib/http
github.com/jeffail/leaps/cmd/leaps
github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/elazarl/go-bindata-assetfs
github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log
github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/golang/protobuf/proto
github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/gabs
github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/amir/raidman/proto
github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/amir/raidman
github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/metrics
github.com/jeffail/leaps/leaps-5cf7328/cmd/leaps
# github.com/jeffail/leaps/leaps-5cf7328/cmd/leaps
go/src/github.com/jeffail/leaps/leaps-5cf7328/cmd/leaps/leaps.go:114: cannot use logger (type "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular) as type "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular in argument to acl.NewFileExists:
        "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular does not implement "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular (wrong type for NewModule method)
                have NewModule(string) "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular
                want NewModule(string) "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular
go/src/github.com/jeffail/leaps/leaps-5cf7328/cmd/leaps/leaps.go:118: cannot use logger (type "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular) as type "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular in argument to curator.New:
        "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular does not implement "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular (wrong type for NewModule method)
                have NewModule(string) "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular
                want NewModule(string) "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular
go/src/github.com/jeffail/leaps/leaps-5cf7328/cmd/leaps/leaps.go:166: cannot use logger (type "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular) as type "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular in argument to "github.com/jeffail/leaps/lib/http".WebsocketHandler:
        "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular does not implement "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular (wrong type for NewModule method)
                have NewModule(string) "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular
                want NewModule(string) "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular
github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/garyburd/redigo/internal
github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/garyburd/redigo/redis
github.com/jeffail/leaps/leaps-5cf7328/lib/acl
github.com/jeffail/leaps/leaps-5cf7328/lib/binder
github.com/jeffail/leaps/leaps-5cf7328/lib/curator
# github.com/jeffail/leaps/leaps-5cf7328/lib/curator
go/src/github.com/jeffail/leaps/leaps-5cf7328/lib/curator/curator.go:256: cannot use c.log (type "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular) as type "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular in argument to binder.New:
        "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular does not implement "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular (wrong type for NewModule method)
                have NewModule(string) "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular
                want NewModule(string) "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular
go/src/github.com/jeffail/leaps/leaps-5cf7328/lib/curator/curator.go:299: cannot use c.log (type "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular) as type "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular in argument to binder.New:
        "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular does not implement "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular (wrong type for NewModule method)
                have NewModule(string) "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular
                want NewModule(string) "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular
go/src/github.com/jeffail/leaps/leaps-5cf7328/lib/curator/curator.go:340: cannot use c.log (type "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular) as type "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular in argument to binder.New:
        "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular does not implement "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular (wrong type for NewModule method)
                have NewModule(string) "github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/jeffail/util/log".Modular
                want NewModule(string) "github.com/jeffail/leaps/vendor/github.com/jeffail/util/log".Modular
github.com/jeffail/leaps/leaps-5cf7328/lib/http
github.com/jeffail/leaps/leaps-5cf7328/lib/register
github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/go-sql-driver/mysql
github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/lib/pq/oid
github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/lib/pq
github.com/jeffail/leaps/leaps-5cf7328/lib/store
github.com/jeffail/leaps/leaps-5cf7328/vendor/github.com/satori/go.uuid
github.com/jeffail/leaps/leaps-5cf7328/lib/util
github.com/jeffail/leaps/lib/register
builder for ‘/nix/store/yishlw4wwp937zkb0vzd39qqmw7l10w5-go1.6-leaps-20160626-5cf7328.drv’ failed with exit code 2
error: build of ‘/nix/store/yishlw4wwp937zkb0vzd39qqmw7l10w5-go1.6-leaps-20160626-5cf7328.drv’ failed

clue

according to golang on irc this might be because the go-modules/generic/default.nix applies various vendor patches i don't understand yet.

do you have a clue what has gone wrong?

Add flag to configure target file.

At this moment, it overwrites the default.nix in a directory. This might not be what users expect. I've been bitten by this twice :)

Unable to find vendored packages

I have some tools for which I'm overriding golang.org/x/crypto/openpgp inside my local codebase (with a directory src/mypackage/vendor/golang.org/x/crypto/openpgp).

go build works correctly, but go2nix save fails with cannot find package "golang.org/x/crypto/openpgp".

A symlink (to mypackage/vendor/golang.org from src/golang.org) works around the issue -- but is this something that should be needed?

version naming conventions

It would be cool, if go2nix would use the naming policy,
we use in nixpkgs for unstable packages:

name = "awesomeapp-unstable-${version}";
version = "<year>-<month>-<day>";

How to generate expressions for Hugo?

I tried to generate the nix expression for Hugo without success. I ran

nix-env -p go git go2nix
mkdir hugobuild
cd hugobuild
export GOPATH=`pwd`
go get -v github.com/gohugoio/hugo
cd src/github.com/gohugoio/hugo
go2nix save

However, I dont't think that all of the dependencies are listed in deps.nix. It seems like the dependencies might be stored in vendor/vendor.json, but I'm not familiar with go at all. How do I generate the nix expressions for Hugo?

Mention dep2nix

https://github.com/nixcloud/dep2nix

go2nix probably can't be called in a builder because nix-prefetch-git wants access to the store

As far as I can tell, go2nix save only uses nix-prefetch-git to get a hash for some things:

go2nix/hash.go

Line 16 in 52270bd

func calculateHash(url, pathType string) (hash string) {

It doesn't actually need the files to end up in the store? So it would be nice if it would just use nix-hash directly and that would mean it would be possible to run in a nix sandbox I think?

Edit: or be able to call nix-prefetch-git without needing to poke the store.

The actual error in question: Command /nix/store/8adaygqps8a6vivrwb2d5l57lz5gnxwr-nix-prefetch-git/bin/nix-prefetch-git [nix-prefetch-git --fetch-submodules file:///build/src/the.domain/go/some-project] failed: exit status 1

use filepath.Join instead of concatenating strings

go2nix/cli.go

Line 57 in df946b0

if strings.HasPrefix(currDir, goPathDir+"/src/") {

This will cause an error if GOPATH=~/go/ (the trailing slash).

Improve documentation on where to run go2nix

I was trying to update Telegraf and it took me hours to find out that you had to run go2nix from
$GHOME/src/github.com/influxdata/telegraf/cmd/telegraf. Running it in $GHOME/src/github.com/influxdata/telegraf/ results in an empty deps.nix. I never came to my mind that the path could be wrong. So maybe the documentation could hint how to find the actual path in those circumstances. Not knowing Go at all I don't really see the pattern there.

Thanks.

Generate fetchFromGitHub instead of fetchgit for github repositories

fetchFromGitHub should be used for all packages hosted on github.

NixOS/nixpkgs#17254 (comment)

go2nix seems overly greedy

Sometimes go2nix requires additional packages beyond those required by go build to actually build. For example, if you are packaging coredns, a go get -d ./ is sufficient to fetch all its dependancies for building their main package. However, running go2nix save requires some other packages before it can successfully compile the Nix expressions.

I ran into this a number of times while packaging for Nix, I ended up creating a script that downloads these extra requirements while go2nix saveing:-

#!/bin/sh

while true; do
    missing_package=$(go2nix save 2>&1 | awk -F'"' '{print $2}')
    [[ -z $missing_package ]] && exit 0
    go get -d -v $missing_package
    echo "Finished downloading $missing_package"
done

Request a better error reporting

go2nix/save.go

Line 59 in 62871dd

return fmt.Errorf("Can't create package for: %v", dep)

Can we also print the err here?

how to use the default.nix/deps.nix?

i've managed to create the files but don't have a clue how to use them.

any advice how to use them?

nix-build

nix-build
error: cannot auto-call a function that has an argument without a default value (‘stdenv’)

nix-shell

nix-shell
error: cannot auto-call a function that has an argument without a default value (‘stdenv’)

default.nix

# This file was generated by go2nix.
{ stdenv, buildGoPackage, fetchgit, fetchhg, fetchbzr, fetchsvn }:

buildGoPackage rec {
  name = "robin-${version}";
  version = "20161024-${stdenv.lib.strings.substring 0 7 rev}";
  rev = "6159f49025fd5500e5c2cf8ceeca4295e72c1de5";

  goPackagePath = "/home/joachim/Desktop/projects/fablab/robin";

  src = fetchgit {
    inherit rev;
    url = "ssh://[email protected]/qknight/selenium_crawler.git";
    sha256 = "1ggqljhlgy6hmq15gzip4m3f6b0q5lig39l2z8xawfkafwqz36z2";
  };

  goDeps = import ./deps.nix { inherit fetchgit fetchhg fetchbzr fetchsvn; };

  # TODO: add metadata https://nixos.org/nixpkgs/manual/#sec-standard-meta-attributes
  meta = {
  };
}

deps.nix

# This file was generated by go2nix.
{ fetchgit, fetchhg, fetchbzr, fetchsvn }:
[
  {
    goPackagePath = "github.com/fatih/structs";
    src = fetchgit {
      url = "https://github.com/fatih/structs";
      rev = "dc3312cb1a4513a366c4c9e622ad55c32df12ed3";
      sha256 = "0wgm6shjf6pzapqphs576dv7rnajgv580rlp0n08zbg6fxf544cd";
    };
  }
  {
    goPackagePath = "github.com/influxdata/influxdb";
    src = fetchgit {
      url = "https://github.com/influxdata/influxdb";
      rev = "6fa145943a9723f9660586450f4cdcf72a801816";
      sha256 = "14ggx1als2hz0227xlps8klhn5s478kczqx6i6l66pxidmqz1d61";
    };
  }
]

kamilchm / go2nix Goto Github PK

go2nix's Introduction

go2nix's People

Contributors

Stargazers

Watchers

Forkers

go2nix's Issues

preparations

go2nix

concept

deps.nix

the package

default.nix

deps.json

the error

clue

nix-build

nix-shell

default.nix

deps.nix

Recommend Projects

Recommend Topics

Recommend Org