rogpeppe / gohack Goto Github PK
View Code? Open in Web Editor NEWMake temporary edits to your Go module dependencies
License: BSD 3-Clause "New" or "Revised" License
Make temporary edits to your Go module dependencies
License: BSD 3-Clause "New" or "Revised" License
We want to be able to set GOHACK to a relative dir, say ./
. But this is not possible at the moment as the initial ./
is removed in the replace directive, creating a broken replacement line for go:
replacement module without version must be directory path (rooted or starting with ./ or ../)
We can work around the problem by manually prefixing the replacement with ./
.
We like the relative path use-case, because it allows enhanced interop in WSL, when the path is on a shared Windows mount (/mnt/c/...), Windows go tools can interpret the same (relative) path as the WSL go tools. With absolute paths, Windows tools obviously cannot interpret the /mnt/c/...
mount.
This is somewhat related to #27.
i want to place my local change somewhere other than $HOME/gohack, so i create a linked file under it.
it reports error already exists; not overwriting
when running gohack get
.
could linked file be treated like a real directory?
Currently the undo command purports to recognize the -rm
and -f
flags but these are ignored.
Just firing in an issue as a reminder more than anything:
gohack
, that it's a playground for ideas around how to "hack" on dependencies etcWould it be reasonable for gohack
to support adding a module to a workspace (if a go.work
file is present) rather than adding a replace to the go.mod
file?
I am getting error
$ gohack get -vcs gopkg.in/danilopolani
cannot get module info: go list -m: can't compute 'all' using the vendor directory
(Use -mod=mod or -mod=readonly to bypass.)
I know the purpose of ~/gohack
, but I'm not a big fan of tools that add stuff to my home directory by default.
Perhaps we could use $GOPATH/hack
or something else instead. Don't have a clear answer to this one.
One of the more frequent comments from those moving from a GOPATH setup to modules is that they lose the ability to know where a given module dependency M is on disk. M is likely a module that they "own" / contribute to, and hence want to make changes and upstream them (frequently). The read-only module cache makes sense, and requiring a step to replace
away from that remains sensible.
But I wonder whether gohack
can do more to help here.
The directory $HOME/gohack
is intentionally named so that it be considered a sort of throw-away area.
One of the early ideas we toyed with was "namespaced" hacks. But we settled on a sensible default for GOHACK
and support for "namespaces" can be achieved by GOHACK=$HOME/mywork gohack $module
.
That said, perhaps for this more common use case of "use the M on disk in its usual location" it makes sense to drive this with some sort of config.
# some sort of config for gohack such that well-known modules
# "hack" to a custom location. e.g. I might configure (a one-off operation)
# github.com/myitcv/gobin to be in $HOME/work/gobin
#
$ gohack config github.com/myitcv/gobin $HOME/work/gobin
# Then everywhere I need to work on github.com/myitcv/gobin:
#
$ gohack github.com/myitcv/gobin
github.com/myitcv/gobin => /home/gopher/work/gobin
I find myself popping back and forth a lot between working on my current project and working on its dependencies.
The most painful part of this is currently when I've decided I am happy with a change in a dependency and want to use it immediately in my project. This requires:
git rev-parse head
of dependency to find version to usego get pkgpath@version
in project to update to latest versionreplace
directive in project (or remove it; I comment out to make it easier to switch back into hack mode)1 will always need to be done manually, but 2-4 could be automated. gohack currently only does 4.
How do you feel about adding 2 and 3 to gohack? Ideas: gohack promote
or gohack undo -get
or gohack undo -promote
.
(I'm starting to lean towards building another tool that most closely matches my current workflow, in which I am actively working a lot on a dependency or two, but I'd like to first investigate whether I can achieve a happy union with gohack.)
When there are no currently gohacked modules, gohack fails:
% gohack -u
failed to remove go.mod replacements: go mod edit: no flags specified (see 'go help mod edit').
error: [
{/home/rog/src/gohack/main.go:302: failed to remove go.mod replacements}
{/home/rog/src/gohack/exec.go:28: go mod edit: no flags specified (see 'go help mod edit').}
]
It should probably do nothing, or say "no current replacements".
It would be useful to be able to print the gohack directory for a module
without actually making any changes.
Perhaps:
gohack -p modulepath
would print the directory for modulepath.
With no arguments, -p could print the directories for all the modules in the current module that have gohack directories.
I think this makes sense; based on an actual scenario.
A module M
I was working on had a dependency on D1
. But I'd already forked D1
to D1'
. Hence my go.mod
already had a replace from D1 => D1'
.
So my desire to hack on D1
translates to needing to hack on D1'
. And hence gohack
should be following replace directives to work out what should ultimately be hacked on. Instead I got:
"github.com/shurcooL/vfsgen" is already replaced; will not override replace statement in go.mod
all modules failed; not replacing anything
error: [
{/home/myitcv/gostuff/src/gopkg.in/errgo.v2/fmt/errors/alias.go:15: all modules failed; not replacing anything}
]
I have the .gitconfig workaround for bitbucket so I can properly get private repositories on bitbucket.
[url "[email protected]:"]
insteadOf = https://bitbucket.org/
and 'gohack get bitbucket.org/xxxx' works, but I'm seeing some issue when using 'gohack get -vcs bitbucket.org/xxxx'
$ ~/go/bin/gohack get -vcs bitbucket.org/yyy/xxx-go
cannot update VCS dir for bitbucket.org/yyy/xxx-go: cannot get info: cannot find module root: https://api.bitbucket.org/2.0/repositories/yyy/xxx-go?fields=scm: 403 Forbidden
Probably go
, golang
, vgo
, modules
, vendor
(?).
Something that makes gohack more discoverable over the github.
I would submit them, but there is no way to do so. :)
When doing undo in the folder <somepath>
, gohack should save what was undone to the file $HOME/gohack/<somepath>
or to the file .gohack-redo
. Command gohack redo
should undo the undoing, so basically turn back what was removed (and delete redo file).
Use case in my mind is the following:
gohack undo
gohack redo
.gohack-redo
file in the .gitignore
to not accidentally commit itThat would be particulary convenient for multi-repo applications, which has a shared codebase in a separate repos, because allows working on the cross-repo feature (/service1 /service2 and /lib1 /lib2 folders cloned under one root), while keeping CI builds intact (which only have /service1 cloned).
If we try to replace two modules in the same repository, the current gohack scheme
of checking out to a single global root won't work well because the two modules
may clash (for example they may well be checked out at different versions).
One possibility might be to detect when there's a clash and use a different
name for the directory that holds the VCS directory. The problem with that
is that names become unpredictable.
when run
gohack github.com/labstack/echo
it show this error message:
creating github.com/labstack/[email protected]+incompatible
cannot update VCS dir for github.com/labstack/echo: error: pathspec 'v3.2.1+incompatible' did not match any file(s) known to git.
all modules failed; not replacing anything
When a go.mod file already contains a replacement of one module by another (not a directory), gohack get -vcs
checks out the original repository, not the current replacement. Also, gohack get
(without -vcs
) does check out the correct code, but puts it in the directory for the original module, not the replacement module, which is probably wrong.
For example, given this go.mod
:
module example.com/foo
require gopkg.in/errgo.v2 v2.0.0
replace gopkg.in/errgo.v2 => github.com/rogpeppe/test2/arble v0.0.0-20181008213029-f6022c873160
If we run gohack get -vcs gopkg.in/errgo.v2
, it checks out gopkg.in/errgo.v2
to $GOHACK/gopkg.in/errgo.v2
, where we actually want it to check out github.com/rogpeppe/test2/arble
to $GOHACK/github.com/rogpeppe/test2/arble
and end up with a go.mod file like this:
module example.com/foo
require gopkg.in/errgo.v2 v2.0.0
replace gopkg.in/errgo.v2 => /home/rog/gohack/github.com/rogpeppe/test2/arble // was gopkg.in/errgo.v2 => github.com/rogpeppe/test2/arble v0.0.0-20181008213029-f6022c873160
I'm not entirely sure this belongs in gohack
, but given the -vcs
flag gohack
makes answering it possible. So raising here for discussion at least.
I'm trying to answer the following question:
Given commit
$commit
, are we currently using a version of module$m
that that has$commit
as an ancestor?
The answer is in effect given by this sequence of commands:
GOHACK=$(mktemp -d) gohack get -vcs golang.org/x/tools
pushd $(go list -m -f {{.Replace.Dir}} golang.org/x/tools)
answer=$(git merge-base --is-ancestor $commit HEAD && echo "yes" || echo "no")
popd
gohack undo golang.org/x/tools
Does it make sense to provide a gohack
command that answers this question?
When we've previously checked out a version of a dependency in a gohack module dir and we want to keep that version instead of switching to the version specified in the go.mod file, it would be nice to have a flag to do that instead of running gohack and then switching back.
I haven't tried gohack yet, but I am curious about this scenario which I have encountered when doing a similar thing manually:
I have two separate projects (let's call them myProjectA
and myProjectB
), each of which happen to use the github.com/jmoiron/sqlx
package.
For project myProjectA
, I discover a need to modify function f
of sqlx
, so I use gohack to get a mutable copy of the repo, and make my changes.
For project myProjectB
, I want to use the original unmodified version of sqlx
, but I want to place a log.Printf(...)
statement inside function g
of sqlx
. If I were to use gohack to replace sqlx
in this project, my understanding is that project myProjectB
would begin using the same copy of sqlx
as project myProjectA
.
Is my reasoning correct?
If so, a way to avoid this seems to be to place gohack'd packages inside, say, myProjectA/.gohack/github.com/jmoiron/sqlx
. This may also resolve #5?
I suppose the main issues with this are:
.gohack
directory would most likely be included in .gitignore
This is how we can reproduce the issue:
$ git clone https://github.com/rogpeppe/go-internal.git
$ gohack get gopkg.in/errgo.v2
$ gohack undo
$ git status
On branch master
Your branch is up to date with 'origin/master'.
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git restore <file>..." to discard changes in working directory)
modified: go.mod
no changes added to commit (use "git add" and/or "git commit -a")
git diff
produces:
diff --git a/go.mod b/go.mod
index 1c11744..e980eb2 100644
--- a/go.mod
+++ b/go.mod
@@ -3,3 +3,5 @@ module github.com/rogpeppe/go-internal
go 1.11
require gopkg.in/errgo.v2 v2.1.0
+
+
What I expected is that it there should be no diff, i.e.
$ git status
On branch master
Your branch is up to date with 'origin/master'.
nothing to commit, working tree clean
Looking at the Go's source code
https://github.com/golang/go/blob/26154f31ad6c801d8bad5ef58df1e9263c6beec7/src/cmd/go/internal/modcmd/edit.go#L209
maybe we could include modf.Cleanup()
before modf.Format()
in
Line 91 in 03d2ff3
Created PR #69 for this. π
Just installed gohack. Trying first hack I get:
open /var/tmp/rayj/gocode/src/github.com/lindenlab/extraction-personas/go.mod: permission denied
The repo I'm trying to hack on did get created in $HOME/gohack - it appears to be failing just opening the file. But:
ls -l go.mod
-rw-r--r-- 1 root root 3075 Nov 8 00:33 go.mod
So why is permission denied?
I think that, for this kind of tool, the best approach would be shell-like tests. For example:
foo/bar
dependencygo build
to update go.mod
gohack foo/bar
gohack -u foo/bar
Hello.
Thought gohack was a great idea but my gohack'ed directory is missing .git. This makes it quite a bit less useful for hacking on things as I'd have to manually transfer the changes back to a version controlled directory.
Am I missing something in terms of usage? go 1.12.1 here, macOS.
TODO:
As the amount of functionality grows, it seems like we should consider having subcommands.
A possible set of commands:
gohack get [-vcs] [-u] [-f] [module...]
Get gets the modules at the current version and adds replace statements to the go.mod file if they're not already replaced.
If the -u
flag is provided, the source code will also be updated to the current version if it's clean.
If the -f
flag is provided with -u
, the source code will be updated even if it's not clean.
If the -vcs
flag is provided, it also checks out VCS information for the modules. If the modules were already gohacked in non-VCS mode, gohack switches them to VCS mode, preserving any changes made (this might result in the directory moving).
With no module arguments and the -u
flag, it will try to update all currently gohacked modules.
gohack diff module
Diff prints (in git style) changes that have been made to the module since it was checked out.
gohack rm [-f] module...
Rm removes the gohack directory if it is clean and then runs gohack undo
. If the -f
flag is provided, the directory is removed even if it's not clean.
gohack undo [module...]
Undo removes the replace statements for the modules. If no modules are provided, it will undo all gohack replace statements. The gohack module directories are unaffected.
gohack dir [-vcs] [module...]
Dir prints the gohack module directory names for the given modules. If no modules are given, all the currently gohacked module directories are printed. If the -vcs
flag is provided, the directory to be used in VCS mode is printed. Unlike the other subcommands, the modules don't need to be referenced by the current module.
Particularly to check that go test ./...
is happy, including vet. If we want this to be more than a one-man project, CI will be useful.
Happy to enable the repo on Travis and send a PR with the yaml file if @rogpeppe agrees. We can use go1.11beta3 for now.
In the README, maybe very briefly explain how to use a fork you created in GitHub, how to push a change back to that fork on GitHub, and then mention you can use that to open a PR on the original project?
For example, if you fork rsc.io/quote
in the GitHub web interface to github.com/thepudds/quote
, then use gohack + git to use that fork:
cd $(mktemp -d) # create an initially empty test module
go mod init testmod
go get rsc.io/quote # not needed if already in go.mod
gohack get -vcs rsc.io/quote
cd $HOME/gohack/rsc.io/quote
git remote rename origin upstream
git remote add origin https://github.com/thepudds/quote
git remote -v
git checkout -b test-branch-on-my-fork
touch new.file # make your edits
git add -A
git commit -am "test commit"
git push origin # push your changes
# then if desired, open a PR to the original project in the GitHub web interface
I purposefully modeled those steps above at least loosely on https://blog.sgmansfield.com/2016/06/working-with-forks-in-go/ because that is a somewhat common resource handed out to people who are new to Go.
@rogpeppe mentioned elsewhere that his personal workflow is usually slightly simpler than that:
cd $(mktemp -d) # create an initially empty test module
go mod init testmod
go get rsc.io/quote # not needed if already in go.mod
gohack get -vcs rsc.io/quote
cd $HOME/gohack/rsc.io/quote/
git remote add rogpeppe https://github.com/rogpeppe/quote
git checkout -b test-branch-on-my-fork
touch new.file # make your edits
git add -A
git commit -am "test commit"
git push origin
I'm not suggesting either of those be the exact text, but I am suggesting that at least mentioning a sample set of steps would be highly beneficial.
Also, people use git in many ways, so I'm not suggesting anything here be presented as "the one true way to do it", but some type of text like "There are multiple ways to do this, but one approach is ..." means someone new to modules and gohack can follow those steps, while someone more advance can map those steps to however they prefer to work with git (but doing that after having the benefit of at least understanding how gohack
works with one git approach, even if it is not their personally preferred git approach).
One nuance is often someone will be using some specific version of a dependency to start in their day-to-day use (e.g., foo v1.2.3
). gohack get -vcs foo
will get that version by default, but that might not always be what you want if you are planning on sending a PR.
Finally, gohack
probably could be tweaked to make some of this simpler, but might be easiest to start by adding something along these lines to the README based on gohack
as it exists today.
When we do gohack get
of a module without specifying the -vcs
flag, we record the checksum so we can later report that the source has been changed, but we have know way of informing the user exactly what has been changed.
We should record the version that's being checked out as well as the checksum. That way we can find out what's changed (using gohack diff
). This also potentially allows us to switch from default to VCS mode while preserving changes made by the user.
One potentially issue with implementing this command is that it's potentially a slippery slope towards adding endless diff features.
$ gohack get -vcs github.com/rogpeppe/go-internal/modfile
module "github.com/rogpeppe/go-internal/modfile" does not appear to be in use
all modules failed; not replacing anything
The module is github.com/rogpeppe/go-internal
. But the intent of the command was clear: I want to look at the package github.com/rogpeppe/go-internal/modfile
. gohack get
should accept a package path and start hacking on the containing module.
Running any gohack
command runs the go
command to find out information on the current modules, but this fails if the modules cannot be resolved (for example because a gohacked module has been removed)
We should be able to undo the gohack even when the target directory no longer exists.
$ gohack get -vcs k8s.io/client-go
fetching k8s.io/[email protected]+incompatible
cannot update VCS dir for k8s.io/client-go: error: pathspec 'v8.0.0+incompatible' did not match any file(s) known to git
all modules failed; not replacing anything
client-go was actually used as an example in the commit message with that pathspec format: https://go-review.googlesource.com/c/go/+/124384.
I'm hoping the fix is simple as just dropping "+incompatible" for the tag.
-*- mode: compilation; default-directory: "~/go/pkg/mod/github.com/stapelberg/[email protected]/" -*-
Compilation started at Fri May 29 21:53:41
goversion -m $(which gohack) && gohack get
/home/michael/go/bin/gohack go1.14
path github.com/rogpeppe/gohack
mod github.com/rogpeppe/gohack v1.0.2
dep github.com/rogpeppe/go-internal v1.0.0
dep golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e
dep gopkg.in/errgo.v2 v2.1.0
cannot determine main module: go list -m: not using modules
Compilation exited abnormally with code 1 at Fri May 29 21:53:41
I have to use gohack get github.com/stapelberg/glog
even though itβs obvious from my working directory which package I mean.
Would a PR for this be accepted?
So the workflow I just used was the following:
Obviously the fork will need to be setup everytime we run GoHack. Curious if you think it's a good idea to add a "remote" or "fork" cli flag to gohack which setups a remote fork to push your changes too in an automated fashion. If so I'd like to take a stab at this PR.
gohack
should check if the dependency already exists in the GOPATH before cloning it into $HOME/gohack
.
Let's say I have github.com/heetch/felice
in my GOPATH.
Instead of having:
replace github.com/heetch/felice => $HOME/gohack/github.com/heetch/felice
I would have
replace github.com/heetch/felice => $GOPATH/src/github.com/heetch/felice
The majority (based on a small sample set) of the time I find myself wanting to make throwaway changes to a dependency. In these situations I really don't want to pollute the shared $HOME/gohack
space.
Hence I propose gohack get -tmp $module
which would place the "hack" in a temporary directory.
Effectively reproduced by:
> [!net] skip
> [!exec:git] skip
> env GOPROXY=
> cd repo
> go get rsc.io/[email protected]
> env GOHACK=/gohack
> gohack get rsc.io/quote
rsc.io/quote => /gohack/rsc.io/quote
> gohack undo
> gohack get -vcs rsc.io/quote
I think this issue also raises the question of how you "downgrade" in the case you subsequently call bare get
(because the assumption would be, in that case, that you aren't working in a VCS checkout).
Many package paths are long. To ease typing, I suggest either/both of:
gohack list
, which will emit all package paths in the module, for easy copy/pasteOpinions?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.