google / go-cmdtest Goto Github PK

Allow tests to express that a command fails with a particular exit code.

I propose that the syntax be extended so that in addition to

cmd --> FAIL

one can instead write

cmd --> FAIL 2

(or any other integer) to specify the desired exit code.

As shown in the comparison between the latest tag and master, some bugfixes have been introduced.

It would be nice to make use of these bugfixes while using a tagged version of the lib.

Hi, i have test suite with absolute path in my stdout:

$ go-arch-lint check --project-path ${PWD}/test/check/project --arch-file invalid_spec.yml --output-color=false --> FAIL
...
[Archfile] path '$.components.not_exist.in': not found directories for 'not_exist' in '/home/neo/go/src/github.com/fe3dback/go-arch-lint/test/check/project/not_exist':
  14 |   main:
  15 |     in: .
...

This test will fall on all another computers, except my )
this lib have any option, to substitute variables in stdout, before comparing with real output?

I tried using ${PWD} but it didn't help.

Hello,

I happened to notice that the github.com/google/go-cmdtest package has a compilation error when targeting the Plan 9 operating system, which has three Go ports:

$ go tool dist list | grep plan9        
plan9/386
plan9/amd64
plan9/arm

The build error can be reproduced on another platform via cross-compilation, for example:

$ GOOS=plan9 GOARCH=amd64 go build ./...
# github.com/google/go-cmdtest
./cmdtest.go:536:17: undefined: syscall.Errno
$ echo $?
1

cmdtest.go imports "syscall" but the API of that package is system-specific. Its documentation says:

On most systems, that error has type syscall.Errno.

But isn't the case on Plan 9.

I don't use Plan 9 myself, but I noticed this when checking that another project (golang.org/x/vuln) at minimum compiles on all Go ports, and that project has go-cmdtest as a dependency. That project doesn't necessarily need to work on Plan 9 and can work around it, so this isn't causing a serious problem for it.

I'm reporting it here in case you find this information useful. If supporting Plan 9 is completely out of scope of go-cmdtest, you can leave the build error on Plan 9 as is and close this issue as not planned.

Thanks.

The problem

Currently, I can't update my ct files' outputs.

The reason

My /tmp is mounted at a different mount point than where my *.ct files are. Therefore, when updating them (by passing true as second argument to (*TestSuite).Run), I get the following error:

rename /tmp/cmdtest196280009 <my-ct-file-at-different-mount-than-tmp>: invalid cross-device link

This is due to using os.Rename to do the copy, which can't handle different mount points.

This happens because os.Rename uses the rename syscall under the hood, which, in its manual page, states:

(...) Linux permits a filesystem to be mounted at multiple points, but rename() does not work across different mount points, even if the same filesystem is mounted on both.

The solution

A possible solution would be to create, copy and remove the file in separate steps, which will avoid calling the rename syscall.

Nonetheless, great work so far, thanks!

I have setup a basic cobra CLI and found if you run if you run cmd --help before cmd --version the command returns help rather than version output. Suggesting that some of the state is being carried over between commands.

CT that functions as expected

$ cmd --version

CT that doesn't function as expected

$ cmd --help
...
$ cmd --version
...

Is there any way to prevent this or is this a bug with cobra?

I have set up an example repo:
https://github.com/Brookke/go-cmdtest-repo/blob/main/testdata/version.ct

I have also tried splitting up the commands into their own ct files, but still seeing the same issue.

Hey 👋

Been using your package here to try to write some cli integration tests for my project box and am running into a few issues that I think could be solved by implementing support for filtering output with either regular expressions or glob patterns.

See for example Mercurial's test runner and the section on Filtering output

Would you consider adding this support? 🤔

#4
started using github.com/google/renameio to rename files.

https://github.com/google/renameio#windows-support indicates that renameio doesn't work on Windows.

Therefore, that commit breaks any use of this package on Windows.

@jba
@gbrlsnchs

Thoughts?