bitfield / script Goto Github PK

I am trying to use script.Concat() to implement 'cat':

        if _, err := script.Slice(flagSet.Args()).Concat().Stdout(); err != nil {
                fmt.Printf("cat: %v\n", err)
                status = 5
        }

Full code: https://github.com/udhos/conbox/blob/master/applets/cat/run.go

However I would like to tell Concat() to keep trying other files in case of failure.

Can we make the behavior configurable?

Hi!

I've translated some of the examples to Next Generation Shell.

If you think it's appropriate, you are welcome to add the following link somewhere:

https://github.com/ngs-lang/ngs/wiki/Code-Snippets-Comparisons#script-go-library

Thank you for publishing this useful tool @bitfield.

Would it be possible to add support for plugging different Regex engines in script? GoLang's regexp package does not support PCRE and without that there is no look-ahead nor look-behind functionality without which some problems cannot be solved.

It would be great if script allowed for 1 or 2. Ideally both, otherwise maybe only 1 due to superior performance.

Hi, This project is great, I've got some idea

Split with columns outputs, with awk mechanism and regexp ?

Which it can set FS input delimiter and OFS output delimiter, specify which columns to output

This may later working directly send output to charts library? (generrate a graph maybe ) or csv

Here are some chart libraries:

1. https://github.com/vdobler/chart
2. https://github.com/chenjiandongx/go-echarts
3. https://github.com/gizak/termui

I used to write this https://github.com/chinglinwen/k8s/blob/master/testing/httpload/funcs, which I think represent some usecases.

So if I have a command that has an obvious error:

func main() {
	p := script.Exec("cat /etc/shels")
	Str, err := p.String()
	if err != nil {
		fmt.Println(Str, err)
	} else {
		fmt.Println(Str)
	}

}

I get the following output:

exit status 1

If I change the first line to:

p := script.Exec("cat /etc/shells")

I get the output from cat.

I looked at the code and when the exec it done it uses exec.CombinedOutput so I am wondering why I don't see the error from cat?

the doc says that WriteFile() is equivalent to > .
but when I use WriteFile() with a file which contains content already, it will not truncate the file but will append it to the begining! overwriting only the bytes I write.
cat file.txt
result: ttttttttttt

script.Echo("zzz").WriteFile("file.txt")
result: zzztttttttt

I think we have to use O_TRUNC here

tested in windows 7 with golang v1.16.2

It's very common to need sudo rights in scripts. We might consider a SudoExec method, for example, or perhaps just a Sudo filter which obtains sudo permissions, if possible, for all subsequent pipe operations.

https://spin.atomicobject.com/2019/07/17/script-find-todos-code/#.XS8eSdFwrY0.hackernews

We should introduce a method that will list all files in a directory and its subdirectories and return a pipe just like the ListFiles Method. Probably the method should be able to filter hidden files using a showHidden argument. I am still yet to come up with a decent name for the method, I
am open to suggestions.

func FindFiles(path string, showHidden bool) *Pipe {
  // content
}

What if we want to compare two text files and display the file differences.

It's common when writing scripts that install or configure things to need input from the user (at a minimum, something like 'Press Enter to continue'; at a maximum, to be able to prompt the user for input, with an optional default value, and return that value).

Let's use this issue to design how that would look, and I invite suggestions!

It would be cool to have a way to write a buffer to multiple sinks, in cases where you want to intercept parts of the incoming data but still pipe the rest for other matches...

script.Stdin().Match("error").Tee(script.File("all_errors")).Match("foo").Stdout()

cat logs | grep error | tee all_errors | grep foo 

In this scenario Tee writes to the File whilst still piping the content to the next item on the pipe Match that eventually pipes to stdout or any other sink.

@bitfield WDYT?

Something like:

fun (*p Pipe) Tee(io.Writer) *p { ... }
// or 
fun (*p Pipe) Tee(p Pipe) *p { ... }

Add Last filter which is analogous to tail command

Modern shell scripts interact often with a REST API or a command line program producing json (kubectl for example). Slicing and dicing the data returned by those APIs is often done with https://stedolan.github.io/jq/.

https://github.com/itchyny/gojq provides a go native implementation of jq.

I propose the following filter:

func (p *Pipe) JQ(query string) *Pipe

An example script:

script.Echo('{"foo": 128}'').JQ(".foo").Stdout()

should return "128" on stdout.

Or to show the IP addresses of the local interface lo a shell user will write:

ip -j a show  | jq '.[] | select(.ifname=="lo") | .addr_info[].local'

With an jq filter in script we can write:

 script.Exec("ip -j a show").JQ('.[] | select(.ifname=="lo") | .addr_info[].local').Stdout)

The same slicing and dicing of data can be done in pure go code, but the jq language provides a well documented and known DSL for this problem.

Hi 👋

It seems that sinks.SHA256Sum() puts the whole file in-memory which may be an issue for hashing large files.
For example when adding a 1GB file to the test:

root@ef78500374de:/workspaces/script# go test
signal: killed
FAIL    github.com/bitfield/script      16.404s

Equivalent to shell xargs. For example, this use case: https://gist.github.com/Skraelingjar/9aabc58aa26f5b814411e69d67b342fe

Based on the discussion in #7 I would like to suggest a subpackage unix. The suggestion is to have functions in this package, which have the same names as the respective unix commands. This would allow to keep the API of script clean and high level.
At the same time this would provide an easy start for people, how are familiar with the unix commands.
Maybe some of the high level commands in the scriptpackage would even decide to use the unix low level commands for their implementation.

can't exec command with spaces such as: bash -c 'docker ps'

p := script.Exec("bash -c 'docker ps'")

I'd like the godoc to be the primary documentation, and rather than explain each method in the README, I'll combine the best of both sets into the doc comments, and have the README just link to the godoc.org page. I also want to make the examples executable examples, if possible.

Hi @bitfield , thanks for this awesome package. I just ran into an issue that is surely an easy thing to improve.

Issue: the output of Pipe.String() may include a newline. FindFiles and ListFiles cannot find a path that ends in a newline. Hence these two methods cannot consume the output of Pipe.String() without prior removal of newlines.

Suggestion: Have FindFiles and ListFiles trim trailing newlines from the input path.

Example:

package main

import (
	"log"
	"github.com/bitfield/script"
)

func main() {
	r, _ := script.Args().String()
	log.Printf("args: [%s]\n", r)
	p := script.ListFiles(r)
	_, err := p.Stdout()
	log.Println(err)
}

Call:

go run main.go .

Output:

2020/09/08 13:57:58 args: [.
]
2020/09/08 13:57:58 stat .
: no such file or directory

Severity: minor annoyance but can cost time to track down.

https://github.com/apps/goreadme

Some methods to set environment varialbes for single command, instead of this:

os.Setenv("XB_AUTHOR_DATE", dateStr)
os.Setenv("XB_COMMITTER_DATE", dateStr)

Maybe in old shell way:

XB_AUTHOR_DATE="2021-04-05" XB_COMMITTER_DATE="2021-04-05" xb_push xxxx

For symmetry with ListFiles().

Like shell [ -f X ].

Yo!
I'm gonna start with an easy one because my golang was never great to begin with, and now I'm rusty :-D

I think you really need an stderr.
There are a lot of use cases. Differentiate between regular output and error output. Output logs to stderr vs data output on stdout, etc. etc.
It's also a very easy addition (again, rusty go).

Unless you have a strong opposition to this, I'll start working on a PR (still reading through your contributor guidelines).

e.g. for command like

script.Exec("ping -c 100 www.google.com").Stdout()

I expect the result shows in stdout line by line just like real Bash script, instead of showing after the run.

Do you have tricks to get this done? or any plan/feature to implement this?

Many sysadmin tasks involve executing commands via SSH. Add an example program that does this, and identify any useful methods that could be added to script to make this easier.

As far as I can see, There is no implementations of directory manipulations. I can see that given API is more about pipelines and one liners, but I can assure that it will be quite handy time to time and can be useful for anyone who will use script.

I would like to suggest PR that will include two new and a must Unix equivalent functions:

1. cd - change directory
2. pwd - print working directory

It appears that EachLine() is only called after a pipe is closed. This prevents using script to write daemon scripts.

If there's a way to perform the equivalent of:

dbus-monitor --system "interface=org.freedesktop.DBus.Properties" | \
    while read -r line; do
        printf "DBus event: %s\n" "$line"
    done

would it be possible to get an example? If not, can we change the behavior of EachLine() to process each line as it is received, or add an EachLineForever() function?

This could be a good companion for script: https://github.com/containous/yaegi

If it works, add some documentation to the README on how to use it, with examples.

Hello,

First, thanks for this project. Really like the API and design!

Instead of an issue, I will more ask a question. I am not sure to understand the strategy of closing for a Pipe when the sink is String() or Bytes().

For String() (https://github.com/bitfield/script/blob/master/sinks.go#L65), the documentation says:

String returns the contents of the Pipe as a string, or an error, and closes the pipe after reading

However, I cannot see in the code any reference to a close method. It only calls Bytes() which itself does not seem to close the reader. Am I missing something? Or is it a documentation mistake?

I can see in a previous commit that the close has been deleted. Is it on purpose?

062e038#diff-1d716f4c8f87036767ecdd0412ad974fL30

Thanks for the clarification!
Loric

there is always a use case to calculate md5hash of a file and compare it with others. Having a single liner will definitely help several use cases.

In #12 I suggested to add a separate sub package called unix with an API similar to the unix coreutils commands. This issue was closed with the suggestion to implement this in it's own package/library, separated from this package.

So I tried to figure out, what would be the best way to do this and therefore I wanted to reach out to you (@bitfield and all other followers of this package) to discuss, how this could be done.

My goal is to leverage this package, especially the Pipe and be able to combine (in an as nice as possible way) functions from this package (e.g. script.Freq()) with functions from an other package (e.g. unix.Head()).

I looked at the example in the README but this does not work in a seperate package, because you can not add new methods to an type defined in an other package.

So the question is, what is the best way to approach this?

Hey, just came across this & thought of a use-case when dealing with converting certs/pems private keys/etc.:

Converting this:

-----BEGIN RSA PRIVATE KEY-----
MIICXQIBAAKBgQC+qUCd8K2bBq0l112W4jcjH0kmByIdpthnSCT/jf/XTQprS43B
dVa6H9wU8LUfZQ5w9Ecz+qRmcxtixMVCYSlAjZXLy74QhYHQdHZbGRhhZLIaN6UF
wti2V/CGsR5lJlRIi1GBDjxdUv3RVUH1ILfmVQT6ujpM52ofEsKnferZTwIDAQAB
AoGAGV5LyrgLYWUyBKbzPPA8hd/Ty8uHLorUoGlpAtfSAsOtbzlOUz9ZmspCbkbY
0qSPl1fpYXEoDrmiGzIzTPHAmytMp3kcw9G/hVfg49x1vWVYkGbrKlq0QvoO6jLp
LOvMOz7YD2DNmJ9Dqef8s/ziYDO3fUkQihdKJ0hj2gX9IIECQQDOISTJa0kRUcG6
/2czCYqLQpv65lkofd2h+tmUgWKVBo4+aQUlsa4ZSagxsFmvF28kYWXd98fVMfr3
ER8avbBDAkEA7MoPlqHIb9y1QnD4rHg87ITuhEXq524sXwFLWvzWvLgmAxB2ey9H
+mZjCvL1WXvpn7zqzcMhurnTtefzMh14BQJBAI8aH2neG5n0gmSKD2E1TIOluJgU
9uzPhOCBQDCDKqd/J51YV4R1uAJCSoxEe968jCJbo9bXwFnYGv0PW+K6sfUCQQDj
KRW7VImNhxb9HpPyIYeRABYyH0EztKYsnnlEWLtJYQBWgDyqALn0prTtlBd8OTvv
WrWHoGODVzKbmGHe+hZhAkAbobPF2robFYv4s40gzkFOP7kQb/xGTZiCSkIhugch
QrDwWNmwR3CYdrFCZWh3VJQhUCSQ3dE7PZCxzybf9kyH
-----END RSA PRIVATE KEY-----

to this:

-----BEGIN RSA PRIVATE KEY-----\nMIICXQIBAAKBgQC+qUCd8K2bBq0l112W4jcjH0kmByIdpthnSCT/jf/XTQprS43B\ndVa6H9wU8LUfZQ5w9Ecz+qRmcxtixMVCYSlAjZXLy74QhYHQdHZbGRhhZLIaN6UF\nwti2V/CGsR5lJlRIi1GBDjxdUv3RVUH1ILfmVQT6ujpM52ofEsKnferZTwIDAQAB\nAoGAGV5LyrgLYWUyBKbzPPA8hd/Ty8uHLorUoGlpAtfSAsOtbzlOUz9ZmspCbkbY\n0qSPl1fpYXEoDrmiGzIzTPHAmytMp3kcw9G/hVfg49x1vWVYkGbrKlq0QvoO6jLp\nLOvMOz7YD2DNmJ9Dqef8s/ziYDO3fUkQihdKJ0hj2gX9IIECQQDOISTJa0kRUcG6\n/2czCYqLQpv65lkofd2h+tmUgWKVBo4+aQUlsa4ZSagxsFmvF28kYWXd98fVMfr3\nER8avbBDAkEA7MoPlqHIb9y1QnD4rHg87ITuhEXq524sXwFLWvzWvLgmAxB2ey9H\n+mZjCvL1WXvpn7zqzcMhurnTtefzMh14BQJBAI8aH2neG5n0gmSKD2E1TIOluJgU\n9uzPhOCBQDCDKqd/J51YV4R1uAJCSoxEe968jCJbo9bXwFnYGv0PW+K6sfUCQQDj\nKRW7VImNhxb9HpPyIYeRABYyH0EztKYsnnlEWLtJYQBWgDyqALn0prTtlBd8OTvv\nWrWHoGODVzKbmGHe+hZhAkAbobPF2robFYv4s40gzkFOP7kQb/xGTZiCSkIhugch\nQrDwWNmwR3CYdrFCZWh3VJQhUCSQ3dE7PZCxzybf9kyH\n-----END RSA PRIVATE KEY-----

and back again.

Converting the "flattened" single-line key can be done like so:

script.File("example.flattened.pem").ReplaceRegexp(regexp.MustCompile(`\\n`), "\n").Stdout()

but converting the "unflattened" multi-line key into the single-line one is a little tricky (or I'm missing something clear, which is definitely possible)

• We can't use ReplaceRegexp to convert \n to "", because ReplaceRegexp goes line-by-line while performing the regex and writes an \n at the end of each line again. So regex in general is effectively out
• We can't use join because although it'll join together the lines, it converts them to spaces instead of a customizable separator key, e.g. \n:

for example:

script.File("example.flattenme.pem").Join().Stdout()

will produce

-----BEGIN RSA PRIVATE KEY----- MIICXQIBAAKBgQC+qUCd8K2bBq0l112W4jcjH0kmByIdpthnSCT/jf/XTQprS43B dVa6H9wU8LUfZQ5w9Ecz+qRmcxtixMVCYSlAjZXLy74QhYHQdHZbGRhhZLIaN6UF wti2V/CGsR5lJlRIi1GBDjxdUv3RVUH1ILfmVQT6ujpM52ofEsKnferZTwIDAQAB AoGAGV5LyrgLYWUyBKbzPPA8hd/Ty8uHLorUoGlpAtfSAsOtbzlOUz9ZmspCbkbY 0qSPl1fpYXEoDrmiGzIzTPHAmytMp3kcw9G/hVfg49x1vWVYkGbrKlq0QvoO6jLp LOvMOz7YD2DNmJ9Dqef8s/ziYDO3fUkQihdKJ0hj2gX9IIECQQDOISTJa0kRUcG6 /2czCYqLQpv65lkofd2h+tmUgWKVBo4+aQUlsa4ZSagxsFmvF28kYWXd98fVMfr3 ER8avbBDAkEA7MoPlqHIb9y1QnD4rHg87ITuhEXq524sXwFLWvzWvLgmAxB2ey9H +mZjCvL1WXvpn7zqzcMhurnTtefzMh14BQJBAI8aH2neG5n0gmSKD2E1TIOluJgU 9uzPhOCBQDCDKqd/J51YV4R1uAJCSoxEe968jCJbo9bXwFnYGv0PW+K6sfUCQQDj KRW7VImNhxb9HpPyIYeRABYyH0EztKYsnnlEWLtJYQBWgDyqALn0prTtlBd8OTvv WrWHoGODVzKbmGHe+hZhAkAbobPF2robFYv4s40gzkFOP7kQb/xGTZiCSkIhugch QrDwWNmwR3CYdrFCZWh3VJQhUCSQ3dE7PZCxzybf9kyH -----END RSA PRIVATE KEY-----% 

which is close! but now we have spaces to deal with. And regex replacing spaces with \n after the join pipe would catch the ---BEGIN RSA--- directives, too

I suppose EachLine could be used (though I haven't goofed with it yet) to write a custom join with separator...?

Alternatively, a JoinWithSeparator(string separator) might be beneficial to join lines together using a custom separator, where instead of only a space it joins on a literal \n, or "," etc

Thoughts / feedback welcome, but this is just a thought that came from goofing around, so if I'm missing something please let me know.

Thoughts?
And of course feel free to close whenever 👍 thanks!

EDIT: Just tried with EachLine

	q := p.EachLine(func(line string, out *strings.Builder) {
		out.WriteString(line + "\\n")
	})

and got

-----BEGIN RSA PRIVATE KEY-----\nMIICXQIBAAKBgQC+qUCd8K2bBq0l112W4jcjH0kmByIdpthnSCT/jf/XTQprS43B\ndVa6H9wU8LUfZQ5w9Ecz+qRmcxtixMVCYSlAjZXLy74QhYHQdHZbGRhhZLIaN6UF\nwti2V/CGsR5lJlRIi1GBDjxdUv3RVUH1ILfmVQT6ujpM52ofEsKnferZTwIDAQAB\nAoGAGV5LyrgLYWUyBKbzPPA8hd/Ty8uHLorUoGlpAtfSAsOtbzlOUz9ZmspCbkbY\n0qSPl1fpYXEoDrmiGzIzTPHAmytMp3kcw9G/hVfg49x1vWVYkGbrKlq0QvoO6jLp\nLOvMOz7YD2DNmJ9Dqef8s/ziYDO3fUkQihdKJ0hj2gX9IIECQQDOISTJa0kRUcG6\n/2czCYqLQpv65lkofd2h+tmUgWKVBo4+aQUlsa4ZSagxsFmvF28kYWXd98fVMfr3\nER8avbBDAkEA7MoPlqHIb9y1QnD4rHg87ITuhEXq524sXwFLWvzWvLgmAxB2ey9H\n+mZjCvL1WXvpn7zqzcMhurnTtefzMh14BQJBAI8aH2neG5n0gmSKD2E1TIOluJgU\n9uzPhOCBQDCDKqd/J51YV4R1uAJCSoxEe968jCJbo9bXwFnYGv0PW+K6sfUCQQDj\nKRW7VImNhxb9HpPyIYeRABYyH0EztKYsnnlEWLtJYQBWgDyqALn0prTtlBd8OTvv\nWrWHoGODVzKbmGHe+hZhAkAbobPF2robFYv4s40gzkFOP7kQb/xGTZiCSkIhugch\nQrDwWNmwR3CYdrFCZWh3VJQhUCSQ3dE7PZCxzybf9kyH\n-----END RSA PRIVATE KEY-----\n

Yeah I suppose that'll work. A JoinWith("..") filter of some kind might be shorter, but EachLine can work fine too

Am new to this package, and am trying to use this for the underlying mechanics of a testing framework for a cli I am building.

Currently I have this function.

func CreatePool(fromUser,sourceChain,symbol,ticker, nativeAmount,externalAmount string) {
	cmd := fmt.Sprintf("sifnodecli tx clp create-pool --from %v --sourceChain %v --symbol %v --ticker %v --nativeAmount %v --externalAmount %v", fromUser,sourceChain,symbol,ticker, nativeAmount,externalAmount )
	p := script.Exec(cmd)
	p.SetError(nil)
	output, err := p.String()
	if err != nil {
		Fail(err.Error())
	}
	fmt.Println(output)
}

However the issue I am having with this cmd is it then requires an input confirmation before executing and the above cancels the cmd. ie

confirm transaction before signing and broadcasting [y/N]: 

Are you able to tell me how I would also automatically pass a Y to said cmd?

Thanks

Mat Ryer's idea of testing main() by having main() call run(os.Stdin, os.Stdout, os.Args) seems cleaner than monkey-patching os.Stdin and os.Stdout.

I really like the idea of this library and it might makes a lot of scripting tasks easy!

The current implementation read all output of each command into memory, and then passes it to the next command and so forth.

For example:

• The usage of CombinedOutput (Also not sure why it was chosen to use the combined output and not stdout?)
• The usage of string builder in the ForEachLine.

It would be nice to avoid it if it is not necessery. Let each reader read as much as needed from previous stage and output whatever necessary to the next stage. This is also how pipes in bash works.

Case

I have two files in separate folders and one empty folder. Here's output of tree

.
├── a1
│   └── 2.txt
├── a\1
│   └── 1.txt
└── empty

3 directories, 2 files

Code

package main

import (
	"fmt"
	"github.com/bitfield/script"
	"os"
)

func main() {
	s, _ := script.ListFiles(os.Args[len(os.Args) - 1]).String()
	fmt.Print(s)
}

Compiled binary list_files_test stored in $PATH

Test

1. $ list_files_test a1
   Output: a1/2.txt
   Expected: a1/2.txt
2. $ list_files_test a\\1
   Output: a1
   Expected: a\1/1.txt (or a\\1/1.txt)
   Maybe i misunderstood expected behavior?

Source of the problem

ListFiles() detect this input as glob by strings.ContainsAny(path, "[]^*?\\{}!"). However that is not glob for sure.
And i don't know is that expected or not.

Additional

This issue related and important to #55.
Let's assume i have input for FindFiles such as /some/abs/path/a\\1/*.txt (assuming we have same case as i provided at the beginning). By this input i would expect library will interpret folder a\\1 as root for filepath.Walk. Then this walking should find all txt files under folder a\\1. But with same way of detecting glob as in ListFiles function will take folder path as root and then will also find all txt files under folder like a1 from provided test case. However these files are not in our interests. (Of course assuming we want to check all directories match glob as well, not just file).
Yes, programm could produce expected output if i would check input string in main but this input is for just illustrating problem related to #55.

Workaround

Change way of glob detecting.

In todays scripting we often have to interact with HTTP APIs. Many APIs provide native Go SDKs, but there are lesser known APIs around without a proper SDK. We should provide a facility to provide similar HTTP capabilities to curl into the go scripting environment. Rolling a own HTTP client in go is not that hard, but it requires a lot of repetitive error checking. Today I wrote such a small script with github.com/bitfield/script and I missed build in HTTP capabilities alot.

I have seen #3. This PR only implements a GET function. Interacting even with the simplest REST APIs requires us to use other methods like POST or PUT. Many APIs requires us to set custom headers.

Therefore I propose the following sources:

// HTTPClient is an interface to allow the user to plugin alternative HTTP clients into the source.
// The HTTPClient interface is a subset of the methods provided by the http.Client
// We use an own interface with a minimal surface to allow make it easy to implement own customized clients.
// Customized clients are required to implement features like oauth2 or TLS client certificate authentication.
type HTTPClient interface {
	Do(r *http.Request) (*http.Response, error)
}

// HTTP executes the given http request with the default HTTP client from the http package. The response of that request,
// is processed by `process`. If process is nil, the default process function copies the body from the request to the pipe.
func HTTP(req *http.Request, process func(*http.Response) (io.Reader, error)) *Pipe {
	return HTTPWithClient(http.DefaultClient, req, process)
}

// HTTP executes the given http request with the given HTTPClient. The response of that request,
// is processed by `process`. If process is nil, the default process function copies the body from the request to the pipe.
func HTTPWithClient(client HTTPClient, req *http.Request, process func(*http.Response) (io.Reader, error)) *Pipe {
}

And the following sink:

// HTTPRequest creates a new HTTP request with the body set to the content of the pipe.
func (p *Pipe) HTTPRequest(method string, url string) (*http.Request, error) 

This allows us to simply post the content of files to remote URL like this:

func main() {
	req, err := script.Args().Concat().HTTPRequest(http.MethodPost, "https://httpbin.org/post")
	if err != nil {
		log.Fatal(err)
	}
	script.HTTP(req, nil).Stdout()
}

This is the equivalent of posting the contents of files via curl to a POST endpoint.

I explored this design in the PR #72.
I'm open for feedback.

First of all, this is a really interesting package, thanks for the effort.

In my scripts I often use the shell commands sort and uniq so I feel these two would be a good additions for this package.

No self-respecting shell-script is complete without at least one call to either dirname or basename to manipulate filenames.

These are two essential filters that would help library users a lot :)

It's common to need to do things like make directories, make symlinks, rename files, and so on in shell scripts. Even though it doesn't quite fit in with the pipe metaphor, it might still be useful to have convenience versions of these in script.

(Rename is an interesting case, as a direct os.Rename isn't allowed in overlay filesystems like Docker containers. You have to do a copy followed by delete, which is by no means straightforward. A Docker-safe rename tool would be useful.)

I have batch file call runall.bat this file will call 3 batches

Call 1.bat this bat file will download ftp files
Call 2.bat
Call 3.bat

I need to know how to prevent calling 2.bat if 1.bat get error.

And stop call 3.bat if 2.bat ger error
Finaly generate a log file shows process made .

What is Exec doing under the hood? Is it invoking a shell, or is it parsing the string itself? And how can we use an explicit argument list and not a string?

Generally, I think a major shortcoming of the readme is that it does not clarify the security considerations of using this library. Shell is very insecure, and one of the major reasons not to use it is to have better security.

Update: I now see this is kind of a duplicate: #32

This possibly happens in some cases only, where the exit code is some special number, but I can reproduce with a script like the following:

artificially lock the git repository

touch .git/index.lock

now try to run something like:

var out, _ = script.Exec("git checkout master").String()
fmt.Println(out) // will be empty

git checkout master would have instead printed sthing like:

fatal: Unable to create '<path>/go/.git/index.lock': File exists.

Another git process seems to be running in this repository, e.g.
an editor opened by 'git commit'. Please make sure all processes
are terminated then try again. If it still fails, a git process
may have crashed in this repository earlier:
remove the file manually to continue.

but that output is not easily accessible. A workaround I am using is to ignore the error status of the pipeline:

var p = script.Exec("git checkout master")
var data, err = ioutil.ReadAll(p.Reader)
var out = string(data)
fmt.Println(out) // will contain the full output

I believe it should be enough to change https://github.com/bitfield/script/blob/master/sinks.go#L69 so that it returns whatever data it was able to read. I suspect the same (or similar) change should be made in Bytes() as well.

I'd be happy to try and provide PR :-)

There has been some valuable proposal work and prototyping on HTTP support for script, notably in #3, #73 , and #72. However, I haven't yet seen a design that looks obvious, and that's how I want it to be: so simple that when you see it, you think "Of course! That's the only way it could work."

Opening this yet-another-issue to track a new approach to HTTP support: not "what could we reasonably include?", but "what could we not reasonably leave out?"

cd is quite common in Shell script. Do you have wrapper/method for this command?