posener / gitfs Goto Github PK

View Code? Open in Web Editor NEW

124.0 3.0 7.0 137 KB

A complete solution for static files in Go code

License: Apache License 2.0

Go 100.00%

go golang filesystem static static-files http no-binary

gitfs's Issues

[idea] filepath.Walk() for folders

Hi!

Would be great to add something like filepath.Walk() (docs) for folders from GitHub fs.

Or, improve fsutil for other files (not *.tmpl).

Huge memory usage

It seems when opening a file with gitfs for example to copy it to another file it use a lot of memory!
but only in the moment you open and read the file, or unless you use precaching
I also notice if use optLocal then no memory is used.

This can somehow be fixed? I really like this lib :)

URL Format When Specifying Path & Non-Master Branch

Hello,
I'm trying to use gitfs to retrieve files from a repository using a path and pointing to a branch that isn't master. What is the correct format for the URL to use with gitfs.New()? The documentation doesn't give a specify example when using both a path and a branch in the URL. I've tried the format github.com/owner-name/project-name/some/path/to/director@heads/PRO-72 but I just get a 404.

Error when `go get github.com/posener/gitfs/cmd/gitfs`

Catch strange bug on v1.1.0 (I use Go v1.13.5):

$ go get github.com/posener/gitfs/cmd/gitfs

# github.com/posener/gitfs/fsutil
../.go/src/github.com/posener/gitfs/fsutil/diff.go:171:18: too many arguments in call to diff.Format
../.go/src/github.com/posener/gitfs/fsutil/diff.go:171:49: undefined: diff.OptSuppressCommon

Ok. Go to github.com/posener/gitfs/fsutil/diff.go:171:49 (link) and see:

// ...
d := diff.Format(string(aData), string(bData), diff.OptSuppressCommon())
// ...

Third arg for function diff.Format() is diff.OptSuppressCommon() and probably it's wrong.

Question: Embedding Static Assets Features?

Hello! I really like your library, and have used it on some private projects.
At work, we use this rubric for evaluating static asset embedding libraries, and I was wondering where gitfs fits on these criteria:

Test binary content

An idea: when binary content is generated, it also generate a test that checks that the binary content is identical to the remote content that it was generated from.

Binary Packing

1. I wan Using Binary Packing. so I test with the following code

func main() {

	ctx := context.Background()
	fs, err := gitfs.New(ctx, "metest/public", gitfs.OptLocal("."))
	if err != nil {
		log.Fatalf("Failed initialize filesystem: %s", err)
	}
	f, err := fs.Open("go.mod")
	if err != nil {
		log.Fatalf("Failed opening file: %s", err)
	}
	io.Copy(os.Stdout, f)
}

2. It can run successfully before Using Binary Packing。The result is：

module test

go 1.13

require (
        github.com/posener/gitfs v1.1.0
        golang.org/x/tools v0.0.0-20200115044656-831fdb1e1868 // indirect
)

3. Using Binary Packing run successfully .The result is：

github.com/posener/gitfs/cmd/gitfs/gitfs .

[gitfs] 2020/01/15 23:06:19 FileSystem "github.com/posener/gitfs/cmd/gitfs/templates" from binary
[gitfs] 2020/01/15 23:06:19 Loaded file binary.go.gotmpl in 0.0s
[gitfs] 2020/01/15 23:06:19 Loaded file test.go.gotmpl in 0.0s
2020/01/15 23:06:22 Encoding project: metest/public
[gitfs] 2020/01/15 23:06:22 FileSystem "metest/public" from local directory
2020/01/15 23:06:22 Encoded path: go.mod
2020/01/15 23:06:22 Encoded size: 332

4. go build the test code ,and run it .run time error is:

panic: Failed decoding project "metest/public": decoding gob: gob: encoded unsigned integer out of range

goroutine 1 [running]:
github.com/posener/gitfs/internal/binfs.Register(0xb12c6f, 0xd, 0x1, 0xb2ec8f, 0x14c)
        /home/star/gopro/pkg/mod/github.com/posener/[email protected]/internal/binfs/binfs.go:61 +0x203
github.com/posener/gitfs/bin.Register(...)
        /home/star/gopro/pkg/mod/github.com/posener/[email protected]/bin/bin.go:10
main.init.0()
        /home/star/gopro/src/test/gitfs.go:7 +0x54

5. I don't know where the mistake happened. Can you help me. Thank you.

Feature Request: Compression for Binary Packing

Feature Request

Several other binary packing libraries support Compression, and it would be nice if gitfs did too.

Bonus Points Feature: Optional Decompression

If your static files are stored in your executable with gzip compression, and you are going to potentially serve up those files to your client with gzip compression, why not just send them the already compressed file data? The ideal library would give you the option, at runtime, to retrieve the already compressed file without decompressing it first.

See Opt. decompression

error when using go run github.com/posener/gitfs/cmd/gitfs ./...

this is the error

Failed encoding filesystem "github.com/segator/transcoderd": opening file .git\COMMIT_EDITMSG: http: invalid character in file path

in my main.go i have this

err := gitfs.New(ctx,
		"github.com/segator/transcoderd",)

the working dir where main.go exists is a git repo poiting to github.com/segator/transcoderd

any idea?

Feature request: support SSH-style git URLs

When the git repo is configured with SSH (i.e. the remote starts with "[email protected]:krokicki/myproject.git") then binary packing fails as follows (gitfs.New fails the same way):

$ gitfs ./...
panic: failed parsing URL: [email protected]:krokicki/myproject.git

goroutine 1 [running]:
github.com/posener/gitfs/internal/localfs.urlProjectName(0xc0001a2c60, 0x26, 0x1, 0x1)
        /Users/tester/go/pkg/mod/github.com/posener/[email protected]/internal/localfs/localfs.go:99 +0x1fb
github.com/posener/gitfs/internal/localfs.computeSubdir(0x189fe74, 0x28, 0xc0001b9540, 0x18, 0x0, 0x0, 0x20, 0x18)
        /Users/tester/go/pkg/mod/github.com/posener/[email protected]/internal/localfs/localfs.go:45 +0x14b
github.com/posener/gitfs/internal/localfs.New(0x189fe74, 0x28, 0xc00003600c, 0x1, 0xc00019a040, 0x1e7c801, 0x2, 0xc00024fb38)
        /Users/tester/go/pkg/mod/github.com/posener/[email protected]/internal/localfs/localfs.go:24 +0x84
github.com/posener/gitfs.New(0x19a95e0, 0xc0001a4000, 0x189fe74, 0x28, 0xc00024fbf0, 0x1, 0x1, 0x30, 0xc000275170, 0x0, ...)
        /Users/tester/go/pkg/mod/github.com/posener/[email protected]/gitfs.go:199 +0x136
project/cmd.generateDockerfile(0x1891c4c, 0x18, 0x17e6760, 0xc000275170)
        /Users/tester/dev/project/cmd/init.go:207 +0xf8
project/cmd.initProjectJavaMaven()
        /Users/tester/dev/project/cmd/init.go:161 +0x17f
project/cmd.glob..func1(0x1ecaf40, 0xc000241110, 0x1, 0x1)
        /Users/tester/dev/project/cmd/init.go:59 +0x1f8
github.com/spf13/cobra.(*Command).execute(0x1ecaf40, 0xc0002410d0, 0x1, 0x1, 0x1ecaf40, 0xc0002410d0)
        /Users/tester/go/pkg/mod/github.com/spf13/[email protected]/command.go:846 +0x29d
github.com/spf13/cobra.(*Command).ExecuteC(0x1ecb1e0, 0x10437aa, 0x1e87fe0, 0xc000000180)
        /Users/tester/go/pkg/mod/github.com/spf13/[email protected]/command.go:950 +0x349
github.com/spf13/cobra.(*Command).Execute(...)
        /Users/tester/go/pkg/mod/github.com/spf13/[email protected]/command.go:887
project/cmd.Execute()
        /Users/tester/dev/project/cmd/root.go:24 +0x31
main.main()
        /Users/tester/dev/project/main.go:6 +0x20

It worked after switching to HTTPS style remote ("https://github.com/krokicki/myproject.git") It would be nice if the SSH style was also supported.

Feature request: Print current ref (branch/commit), if one was not provided explicitly

Hi,

I'd find it very handy to be able to print the current ref (latest branch/commit), if one was not provided explicitly.

type HasCommitSHA interface {
    CommitSHA() string
}

fs, _ := gitfs.New(ctx, "github.com/x/y/static")

if v, ok := fs.(HasCommitSHA); ok {
    fmt.Println("current commit:",  v.CommitSHA())
}

Is this somehow possible right now? Would you be open to such functionality?

Thanks,
Vojtech

P.S.: gitfs works very well, thank you for this project! We started using it in https://github.com/webrpc/webrpc.

Add example for go generate Binary Packing of Static Assets

Per discussion in #16

It would be nice to have an example for go generate Binary Packing of Static Assets. Thanks again for a great library!