sktan / aws-codeartifact-proxy Goto Github PK

View Code? Open in Web Editor NEW

29.0 2.0 8.0 62 KB

An AWS code artifact proxy to allow unauthenticated read access to your code artifacts

License: MIT License

Dockerfile 3.65% Shell 0.36% Go 46.62% Python 37.91% Nix 11.45%

aws code artifact proxy codeartifact npm pip

aws-codeartifact-proxy's People

Contributors

Stargazers

Watchers

Forkers

andrewslabsit beplus lafrenierejm arinet shurshun interopio tomertcherniak

aws-codeartifact-proxy's Issues

Does this project support caching

Hi team,

I'm currently using aws-codeartifact-pro as a proxy for my CodeArtifact private npm registry. I wanted to understand how the caching mechanism works in this setup. Does the proxy cache package data from CodeArtifact? If so, how long are the packages cached, and is there a way to configure this cache?

Thanks for your assistance!

Nuget header not adding Authentication

Hello and thank you for your work on this.

We're trying to use nuget to push artifacts. The container starts and we get the Authorization successful result.
When we attempt to nuget push or install we get a prompt for credentials. Looking at the code and doing some packet captures to try and find the issue.

"fatal error: concurrent map writes" in maven proxy mode

Hi, thanks for the great tool!
I'm using it to proxy to my maven codeartifact proxy. Noted gradle builds fails with Received status code 503 from server: Service Temporarily Unavailable error. aws-codeartifact-proxy deployed to the k8s cluster and during the investigation I noted that pod has been restarted. Logs:

fatal error: concurrent map writes

goroutine 1079 [running]:
runtime.throw({0x7c3a56?, 0xc00062c8d8?})
	/usr/local/go/src/runtime/panic.go:992 +0x71 fp=0xc00062c8e0 sp=0xc00062c8b0 pc=0x433451
runtime.mapassign_faststr(0x759f40, 0xc0000a5d40, {0xc00014c120, 0x15})
	/usr/local/go/src/runtime/map_faststr.go:295 +0x38b fp=0xc00062c948 sp=0xc00062c8e0 pc=0x4129ab
github.com/sktan/aws-codeartifact-proxy/tools.ProxyRequestHandler.func1({0x8c1168, 0xc0001d6540}, 0xc000536500)
	/go/src/app/tools/proxy.go:22 +0x77 fp=0xc00062ca20 sp=0xc00062c948 pc=0x71aab7
net/http.HandlerFunc.ServeHTTP(0x7fc0b32d9338?, {0x8c1168?, 0xc0001d6540?}, 0x40cbc5?)
	/usr/local/go/src/net/http/server.go:2084 +0x2f fp=0xc00062ca48 sp=0xc00062ca20 pc=0x62d2af
net/http.(*ServeMux).ServeHTTP(0x0?, {0x8c1168, 0xc0001d6540}, 0xc000536500)
	/usr/local/go/src/net/http/server.go:2462 +0x149 fp=0xc00062ca98 sp=0xc00062ca48 pc=0x62ec49
net/http.serverHandler.ServeHTTP({0xc000607d70?}, {0x8c1168, 0xc0001d6540}, 0xc000536500)
	/usr/local/go/src/net/http/server.go:2916 +0x43b fp=0xc00062cb58 sp=0xc00062ca98 pc=0x62ff5b
net/http.(*conn).serve(0xc0005f2a00, {0x8c13f0, 0xc000607cb0})
	/usr/local/go/src/net/http/server.go:1966 +0x5d7 fp=0xc00062cfb8 sp=0xc00062cb58 pc=0x62bd57
net/http.(*Server).Serve.func3()
	/usr/local/go/src/net/http/server.go:3071 +0x2e fp=0xc00062cfe0 sp=0xc00062cfb8 pc=0x6308ae
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1571 +0x1 fp=0xc00062cfe8 sp=0xc00062cfe0 pc=0x462361
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:3071 +0x4db

goroutine 1 [IO wait]:
internal/poll.runtime_pollWait(0x7fc0b32d98d8, 0x72)
	/usr/local/go/src/runtime/netpoll.go:302 +0x89
internal/poll.(*pollDesc).wait(0xc0001ba880?, 0xc000036500?, 0x0)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:83 +0x32
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:88
internal/poll.(*FD).Accept(0xc0001ba880)
	/usr/local/go/src/internal/poll/fd_unix.go:614 +0x22c
net.(*netFD).accept(0xc0001ba880)
	/usr/local/go/src/net/fd_unix.go:172 +0x35
net.(*TCPListener).accept(0xc00000c798)
	/usr/local/go/src/net/tcpsock_posix.go:139 +0x28
net.(*TCPListener).Accept(0xc00000c798)
	/usr/local/go/src/net/tcpsock.go:288 +0x3d
net/http.(*Server).Serve(0xc0001502a0, {0x8c0f88, 0xc00000c798})
	/usr/local/go/src/net/http/server.go:3039 +0x385
net/http.(*Server).ListenAndServe(0xc0001502a0)
	/usr/local/go/src/net/http/server.go:2968 +0x7d
net/http.ListenAndServe(...)
	/usr/local/go/src/net/http/server.go:3222
github.com/sktan/aws-codeartifact-proxy/tools.ProxyInit()
	/go/src/app/tools/proxy.go:121 +0xce
main.main()
	/go/src/app/main.go:16 +0x2a

goroutine 66 [sleep]:
time.Sleep(0x37e11d600)
	/usr/local/go/src/runtime/time.go:194 +0x12e
github.com/sktan/aws-codeartifact-proxy/tools.CheckReauth()
	/go/src/app/tools/aws.go:103 +0x29
created by main.main
	/go/src/app/main.go:13 +0x25

goroutine 1032 [IO wait]:
internal/poll.runtime_pollWait(0x7fc0b32d9248, 0x72)
	/usr/local/go/src/runtime/netpoll.go:302 +0x89
internal/poll.(*pollDesc).wait(0xc000624380?, 0xc00064e000?, 0x0)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:83 +0x32
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:88
internal/poll.(*FD).Read(0xc000624380, {0xc00064e000, 0x1000, 0x1000})
	/usr/local/go/src/internal/poll/fd_unix.go:167 +0x25a
net.(*netFD).Read(0xc000624380, {0xc00064e000?, 0xc000076160?, 0x46b57e?})
	/usr/local/go/src/net/fd_posix.go:55 +0x29
net.(*conn).Read(0xc0000b8028, {0xc00064e000?, 0x61f16c?, 0xc0000bc400?})
	/usr/local/go/src/net/net.go:183 +0x45
net/http.(*connReader).Read(0xc000606330, {0xc00064e000, 0x1000, 0x1000})
	/usr/local/go/src/net/http/server.go:780 +0x16d
bufio.(*Reader).fill(0xc00062e420)
	/usr/local/go/src/bufio/bufio.go:106 +0x103
bufio.(*Reader).ReadSlice(0xc00062e420, 0x0?)
	/usr/local/go/src/bufio/bufio.go:371 +0x2f
bufio.(*Reader).ReadLine(0xc00062e420)
	/usr/local/go/src/bufio/bufio.go:400 +0x27
net/textproto.(*Reader).readLineSlice(0xc000606360)
	/usr/local/go/src/net/textproto/reader.go:57 +0x99
net/textproto.(*Reader).ReadLine(...)
	/usr/local/go/src/net/textproto/reader.go:38
net/http.readRequest(0xc0000b8028?)
	/usr/local/go/src/net/http/request.go:1029 +0x79
net/http.(*conn).readRequest(0xc0005f20a0, {0x8c1348, 0xc0000bc2c0})
	/usr/local/go/src/net/http/server.go:988 +0x24a
net/http.(*conn).serve(0xc0005f20a0, {0x8c13f0, 0xc000607cb0})
	/usr/local/go/src/net/http/server.go:1891 +0x32b
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:3071 +0x4db

goroutine 1076 [IO wait]:
internal/poll.runtime_pollWait(0x7fc0b32d8ca8, 0x72)
	/usr/local/go/src/runtime/netpoll.go:302 +0x89
internal/poll.(*pollDesc).wait(0xc0001ba600?, 0xc00066d980?, 0x0)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:83 +0x32
internal/poll.(*pollDesc).waitRead(...)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:88
internal/poll.(*FD).Read(0xc0001ba600, {0xc00066d980, 0x1826, 0x1826})
	/usr/local/go/src/internal/poll/fd_unix.go:167 +0x25a
net.(*netFD).Read(0xc0001ba600, {0xc00066d980?, 0xc0002b4480?, 0xc00066da7d?})
	/usr/local/go/src/net/fd_posix.go:55 +0x29
net.(*conn).Read(0xc0000b8048, {0xc00066d980?, 0xd?, 0x9?})
	/usr/local/go/src/net/net.go:183 +0x45
crypto/tls.(*atLeastReader).Read(0xc0003b8750, {0xc00066d980?, 0x0?, 0x986?})
	/usr/local/go/src/crypto/tls/conn.go:784 +0x3d
bytes.(*Buffer).ReadFrom(0xc000660978, {0x8bd320, 0xc0003b8750})
	/usr/local/go/src/bytes/buffer.go:204 +0x98
crypto/tls.(*Conn).readFromUntil(0xc000660700, {0x8bdf60?, 0xc0000b8048}, 0x1736?)
	/usr/local/go/src/crypto/tls/conn.go:806 +0xe5
crypto/tls.(*Conn).readRecordOrCCS(0xc000660700, 0x0)
	/usr/local/go/src/crypto/tls/conn.go:613 +0x116
crypto/tls.(*Conn).readRecord(...)
	/usr/local/go/src/crypto/tls/conn.go:581
crypto/tls.(*Conn).Read(0xc000660700, {0xc00015b000, 0x1000, 0x5fd7e0?})
	/usr/local/go/src/crypto/tls/conn.go:1284 +0x16f
bufio.(*Reader).Read(0xc000110de0, {0xc0001d64a0, 0x9, 0x618a82?})
	/usr/local/go/src/bufio/bufio.go:236 +0x1b4
io.ReadAtLeast({0x8bd240, 0xc000110de0}, {0xc0001d64a0, 0x9, 0x9}, 0x9)
	/usr/local/go/src/io/io.go:331 +0x9a
io.ReadFull(...)
	/usr/local/go/src/io/io.go:350
net/http.http2readFrameHeader({0xc0001d64a0?, 0x9?, 0xc000607a70?}, {0x8bd240?, 0xc000110de0?})
	/usr/local/go/src/net/http/h2_bundle.go:1566 +0x6e
net/http.(*http2Framer).ReadFrame(0xc0001d6460)
	/usr/local/go/src/net/http/h2_bundle.go:1830 +0x95
net/http.(*http2clientConnReadLoop).run(0xc000242f98)
	/usr/local/go/src/net/http/h2_bundle.go:8815 +0x130
net/http.(*http2ClientConn).readLoop(0xc00065e480)
	/usr/local/go/src/net/http/h2_bundle.go:8711 +0x6f
created by net/http.(*http2Transport).newClientConn
	/usr/local/go/src/net/http/h2_bundle.go:7439 +0xa65

goroutine 1080 [runnable]:
net/http.(*connReader).startBackgroundRead.func2()
	/usr/local/go/src/net/http/server.go:668
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1571 +0x1
created by net/http.(*connReader).startBackgroundRead
	/usr/local/go/src/net/http/server.go:668 +0xca

goroutine 1061 [runnable]:
net/http.(*connReader).startBackgroundRead.func2()
	/usr/local/go/src/net/http/server.go:668
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1571 +0x1
created by net/http.(*connReader).startBackgroundRead
	/usr/local/go/src/net/http/server.go:668 +0xca

goroutine 1078 [runnable]:
github.com/sktan/aws-codeartifact-proxy/tools.ProxyRequestHandler.func1({0x8c1168, 0xc0001500e0}, 0xc000330300)
	/go/src/app/tools/proxy.go:23 +0xde
net/http.HandlerFunc.ServeHTTP(0x7fc0b32d8d98?, {0x8c1168?, 0xc0001500e0?}, 0x40cbc5?)
	/usr/local/go/src/net/http/server.go:2084 +0x2f
net/http.(*ServeMux).ServeHTTP(0x0?, {0x8c1168, 0xc0001500e0}, 0xc000330300)
	/usr/local/go/src/net/http/server.go:2462 +0x149
net/http.serverHandler.ServeHTTP({0xc0001b0540?}, {0x8c1168, 0xc0001500e0}, 0xc000330300)
	/usr/local/go/src/net/http/server.go:2916 +0x43b
net/http.(*conn).serve(0xc0005f2960, {0x8c13f0, 0xc000607cb0})
	/usr/local/go/src/net/http/server.go:1966 +0x5d7
created by net/http.(*Server).Serve
	/usr/local/go/src/net/http/server.go:3071 +0x4db

I'm not too familiar with Go unfortunately. But it looks like I can reproduce the problem as fail happens all the time I'm building this specific android project.

Add ARM64 support

Add ARM64 support for images and binaries for those people who want to run on a more power-efficient platform.

Support pnpm user agent

In proxy.go, there is a conditional which checks if the user agent begins with npm to apply specific npm logic, like switching out the registry URL for the proxy URL in responses.

When it comes to registry access, pnpm works the same as npm, so should be fully compatible with this proxy. However, the user agent which pnpm sends is below.

pnpm/8.6.3 npm/? node/v16.17.1 darwin arm64

This, coupled with the conditional, means that when using pnpm, the proxy responds with the original CodeArtifact URLs for the dist files, resulting in an authentication error, since clients are not authenticated.

I would open a PR for this, but my Go skills aren't too sharp 🙏

Workaround

It's possible to work around this issue, by setting the below in an .npmrc file local to the project. This updates the user agent which pnpm uses to just npm and thus satisfies the conditional in the proxy. However, due to an issue with pnpm, it only works with a project specific .npmrc file.

user-agent=npm

404 error from NPM Install.

Hello,

I tried to use this proxy (running localhost for testing purposes) to install a scoped package from codeartifact and I keep getting the error message below:

'@my-org/<package-name>@http://localhost:8080/@my-org/<package-name>/-/<package-name>-1.26.1.tgz' is not in this registry.

The NPM logs shows a 404 error and my .npmrc file looks like this:

registry=https://registry.npmjs.org
@my-org:registry=http://localhost:8080/

Is there something I am missing?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.