Allow usage of GOPROXY variable for go module downloads about ort HOT 10 CLOSED

By strength you are referring to execution time / speed (or anything else)? The choice whether to disable the proxy was a trade-off:

* speed vs.  correctness of result + much less complex implementation

..we've deliberately chosen the latter.

Then I need to add a SSH key and this configuration when I run ORT in a Gitlab-CI and this method skips the strengths of using a Go proxy.

This would already solve the entire issue, right? (Maybe an alternative could be to make the repos clonable without authentication, as they are accessible already anyway through the proxy)

Yeah, the execution time was my concern in this case. And we typically run our Artifactory with more permissive permissions instead of the Gitlab instance, so the usage of the Go proxy would have been a nice option and is our state of truth in many cases. But I will have fallback to git over SSH in the current state.

I looked into some packages that are served by our private Go proxy and found license files and much more which are needed by ort and scancode to determine the relevant information.

You say this to emphasize that skipping private packages is not an option?

I am not aware that I can skip private packages. Please give me a quick hint on how to do that.
I think this is no option because of the nested dependencies that are used in the private packages.

And I personally like the use of the Go proxy to avoid getting rate limited by global VCS like github.com 😄

To my knowledge rate limits apply to the use of GitHubs REST Api, but not to normal Git operations, such as clones. Would you have some evidence that using Go tooling with disabled proxy would run into rate limiting issues?

I have looked up this and you are right!

A quick summary: I will do a fallback to git over SSH for Go dependencies.
A huge thank you for the quick response. That is not really common in many GitHub repositories.

from ort.

I am not aware that I can skip private packages. Please give me a quick hint on how to do that.
I think this is no option because of the nested dependencies that are used in the private packages.

I believe there was a misunderstanding. I was just trying to clarify why you mentioned this in the first place,
because I didn't get it.

A quick summary: I will do a fallback to git over SSH for Go dependencies.

@elivlo : Great, so it's fine to close this issue as "clarified" ?

from ort.

Maybe @fviernau or @haikoschol can assist here?

from ort.

Go proxy has been deliberately disabled in Ort's Go mod integration, solely to derive the VCS info entirely from the tooling. At the time of doing so (with the current Go version) this was the only way of figuring out the VCS info without re-implementing the rather complex logic from the Go tooling.

I'd look for options which work without the use of a Go proxy first. E.g. you say

are not downloadable with git over HTTP

So, how can you download sources from the corresponding VCSes? via SSH ?

from ort.

So, how can you download sources from the corresponding VCSes? via SSH ?

I tried using this configuration but then I need authentication with an SSH key:
git config --global url.git@<self-hosted-git-domain>:.insteadOf https://<self-hosted-git-domain>/

Then I need to add a SSH key and this configuration when I run ORT in a Gitlab-CI and this method skips the strengths of using a Go proxy. I looked into some packages that are served by our private Go proxy and found license files and much more which are needed by ort and scancode to determine the relevant information.

And I personally like the use of the Go proxy to avoid getting rate limited by global VCS like github.com 😄

from ort.

Then I need to add a SSH key and this configuration when I run ORT in a Gitlab-CI and this method skips the strengths of using a Go proxy.

By strength you are referring to execution time / speed (or anything else)?
The choice whether to disable the proxy was a trade-off:

• speed vs. correctness of result + much less complex implementation

..we've deliberately chosen the latter.

I looked into some packages that are served by our private Go proxy and found license files and much more which are needed by ort and scancode to determine the relevant information.

You say this to emphasize that skipping private packages is not an option?

And I personally like the use of the Go proxy to avoid getting rate limited by global VCS like github.com 😄

To my knowledge rate limits apply to the use of GitHubs REST Api, but not to normal Git operations, such as clones.
Would you have some evidence that using Go tooling with disabled proxy would run into rate limiting issues?

Then I need to add a SSH key and this configuration when I run ORT in a Gitlab-CI and this method skips the strengths of using a Go proxy.

This would already solve the entire issue, right? (Maybe an alternative could be to make the repos clonable without authentication, as they are accessible already anyway through the proxy)

from ort.

• speed vs. correctness of result + much less complex implementation

..we've deliberately chosen the latter.

True. However, also see #8361 in this context.

from ort.

True. However, also see #8361 in this context.

@elivlo : I've wrote above assuming the discussion is around ORT's currently existing GoMod analyzer only, not around any not yet existing alternative implementations.

from ort.

@elivlo : I've wrote above assuming the discussion is around ORT's currently existing GoMod analyzer only, not around any not yet existing alternative implementations.

I guess this was actually directed at me. Yes, I'm aware of that, but I wanted to point out that as least my perspective us unconstitutionally favoring accuracy over speed has relativized, and I think there's a fair use-case for favoring speed over accuracy. But let's continue any such discussion in #8361.

from ort.

fair use-case for favoring speed over accuracy.

In this case of GoMod the difference in implementation + maintainance cost is huge IMO, which is why
(given the development power we currently have) I believe this is very unrealistic to happen.
But sure let's not discuss this here.

from ort.