Support any composer-type repository about nexus-repository-composer HOT 8 OPEN

Hi, will this issue get some traction any time soon?

from nexus-repository-composer.

Thanks for writing this up. This is actually a known issue with the implementation as we're essentially trying to get a working MVP out there for the community to build on rather than trying to make it perfect. What you describe very much parallels many of our early learnings in reverse-engineering/studying this format; there are a couple of primary reasons we didn't use the approach you described as the MVP, so here's a bit of background:

One reason is that there's some significant overhead in trying to process and rewrite those nested layers of provider files, and it's not something that our current formats infrastructure is necessarily the best match for. For example, Nexus has to rewrite download links in order to redirect the client through the repository manager, but rewriting one of the JSON files changes its hash, which means that a change at any "leaf" JSON file will propagate up the entire chain. Also consider that we don't know what files will be requested, so we cannot do the rewrite in advance, thus necessitating some sort of mirror or sync mechanism to keep things matching the upstream repository. This is not unheard of, but it's a poor fit for the normal mechanisms of Nexus proxy repositories. In many respects, the sort of design that went into making Composer a more challenging target for a man-in-the-middle attack at the index-file/registry level also makes it more challenging for us to effectively go in and change things.

The other reason is that flattening the list of providers also helps us in other ways, such as with implementing the group merging feature of the Nexus repo. Group repositories have to merge the contents of all member repositories, and in different repositories with different provider/nesting structures and filenames, the group merge could quickly get a lot more expensive and certainly won't be something that we could do on a lazy basis. We have other formats that have to do this kind of more complicated metadata merging across different repositories and while we could figure it out, it would have taken a lot longer than we had available to devote to what's essentially a spare-time project done under the Sonatype Community aegis.

That said, this is something that's been on my radar that I'm hoping to get around to once we have a minimal implementation of proxy, hosted, and group working for Composer. Until then, I'd say that our Composer proxy also requires the additional API method provided by Packagist at this time, though we're always happy if someone wants to start picking up one of these issues and coming up with possible paths forward.

(One suggestion I might have would be to see how difficult in terms of implementation and how expensive in terms of performance it might be to synthesize the equivalent of a list.json file by traversing the providers as you describe, using that as the reference to build the flattened JSON file that proxy is currently serving up as a substitute. That said, that's not a complete solution because you will need the sha values in order to access the packages hosted by Drupal even though we don't need those when accessing the same files on Packagist, but it might be a place to start an investigation.)

from nexus-repository-composer.

I've given this a bit more thought and another option might be to rely on the same "hack" that we currently use when building the provider file, namely that if the sha is blank (but the key is still present) the composer client effectively ignores the sha and fetches the file without an associated hash in the filename.

So long as this holds we might be able to minimize the rewrites we have to coming up from the leaf nodes, but we'd have to have a way to back-associate the sha values in order to proxy the targeted file. I think we'd also have to still rewrite the provider-includes section, but that's not that much worse than what we currently have to do for other formats.

Note that I have not come up with a POC for this and likely won't for some time. Also note that this solution doesn't take into account the group merge behavior, but perhaps we could combine the approach with a "flattening" sort of transformation like we're currently doing. Once I get done with hosted (and perhaps group) I may investigate this further if nobody else has grabbed it by then.

from nexus-repository-composer.

One suggestion I might have would be to see how difficult in terms of implementation and how expensive in terms of performance it might be to synthesize the equivalent of a list.json file by traversing the providers as you describe, using that as the reference to build the flattened JSON file that proxy is currently serving up as a substitute

That's exactly what I was thinking in the first place. I don't think we need at first a "copy" of the way composer is working, but rather a way for this plugin to use other composer repositories (particularly the Drupal one)

Note that I have not come up with a POC for this and likely won't for some time

No problem, we've come with a temporary solution with this issue. I've forked the repo and wanted to fix this issue, but I've discovered that I don't have enough knowledge with Nexus nor with Java, neither do I have enough time to learn everything to fix this :-/

from nexus-repository-composer.

@Elendev if you got it working, post up a PR. We can always work together to try and get it done together :)

from nexus-repository-composer.

@DarthHater unfortunately I just took a look at the code but I wasn't able to find time to start a PR about this, so feel free to start working on this issue :-)

from nexus-repository-composer.

Some improvements will be made as part of #22 but will not solve the matter for all repositories. In particular, I think we have to make some other significant changes (supporting sources) to be able to get it working with the Drupal repositories, which we'll have to leave for a later date. Refer to the conversation on that PR for details.

from nexus-repository-composer.

Still no news ?

from nexus-repository-composer.