eclipse-orbit / orbit Goto Github PK

View Code? Open in Web Editor NEW

2.0 2.0 3.0 1.87 MB

Third-party libraries bundled using Eclipse Bundle Recipes.

HTML 91.90% Shell 1.34% Batchfile 0.42% Perl 0.19% Python 0.13% XSLT 4.65% Java 0.47% PHP 0.89% JavaScript 0.01%

orbit's Issues

New version of Ant without jai

We need a new version of Ant in Orbit without Jai as we can't distribute Jai from EF - see IP lab

Move this repo to `eclipse-orbit` organization along with Eclipse EBR once `eclipse-orbit` can be registered.

Bundle names do not follow the pattern

Some IPLab issues recently created by the Orbit build bot have bundle IDs that do not match the established pattern.

e.g., https://gitlab.eclipse.org/eclipsefdn/emo-team/iplab/-/issues/10727

The bundle name org.apache.poi.poi.ooxml doesn't match the established pattern org.apache.poi.ooxml (that is, there is one fewer "poi" in the EBR pattern expressed here).

Is this intentional?

Orbit PMI developer page needs to be updated

https://projects.eclipse.org/projects/tools.orbit/developer mentions that Orbit uses Gerrit linking to http://wiki.eclipse.org/Gerrit.
Later it says the project moved to github.

This should be cleaned up.

I guess we can also remove the note about old versions which were built from CVS which is ancient history not really relevant anymore for today's Orbit contributors.

Complete migration to GitHub

The Eclipse Orbit Project is now in GitHub (this repo!). This issue is to track the items left to do to complete the migration with a general goal of getting this all done by 2023-06 M1 on 4 April.

Update CI jobs with new URL
Move releasing steps from wiki.eclipse.org to GitHub
Consider closing out old bugs in Bugzilla (with Resolved -> Moved)
Find and update all locations that say file bugs in bugzilla and migrate that to GitHub issues
#6

Replace Orbit recipes with M2E's 'Maven' target locations fetching from Maven-Central

In order to ease the transition to use M2E's Maven target locations in projects directly and to simplify the update of existing artifacts in Orbit and reduce the technology stack, I want to propose to use Maven type target locations in Orbit.

When migrating a Orbit recipe to a dependency in a Maven target location one has to distinguished if the artifact at Maven-Central already has a OSGi compliant manifest or not:

OSGi compliant artifacts can be used as they are from Maven-Central as simple dependency
For not OSGi compliant artifacts, use instructions element of a Maven-Target. The instructions should probably be similar to the previous Orbit recipe (osgi.bnd).

In some cases this can leads to subtle differences in OSGi metadata and the artifact content. In Orbit some artifacts were split into two (e.g. logback 1.2) or mutliple into one (e.g. guava and its failureaccess dependency).
The generated artifact will also not contain the Eclipse license files. But if they are a requirement the build could be adjusted to add them to the artifacts with generated manifest.

To ensure license compliance of all artifacts in Orbit the reusable eclipse.dash-licenses license-check workflow can be used:

Ed's new SimRel-Maven service (which is great :) ) would then mainly serve the purpose of providing updates for Maven-targets used in SimRel projects until dependabot can do it (dependabot/dependabot-core#6913). At the moment I have the impression that it is also used as extended Orbit with more recent Maven-artifacts from Maven-targets (merks/simrel-maven#3).

If you are interested I can make an initial migration for a simple case, but migrating all recipes will be a greater task and will probably also need some adjustments in consuming projects because of the subtle differences mentioned above.

At the same time this could also be a good opportunity to clean up unused artifacts. But it is probably not sufficient to only look into SimRel is it?

Bundle version number does not match what's in the GAV

We have two IP review requests created by the Orbit build bot for a bundle ID com.github.virtuald.curvesapi.

e.g., https://gitlab.eclipse.org/eclipsefdn/emo-team/iplab/-/issues/10700

In both cases, the version number of the bundle does not match the version number of the corresponding Maven component.

For example, AFAICT, com.github.virtuald.curvesapi_1.7.0 maps to com.github.virtuald:curvesapi:1.07.

Should I expect that the version numbers align?

removed httpmime dependencies cause failures

From https://bugs.eclipse.org/bugs/show_bug.cgi?id=581940

4.5.14 is bundling less:

https://github.com/eclipse/orbit/blob/71e90fcd86da88f8ff3aeb1b10f4429c74a57360/apache/httpcomponents/org.apache.httpcomponents.httpclient_4.5.14/pom.xml#L14-L20

than 4.5.13 which was shipped with 2023-03:

https://github.com/eclipse/orbit/blob/b7acb75ba68159d43d742f7bd166f69772f4d04d/apache/httpcomponents/org.apache.httpcomponents.httpclient_4.5.13/pom.xml#L17-L38

This results in other plugins built against 2023-03 failing when the httpclient is updated, for example when the EGit plugin is installed.

Remove com.ibm.icu4j from Orbit

I've updated the Platform to version 73.1 and added it to my target:

https://github.com/merks/simrel-maven/blob/17fcd1ac411cd73d9c733093981b265a9b8fe49d/merged-target/updated.target#L240-L245

It's available from here:

com.ibm.icu
- icu4j 73.1

And I've done builds to make it available

This would seem like a good first opportunity to remove something from Orbit...

use dependabot

Is it possible to use dependabot or some tool alike to monitor the used libraries?
I manually stumbled across a vulnerable library in orbit which already has a fixed version.

Rename "master" to "main"?

The term "master" as branch name is deprecated. We should rename to "main".

Status and future of javax.xml

I wanted to record some observations about java.xml. There are two versions on the train:

I'm not sure why WTP is explicitly including/requiring the older version.

Note that some other older legacy dependencies also require the older version, so I wanted to figure out why that is...

Looking at the 1.3.4 version, the pretty printed MANIFEST.MF looks like this:

Bundle-Localization                     plugin
Bundle-ManifestVersion                  2
Bundle-Name                             %Bundle-Name.0
Bundle-RequiredExecutionEnvironment     J2SE-1.2
Bundle-SymbolicName                     javax.xml
Bundle-Vendor                           %Bundle-Vendor.0
Bundle-Version                          1.3.4.v201005080400
Export-Package                          javax.xml.datatype;version="1.3"
                                        javax.xml.namespace;version="1.3"
                                        javax.xml.parsers;version="1.3"
                                        javax.xml.transform.dom;version="1.3"
                                        javax.xml.transform.sax;version="1.3"
                                        javax.xml.transform.stream;version="1.3"
                                        javax.xml.transform;version="1.3"
                                        javax.xml.validation;version="1.3"
                                        javax.xml.xpath;version="1.3"
                                        javax.xml;version="1.3"
                                        org.apache.xmlcommons;version="1.3.4"
                                        org.w3c.dom.bootstrap;version="3.0"
                                        org.w3c.dom.css;version="2.0"
                                        org.w3c.dom.events;version="2.0"
                                        org.w3c.dom.html;version="2.0"
                                        org.w3c.dom.ls;version="2.0"
                                        org.w3c.dom.ranges;version="2.0"
                                        org.w3c.dom.stylesheets;version="2.0"
                                        org.w3c.dom.traversal;version="2.0"
                                        org.w3c.dom.views;version="2.0"
                                        org.w3c.dom.xpath;version="3.0"
                                        org.w3c.dom;version="3.0"
                                        org.xml.sax.ext;version="2.0.2"
                                        org.xml.sax.helpers;version="2.0.2"
                                        org.xml.sax;version="2.0.2"
Manifest-Version                        1.0
Require-Bundle                          system.bundle

Note the exports are not imported. Note too the version numbers on the packages.

Compare that to the Orbit 1.4.1 version:

Ant-Version                             Apache Ant 1.7.0
Automatic-Module-Name                   javax.xml
Build-Jdk-Spec                          11
Bundle-ManifestVersion                  2
Bundle-Name                             %bundleName
Bundle-SymbolicName                     javax.xml
Bundle-Vendor                           %bundleVendor
Bundle-Version                          1.4.1.v20220503-2331
Created-By                              Eclipse Bundle Recipe Maven Plug-in
Export-Package                          javax.xml.datatype;version="1.4.1";uses:="javax.xml.namespace"
                                        javax.xml.namespace;version="1.4.1"
                                        javax.xml.parsers;version="1.4.1";uses:="javax.xml.validation,org.w3c.dom,org.xml.sax,org.xml.sax.helpers"
                                        javax.xml.stream.events;version="1.4.1";uses:="javax.xml.namespace,javax.xml.stream"
                                        javax.xml.stream.util;version="1.4.1";uses:="javax.xml.namespace,javax.xml.stream,javax.xml.stream.events"
                                        javax.xml.stream;version="1.4.1";uses:="javax.xml.namespace,javax.xml.stream.events,javax.xml.stream.util,javax.xml.transform"
                                        javax.xml.transform.dom;version="1.4.1";uses:="javax.xml.transform,org.w3c.dom"
                                        javax.xml.transform.sax;version="1.4.1";uses:="javax.xml.transform,org.xml.sax,org.xml.sax.ext"
                                        javax.xml.transform.stax;version="1.4.1";uses:="javax.xml.stream,javax.xml.transform"
                                        javax.xml.transform.stream;version="1.4.1";uses:="javax.xml.transform"
                                        javax.xml.transform;version="1.4.1"
                                        javax.xml.validation;version="1.4.1";uses:="javax.xml.transform,org.w3c.dom,org.w3c.dom.ls,org.xml.sax"
                                        javax.xml.xpath;version="1.4.1";uses:="javax.xml.namespace,org.xml.sax"
                                        javax.xml;version="1.4.1"
                                        org.apache.xmlcommons;version="1.4.1"
                                        org.w3c.dom.bootstrap;version="1.4.1";uses:="org.w3c.dom"
                                        org.w3c.dom.css;version="1.4.1";uses:="org.w3c.dom,org.w3c.dom.stylesheets,org.w3c.dom.views"
                                        org.w3c.dom.events;version="1.4.1";uses:="org.w3c.dom,org.w3c.dom.views"
                                        org.w3c.dom.html;version="1.4.1";uses:="org.w3c.dom"
                                        org.w3c.dom.ls;version="1.4.1";uses:="org.w3c.dom,org.w3c.dom.events,org.w3c.dom.traversal"
                                        org.w3c.dom.ranges;version="1.4.1";uses:="org.w3c.dom"
                                        org.w3c.dom.stylesheets;version="1.4.1";uses:="org.w3c.dom"
                                        org.w3c.dom.traversal;version="1.4.1";uses:="org.w3c.dom"
                                        org.w3c.dom.views;version="1.4.1"
                                        org.w3c.dom.xpath;version="1.4.1";uses:="org.w3c.dom"
                                        org.w3c.dom;version="1.4.1"
                                        org.xml.sax.ext;version="1.4.1";uses:="org.xml.sax,org.xml.sax.helpers"
                                        org.xml.sax.helpers;version="1.4.1";uses:="org.xml.sax"
                                        org.xml.sax;version="1.4.1"
Import-Package                          javax.xml.namespace;version="[1.4,2)"
                                        javax.xml.stream.events;version="[1.4,2)"
                                        javax.xml.stream.util;version="[1.4,2)"
                                        javax.xml.stream;version="[1.4,2)"
                                        javax.xml.transform.stream;version="[1.4,2)"
                                        javax.xml.transform;version="[1.4,2)"
                                        javax.xml.validation;version="[1.4,2)"
                                        org.w3c.dom.events;version="[1.4,2)"
                                        org.w3c.dom.ls;version="[1.4,2)"
                                        org.w3c.dom.stylesheets;version="[1.4,2)"
                                        org.w3c.dom.traversal;version="[1.4,2)"
                                        org.w3c.dom.views;version="[1.4,2)"
                                        org.w3c.dom;version="[1.4,2)"
                                        org.xml.sax.ext;version="[1.4,2)"
                                        org.xml.sax.helpers;version="[1.4,2)"
                                        org.xml.sax;version="[1.4,2)"
Manifest-Version                        1.0
Originally-Created-By                   1.3.1 (IBM Corporation)
Require-Bundle                          system.bundle
Require-Capability                      osgi.ee;filter:="(&(osgi.ee=JavaSE)(version=1.2))"

Note the package versions and note the non-optional imports of the exports with bounded version ranges.

This is not really a good drop-in replacement for the 1.3.4 version and I believe that the upper bounds on the imported exports is what makes package imports requirement version 2.x or higher require the older 1.3.4 version instead of the newer one.

Here is a new version that I created using an m2e location:

Ant-Version                             Apache Ant 1.7.0
Bundle-ManifestVersion                  2
Bundle-Name                             Bundle xml-apis : xml-apis
Bundle-SymbolicName                     javax.xml
Bundle-Version                          1.4.1.v20230703-0729
Created-By                              1.3.1 (IBM Corporation)
Eclipse-Wrapped-Bundle                  xml-apis:xml-apis:1.4.1
Export-Package                          javax.xml.datatype;version="1.4.1";uses:="javax.xml.namespace"
                                        javax.xml.namespace;version="1.4.1"
                                        javax.xml.parsers;version="1.4.1";uses:="javax.xml.validation,org.w3c.dom,org.xml.sax,org.xml.sax.helpers"
                                        javax.xml.stream.events;version="1.4.1";uses:="javax.xml.namespace,javax.xml.stream"
                                        javax.xml.stream.util;version="1.4.1";uses:="javax.xml.namespace,javax.xml.stream,javax.xml.stream.events"
                                        javax.xml.stream;version="1.4.1";uses:="javax.xml.namespace,javax.xml.stream.events,javax.xml.stream.util,javax.xml.transform"
                                        javax.xml.transform.dom;version="1.4.1";uses:="javax.xml.transform,org.w3c.dom"
                                        javax.xml.transform.sax;version="1.4.1";uses:="javax.xml.transform,org.xml.sax,org.xml.sax.ext"
                                        javax.xml.transform.stax;version="1.4.1";uses:="javax.xml.stream,javax.xml.transform"
                                        javax.xml.transform.stream;version="1.4.1";uses:="javax.xml.transform"
                                        javax.xml.transform;version="1.4.1"
                                        javax.xml.validation;version="1.4.1";uses:="javax.xml.transform,org.w3c.dom,org.w3c.dom.ls,org.xml.sax"
                                        javax.xml.xpath;version="1.4.1";uses:="javax.xml.namespace,org.xml.sax"
                                        javax.xml;version="1.4.1"
                                        license;version="1.4.1"
                                        org.apache.xmlcommons;version="1.4.1"
                                        org.w3c.dom.bootstrap;version="1.4.1";uses:="org.w3c.dom"
                                        org.w3c.dom.css;version="1.4.1";uses:="org.w3c.dom,org.w3c.dom.stylesheets,org.w3c.dom.views"
                                        org.w3c.dom.events;version="1.4.1";uses:="org.w3c.dom,org.w3c.dom.views"
                                        org.w3c.dom.html;version="1.4.1";uses:="org.w3c.dom"
                                        org.w3c.dom.ls;version="1.4.1";uses:="org.w3c.dom,org.w3c.dom.events,org.w3c.dom.traversal"
                                        org.w3c.dom.ranges;version="1.4.1";uses:="org.w3c.dom"
                                        org.w3c.dom.stylesheets;version="1.4.1";uses:="org.w3c.dom"
                                        org.w3c.dom.traversal;version="1.4.1";uses:="org.w3c.dom"
                                        org.w3c.dom.views;version="1.4.1"
                                        org.w3c.dom.xpath;version="1.4.1";uses:="org.w3c.dom"
                                        org.w3c.dom;version="1.4.1"
                                        org.xml.sax.ext;version="1.4.1";uses:="org.xml.sax,org.xml.sax.helpers"
                                        org.xml.sax.helpers;version="1.4.1";uses:="org.xml.sax"
                                        org.xml.sax;version="1.4.1"
Manifest-Version                        1.0
Originally-Created-By                   1.3.1 (IBM Corporation)
Require-Capability                      osgi.ee;filter:="(&(osgi.ee=JRE)(version=1.1))"

Note the extremely low generated BREE. It uses this recipe, hence to imported exports:

Bundle-Name:            Bundle ${mvnGroupId} : ${mvnArtifactId}
version:                ${versionmask;===;${version_cleanup;${mvnVersion}}}
Bundle-SymbolicName:    javax.xml
Bundle-Version:         ${version}.v20230703-0729
Eclipse-Wrapped-Bundle: ${mvnGroupId}:${mvnArtifactId}:${version}
Import-Package:         *
Export-Package:         *;version="${version}";-noimport:=true
-noextraheaders:        true
-removeheaders:         Private-Package

Probably this version is a drop-in replacement for the 1.3.4 version...

Remove bundles that have OSGi metadata upstream

For a long time Orbit's re-bundling was the way to get third-party libraries from Maven into an Eclipse p2 consumable state.

For a while now there has been a better way for getting third-party dependencies that already contains OSGi metadata. That is to use the M2E's maven target locations and sign the bundles with PGP instead of jarsigning. There has been much communication on mailing lists and in issues on this topic in the the last couple of years.

The next step now is to stop maintaining the Orbit bundles that rewrite the OSGi metadata when the upstream bundle has the OSGi metadata in it correctly already.

The goal is to have the first step of this process done for 2023-09 M1 by removing the bundles. In addition @merks's work in https://github.com/eclipse-oomph/oomph.incubator will be migrated into the Orbit project so that Orbit provides p2 repos with the PGP signed third-party dependencies already in them.

Here are the bundles that have valid OSGi metadata in the upstream version that we plan to remove. This list will be added to as we vet additional bundles.

Did the BundleID change?

The Eclipse Dash License Tool uses this repository to figure out how to map a bundle Id (e.g., com.atlassian.commonmark-gfm-tables) to the corresponding GAV (e.g., com.atlassian.commonmark:commonmark-ext-gfm-tables). The scanner walks this repository and pulls this information out of the pom.xml files. It generally works (with some small amount of hinting).

The expectation is that as we encounter new versions of things, they are likely to have similar IDs to the older versions. The exceptions are handled with the aforementioned small amount of hinting.

The Orbit build (the only information that I have is that the issue "was created 2 days ago by orbit bot user) recently created a handful of reviews for which the scanner cannot find source code.

For example: https://gitlab.eclipse.org/eclipsefdn/emo-team/iplab/-/issues/10695 (there's actually a bunch of similar "commonmark" entries).

Here... we need to map the bundle ID org.commonmark-gfm-tables. The closest that I can find in this repository is a bundle named com.atlassian.commonmark-gfm-tables.

Are these the same thing? Did the name change intentionally? Is this a mistake?

Eliminate "Legacy" Bundle

In order to produce this aggregation:

https://download.eclipse.org/tools/orbit/simrel/orbit-aggregation

these really, really old bundles are dragged in:

These all need to be removed or replaced.

A bunch of these are dragged in to make batik happy, i.e., the 3 selected bundles above. They could be replaced by one of these:

But those things do not provide sources and I don't think it's okay to not provide sources. It's certainly not nice and not ideal.

Note that I tried to find if anything on Maven Central provides sources and have reached the conclusion that the only thing providing sources are the old Orbit source bundles themselves.

The problem isn't just that these are old, they also have package imports that could be satisfied by the JDK, but not when there is a version constraint because the JDK's packages are not versioned.

I would like to propose that we create a new Git repository to maintain bundles in source form for bundles that cannot be created by bundling a Maven Central artifact. Of course we want as few of those as possible!

I've prototyped the following repository contents:

I've take the three source bundles and combined them into a single bundle with the same combined contents as the above two Maven artifacts.

If I use the resulting repository in the aggregation, we can see that the new bundle eliminates the org.w3c.dom.events bundle too, it satisfies all the batik constraints, and it will allow us to eliminate the javax.xml bundle too.

#48

@jonahgraham

Shall I create a new orbit-legacy Git repository with the initial contents as outlined above to produce a p2 repository https://download.eclipse.org/tools/orbit/simrel/orbit-legacy that can then be used in the aggregation to replace using an actual older orbit ebr repository?

Do you have any questions or concerns?

Remove EJML (v0.29) bundles from Orbit

A newer version (v0.43) is available here:
http://ejml.org

Furthermore, we faced some problems with conflicting classes when consuming EJML via Orbit. Nowadays, it works fine to reference the EJML dependencies in the product definition by using the Maven dependency option.

If no one else has objections, I will remove the bundles soon.

CD for Orbit

Orbit should have a CD pipeline, i.e. after a PR is merged, a new repository should be provided and published under a "latest" or "main" (as in "main" branch) URL.

eclipse-orbit / orbit Goto Github PK

orbit's Issues

Recommend Projects

Recommend Topics

Recommend Org