We need to take a sample of packages with (at least 1 release after 2019?) and the timestamp property in the POM. The rest we know for sure are not reproducible due to different timestamps of build artefacts on each build. This means re-running everything for these new packages!!!

Found out that URLs with the git:// prefix always time out when using ls-remote. Converting it to https:// makes the links work -_-

Fix the indexer reader to only accept packaging type of executables.
Change the query in the database accordingly

Build successful, partially reproducible

• Only pom and resource files reproducible, not jar (Maybe wrong JDK used?)
• Reproducible files belong to other packages in the same repo
  org.apache.maven.wagon,wagon-http-lightweight,3.5.0

Build successful, not reproducible

• Non-default build params?
• Wrong JDK used

Build fail

• Not built with Maven
• Correct JDK Toolchain not present
• Build command should include location of pom file (common in repos with multiple projects)
• Other packages/dependencies in same repository failing (we need to ONLY build our package) org.apache.atlas,impala-bridge,2.3.0
• (Maven Enforcer Plugin) Detected Maven Version: 3.6.3 is not in the allowed range [3.8, ) eu.maveniverse.maven.mima,context,1.0.3
• Some dependencies not in Maven Central
  com.adobe.dx,core,0.0.12
• Repository empty (project split into multiple smaller repositories)
  com.alibaba.lindorm,lucene-facet,8.9.0
• Failed to set up Docker container environment
  de.jflex,benchmark,1.9.0 failed because of a missing apt dependency but we can build it normally
• Builds correctly but other packages in same repo that come after it fail. But the .buildcompare should STILL be there!!?
• Failed to clone (possibly network downtime)
  org.simplify4u,pgp-keys-map,2021.07.02

Conclusions

• Ideally we tell Maven to build only our package (+ only required dependencies in the same repo)
• We might want to keep the build cache so that we can run diffoscope. Otherwise, we have to rebuild each project we want to inspect. Alternatiely, only keep the buildcache for non-/partially-reproducible packages to save space.

Either by command line or from a config file

https://maven.apache.org/resolver/about-checksums.html

• Record all packages without a checksum.
• Make sure we actually resolve the checksum files from Maven repo

For packages that have repository url in the database, search the content of their repository for build.gradle and build.xml. This logic should be implemented in the python code before we do the building and after the repository cloning. We can also store two boolean columns on the data base for this:isGradle and isAnt.

Pick a random sample of packages and check the correctness of the tags found.
Make an exhaustive list of correct common tag naming conventions found in the dataset and remove all that don't match.

We could write an extractor to get the required maven version to build.

Examples:

• com.io7m.calino,com.io7m.calino.vanilla,0.0.1
  Detected Maven Version: 3.6.3 is not in the allowed range [3.8.2,4.0.0).

Running the artifact in a docker container does not work as it is missing the required class definitions from dependencies.
Configure Maven shade plugin to include dependencies in the final jar or include them in the classpath when running the jar.

Rerun all from RC, 100 with timestamp but not RC, 100 without timestamp

Blocked by #97

Bare minimum

• Insert build params and result to db
• Change db query to resume from unprocessed (and add on conflict ignore)
• Look for .buildcompare, if exists, build was successful
• Read .buildcompare, save okFiles and koFiles
• Parse and convert major java versions to format accepted in .buildspec
• Check if any file corresponds to the jar of the package

Extra

• Parse class file java version hex into major version
• If no java version found, try each individually
• Merge projects with same URL to avoid rebuilding
• Guess newline from pom.properties file (Example https://github.com/apache/maven-artifact-plugin/blob/maven-artifact-plugin-3.4.1/src/main/java/org/apache/maven/plugins/artifact/buildinfo/ReferenceBuildinfoUtil.java)

Devise a strategy to calculate a reproducibility score.
Initial idea: For each archive, calculate the MD5 of its constituents and compare to the reference version. Save the number of reproducible files and their respective types.. Might need to change pkey of builds table to a generated id and create a new table with entries for each filetype for build.

We can assign different weights to filetypes eg. more weight to class files vs resource files

Examples:
io.wcm:io.wcm.testing.aem-mock:2.2.16 (${site.url} is 2 parents up)

-use docker

Instead of running packages one by one, the process can be parallelized.

Perhaps the compiling JDK version is the same as the target JDK version. Worth a try. Use Gideon's results for this

Currently, if any build for a specific package is already present in the builds db, it is not fetched anymore. Since we are now building each package multiple times, if the experiment is stopped halfway or crashes, any builds left for the package are not resumed.

How to fix:

1. Fetch ALL packages from tags table matching our requirements
2. Let all build parameters be determined, but right before building check whether a build with those exact parameters already exists. If yes, skip this build.

Why? Multimodule projects are stored under an arbitrary artifactid of any of the submodules due to a simplifacation in the RC project.
If our package isn't exactly this arbitrary artifactid, then we don't find its buildspec EVEN though another sub-module in the same project might have the buildspec for the entire project.

This requires an overhaul of how the buildspec is searched for, parsing the READMEs is probably the best way to go about it.

Figure out a way to extract the reason for:

• Build failures
• Non-reproducibility
• Partial reproducibility

1. Try to condense stdout and stderr for each build to make them human-readable and see if there are common failure outputs that we can group together for further analysis. #89
2. Find only the relevant files for each package if it is part of a larger repository. (perhaps we can find metadata containing all the filenames on Maven Central) (#link-issue)
3. Use diffoscope on non/partially-reproducible packages to figure out which files are not the same (#link-issue)

I realised that some developers might do strange things and put their repo url in some other fields but not in scm.url. Need to add the following fields in VCExtractor

• scm.developerConnection
• scm.connection

• Save unparseable hostname urls to an error table

Not sure if it was just a fluke or not but when we ran index 720 today none of the artifacts had these checksums. However on previous runs, about 10% had them. We need investigate this further.

SQL query is broken, currently only eliminates different versions of the same package in the same year, not overall.

Create README.md in both the root and the analyzer module.

Some packages are listed in the index as having the packagingType 'module', but they actually do have a .jar executable. Code will fail because it will try to open the .module file instead of the .jar.

Crashes when trying to run on Docker because it can't find the submodule, as it is not copied. The working Dockerfile is on the server, I edited it manually.

Remove pkey constraint, add a serial build_id pkey.

Packages where line_ending_lf and/or line_ending_crlf are null in 'packages' table implictly don't have pom.properties. As such, they are 99% likely not built with Maven.

• Remove existing non-Maven builds
• Change builder SQL fetch query

The database image is started after the executable is ran, resulting in database connections timing out / failing.

Select only 1 version from each package