seancrowe / chili-pdf-compare Goto Github PK

chili-pdf-compare was created as a Windows tool to compare PDFs for changes between CHILI versions. It is built on diff-pdf, an open-source visual PDF comparison tool.

There are many uses cases for such a tool:

• Creating automatic testing for CHILI publish Online releases
• Comparing PDFs for output consistence testing
• Comparing PDF consistence across different environments or different versions

At the current moment chili-pdf-compare is considered to be "pre-released", and will most likely stay in "pre-released" until someone volunteers to use this in production.

While the main features are covered by tests, there are still some code that needs coverage and situations that need tests written.

In addition to adding test cases, there could be some improvements to the tool such as a better way to present PDF downloads as a progress bar, better error handling for the generating PDF part, and better handling of edge case errors.

In addition, some things have not been tested or do not work such as: password-protected PDFs, PDFs with hundreds of pages, zip output, and PDFs with paragraph anchors.

With that written, this tool was targeted for testing small multiple-page PDFs and should work out of the box with only minor (hopefully 🤞 no) bugs for that use case.

Installation is easy because this is a portable app:

1. Download the latest release from the Github page
2. Unzip your package to any folder you wish

There you go you just installed it, and to remove, just delete the folder.

⛔ Warning: This executable is not signed nor is it famous enough to be known by anti-virus. Therefore, some very strict environments may falsely flag this dangerous. Last I checked, VirusTotal rates this "Undetected" threat from all their engines. If you are concerned, feel free to build this yourself with the instructions below.

chili-pdf-compare works off of two folders, and a configuration file.

Source/Compare Folders The first folder is called the "source" folder. This folder should contain all the PDFs which will be used for comparison. These are all the "correct" PDFs - the ones where the visual output is accepted and approved.

The second folder is called the "compare" folder. This folder should contain all the PDFs which you want to compare to their approved counterparts in the "source" folder. Because chili-pdf-compare compares PDFs from the "compare" folder to the "source" folder, it is important that the names which you want to compare match.

For example, if your compare folder contained:

-📁 Compare
 -📄 Very Cool.pdf
 -📄 Neato.pdf
 -📄 Awesome.pdf

Then your source folder must contain the same named PDFs

-📁 Source
 -📄 Very Cool.pdf
 -📄 Neato.pdf
 -📄 Awesome.pdf

If you are missing one or more PDFs in the source the folder then the results will be labeled as "missing" to notify you that the there was no source file to compare.

Inversely, if you have more PDFs in your source folder than in your compare folder, then that is okay. chili-pdf-compare will only compare the files that exist in the compare folder to their counterparts in source folder.

📝Side Note If you have not figured out already, chili-pdf-compare relies heavily on naming. To compare two PDFs, one must exist in the source folder with the same name as the second, which exists in the compare folder.

Configuration Files To tell chili-pdf-compare where your source folder and compare folder are located, you must use a configuration file.

A configuration file is a JSON file that is formatted like so:

{
	"sourcePath": "path to source folder",
	"comparePath": "path to compare folder"
}

The path can be absolute or relative to the executable. For example:

{
	"sourcePath": "C:\\Users\\Sean\\CHILI\\Source", <-- Absolute
	"comparePath": ".\\Compare" <-- Relative
}

Your folders do not need to be named "Source" and "Compare", but it is nice to keep similar naming conventions for clarity.

The name and location of your configuration file is not really important, as we will tell chili-pdf-compare the location at launch, but keeping with naming conventions, I suggest calling it "config.json".

Generate PDFs While the tool was originally designed to compare PDFs that were generated by the Editor or another application, you can instead tell chili-pdf-compare to generate the PDFs.

In your configuration file, add the following "server" property with "url", "apiKey", and "documents".

url - must be the URL to the base of your CHILI REST API. For example, ft-nostress BackOffice is lcoated at:

https://ft-nostress.chili-publish-sandbox.online/ft-nostress/interface.aspx

So, that means the REST API is located at:

https://ft-nostress.chili-publish-sandbox.online/rest-api/version/

Therefore, our URL would be

{
	"sourcePath": "C:\\Users\\Sean\\CHILI\\Source",
	"comparePath": ".\\Compare",
	"server": {
		"url": "https://ft-nostress.chili-publish-sandbox.online/"
	}
}

apiKey - must be an active API key for the particular environment you want to generate PDFs from.

You can get the API key using REST, or you can snag it in the BackOffice after logging in and pressing the "Get URL" button for any Document. (You could also snag it from the network tab.)

Because CHILI publish Online currently does not support environment switching, chili-pdf-compare only supports one environment (the one tied to your API key). If you want to test multiple environments, you would need to either run multiple tests or modify the source code to allow environment switching.

{
	"sourcePath": "C:\\Users\\Sean\\CHILI\\Source",
	"comparePath": ".\\Compare",
	"server": {
		"url": "https://ft-nostress.chili-publish.online/",
	    "apiKey": "GX0ohac4kIeDCqbd5zV3ZWwR7_nJDKodjRf_T4NgxBgB8rpdIWO3nH4BkegKq7rMXFT9qD_RpP81ptrJ_dUPUg=="
    }
}

documents - is any array of document objects with the id of the CHILI Document to generate using the id of the PDF Export Setting.

Failing to provide both will cause in the PDF being skipped.

{
	"sourcePath": "C:\\Users\\Sean\\CHILI\\Source",
	"comparePath": ".\\Compare",
	"server": {
		"url": "https://ft-nostress.chili-publish.online/",
	    "apiKey": "GX0ohbc4kIeDCqbd5zV3ZWwR7_nJDKodjRf_T4NgxBgB8rpdIWO3nH4BkegKq7rMXFT9qD_RpP81ptrJ_dUPUg==",
	    "documents": [
	      {
	        "id": "2ff3bb78-e727-4cb6-b89c-fad9ad9e52ff",
	        "pdfId": "69cac599-8bee-460f-d04e-34b993eaef3f"
	      },
	      {
	        "id": "6b43db91-2293-42f3-a762-6670772081f4",
	        "pdfId": "69cac599-8bee-460f-d04e-34b993eaef3f"
	      }
	    ]
	}
}

• Always test PDFs using the same PDF Export Settings. Failing to do so will cause false positives
• Any PDF Export Settings which returns a zip is not supported (no unzipping at the moment)
• Generating VDP is not tested
• Large PDF lengths (like hundreds to thousands of pages) have not been tested and would cause extreme slowdown

Once you have:

• configuration file setup and saved to disk
• source folder created with the accepted and approved PDFs
• compare folder created either with PDFs or expecting PDFs to be generated -- If you do not provide the compare folder in configuration file, a folder called "tempDiffDownload" will be created for you in the executable's folder.

...You are ready to run chili-pdf-compare in 3 simple steps:

1. Open up your command prompt and navigate to your chili-pdf-compare.exe location
2. In the terminal type chili-pdf-compare.exe
3. Then add the path to the configuration file using the --config or -c flag chili-pdf-compare.exe -c "./config.json"

The results will then output in the same folder at the executable.
You can change the results output location by providing the --output or -o flag. chili-pdf-compare.exe -c "./config.json" -o "./path/to/source"

Example In this example, we have our source folder located at

C:\CHILI_test\source_pdfs

In this folder with have all of our approved PDFs.

We are going to a mixture of generating documents and using documents we already generated as PDFs from out system. We place the already generated documents in new folder called "compare_pdfs" located at

C:\CHILI_test\compare_pdfs

We download the zip and place the executable at

C:\Users\Sean\Downloads\chili-pdf-compare

We are also going to create a configuration file at this location called "config.json". The file is going to contain the following JSON with our documents we wish to generate.

{
	"sourcePath": "C:\\CHILI_test\\compare_pdfs",
	"comparePath": "C:\\Users\\Sean\\Downloads\\chili-pdf-compare",
	"server": {
		"url": "https://ft-nostress.chili-publish.online/",
		"apiKey": "GX0ohbc4kIeDCqbd5zV3ZWwR7_nJDKodjRf_T4NgxBgB8rpdIWO3nH4BkegKq7rMXFT9qD_RpP81ptrJ_dUPUg==",
		"documents": [
			{
				"id": "88bf4478-070d-4099-ae45-e73c81ead8ae",
				"pdfId": "be1361c5-82ee-4daa-8e56-c83c824eac81"
			},
			{
				"id": "babef5df-209f-4b0f-be55-4b28fc02918d",
				"pdfId": "be1361c5-82ee-4daa-8e56-c83c824eac81"
			},
			{
				"id": "2dd3ce3c-dc4a-4047-a42b-e9b26df12530",
				"pdfId": "be1361c5-82ee-4daa-8e56-c83c824eac81"
			}
		]
	}
}

cd C:\Users\Sean\Downloads\chili-pdf-compare\

.\chili-pdf-compare.exe -c ".\config.json"

Then we will get a "results.csv" file in the C:\Users\Sean\Downloads\chili-pdf-compare\ directory when complete.

If we did not want to generate PDFs, but just test the ones locally in the C:\Users\Sean\Downloads\chili-pdf-compare folder, we could remove the "server" object from our JSON conifurgation.

{
	"sourcePath": "C:\\CHILI_test\\compare_pdfs",
	"comparePath": "C:\\Users\\Sean\\Downloads\\chili-pdf-compare"
}

After successfully running chili-pdf-compare you will get a results.csv file which will contain two columns: different and path.

• different allows for three values: true, false, or missing -- true indicates the source PDF is different from the compare PDF -- false indicates the source PDF is the same as the compare PDF -- missing indicates no source PDF was found
• path is the path of the PDF in the compare folder

If a PDF is found to be "true", then that should signal a further investigation as the PDFs were found to be different. Potentially the difference is caused by a change in the rendering engine, but it could also be caused by using the wrong PDF export settings, a change in variables, or someone editing the document.

You can compare two PDFs visually by using diff-pdf.exe found in the lib folder. See diff-pdf for more info. However, I tend to find that if the PDFs are off by a few pixels, the visual difference will not be noticable, and you will get a false highlighting.

If the difference is verified to be legitament, then I suggest retesting, determining if the difference is acceptiable, and potentially update your source files.

Building is simple, but can only be done on Windows with this repository. Potentially you can swap out diff-pdf with the Linux build.

Steps to build from source:

1. Install Node v14 or higher
2. Download this repository
3. Run npm install in the root of this repository
4. Run npm run setup in the root of this repository
5. Run npm run build in the root of this repository
6. Find the built executable in the ./dist-pkg folder