versione italiana - italian version

This project is born from the need to simply extract .P7M files content in Delphi / Free Pascal / Lazarus applications.
Starting January 1st, 2019 italian developers have faced the new centralized digital invoicing system which has become mandatory by law.
Digital invoices are basically represented into XML files that are then digitally signed using CAdES-BES mode.
Developers managing invoices need to generate those XMLs and be able to read them back, but when a digitally signed invoice is enveloped into a .P7M file many of them ended up stripping it down by finding the position of the opening XML tag and the closing tag and then copying that part of the file.
We thought this couldn't be the professional way of managing this task, so we started investigating and we ended up creating this work.

• PKCS#7 Extractor library for Delphi / Free Pascal / Lazarus
• Summary
• Included files
  • PKCS7Extractor.pas
    • What it is and how does it work
    • Exported functions
    • Exported class
      • OnStreamCreate event
• Sample application
• Unit tests
  • Save a tests report
  • Get a list of the OpenSSL libraries
  • Run test only on certain versions of OpenSSL
• Required files
• Notes about Base64-encoded files
• Version history
• Compiler compatibility
• Projects for the future
• Thanks
• Projects using this library
• Comments

This is the main unit developers will deal with. It exports some useful functions to deal with PKCS#7 messages. This is a wrapper for the OpenSSL's Libeay32.dll file. Please keep in mind that this is a minimal wrapper written to include only the functions required for PKCS#7 extraction and it's not intended to become a full wrapper.
It also does not work like a regular wrapper: when loaded it it first searches for already present instances of the library and attaches to that, if library is not currently in memory it will try to load it like a normal wrapper would do.
But when unloaded this wrapper will only clear it's variables and will not free loaded library from memory. This was designed in this way because this shouldn't be used as a main source for OpenSSL interfacing and considering that other components/libraries could be present in the application unloading the library could drive to catastrophic results.
This behaviour will be safe for working along to others' works.

Every exported function has an history so long nobody would believe us. So let be clear, this was not so easy after all, but now it will be easy to go as follows. Let's begin decribe the functions for the wrapper of OpenSSL functions.

function Load: Boolean;

Call the Load function to initialize the wrapper, this will eventually cause the load of the library. If a valid folder was specified using the SetFolder procedure, it will try to load the library from that folder.
The return value represents the successfull attach to the library or the load of the library and involves it's functions being found. Calling this more times will have no results as long as the library resides in memory.

function Loaded: Boolean;

Returns if the wrapper is currently correctly initialized.

procedure Unload;

The Unload procedure causes the wrapper to be uninitialized but, as said before, the library will be left loaded in memory.

procedure SetFolder(const S: String);

Call SetFolder prior of the Load to set the folder where to load the library. Note that when the library has already been loaded from this wrapper or from another component/library calls to this function will have no effect at all.
If an invalid folder has been specified or the library is not found in the folder, normal system paths priority will be used to load the library.
Does not have effect when library is statically linked into executable.

function GetFolder: String;

Function GetFolder will return the absolute path of the library being actually loaded into memory or an empty string if the library was not loaded.
This can differ from the folder specified by SetFolder when the specified folder is invalid or the library is not found in the folder.

function GetVersion: String;

Utility function GetVersion does simply return the string containing the library version or an empty string if the wrapper has not been initialized.
The corresponding function has been introduced since OpenSSL version 0.9.7, hence this function will return empty with older versions.

function GetErrorCode: Cardinal;

Returns a numeric value representing last OpenSSL library internal error.

function GetError(const ErrorCode: Cardinal): String;

Returns the description of a given error code.

function GetError: String;

Returns the description of last OpenSSL library internal error./ / Before continuing, let's take a look at two fundamental types.

  TVerifyStatus = (vsUnknown, vsFull, vsPartial);

  TSignatureMode = (smUnknown, smPKCS7, smCMS);

What follows are the utility functions.

function Extract(InStream, OutStream: TStream): Boolean;

Given a PKCS#7 message in a stream outputs the content extracted to the output stream.
Returns if the operation was successful.
Please keep in mind that InStream.Position has to be set to the beginning of the PKCS#7 data prior to calling this function for this to work. The Position property will be at the end of the stream once this returns, also when the function doesn't produce a positive result.
The resulting data - if any - is appended to the current content of the output stream, that needs to be cleared before being passed to this function to only receive the content extracted.

function Extract(InFilename, OutFilename: String): Boolean;

Overloaded function that will extract from a source PKCS#7 message file to a output file and return if the operation was successful or not.

function Extract(Filename: String): Boolean;

Extracts the content of the PKCS#7 message file using as output file the guessed destination filename. It will return if the operation was successful or not. Keep in mind that if the destination filename can't be guessed, the operation will fail.
To see how the filename is guessed look at the GuessFilename function below.

function GuessFilename(Filename: String): String;

Tries to guess the destination filename from a given original filename, which means: remove the appended ".p7m" extensions from the file.
To check if this extension was appended instead of replaceing the original one, if the resulting filename does not contain another extension, the function will fail and return the original file name.

function Verify(Stream: TStream): TVerifyStatus;

Utility function to check if a given stream does contain a valid PKCS#7 message. As for Extract(TStream, TStream) please keep in mind that input stream Position property must be set to the beginning of the PKCS#7 message prior to calling this.
Unlike Extract(TStream, TStream) however, this function will preserve stream position to it's current state.

function Verify(Filename: String): TVerifyStatus;

Simply checks if given file contains a valid PKCS#7 message.

function Extract(Stream: TStream; var S: String): Boolean;

Utility function to have the content of a PKCS#7 message extracted from a stream and put directly into a string. If the operation fails the return value will be false and an empty string will be set.

function ExtractToString(Filename: String; var S: String): Boolean;

Utility function to have the content of a PKCS#7 message extracted from a file and put directly into a string. If the operation fails the return value will be false and an empty string will be set.

function SignatureMode(Stream: TStream): TSignatureMode; overload;

Returns the crypto functions type used for enveloping data. Preserves value of the Position property of the stream.

function SignatureMode(const Filename: String): TSignatureMode; overload;

Returns the crypto functions type used for enveloping a file.

This unit exports a handle class TPKCS7Message that is the main core where the job is done. If you prefeer to deal with this, here's how it works. Please be advised that unlike utilities functions above, this class does not try to initialize the wrapper by itself, so always remember to call Load() before using this.

constructor TPKCS7Message.Create;

Creates an instance of the class.

procedure TPKCS7Message.Clear;

Clears all whe local variables discarding eventually loaded data.

function TPKCS7Message.LoadFromStream(Stream: TStream): Boolean;

Loads a PKCS#7 message from a stream, returns True on success.

function TPKCS7Message.LoadFromFile(Filename: String): Boolean;

Loads a PKCS#7 message file, returns True on success.

function TPKCS7Message.SaveToStream(Stream: TStream): Boolean;

Saves content previously loaded with LoadFromStream or LoadFromFile methods to a stream, returns True on valid data.

function TPKCS7Message.SaveToFile(Filename: String): Boolean;

Saves content previously loaded with LoadFromStream or LoadFromFile methods to a file, returns True on valid data.

property SignatureMode: TSignatureMode;

Read-only property representing the crypto functions type used for enveloping data.

property VerifyStatus: TVerifyStatus;

Read-only property representing the verification mode used for data.

The class extracts the whole content from the data and stores it to temporary memory inside the LoadFromStream or LoadFromFile functions. We chose to store that data in the RAm memory by using a TMemoryStream because the initial target of this class were small files. It's possible to redirect that data to other destinations, for this reason an event has been introduces, which gets called when the class needs to allocate space. In this way a redirection to every writable TStream descendant is possible.

property OnStreamCreate: TStreamCreateEvent;

As an example, we could define an event on out Form to save it to a temporary file.

type
  TMyForm = class(TForm)
    ...
    procedure FormCreate(Sender: TObject);
  private
    FPKCS7Message: TPKCS7Message;
  protected
    procedure PKCS7Stream(Sender: TObject; var AStream: TStream);
    ...

...
procedure TMyForm.PKCS7Stream(Sender: TObject; var AStream: TStream);
begin
  AStream := TFileStream.Create('temp.tmp', fmCreate or fmOpenReadWrite or fmShareDenyWrite)
end;

...

procedure TMyForm.FormCreate(Sender: TObject);
begin
  FPKCS7Message := TPKCS7Message.Create;
  FPKCS7Message.OnStreamCreate := PKCS7Stream;
...
end;
...

The class will call the Free method of the stream when this is not longer necessary.

ADDENDUM: in the "Utils" folder you can find the file CCLib.TempFileStream.pas which is not part ot this library but freely available. It simply exports a TStream descendant class that creates a temporary file in system's temporary folder with a randomic filename, denying access to the file to other applications. This file will be automatically deleted once the Free method is called.

The sample application contained into Demo folder is a simple extractor which allows to change the folder where the OpenSSL libraries are loaded from and pick up a file, extract it to destination file and view it's contents into a TMemo.

The unit tests contained in Tests folder is a console application project that will run a whole serie of test on the library and the wrapper.

The software requires some folders and files to be in the same folder from where the compiled executable is run from, here's the list of the folders and what they need to contain.

To save a tests report to a text file for further investigation - or just to stop wasting time scrolling the console - the use of output redirection comes handy.

 PKCS7ExtractorTest.exe >report.txt

Will generate a text file called "report.txt" containing all the information needed.

To obtain a list of the OpenSSL libraries in the .\OpenSSL\x32 and .\OpenSSL\x64 folders the tests program can be run as:

 PKCS7ExtractorTest.exe /list

This will cause the program to only output a list of the libraries versions it is able to load. To save this list to a text file simply use output redirection by running the command:

 PKCS7ExtractorTest.exe /list >output.txt

This will generate a file named "output.txt".

Having a lot of different version of OpenSSL libraries in the .\OpenSSL folder in sub-folders we ended up losing a lot of time running tests for certain versions. For this reason we implemented a way to specify which versions to run.
Just run the command specifying the folders names where to load from, for example:

 PKCS7ExtractorTest.exe 0.9.6 openssl-0.9.8x-i386-win32 "openssl latest"

Will execute tests only on the folders named "0.9.6", "openssl-0.9.8x-i386-win32" and "openssl latest".

This project only requires GNU Win32/64 binary libraries for Windows. Minimum supported version should be 0.9.6 released on September, 24th 2000. We're unable to test binaries before this version.
Starting from version 1.1.0 OpenSSL libraries had a mayor rewrite and we're currently investigatin on how to maker this libary compatible with both versions. We don't guarantee this will be possible in the future.

This unit has been tested with these binary versions:

Most of these binaries can be found at indy.fulgan.com.
We're currently looking for binaries not listed here so please let us know if there're some around, we'll be happy to add them to our tests.
Every compatibility issues reported will be appreciated. We decided to remove versions 1.1.0+ in this table, as well as beta versions which are not suitable for distribution.

This library does not manage Base64-encoded files, read this chapter to discover how to enable management.

Since this library has been made available, we got in touch with people using it found files that they stated where not correctly managed, telling us that files appeared corrupted or "strange". After analysing all that files we discovered those where simply encoded using the Base64 algorithm. Since there's a great availability of libraries, components and simple units that manage that encoding, we just addressed people that way.
This way of managing the problem has been decided based on the results of a public poll on Delphi Club Italia from which emerged that this functionality should not be part of this library because it doesn't belong to the process. Many pointed that most developers had already solved this problem with one of the implementations of the Base64 decoder available.

However, recent developments have pushed us in the direction of integratic a control and management of these files, mostly because the OpenSSL libraries already include all the functionalities for decoding this format, but they do not apply automatic recognition of the incoming format on functions.

In fact, the library functions expect the data to be enclosed between two block separators as shown here, to indicate how to interpret the data.

-----BEGIN PKCS7-----
V2UgYXJlIHBsZWFzZWQgYnkgeW91IGN1cmlvc2l0eSB0byB1bmNvdmVyIHRoZSBjb250ZW50IG9m
IG91ciBzYW1wbGUgdGV4dCwgaXQgaXMgbm90IGEgc2ltcGxlIGxvcmVtIGltcHN1bSwgYnV0IGEg
dGFpbG9yZWQgdGV4dCB3ZSB3cm90ZSBmb3IgeW91LiBXZSB3aXNoIHlvdSBhIGdvb2QgZGF5Lg==
-----END PKCS7-----

For this reason, a function has been introduced when opening the files, that checks if they are in Base64 format and in this case adds the opening and closing tags.

Although the OpenSSL libraries do not support the URL safe variant alfabet of the Base64, this procedure recognizes both alphabets and when needed converts to the standard alphabet.
This introduces a possible erroneous behavior in which a file where there is the mixed presence of distinctive elements of the two alphabets should not be considered a valid Base64 encoded file. However, it has been considered and weighted that the occurrence of a binary file falling into this case is so remote that it does not introduce any kind of risk in the use of this library.

This feature does not have any effect on all the files that where managed before, taking into account that:

• every binary file containing values outside the aforementioned alfabets of the Base64 algorithm - excluding CR and LF codes - gets not altered;
• every Base64-encoded file already containing the tags falls under the aforementioned rule.

Anyone wishing to enable this new feature can simply define the compilation directive.

{$DEFINE PKCS7MANAGEBASE64}

Also present at the top of the PKCS7Extractor.pas file.

• Version 1.3.0.0 released June, 15th 2021

  • added management of Base64-encoded files using OpenSSL libraries' internal method
    this functionality is not provided by default but can be enabled by defining the PKCS7MANAGEBASE64 compiler directive.

• Version 1.2.0.0 released March, 2nd 2020

  • added support for files signed using CMS cryptography (requires OpenSSL libraries 0.9.8h+)
  • now compatible with Lazarus / Free Pascal
  • lost compatibility with Delphi compiler prior to Delphi 6
  • all the magic has been moved to the LoadFromStream function removing a logical bug, less class variables are used now
  • TPKCS7Message.Verify has been replaced by property TPKCS7Message.VerifyStatus
  • new property TPKCS7Message.SignatureMode
  • new SignatureMode(Stream) and SignatureMode(String) utility functions
  • new event TPKCS7Message.OnStreamCreate allows to override default TStream descendant
  • Extract(Stream, S) and ExtractToString now manage BOMs and encodings (Delphi 2009+)
  • added functions GetErrorCode, GetError(Cardinal) e GetError
  • no need to specify verifications options anymore
  • doesn't leave an empty file while creating a new file with failing SaveToFile function
  • moved all the annoying structures definition in implementation
  • avoided a compiler hint on Delphi (still there are some in Lazarus / Free Pascal, but very odd)
  • added file CCLib.TempFileStream.pas
  • tested on OpenSSL libraries versions 1.0.2q, 1.0.2r, 1.0.2s, 1.0.2t and 1.0.2u
  • minor bugfixing and code refactoring, removed syntactic sugar
  • added batch file for automatic creation of test expected results
  • updated README.md, LEGGIMI.md and demo screenshot
  • restored DCI icon's file in the demo folder

• Version 1.0.0.0 released November, 27th 2018

  • first public release
  • minor bugfixes
  • code cleanup and commented
  • tested with more versions of OpenSSL binaries

• Version 0.9.5.0 - November, 26th 2018

  • bugfixes

• Version 0.9.3.0 - November, 25th 2018

  • verification now supports full signature method
  • added verification parameter to every function

• Version 0.9.0.0 - November, 25th 2018

  • changed structure of the library
  • only one file required
  • the two static classes (Libeay32Wrapper and PKCS7Message) are gone
  • a new class TPKCS7Message has been introduced, utility functions use that class

• Version 0.8.2.0 - November, 24rd 2018

  • removed PKCS7LibraryLocation from PKCS7Extractor.pas use Libeay32Wrapper.GetLibraryFolder
  • removed PKCS7LibraryVersion from PKCS7Extractor.pas use Libeay32Wrapper.SSLeay_version

• Version 0.8.0.0 - November, 24rd 2018

  • added x64 compatibility

• Version 0.7.0.0 - November, 23rd 2018

  • removed the Loaded function from Libeay32Wrapper, just use Load
  • completly rewritten the Load function in Libeay32Wrapper
  • added function GetLibraryFolder to get location of the library loaded
  • SSLeay_version is no more a replication of the OpenSSL specification but a more convenient function returning a string containing version

• Version 0.5.0.0 - November, 23rd 2018

  • abandoned dependency on Indy project, now uses a minimal wrapper
  • removed PKCS7Check from PKCS7Extractor.pas use Libeay32Wrapper.Load and .Loaded functions
  • added Libeay32Wrapper.pas
  • removed PKCS7Check function

• Version 0.2.1.0 - November, 22nd 2018

  • added PKCS7GuessFilename function
  • PKCS7Extract(Filename) function now returns destination filename or an empty string on failure
  • compatible with Delphi 2007

• Version 0.2.0.0 - November, 22nd 2018

  • added PKCS7ExtractorTest.dpr
  • added PKCS7Check function
  • added PKCS7LibraryLocation function
  • added PKCS7LibraryVersion function
  • added IsPKCS7 functions

• Version 0.1.1.0 - November, 21st 2018

  • fixed missing Libeay32.dll library initialization

• Version 0.1.0.0 - November, 20st 2018

  • first working version

We would like to be able to achieve the full compatibility to every Delphi / Free Pascal / Lazarus, but we need others to help us testing with compilers we don't own. For each unchecked compiler we're waiting for someone to get in touch with us sending the compiled binaries.

*: version 1.1.0.0 never made out of alpha version.

This library was born for a goal which at the moment appears to be fully achieved.
Given the longevity of the version we believe that a this time the library can be considered stable and complete.
Therefore no other features will be added, some sub-version could be published for the following reasons:

• Fixes and improvements.
• Making this unit compatible with all versions of Delphi or other Object Pascal compilers.
• Compatibility with all possible versions of OpenSSL.

• Delphi Club Italia Facebook page
• Christian Cristofori
• Giancarlo Oneglio
• Diego Rigoni
• Gianni Giorgetti
• Marcello Gribaudo

Feel free to use this library where and how you wish, without any obligations, but we really would like to hear from you and put a link to your project in this list.

• FExpolorer (Fattura Elettronica in Windows Explorer) by Carlo Barazzetta and Andrea Magni
  Open-Source Delphi project of a fully functional Legal Invoices viewer integrated with Windows' shell.

Any suggestion, contribution and comment will be appreciated.