Statically linked openssl and/or OpenSSL libraries: Enable simple use of static third party provider(s) about openssl HOT 10 OPEN

Isn't what you're describing enabled by the OSSL_PROVIDER_add_builtin call?

That is to say, if you want a provider that is statically linked in with your application, you can just:

1. Create the provider, and build it as a .a file
2. link it to your application along with -lcrypto and -lssl
3. Have the application call OSSL_PROVIDER_add_builtin(ctx, "provider name", init_fn);

Then it should work like any other builtin provider, no need to modify the build infrastructure of openssl at all.

from openssl.

@nhorman Yes, I understand the available concepts for adding/loading a statically linked provider in an application.

The first application I tried out was openssl itself.
I tried really hard to insert combinations of the triple OSSL_PROVIDER_available, OSSL_PROVIDER_add_builtin and OSSL_PROVIDER_load into the OpenSSL source code to get a static version of the openssl application with OQS provider. With the the concept of a "default library context" in combination with threading used in the openssl application, I wasn't successful. Maybe you can provide hints how/where in the OpenSSL source code to load a statically linked provider for the openssl application? (it is understood that this implies code changes in OpenSSL as well. Btw, using the undocumented feature via environment variable OPENSSL_TEST_LIBCTX does not cut it, neither does adding provider loading to OSSL_LIB_CTX_new).

The next applications were cURL, HAproxy, Nginx. Doable using the available functions - but with code changes in each of these widely used applications.

I then got to the approach outlined above, which - without any further code changes - enables statically linked versions for all of these applications. And as a side benefit, a statically linked openssl itself can also make use of statically linked provider(s). I'd like to emphasize the latter because I would not know how to achieve that otherwise.

One important aspect I didn't mention so far because I thought it might be obvious once a third party provider looks and feels like a built-in provider: There's the option for semi-static linking. This means static linking of provider(s) when building the dynamic OpenSSL libs, but all dynamic linking when building applications - or when using pre-built, packaged applications after replacing the default OpenSSL libraries with their provider enhanced versions (plus provider enablement in openssl.cnf). Verified both via cURL from package and built from source code.

In other words: With a code change in one file of OpenSSL, the above outlined approach enables addition of statically linked provider(s) to every application with makes use of the OpenSSL libraries, irrespective of whether these apps are statically or dynamically linked and without requiring [to add] an "ossl-modules" directory. This in contrast to required code changes in every application.

This is why I argue in favor of OpenSSL enabling transparent (= no app code changes) use of statically linked provider(s), which btw is the equivalent usability level that dynamically loaded providers achieve.

from openssl.

So, you're approach above, as you've noted only works for a single additional provider. There would have to be some significant work done to enable multiple provider linking, which the existing api already allows. And the only advantage that I see to your approach is that it offer some mild level of convenience in that you only have to link -lcrypto instead of -lcrypto -l, which seems negligible in that the process is automated in most build environments.

You're welcome to create and submit a PR, but I think you'll run into challenges getting it accepted, as I don't see it offering a significant advantage over the programatic approach thats available now.

I'd be interested to know what failed with your approach using OSSL_PROVIDER_add_builtin(). That approach shouldn't require any additional changes to openssl code at all. The application just links in your provider, and during application startup makes a call to OSSL_PROVIDER_add_builtin, after which it should be loadable via OSSL_PROVIDER_load, like any other provider, or via a customized config file. Can you elaborate on what you did there and what went wrong?

from openssl.

The entire feature request here is about avoiding code changes - which I claim nobody likes. As such, and with all due respect, I would not classify the benefits this provides as "some mild level of convenience". For example, there's a plethora of statically linked custom applications out there which must be upgraded to Q-safe TLS operation. The existing API is enforcing code changes for each of those. Which leads to the question of why enforcing API use in the first place, thereby enforcing code changes.

From your comment "shouldn't require any additional changes to openssl code" I conclude that I wasn't successful in making clear that the openssl binary itself was the application to receive static provider functionality.

int OSSL_PROVIDER_add_builtin(OSSL_LIB_CTX *libctx, const char *name, ossl_provider_init_fn *init_fn); requires a library context. Hence, one can't simply add this anywhere in the application code.

The question for the openssl app was therefore not how to use the existing API in an application, but where.

So I tried to add loading the provider here (inside an else statement) to 'sneak it in' whenever a new library context is generated. This was leading to a successful build and I was able to use openssl req ... - but wasn't able to successfully use it with openssl X509 ...

I then realized that openssl is making use of a "default library context", which is replicated to threads. I tried a couple of code places to explicitly load the provider for the default context, but never got openssl to build.

Finally, I tried to sneak it in here (including specifying the environment variable OPENSSL_TEST_LIBCTX), which built but again without success for all openssl sub-commands.

Last-not-least, related to "significant work done to enable multiple provider linking": Rather than changing the file provider_predefined.c explicitly, one most probably could change it implicitly by throwing in some simple sed statements via makefile.

from openssl.