Update OS used to build .NET package about libsodium HOT 20 CLOSED

1.0.18.3 has been published :)

from libsodium.

On a failed decryption, the message buffer is now filled with 0xd0 instead of 0x00 (in the tradition of OpenBSD, making it more obvious that something went wrong instead of an all-null content).

The NSec test should check that the output buffer doesn't match the plaintext rather than expect a specific output on failure.

from libsodium.

The issue is the run-test-binaries-os-versions job, which tests compatibility with CentOS 7 and Debian 10.

So far I have used the following for the build-linux-glibc-xx jobs:

• Ubuntu 20.04 fails
• Ubuntu 22.04 fails
• Debian 10 fails

Each time the error is:

Run chmod +x .libsodium-builds/linux-x64/Tests
  chmod +x .libsodium-builds/linux-x64/Tests
  .libsodium-builds/linux-x64/Tests
  shell: sh -e {0}
  env:
    DOTNET_SYSTEM_GLOBALIZATION_INVARIANT: 1
Unhandled exception. System.DllNotFoundException: Unable to load shared library 'libsodium' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: liblibsodium: cannot open shared object file: No such file or directory
   at Program.sodium_version_string()
   at Program.Main() in /__w/libsodium/libsodium/.libsodium-test/Program.cs:line 8
/__w/_temp/10cd1428-5f50-4743-9b21-01f1bbd2452a.sh: line 2:    39 Aborted                 (core dumped) .libsodium-builds/linux-x64/Tests
Error: Process completed with exit code 134.

However using Debian 10 as the build platform (supported until 30 June 2024) means the Debian 10 tests pass (all the other tests pass on all the above) - so the main problem is CentOS 7.

If CentOS 7 compatibility is important, is it worth considering a build process for CentOS 7 separate to other platforms?

from libsodium.

I'd love to ditch CentOS 7 compatibility, but people are going to complain that it is still supported by the vendor.

from libsodium.

.NET unfortunately masks the actual error: It tries to load libsodium from a number of places and using different spellings (e.g., by adding a .so suffix and/or a lib prefix to the library name). If it cannot load and initialize the library, it throws a DllNotFoundException saying "liblibsodium: cannot open shared object file: No such file or directory", even if the library exists in one of the places and spellings.

The solution to finding the actual error is given in the exception message: "In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable". My guess is that this will show an "Undefined reference to memcpy@GLIBC_2.14" error.

from libsodium.

When compiling libsodium.so on ubuntu:22.04, loading that file fails on both centos:7 and debian:10 with the following error:

checking for version `GLIBC_2.3' in file /lib64/ld-linux-x86-64.so.2 [0] required by file /__w/libsodium/libsodium/.libsodium-builds/linux-x64/libsodium.so [0]
checking for version `GLIBC_2.33' in file /lib64/libc.so.6 [0] required by file /__w/libsodium/libsodium/.libsodium-builds/linux-x64/libsodium.so [0]
/lib64/libc.so.6: error: version lookup error: version `GLIBC_2.33' not found (required by /__w/libsodium/libsodium/.libsodium-builds/linux-x64/libsodium.so) (fatal)

This is independent of whether __asm__(".symver memcpy,memcpy@GLIBC_2.2.5"); is present or not.

When compiling on ubuntu:20.04 or ubuntu:18.04 and loading on centos:7:

checking for version `GLIBC_2.3' in file /lib64/ld-linux-x86-64.so.2 [0] required by file /__w/libsodium/libsodium/.libsodium-builds/linux-x64/libsodium.so [0]
checking for version `GLIBC_2.2.5' in file /lib64/libpthread.so.0 [0] required by file /__w/libsodium/libsodium/.libsodium-builds/linux-x64/libsodium.so [0]
checking for version `GLIBC_2.14' in file /lib64/libc.so.6 [0] required by file /__w/libsodium/libsodium/.libsodium-builds/linux-x64/libsodium.so [0]
checking for version `GLIBC_2.3.4' in file /lib64/libc.so.6 [0] required by file /__w/libsodium/libsodium/.libsodium-builds/linux-x64/libsodium.so [0]
checking for version `GLIBC_2.4' in file /lib64/libc.so.6 [0] required by file /__w/libsodium/libsodium/.libsodium-builds/linux-x64/libsodium.so [0]
checking for version `GLIBC_2.25' in file /lib64/libc.so.6 [0] required by file /__w/libsodium/libsodium/.libsodium-builds/linux-x64/libsodium.so [0]
/lib64/libc.so.6: error: version lookup error: version `GLIBC_2.25' not found (required by /__w/libsodium/libsodium/.libsodium-builds/linux-x64/libsodium.so) (fatal)

When compiling on ubuntu:16.04, everything works (without __asm__(".symver memcpy,memcpy@GLIBC_2.2.5");).

So it seems that, until CentOS 7 reaches end-of-life (or .NET drops support for it), we're stuck with compiling libsodium.so for .NET on ubuntu:16.04...

glibc versions of distributions:

libc compatibility of .NET itself:

• x64: glibc 2.17 (from CentOS 7)
• Arm32, Arm64: glibc 2.23 (from Ubuntu 16.04)
• Alpine (x64 and Arm64): musl 1.2.2 (from Alpine 3.13)

from libsodium.

Another option is to compile with Zig, that can cross-compile to many targets, including specific libc versions:

Example:

zig build -Drelease-fast -Dtarget=x86_64-linux-gnu.2.17

from libsodium.

Looks like this is the best way to go.

dotnet-core.yml is now using zig to build Linux binaries.

Compilation can thus be done on current Linux distributions, while targeting older glibc versions. So, we target glibc 2.17 for x86_64 and glibc 2.23 for ARM, according to your table.

Compilation to aarch64 and arm with musl was fixed by the way (previously, the aarch64 build was actually an x86_64 one).

from libsodium.

Adding Windows ARM would probably also be nice.

zig build -Drelease-fast -Dtarget=aarch64-windows gives us a DLL and a .LIB for Windows ARM. But I guess this is impossible to test on GitHub CI, right?

from libsodium.

Is 1.0.18.3 waiting on this?

from libsodium.

Some unit tests in NSec that passed with 1.0.18.2 are now failing with 1.0.18.3 ☹️

It looks like something changed in or around crypto_aead_aes256gcm_decrypt?

from libsodium.

And it seems the pre-compiled binary built for linux-musl-arm isn't actually included in the NuGet package...

from libsodium.

Ah, looks like that has to be added to libsodium.pkgproj.

from libsodium.

Ahh, that must also have got clobbered like the version bump I did.

from libsodium.

Some unit tests in NSec that passed with 1.0.18.2 are now failing with 1.0.18.3 ☹️

It looks like something changed in or around crypto_aead_aes256gcm_decrypt?

Is this to do with 408125a?

from libsodium.

Some unit tests in NSec that passed with 1.0.18.2 are now failing with 1.0.18.3 ☹️
It looks like something changed in or around crypto_aead_aes256gcm_decrypt?

Is this to do with 408125a?

Very unlikely. This code doesn't exist in stable.

from libsodium.

What do the other failing tests do?

from libsodium.

All three of the other failing tests are decrypting a valid ciphertext using crypto_aead_aes256gcm_decrypt. However:

• In DecryptWithAdOverlapping, m (length mlen) and ad (length adlen) overlap in memory (m < ad < ad+adlen < m+mlen).
• In DecryptWithSpanInPlace, m and c point at the same memory location (m == c).
• In DecryptWithSpanOutOfPlace, none of the buffers overlap.

Previously, in each case, the function returned zero; now it returns a non-zero value.

The last case puzzles me the most because it is the simplest possible test case: encrypt a plaintext and decrypt the resulting ciphertext with exactly the same key, nonce and additional data. How could that possibly fail? 😕

from libsodium.

The simplest possible test case fails if clen >= 256.

from libsodium.

Oops! That should be fixed by a5ea347

Our test suite was missing tests with long inputs. This case is now tested.

from libsodium.