I'm testing the <a href="https://gitlab.com/gnutls/gnutls/-/merge_requests/1404" rel="

Maximum input buffer size for AEAD about libkcapi HOT 12 CLOSED

ueno commented on July 21, 2024

Maximum input buffer size for AEAD

from libkcapi.

Comments (12)

smuellerDD commented on July 21, 2024

Am Donnerstag, dem 15.04.2021 um 23:00 -0700 schrieb Daiki Ueno:

I'm testing the [MR](https://gitlab.com/gnutls/gnutls/-/merge_requests/1404) in GnuTLS and noticed the following error in `tests/slow/test-ciphers- openssl.sh`: ```c libkcapi - Error: vmsplice: not all data received by kernel (data received: 61255 -- data sent: 66579) test-gcm: test-gcm.c:44: main: Assertion `kcapi_aead_stream_update_last(handle, iov, 2) >= 0' failed. ```

This is the following issue - it is taken from the libkcapi test: /* * vmsplice: AAD must be at most 14 pages as otherwise the * plaintext and tag cannot be sent - shrink the AAD by two * pages - for sendmsg, this would not be necessary * * AAD is 65504 bytes (in stream mode together with the 32 bytes * plaintext we have 65536 bytes, i.e. 16 pages). We shrink the * AAD to 14 pages */ It sounds like the input data that was provided is one large contiguous chunk. Let me think whether this could be best handled in libkcapi. But so far I have not found a right solution as we handle with IOVECs: What happens if we have 2 IOVECs where the first is way below the threshold and the 2nd is either larger as the threshold. Or both separately are smaller and yet the sum is larger than the threshold... Ciao

…

It seems that the `kcapi_aead_stream_update_last` function has some implicit limit on the input dat, around 61255. I'm attaching a reproducer ([test-gcm.c.gz]( https://github.com/smuellerDD/libkcapi/files/6322795/test-gcm.c.gz)).

from libkcapi.

ueno commented on July 21, 2024

Thank you for the hint. I tried to split the input data into 4096-byte (= page size) chunks in IOVECs, but got the following error instead; the gap is even larger:

libkcapi - Error: vmsplice: not all data received by kernel (data received: 32583 -- data sent: 66579)
test-gcm: test-gcm.c:56: main: Assertion `kcapi_aead_stream_update_last(handle, iov, iovlen) >= 0' failed.

from libkcapi.

smuellerDD commented on July 21, 2024

Am Freitag, dem 16.04.2021 um 00:17 -0700 schrieb Daiki Ueno:

Thank you for the hint. I tried to split the input data into 4096-byte (= page size) chunks in IOVECs, but got the following error instead; the gap is even larger: ```console libkcapi - Error: vmsplice: not all data received by kernel (data received: 57159 -- data sent: 66579) test-gcm: test-gcm.c:47: main: Assertion `kcapi_aead_stream_update_last(handle, iov, 3) >= 0' failed. ```

Well, do you use update and op operation must be used in the loop? Note, the update only sends the data to the kernel but it is not processed. The update operation does that. * kcapi_aead_stream_update() - send more data for processing (stream) * * @handle: [in] cipher handle * @Iov: [in] scatter/gather list with data to be processed by the cipher * operation. * @iovlen: [in] number of scatter/gather list elements. * * Using this function call, more plaintext for encryption or ciphertext for * decryption can be submitted to the kernel. * * Note, see the order of input data as outlined in * kcapi_aead_stream_init_dec(). * * This function may cause the caller to sleep if the kernel buffer holding * the data is getting full. The process will be woken up once more buffer * space becomes available by calling kcapi_aead_stream_op(). * * Note: The last block of input data MUST be provided with * kcapi_aead_stream_update_last() as the kernel must be informed about the * completion of the input data. * * With the separate API calls of kcapi_aead_stream_update() and * kcapi_aead_stream_op() a multi-threaded application can be implemented * where one thread sends data to be processed and one thread picks up data * processed by the cipher operation. * * IMPORTANT NOTE: The kernel will only process * sysconf(_SC_PAGESIZE) * ALG_MAX_PAGES at one time. Longer input data cannot * be handled by the kernel. * * WARNING: The memory referenced by @Iov is not accessed by the kernel * during this call. The memory is first accessed when kcapi_cipher_stream_op() * is called. Thus, you MUST make sure that the referenced memory is still * present at the time kcapi_cipher_stream_op() is called. Thanks Stephan

…

from libkcapi.

ueno commented on July 21, 2024

Well, do you use update and op operation must be used in the loop?

I construct a single iovec array with many chunks of data, and call kcapi_aead_stream_update_last just once. Even if I change the call to kcapi_aead_stream_update (not _last), I get the same error.

I'm attaching the test program I'm using:
test-gcm2.c.gz.

from libkcapi.

smuellerDD commented on July 21, 2024

Am Freitag, dem 16.04.2021 um 00:55 -0700 schrieb Daiki Ueno:

> Well, do you use update and op operation must be used in the loop? I construct a single iovec array with many chunks of data, and call `kcapi_aead_stream_update_last` just once. Even if I change the call to `kcapi_aead_stream_update` (not `_last`), I get the same error. I'm attaching the test program I'm using: [test- gcm2.c.gz](https://github.com/smuellerDD/libkcapi/files/6323396/test-gcm2.c.gz ).

I just checked with the kernel code of algif_aead.c: since we talk about AEAD I think we have a general limit: The AEAD ciphers are not implemented to handle init/update/final operations in the kernel. That said, the kernel must require that all data to be encrypted is present for the cipher operation to calculate the tag. Yet, the kernel has only a limited amount of memory available. The maximum size is defined with /proc/sys/net/core/optmem_max. When using splice, only meta data about the memory is kept in the kernel. When using sendmsg, the full data is copied into the kernel. That said, just for testing, can you increase the value in /proc/sys/net/core/optmem_max and see whether this works? If this works, we have to make a choice: 1. should GnuTLS reject larger buffers? 2. should GnuTLS simply return the libkcapi / kernel error? 3. should GnuTLS invoke a fallback implementation? Note, the chunking up of data only works for ciphers that can be invoked multiple times like symmetric non-AEAD ciphers. In this case, your chunking would look like: kcapi_cipher_stream_init_enc while(data) { // Make sure that the input/output buffers have the same size // as otherwise the kernel may block the process to either wait // for input or to wait for having the output data picked up kcapi_cipher_stream_update(iov) kcapi_cipher_stream_op(iov) } kcapi_cipher_stream_update_last(iov) kcapi_cipher_stream_op(iov) Ciao Stephan

from libkcapi.

ueno commented on July 21, 2024

Yet, the kernel has only a limited amount of memory available. The maximum
size is defined with /proc/sys/net/core/optmem_max. When using splice, only
meta data about the memory is kept in the kernel. When using sendmsg, the full
data is copied into the kernel.

I see, thank you for looking into it.

That said, just for testing, can you increase the value in
/proc/sys/net/core/optmem_max and see whether this works?

I'm afraid it doesn't work. With the original program attached (test-gcm.c.gz):

$ cat /proc/sys/net/core/optmem_max
81920
$ expr `cat /proc/sys/net/core/optmem_max` '*' 2 | sudo tee /proc/sys/net/core/optmem_max
163840
$ cat /proc/sys/net/core/optmem_max
163840
$ ./test-gcm
libkcapi - Error: vmsplice: not all data received by kernel (data received: 61255 -- data sent: 66579)
test-gcm: test-gcm.c:44: main: Assertion `kcapi_aead_stream_update_last(handle, iov, 2) >= 0' failed.

I'm trying with the loop approach, but I wonder what input/output data format are expected in the loop, in particular:

should I send AAD with _update every time,
does _op return tag every time

from libkcapi.

smuellerDD commented on July 21, 2024

Am Freitag, dem 16.04.2021 um 09:29 -0700 schrieb Daiki Ueno:

> Yet, the kernel has only a limited amount of memory available. The maximum size is defined with /proc/sys/net/core/optmem_max. When using splice, only meta data about the memory is kept in the kernel. When using sendmsg, the full data is copied into the kernel. I see, thank you for looking into it. > That said, just for testing, can you increase the value in /proc/sys/net/core/optmem_max and see whether this works? I'm afraid it doesn't work. With the original program attached (test- gcm.c.gz): ```console $ cat /proc/sys/net/core/optmem_max 81920 $ expr `cat /proc/sys/net/core/optmem_max` '*' 2 | sudo tee /proc/sys/net/core/optmem_max 163840 $ cat /proc/sys/net/core/optmem_max 163840 $ ./test-gcm libkcapi - Error: vmsplice: not all data received by kernel (data received: 61255 -- data sent: 66579) test-gcm: test-gcm.c:44: main: Assertion `kcapi_aead_stream_update_last(handle, iov, 2) >= 0' failed. ``` I'm trying with the loop approach, but I wonder what input/output data format in the loop, in particular: - should I send AAD with `_update` every time, - should I expect `_op` returns tag ?

Gosh, I found the issue: the issue is the maximum pipe size. The splice operation will add the sent data to the size of the pipe (even though the kernel pipe buffer is not really occupied). After allocation of a pipe, its size is 16 * PAGE_SIZE. Thus, it can hold at max 64kB on Intel. It seems that there is already some data in the pipe, so the effective buffer size for us is less. To handle that, either - the pipe is cleared before new data is added (which is not possible for AEAD data as discussed before), or - the pipe size is increased (which I can add to libkcapi with a call like fcntl(handle->pipes[0], F_SETPIPE_SZ, 128000); - this however can easily bump against resource constraints as the ultimate maximum size is given in /proc/sys/fs/pipe-max-size - which is usually 1 MB So, when working with the kernel, we have a maximum data buffer size that can be processed at once. As for AEAD data this implies that this limit is the maximum buffer limit of the data to be processed. Now, how do you think what would be the ultimate maximum number of bytes we should support for AEAD (i.e. the buffer size for AAD || CT || TAG). PS: I tested with a modified libkcapi which includes the fcntl call mentioned above and this allows the test code to succeed. Ciao Stephan

from libkcapi.

ueno commented on July 21, 2024

Is there a way to get the limit accurately before sending data to the kernel? If so, we could fallback to the default implementation based on that.

from libkcapi.

smuellerDD commented on July 21, 2024

Am Samstag, 17. April 2021, 07:56:25 CEST schrieb Daiki Ueno: Hi Daiki,

In practice, 64KB is sufficient for the use with TLS, as the limit of record size is around 2^14 (16KB). On the other hand, as GnuTLS also provides generic crypto API with in-place encryption/decryption, it would be a bit surprising if the operation fails because of the internal limit. Is there a way to get the limit accurately before sending data to the kernel? If so, we could fallback to the default implementation based on that.

That can be achieved with F_GETPIPE_SZ. However, I first want to check what is already occupying the pipe, because I expected that the pipe is empty and we would have exactly 64KB available. But the kernel reports that only 61KB can be filled. In any case, I am thinking to add libkcapi APIs around getting/setting the pipe size which then can be used by GnuTLS. Ciao Stephan

from libkcapi.

smuellerDD commented on July 21, 2024

Am Samstag, 17. April 2021, 07:56:25 CEST schrieb Daiki Ueno: Hi Daiki,

I just pushed an update to libkcapi. This update should imply that GnuTLS should not need to make any changes to its code. Using libkcapi, it should simply work. Of course, the kernel will enforce a maximum buffer size as defined with / proc/sys/net/core/optmem_max With this update, your test app does not cause any error any more and if you play around with it by setting kcapi_set_verbosity(KCAPI_LOG_DEBUG); you will see: - if the data is less or equal to 16 IOVECs with at most a page per IOVEC, the vmsplice call is used (i.e. the libkcapi log will not display special handling) - if the data is larger than 16 IOVECs and/or we have more than 64KB, the log will indicate that the sendmsg fallback is used which has a larger buffer Is that a solution that you think you can live with? Ciao Stephan

from libkcapi.

ueno commented on July 21, 2024

Thank you, that is awesome! I tried the patch and confirmed that all the GnuTLS tests pass now.

from libkcapi.

smuellerDD commented on July 21, 2024

Thank you, closing the issue.

from libkcapi.

Maximum input buffer size for AEAD about libkcapi HOT 12 CLOSED

Comments (12)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent