aloneguid / ironcompress Goto Github PK

X64 Windows 10 .NET 7, this LINQPad code:

var bytes = File.ReadAllBytes(@"...\bytes.bin");
var iron = new Iron();
using (var ironCompressResult = iron.Decompress(Codec.Brotli, bytes))
{
ironCompressResult.Dump();
Encoding.UTF8.GetString(ironCompressResult).Dump();
}

Then ironCompressResult.Length = 0 for 1.30 version, but ironCompressResult.Length > 0 for 1.28 (which works as expected).

I tried .NET 7, .NET 6 and netstandard 2.0, the same result.

If available, this may be faster than managed gzip.

Hello,

Is there a way to set a compression level?

It seems that the LZ4 compressor only actually compresses if given the CompressionLevel.Fastest flag:

Iron iron = new Iron(ArrayPool<byte>.Shared);
using (IronCompressResult lz4 = iron.Compress(Codec.LZ4, uncompressed.AsSpan()))
    Console.WriteLine("LZ4 (optimal): {0} bytes ({1:0.00}%)", lz4.Length, lz4.Length * 100.0 / uncompressed.Length);
using (IronCompressResult lz4 = iron.Compress(Codec.LZ4, uncompressed.AsSpan(), compressionLevel: CompressionLevel.SmallestSize))
    Console.WriteLine("LZ4 (smallest): {0} bytes ({1:0.00}%)", lz4.Length, lz4.Length * 100.0 / uncompressed.Length);
using (IronCompressResult lz4 = iron.Compress(Codec.LZ4, uncompressed.AsSpan(), compressionLevel: CompressionLevel.Fastest))
    Console.WriteLine("LZ4 (fastest): {0} bytes ({1:0.00}%)", lz4.Length, lz4.Length * 100.0 / uncompressed.Length);

This results in the following output:

LZ4 (optimal): 206368 bytes (100,39%) native
LZ4 (smallest): 206368 bytes (100,39%) native
LZ4 (fastest): 109516 bytes (53,28%) native

That is, only CompressionLevel.Fastest actually does any compression.

The issue seems to be that ToNativeCompressionLevel() converts CompressionLevel.Optimal to the value 2, but then compress_lz4() in api.cpp converts the value 2 to level = 32768. This is passed to LZ4 as the acceleration_level where a higher number means less compression.

So looks like the compress_lz4() interprets the converted-to-native compression level in an incorrect fashion.

For reference, the Zstandard seems to handle the levels as expected:

Zstd (optimal): 64226 bytes (31,24%)
Zstd (smallest): 63076 bytes (30,68%)
Zstd (fastest): 73317 bytes (35,67%)