respeak / quicklz Goto Github PK

Hello. I've been decompressing about 800 files weekly and was playing with trying to use your rust implementation of quicklz.

I altered the new() method to use a different key but everything else is the same. [that isn't involved with this, sorry]

81 of these files comes out with differing bytes. All are compressed lvl 3 and the differing bytes aren't scattered about but are in chunks (if that means anything). No errors are being returned and my rust code to call is simply...

fn decompress(data: &[u8], size: u32) -> Result<Vec<u8>, std::io::Error> {
    let mut r = std::io::Cursor::new(data.as_ref());
    match quicklz::decompress(&mut r, size) {
        Ok(r) => Ok(r),
        Err(e) => {
            eprintln!("{}", e);
            Err(std::io::Error::new(
                std::io::ErrorKind::InvalidData,
                "quicklz error",
            ))
        }
    }
}

I did notice that your implementation is not a "full" implementation but rather customized to suit your needs but I was wondering if you might have some input on this.

Attached is a sample of a file that was compressed with some embedded c++ version of quicklz. I can successfully decompress it with c++ code and python (cython) code via the pyquicklz package.

Also attached is the known 'good' decompressed file for comparison.

(renamed from .bin to .txt to attach...)
compressed.txt
decompressed.txt

Recently I use rust to refactoring a decompress tool, and I found there might be a problem of hashtable.
In C version, the function qlz_decompress() will transmit a parameter called state_decompress, which contains the array of hashtable. That means the hashtable might be used in different compressed packages(the pointer of offset might point to decompressed datas to get value if matchlen != 0).
But in your rust implementation of quicklz, hashtable is defined only when program enter the function quicklz::decompress(). So, in the decompress progress of second and later compressed packages, the problem is more likely to cause a error, because the hashtable is empty while matchlen is not zero.
I do not know if I said clearly, and I am very happy to contact with you for more details.

Check here: