Comments (6)
I think I found the bug which was in my code. I assigned output length more than the size of the output array accidentally. It works now , sorry for bothering you with my own bug, and thank you very much...
from aircompressor.
It is likely a bug. Can you share a file that reproduces this? These kinds of bugs are generally impossible to fix without a reproduction.
from aircompressor.
Also a stack trace would help.
BTW did you try decoding the file to make sure it was not corrupted? Some of the native libraries for zstd disabled the corruption checks to improve performance, so it "works" but the output is actually corrupted.
from aircompressor.
I was using this along with the jhdf (https://github.com/jamesmudd/jhdf), as a filter.
When I reported this issue I thought it was the decoder's problem, but later when I tried to isolate a frame of data which reported an error to a separate file, and used AC to decode it, it didn't report an error.
So I thought it might be threading issue. So then I tried to use ThreadLocal, and just create a new instance every time when I needed to, the old corruption error disappeared, instead it gets a coredump, which I could not comprehence.
The original file is about 1.7Gb, I'm not sure hot to send it to you.
from aircompressor.
The hs error log doesn't really help, because it just says that the VM crashed in the core JVM binary with no reference to aircompressor.
I'm not going to be able to dig through a 1.7GB file in a format I'm unfamiliar with. Can you isolate the problematic compressed frame and post just that frame?
from aircompressor.
the problem is, when I did so, there was no problem. I had to guess it has something to do with the jhdf lib as well. Let me try to bring this issue to the jhdf guy and see if he/she has some ideas. Will update when more to come.
from aircompressor.
Related Issues (20)
- Snappy Decompress throw MalformedInputException HOT 2
- Implement other ZSTD compression levels
- Lz4 double buffering
- compile and generate class files for the given Java implementaion. HOT 1
- ZstdOutputStream writes empty stream HOT 2
- Support Zstd seekable format
- Hive fails opening split for zst compressed files
- ZStd JNI vs Aircompressor pure java performance question HOT 3
- The result is different HOT 1
- Add `SECURITY.md` file and enable vulnerability reporting HOT 2
- Set up OSS-Fuzz
- Decompression: How to determine the size of the output buffer? HOT 1
- Remove dependency on sun.misc.Unsafe? HOT 1
- May I ask if Android is usable? HOT 1
- Security vulnerability: Snappy decompressor can be made to crash JVM HOT 5
- Compression ratio is different in ZSTD algorithm between ZstdOutputStream and ZstdCompressor.compress(Bytebuffer) HOT 2
- Support lz4 framing decompression HOT 1
- ZSTD : Drive the end of decompression with both "inputLimit" and "outputLimit"
- Request: Generic Decompressor for all algos based on magic number header
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aircompressor.