Comments (18)
Probably related to #82 , I also have the suite benchmark hanging
Also, x86 doesn't work at all with AccessViolationException on the first compression call. I wonder if it is expected that MinGW build should work out of the box on x86 smoothly?
from c-blosc.
After some further reading this is obviously related to #74, my MinGW gcc version is 4.9.2. But the test fails even with the default BloscLZ compressor.
from c-blosc.
That's strange. At least on Linux, the only compressor that was affected was LZ4, but perhaps MinGW is uncovering other side effects by GCC 4.9 series :(. What's the version of the c-blosc library that you are testing? 1.5.2 was known to work with GCC 4.9.1.
from c-blosc.
@FrancescAlted I am using master head as of yesterday
from c-blosc.
Just recompiled 1.5.2 - same thing, with O3 and number of threads above 1 blosc_decompress_ctx
throws AccessViolationException
from c-blosc.
I was incorrect, all modes -O0..-O3 throw an exception with multiple decompression threads, -O3 throws AccessViolation, -O0 throws SEHException. I need to review the source and my interop code (if they match) and cross-compile on Linux for Windows with MinGW to be somewhat sure it is not my code.
What about x86? Is the code supposed to work without additional flags or anything when compiled with MinGW (i.e. was it tested by someone)? If yes then my build is likely broken, if no - then there is probably some issue with Windows/MinGW.
Single threaded is quite fast - faster than any other alternative I had before I found this fascinating library! My main concern is stability though.
from c-blosc.
Can you try with some pre 1.5 version?
Enviat des del meu iPhone
El 15/01/2015, a les 19:47, Victor Baybekov [email protected] va escriure:
I was incorrect, all modes -O0..-O3 throw an exception with multiple decompression threads, -O3 throws AccessViolation, -O0 throws SEHException. I need to review the source and my interop code (if they match) and cross-compile on Linux for Windows with MinGW to be somewhat sure it is not my code.
What about x86? Is the code supposed to work without additional flags or anything when compiled with MinGW (i.e. was it tested by someone)? If yes then my build is likely broken, if no - then there is probably some issue with Windows/MinGW.
Single threaded is quite fast - faster than any other alternative I had before I have found this fascinating library! My main concern is stability though.
—
Reply to this email directly or view it on GitHub.
from c-blosc.
same with the first commit where the _ctx functions were introduced. ff6ca8e
I haven't tried older functions which require init(). will come back after double check if my build is broken or not. The simplest way is to write the same test in pure C and debug.
from c-blosc.
When I set this line to two threads the test_basics.exe
crashes, with 1 thread it is OK. My experience with C and tooling is not enough yet to debug it.
from c-blosc.
Good to know, thanks. Yeah, I strongly suspect that we introduced some multithreading glitch during the big revamp for 1.5 series.
from c-blosc.
Without digging into it too much but judiging from the recent issues we have been seeing with bcolz, that is my gut feeling also.
from c-blosc.
@buybackoff Can you check whether this issue is fixed by #87?
from c-blosc.
@jack-pappas together with the latest commit it doesn't help. Same crash after a couple of thousands iterations of compress/decompress calls via P/Invoke to _ctx versions on 2+ threads.
from c-blosc.
Ah, that's unfortunate. The change in #87 seems to fix the hanging issue in the benchmark tool I reported in #82.
Btw, in the C# P/Invoke signature you provided above for blosc_decompress_ctx
-- I think the type of the numinternalthreads
parameter should be an int
or uint
, not IntPtr
.
from c-blosc.
Thanks, fixed that. As long as I do not box integers and use a pointer to boxed int, but use just new IntPtr(2)
could that matter?
BTW, I have realized that Blosc blocksize blocksize = L1 * 4;
with clevel 9 blocksize *= 2
is 16-32 times larger than my typical data (8-16Kb), since I cut large data sets into chunks myself before calling Blosc. So the issue became irrelevant from practical point of view as I do not need 2+ threads. But the error persists when data is larger than a single block.
from c-blosc.
With MinGW, x86 version crashed even with single thread. I recompiled the library with CMake+VS12, now x86 works fine with one thread, but the crash with 2+ threads remains with the same behavior.
from c-blosc.
Version 1.7 (compiled with CMake + VS2015RC) works great without exceptions, both x86 and x64
from c-blosc.
Cool. Thanks for the feedback!
from c-blosc.
Related Issues (20)
- Unable to build on macOS with Apple Clang 12 HOT 6
- build for arm64 error HOT 2
- c-blosc needs a CMake package file HOT 3
- Is there a function like compress_bound() ? HOT 1
- Files referencing non existing `LICENSE.txt`
- Conda package wants unprovided compression functions HOT 2
- Update conda package of latest release 1.21.1? HOT 3
- Support Quantile Compression codec HOT 3
- Linker error
- macOS: Universal2 build fails on SSE2 HOT 4
- Switching from `master` to `main` HOT 1
- Uncompressable data set? HOT 6
- THOUGHTS_FOR_2.0.txt
- 1.21.1: test suite is failing HOT 6
- zlib / zlib-ng licence file HOT 2
- CMake: Enable external ZLIB from CMake variable ZLIB_ROOT
- Upgrade ZSTD to 1.5.5 due to potential corruption HOT 4
- Illegal Instruction vinserti128 in set_host_implementation HOT 11
- Troubles while building HOT 1
- Help compiling example HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from c-blosc.