maxi-k / btrblocks Goto Github PK
View Code? Open in Web Editor NEWBtrBlocks: Efficient Columnar Compression for Data Lakes (SIGMOD 2023 Paper)
License: MIT License
btrblocks's People
Contributors
Stargazers
Watchers
Forkers
liushengxuan cxz wanglei4687 paullampe jokerzh omar-saleh jaicewizard leochencipher distributedsystemresearch xarantolus seb711 vderic malkia lawbenbtrblocks's Issues
Can't build on Fedora 40 (GCC 14)
I realize Fedora is not a primary target, but people on other early-adopter distros like Arch are likely to hit the same issues. :)
This is after:
mkdir build; cd build; cmake ..
make -j10
Intel TBB
According to this similar issue this has been fixed upstream in TBB, but in the vendored version:
In file included from /home/alex/dev/btrblocks/build/vendor/intel/tbb/src/tbb_src/src/tbb/scheduler_common.h:43,
from /home/alex/dev/btrblocks/build/vendor/intel/tbb/src/tbb_src/src/tbb/scheduler.h:24,
from /home/alex/dev/btrblocks/build/vendor/intel/tbb/src/tbb_src/src/tbb/governor.cpp:25:
/home/alex/dev/btrblocks/build/vendor/intel/tbb/src/tbb_src/include/tbb/task.h:266:20: error: declaration of ‘tbb::task& tbb::internal::task_prefix::task()’ changes meaning of ‘task’ [-Wchanges-meaning]
266 | tbb::task& task() {return *reinterpret_cast<tbb::task*>(this+1);}
| ^~~~
The workaround for this is to change line 225 of vendor/intel/tbb/src/tbb_src/include/tbb/task.h from:
task* next_offloaded;
to:
tbb::task* next_offloaded;
Missing includes in playground.cpp
[ 82%] Linking CXX executable double_benchmarking
/home/alex/dev/btrblocks/tools/playground/playground.cpp: In function ‘int main(int, char**)’:
/home/alex/dev/btrblocks/tools/playground/playground.cpp:40:42: error: ‘VariableByte’ was not declared in this scope
40 | new CompositeCodec<SIMDFastPFor<4>, VariableByte>());
| ^~~~~~~~~~~~
/home/alex/dev/btrblocks/tools/playground/playground.cpp:39:50: error: expected primary-expression before ‘(’ token
39 | auto fast_pfor = std::shared_ptr<IntegerCODEC>(
| ^
/home/alex/dev/btrblocks/tools/playground/playground.cpp:40:10: error: ‘CompositeCodec’ does not name a type
40 | new CompositeCodec<SIMDFastPFor<4>, VariableByte>());
| ^~~~~~~~~~~~~~
/home/alex/dev/btrblocks/tools/playground/playground.cpp:40:25: error: ‘SIMDFastPFor’ was not declared in this scope
40 | new CompositeCodec<SIMDFastPFor<4>, VariableByte>());
| ^~~~~~~~~~~~
/home/alex/dev/btrblocks/tools/playground/playground.cpp:40:40: error: expected primary-expression before ‘,’ token
40 | new CompositeCodec<SIMDFastPFor<4>, VariableByte>());
| ^
/home/alex/dev/btrblocks/tools/playground/playground.cpp:40:56: error: expected primary-expression before ‘)’ token
40 | new CompositeCodec<SIMDFastPFor<4>, VariableByte>());
| ^
/home/alex/dev/btrblocks/tools/playground/playground.cpp:41:46: error: expected primary-expression before ‘(’ token
41 | auto codec = std::shared_ptr<IntegerCODEC>(
| ^
/home/alex/dev/btrblocks/tools/playground/playground.cpp:42:51: error: ‘CompositeCodec’ does not name a type
42 | new CompositeCodec<FastBinaryPacking<32>, VariableByte>()); //
| ^~~~~~~~~~~~~~
/home/alex/dev/btrblocks/tools/playground/playground.cpp:42:66: error: ‘FastBinaryPacking’ was not declared in this scope
42 | new CompositeCodec<FastBinaryPacking<32>, VariableByte>()); //
| ^~~~~~~~~~~~~~~~~
/home/alex/dev/btrblocks/tools/playground/playground.cpp:42:87: error: expected primary-expression before ‘,’ token
42 | new CompositeCodec<FastBinaryPacking<32>, VariableByte>()); //
| ^
/home/alex/dev/btrblocks/tools/playground/playground.cpp:42:103: error: expected primary-expression before ‘)’ token
42 | new CompositeCodec<FastBinaryPacking<32>, VariableByte>()); //
| ^
There are no FastPFOR includes in this file; I'm not sure where they're supposed to come from.
CURLOPT_PUT Deprecated
edit: Updating to https://github.com/aws/aws-sdk-cpp/commits/1.10.53 seems to avoid this error but I don't know if it's safe 😅
My libcurl is version 8.6.0 and it complains that CURLOPT_PUT is deprecated and CURLOPT_UPLOAD should be used instead.
Indeed, changing line 414 of vendor/libawscpp-download/src/libawscpp-download/aws-cpp-sdk-core/source/http/curl/CurlHttpClient.cpp seems to help;
before:
curl_easy_setopt(requestHandle, CURLOPT_PUT, 1L);
after:
curl_easy_setopt(requestHandle, CURLOPT_UPLOAD, 1L);
Unable to build on Mac M1 Pro
output of make command:
[ 0%] Performing build step for 'fastpfor_src'
[ 3%] Building CXX object CMakeFiles/FastPFOR.dir/src/bitpacking.cpp.o
clang: error: the clang compiler does not support '-march=native'
make[5]: *** [CMakeFiles/FastPFOR.dir/src/bitpacking.cpp.o] Error 1
make[4]: *** [CMakeFiles/FastPFOR.dir/all] Error 2
make[3]: *** [all] Error 2
make[2]: *** [vendor/lemire/fastpfor/src/fastpfor_src-stamp/fastpfor_src-build] Error 2
make[1]: *** [CMakeFiles/fastpfor_src.dir/all] Error 2
make: *** [all] Error 2
clang version:
Apple clang version 14.0.0 (clang-1400.0.29.202)
Target: arm64-apple-darwin22.3.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
Could you let me know if there are any workarounds to do the setup so that I can run the tests to compare against Parquet files ?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.