ustcadsl / diffkv Goto Github PK

View Code? Open in Web Editor NEW

58.0 58.0 18.0 5.69 MB

License: Apache License 2.0

CMake 1.44% C++ 84.28% Shell 1.46% C 0.01% Ruby 12.82%

diffkv's People

Contributors

Stargazers

Watchers

Forkers

zacharyliu-cs sz-npe rgmacedo iam57-2020 lsmdb iridiumine lroethan zhichao-cao ysjyx7 xuangestallone distributedsystemresearch o4thkeeper cscetbon tc-tuffy m561247 sunzhoujia shanting98 smartrabbit520

diffkv's Issues

titan foreground gc

can't find the code about titan foreground gc, but you mentioned in the experiment

What is the merge policy of key LSM-tree in DiffKV?

leveling merge policy or Partitional leveling merge policy?

Why the write performance of PebblesDB and TitanDB is significantly lower than RocksDB ?

Why is it that when I conduct microbenchmark tests using the ycsb-c, the results show that the write performance of PebblesDB and TitanDB is significantly lower than RocksDB? This clearly doesn't seem reasonable. I haven't made any changes to the configuration or workload; I'm just conducting the tests. What could be causing this?

how to using diffkv?

When I used YCSBC

“./ycsbc -db diffkv -dbfilename /home/eros/workspace/tmp  -threads 16 -P workloads/workloadpareto1KB100GB.spec  -phase load -configpath configDir/diffkv_config.ini ”

error

terminate called after throwing an instance of 'utils::Exception'
what(): File not open!
已放弃 (核心已转储)

I think there's an error like this when I use Benchmark.sh.

WAL_DIR is not defined

./benchmark.sh [bulkload/fillseq/overwrite/filluniquerandom/readrandom/readwhilewriting/readwhilemerging/updaterandom/
mergerandom/randomtransaction/compact]

Could you tell me how to test diffkv in two ways?

YCSBC
benchmark or db_bench

Compile issue

When I tried to build YCSB-C, I encountered the following problems:

cannot find "pebblesdb/db.h"

In file included from db_factory.cc:17:
../db/leveldb_db.h:8:10: fatal error: pebblesdb/db.h: No such file or directory
    8 | #include "pebblesdb/db.h"
      |

I copied the code of pebblesdb and changed the makefile in the YSCB-C/db directory, and finally solved the problem.

Static assertion over incorrect types

g++ -std=c++11 -c -g -Wall -fgnu-tm -I../ db_factory.cc -o db_factory.o
In file included from /usr/include/c++/8/unordered_map:46,
                 from ../lib/stl_hashtable.h:13,
                 from ../lib/lock_stl_hashtable.h:11,
                 from ../db/lock_stl_db.h:16,
                 from db_factory.cc:13:
/usr/include/c++/8/bits/hashtable.h: In instantiation of ‘class std::_Hashtable<vmp::String, std::pair<const vmp::String, vmp::StringHashtable<const char*>*>, std::allocator<std::pair<vmp::String, vmp::StringHashtable<const char*>*> >, std::__detail::_Select1st, vmp::StlHashtable<vmp::StringHashtable<const char*>*, MemAlloc, std::allocator<std::pair<vmp::String, vmp::StringHashtable<const char*>*> > >::Equal, vmp::StlHashtable<vmp::StringHashtable<const char*>*, MemAlloc, std::allocator<std::pair<vmp::String, vmp::StringHashtable<const char*>*> > >::Hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >’:
/usr/include/c++/8/bits/unordered_map.h:105:18:   required from ‘class std::unordered_map<vmp::String, vmp::StringHashtable<const char*>*, vmp::StlHashtable<vmp::StringHashtable<const char*>*, MemAlloc, std::allocator<std::pair<vmp::String, vmp::StringHashtable<const char*>*> > >::Hash, vmp::StlHashtable<vmp::StringHashtable<const char*>*, MemAlloc, std::allocator<std::pair<vmp::String, vmp::StringHashtable<const char*>*> > >::Equal, std::allocator<std::pair<vmp::String, vmp::StringHashtable<const char*>*> > >’
../lib/stl_hashtable.h:45:13:   required from ‘class vmp::StlHashtable<vmp::StringHashtable<const char*>*, MemAlloc, std::allocator<std::pair<vmp::String, vmp::StringHashtable<const char*>*> > >’
../lib/lock_stl_hashtable.h:19:7:   required from ‘class vmp::LockStlHashtable<vmp::StringHashtable<const char*>*>’
../db/lock_stl_db.h:23:16:   required from here
/usr/include/c++/8/bits/hashtable.h:192:21: error: static assertion failed: unordered container must have the same value_type as its allocator
       static_assert(is_same<typename _Alloc::value_type, _Value>{},
                     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/include/c++/8/bits/hashtable.h: In instantiation of ‘class std::_Hashtable<vmp::String, std::pair<const vmp::String, const char*>, std::allocator<std::pair<vmp::String, const char*> >, std::__detail::_Select1st, vmp::StlHashtable<const char*, MemAlloc, std::allocator<std::pair<vmp::String, const char*> > >::Equal, vmp::StlHashtable<const char*, MemAlloc, std::allocator<std::pair<vmp::String, const char*> > >::Hash, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<true, false, true> >’:
/usr/include/c++/8/bits/unordered_map.h:105:18:   required from ‘class std::unordered_map<vmp::String, const char*, vmp::StlHashtable<const char*, MemAlloc, std::allocator<std::pair<vmp::String, const char*> > >::Hash, vmp::StlHashtable<const char*, MemAlloc, std::allocator<std::pair<vmp::String, const char*> > >::Equal, std::allocator<std::pair<vmp::String, const char*> > >’
../lib/stl_hashtable.h:45:13:   required from ‘class vmp::StlHashtable<const char*, MemAlloc, std::allocator<std::pair<vmp::String, const char*> > >’
../lib/lock_stl_hashtable.h:19:7:   required from ‘class vmp::LockStlHashtable<const char*>’

This is a bug in ycsb and has been fixed in the new version. The version used here does not fix this bug. Check this

Miss a lot of libs, like rocksDB,snappy......
Options does not have the member exp_ops
This error occurs in ./bench_tools/YCSB-C/db/leveldb_db.cc line24. Maybe you forgot to comment this line during coding.

What do you mean by you commit to mainstream Titan ?

In your README you say

And we also commit our implementation to mainstream in Titan.

What do you mean ?

Why does the DiffKV outperform than PebblesDB for range query performance?

After reading the paper, I found that DiifKV use the vTree structure, similar to PebblesDB, to organize the middle-size key-value pairs. For range query, compared with PebblesDB, DiffKV seems to have no obvious advantage, except for the scan-optimized merge.

Why does the DiffKV outperform than PebblesDB for range query performance? Is this due to the scan-optimized merge strategy? Or is it caused by the mixed size workload? For middle-size key-value pairs, does the performance advantage of DiffKV range queries still exist?

compile issue

Thanks for sharing your greate work!
When build DiffKV, I met errors

[root@localhost build]# make titan -j
make: *** No rule to make target 'titan'.  Stop.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.