pmem / kvdk Goto Github PK
View Code? Open in Web Editor NEWKey Value Development Kit
License: BSD 3-Clause "New" or "Revised" License
Key Value Development Kit
License: BSD 3-Clause "New" or "Revised" License
Why not use libpmemobj ?
I do not think kvdk can support atomic operation only using APIs in libpmem.
batchWrite can write multiple KV pairs one by one, but could not roll back if failed.
DLinked_list could not update prev/next pointer in one transaction.
But it shows "Provide APIs to write multiple key-value pairs in an atomic batch. ",
Is there something wrong ?
The problem/use-case that the feature addresses
The Redis commands such as KEYS, DEL require the storage system to provide the type of the key.
Description of the feature
E.g. DEL key1,----> kvdk return key1 is belong to String? List? Hash? Set? etc
------- ops in seconds -----------
time (ms), read ops, not found, write ops, total read, total write
1000 88150000 23965000 1744000 88150000 1744000
2001 113761000 113763000 1894000 201911000 3638000
3002 111013000 111011000 1901000 312924000 5539000
4003 110966000 110969000 1376000 423890000 6915000
5003 110821000 110813000 576000 534711000 7491000
6004 110837000 110847000 565000 645548000 8056000
7005 110834000 110831000 577000 756382000 8633000
8006 110812000 110808000 574000 867194000 9207000
9007 110768000 110771000 573000 977962000 9780000
10008 110698000 110701000 571000 1088660000 10351000
"Set error" happened when using sorted-type and latency enable, option type inclues "Fill, update, Insert",for example:
Insert new sorted-type kv
Write latency overflow: 12191390 us
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
Set error
sh: line 1: 71030 Segmentation fault (core dumped) numactl --cpunodebind=0 --membind=0 ./bench -latency=1 -populate=1 -value_size=128 -threads=96 -time=10 -path=/mnt/pmem0/kvdk -num=789516047 -space=536870912000 -max_write_threads=96 -fill=0 -type=sorted -read_ratio
=0 -existing_keys_ratio=0 > ./results/sorted_vs128_insert_thread96
what is the difference between kvdk and pmemkv?
I cannot compile the kvrocks correctly by following the steps mentioned in the example/kvrocks/README
I found that the README is updated 9 months ago. However, the patch is updated three days ago.
When I compile with running ‘make’, there's an error:
incubator-kvrocks/external/kvdk/include/types.hpp:7:10: fatal error: libpmemobj++/string_view.hpp: No such file or directory
This error could be fixed by copying libpmemobj++/
directory from kvdk to the kvrocks.
But there is another error:
In file included from redis_bitmap_string.cc:5:
redis_string.h:9:10: fatal error: namespace.hpp: No such file or directory
#include "namespace.hpp"
I found that you are developing kvdk and there is no namespace.h
file in the latest version. I would be grateful if you fixing the issue and update the README.
The problem/use-case that the feature addresses
The Redis::List commands require KVDK::List provide more functions.
Description of the feature
E.g.
Redis::List::LLEN ---> KVDK return the length of the list
Redis::List::LPOS ---> KVDK returns the index of matching elements inside a KVDK::List
etc.
problem: Fsdax model in kvdk will let request go into the kernel vfs system and this model will extend the critical path of the request.
improv: Devdax could bypass the os-fs,and access the pmem directly, it's benefit for kvdk's work model.
We will upload the pr for support devdax in kvdk.
commit 14a9aa9
Date: Wed Dec 1 15:06:56 2021
CPU: Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz
PMEM: 128 GB x 8
numactl --cpunodebind=1 --membind=1 ../build/bench -type=string -value_size=120 -threads=64 -max_write_threads=64 -time=10 -path=/mnt/pmem1/kvdk_string -num=805306368 -space=412316860416 -collections=16 -fill=1 -populate=1
numactl --cpunodebind=1 --membind=1 ../build/bench -type=string -value_size=16 -threads=64 -max_write_threads=64 -time=10000 -path=/mnt/pmem1/kvdk_string -num=805306368 -space=412316860416 -collections=16 -fill=0 -read_ratio=0 -key_distribution=random
[INFO] time 3894 ms: Map pmem space done
[INFO] time 4941 ms: RestorePendingBatch done: iterated 0 records
[INFO] time 11615 ms: RestoreData done: iterated 806092662 records
[INFO] time 11615 ms: Rebuild skiplist done
init 64 write threads
init 0 read threads
------- ops in seconds -----------
time (ms), read ops, not found, write ops, total read, total write
1000 0 0 38951000 0 38951000
2000 0 0 38459000 0 77410000
3000 0 0 39947000 0 117357000
4000 0 0 29682000 0 147039000
5000 0 0 2153000 0 149192000
6000 0 0 2128000 0 151320000
7000 0 0 2058000 0 153378000
8000 0 0 2053000 0 155431000
9001 0 0 2104000 0 157535000
10001 0 0 2002000 0 159537000
11001 0 0 1668000 0 161205000
12001 0 0 961000 0 162166000
13001 0 0 941000 0 163107000
14001 0 0 964000 0 164071000
15001 0 0 947000 0 165018000
16002 0 0 963000 0 165981000
17002 0 0 1314000 0 167295000
18002 0 0 1324000 0 168619000
19002 0 0 1303000 0 169922000
20002 0 0 1285000 0 171207000
21002 0 0 1306000 0 172513000
write ops doesn't go down.
write ops slow down from 30Mops to 1Mops.
size_t num_threads = 1;
batch_size = 150;
no put before BatchWrite:
auto Put = [&](size_t tid) {
// for (size_t i = 0; i < count; i++) {
// values[tid][i] = GetRandomString(120);
// ASSERT_EQ(
// engine->SortedPut(collection_name, elems[tid][i],
// values[tid][i]), Status::Ok);
// }
};
pass the test
for (size_t i = 0; i < count; i++) {
std::string val_resp;
if (values[tid][i].empty()) {
ASSERT_EQ(
engine->SortedGet(collection_name, elems[tid][i], &val_resp),
Status::NotFound);
} else {
ASSERT_EQ(
engine->SortedGet(collection_name, elems[tid][i], &val_resp),
Status::Ok);
ASSERT_EQ(values[tid][i], val_resp);
}
}
fail at :
ASSERT_EQ(engine->SortedGet(collection_name, elems[tid][i], &val_resp),Status::Ok);
output:
/home/fangsl/kvdk/tests/tests.cpp:840: Failure
Expected equality of these values:
engine->SortedGet(collection_name, elems[tid][i], &val_resp)
Which is: 1
Status::Ok
Which is: 0
The problem/use-case that the feature addresses
A description of the problem that the feature will solve, or the use-case with which the feature will be used.
The KVDKModify functions will erase the exist TTL by default. But sometime we prefer to keep current TTL.
Description of the feature
A description of what you want to happen.
Can we provide an option to indicate that whether the KVDKModify function will erase the original TTL or keep it.
Alternatives you've considered
Any alternative solutions or features you've considered, including references to existing open and closed feature requests in this repository.
Additional information
Any additional information that is relevant to the feature request.
As a kv system, it may save any type of data except for std::basic_string_view, will kvdk support other types?
Following codes only support c-like string as input data, why not support any data with size ?
/**
The problem/use-case that the feature addresses
The Redis::List commands require KVDK::Hash provide more functions.
Description of the feature
E.g.
Redis::Hash::HLEN ---> KVDK returns the length of the Hash
Redis::Hash::HKEYS ---> KVDK returns all field names in the hash stored at key.
Redis::Hash::HRANDFIELD ---> KVDK returns a random field from the hash value stored at key.
Redis::Hash:: HVALS ---> KVDK returns all values in the hash stored at key.
kvdk::Configs engine_configs;
{
engine_configs.pmem_file_size = 40 * 1024UL * 1024UL * 1024UL;
engine_configs.pmem_segment_blocks = (1ull << 8);
engine_configs.hash_bucket_num = (1ull << 10);
engine_configs.log_level = kvdk::LogLevel::Debug;
}
std::string engine_path{kvpmem_data_file_path};
// Purge old KVDK instance
system(std::string{"rm -rf " + engine_path + "\n"}.c_str());
status = kvdk::Engine::Open(engine_path, &engine, engine_configs, stdout);
assert(status == kvdk::Status::Ok);
DBUG_PRINT("KVDK",
("Successfully created KVDK engine %s", kvpmem_data_file_path));
status = engine->SSet("hi", "yo", "peeep");
ASSERT(status == kvdk::Status::Ok, (int)status);
}
Should put value in the ordered set.
Assert fails with InvalidDataSize
. If I print the size of the string view inside the engine I get following output [ERROR] time 40009 ms: SSET size collection: 1147430597 userkey: 1147430594 value: 1147430588
.
I think there is an issue with the string view but I don't understand why since the usage is very similar to the usage guide. I also tried to pass a static string. Any ideas?
Can you update the wechat group QR code?
CPU: Intel(R) Xeon(R) Platinum 8358 CPU @ 2.60GHz x2
DRAM: DIMM DDR4 Synchronous Registered (Buffered) 3200 MHz 16GBx16
PMEM: DIMM Synchronous Non-volatile LRDIMM 3200 MHz 256GBx16
--- a/engine/kv_engine.cpp
+++ b/engine/kv_engine.cpp
@@ -1314,7 +1314,7 @@ Status KVEngine::StringSetImpl(const StringView &key, const StringView &value) {
void *block_base = pmem_allocator_->offset2addr(sized_space_entry.offset);
uint64_t new_ts = get_timestamp();
- assert(!found || new_ts > data_entry.meta.timestamp);
+ kvdk_assert(!found || new_ts > data_entry.meta.timestamp, "old record has newer timestamp!");
StringRecord::PersistStringRecord(block_base, sized_space_entry.size,
new_ts, StringDataRecord, key, value);
Build with DCMAKE_BUILD_TYPE=FastDebug
Then run
python3 run_benchmark.py string
update_random and read_write_random will fail.
Update random string
bench: ../engine/kv_engine.cpp:1317: kvdk::Status kvdk::KVEngine::StringSetImpl(const StringView&, const StringView&): Assertion `(!found || new_ts > data_entry.meta.timestamp) && "old record has newer timestamp!"' failed.
bench: ../engine/kv_engine.cpp:1317: kvdk::Status kvdk::KVEngine::StringSetImpl(const StringView&, const StringView&): Assertion `(!found || new_ts > data_entry.meta.timestamp) && "old record has newer timestamp!"' failed.
Aborted (core dumped)
In the code below I create collection with one element using SSet()
, and than I delete this element. When I call SGet()' on such element, the status is
NotFound, and everything seems to be OK. However If I try to seek such element through iterator (both
Seek()and
SeekToFirst()`), I may get it with empty string as a value - so when I use iterator I cannot distinct deleted element from element with empty string as value.
TEST_F(EngineBasicTest, TestSeekToFirst) {
const std::string collection = "col";
std::string val;
ASSERT_EQ(Engine::Open(db_path.c_str(), &engine, configs, stdout),
Status::Ok);
ASSERT_EQ(engine->SSet(collection, "foo" , "bar"), Status::Ok);
ASSERT_EQ(engine->SGet(collection, "foo", &val), Status::Ok);
ASSERT_EQ(engine->SDelete(collection, "foo"), Status::Ok);
ASSERT_EQ(engine->SGet(collection, "foo", &val), Status::NotFound);
auto iter = engine->NewSortedIterator(collection);
ASSERT_NE(iter, nullptr);
iter->SeekToFirst();
if(iter->Valid()) {
std::cout << iter->Key() << std::endl;
std::cout << iter->Value() << std::endl;
}
}
Test output:
localhost/kvdk:build# PMEM_IS_PMEM_FORCE=1 ./dbtest --gtest_filter="*SeekToFirst*"
Note: Google Test filter = *SeekToFirst*
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from EngineBasicTest
[ RUN ] EngineBasicTest.TestSeekToFirst
[LOG] time 0 ms: Initializing PMEM size 17179869184 in file /mnt/pmem0/data/data
[LOG] time 1977 ms: Map pmem space done
[LOG] time 1979 ms: In restoring: iterated 0 records
foo
[ OK ] EngineBasicTest.TestSeekToFirst (2034 ms)
[----------] 1 test from EngineBasicTest (2034 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (2034 ms total)
[ PASSED ] 1 test.
I may workaround this problem by calling additional SGet()
after each SeekToFirst()
, but it would be much more intuitive, less error prone and probably faster if such iterator would be invalid after removal of all elements in collection.
when will the next release version of kvdk be committed?
The release version now is v_0.2. I noticed this version's batchwrite was incompeleted and recently you have added the feature of batchwrite. I would like to know the current stage of this project and what I need to pay attention to if implementing kvdk into my project?
2 * Intel(R) Xeon(R) Platinum 8269C CPU @ 2.50GHz
CPU: 2 * Intel(R) Xeon(R) Platinum 8269C CPU @ 2.50GHz
DDR: 12 * DDR4 16384 MB 2666 MT/s
PMEM: 4 * 129408 MB 2666 MT/s
cd build
cmake .. -DCMAKE_BUILD_TYPE=FastDebug -DCHECK_CPP_STYLE=ON
cd ..
./build/dbstress_test --gtest_filter="*Sorted*" --gtest_break_on_failure --gtest_catch_exceptions=1 --gtest_repeat=-1
Test Case success
assertion fail
[Testing] Execute SSet and SDelete in GlobalHashesCollection.
264200/1048576 [############--------------------------------------]
dbstress_test: /home/shizy/kvdk_folk/kvdk/engine/skiplist.cpp:224: bool kvdk::Skiplist::FindAndLockWritePos(std::vector<kvdk::SpinMutex*>&, kvdk::Splice*, const string_view&, const kvdk::HashTable::KeyHashHint&, const kvdk::DLRecord*): Assertion `prev == header_->record || compare_string_view(Skiplist::UserKey(prev->Key()), insert_key) < 0' failed.
Populate space sometimes fails to initialize the pmem space. In PMEMAllocator::PopulateSpace
it is possible that pmem_memset
gets called on memory that is not part of the file. It happens if pu
(cpu count) is not a power of 2. In my case pu
was 12 and I used linux memmap. This caused a segmentation fault.
Reproduce:
pu
is 12Solutions:
pu
to 16
as done here and check file is the power of 2The problem/use-case that the feature addresses
If one PM disk space is not enough, how can multiple PM disks be mapped into one KVDK?
Description of the feature
KVDK can take use of multiple PM disk.
Alternatives you've considered
Additional information
Any additional information that is relevant to the feature request.
2 * Intel(R) Xeon(R) Platinum 8269C CPU @ 2.50GHz
CPU: 2 * Intel(R) Xeon(R) Platinum 8269C CPU @ 2.50GHz
DDR: 12 * DDR4 16384 MB 2666 MT/s
PMEM: 4 * 129408 MB 2666 MT/s
Add following code to test.cpp. Build and run.
TEST_F(EngineBasicTest, SortedCollectionSecurityIssue1) {
std::string const collection_name = "SortedCollection_01";
configs.pmem_segment_blocks = 16;
int num_threads = 1;
configs.max_write_threads = num_threads;
ASSERT_EQ(Engine::Open(db_path.c_str(), &engine, configs, stdout),
Status::Ok);
auto Check = [&]()
{
auto iter = engine->NewSortedIterator(collection_name);
size_t cnt = 0;
for (iter->SeekToLast(); iter->Valid(); iter->Prev())
++cnt;
ASSERT_EQ(cnt, 2)
<< "Two keys, aka \"c\" and \"d\" should be in the collection";
};
for (size_t i = 0; i < 10; i++)
{
std::uint64_t fake_prev = i;
std::uint64_t fake_next = 13;
std::string fake_key;
fake_key.append(std::string_view{reinterpret_cast<char*>(&fake_prev), sizeof(decltype(fake_prev))});
fake_key.append(std::string_view{reinterpret_cast<char*>(&fake_next), sizeof(decltype(fake_next))});
engine->Set(fake_key, "Vicious entries!");
}
engine->SSet(collection_name, "a", "To be deleted.");
engine->SSet(collection_name, "b", "To be deleted.");
engine->SSet(collection_name, "c", "Always here.");
engine->SSet(collection_name, "d", "Always here.");
engine->SDelete(collection_name, "b");
engine->SDelete(collection_name, "a");
for (size_t i = 10; i < 11; i++)
{
std::uint64_t fake_prev = i;
std::uint64_t fake_next = 13;
std::string fake_key;
fake_key.append(std::string_view{reinterpret_cast<char*>(&fake_prev), sizeof(decltype(fake_prev))});
fake_key.append(std::string_view{reinterpret_cast<char*>(&fake_next), sizeof(decltype(fake_next))});
engine->Set(fake_key, "Vicious entries!");
}
Check();
delete engine;
ASSERT_EQ(Engine::Open(db_path.c_str(), &engine, configs, stdout),
Status::Ok);
Check();
}
Test case Success
Test case Fail
User may use this access pattern to break into internals of KVEngine.
2 * Intel(R) Xeon(R) Platinum 8269C CPU @ 2.50GHz
CPU: 2 * Intel(R) Xeon(R) Platinum 8269C CPU @ 2.50GHz
DDR: 12 * DDR4 16384 MB 2666 MT/s
PMEM: 4 * 129408 MB 2666 MT/s
Modify kv_engine.hpp
inline uint64_t get_timestamp() {
auto res = get_cpu_tsc() - ts_on_startup_ + newest_version_on_startup_;
return res;
}
to
inline uint64_t get_timestamp() {
return 0;
}
Add test case to stress_test.cpp
TEST(HotspotTest, StringABATest)
{
const std::string path_db{"/mnt/pmem0/kvdk_test_hotspot"};
std::string cmd = "rm -rf " + path_db + "\n";
int _sink = system(cmd.data());
kvdk::Configs configs;
configs.pmem_file_size = 1ULL << 30;
configs.max_write_threads = 4;
kvdk::Engine* engine = nullptr;
kvdk::Engine::Open(path_db.data(), &engine, configs, stderr);
size_t n_skip = 636031;
size_t v_size = (1ULL << 12);
ProgressBar bar{std::cout, "", n_skip};
for (size_t i = 0; i < n_skip; i++)
{
GetRandomString(v_size);
bar.Update(i + 1);
}
std::string key{"some_key"};
key.resize(40);
std::string zero_filled(v_size, 0);
std::string value{GetRandomString(v_size)};
for (size_t i = 3456; i < value.size(); i++)
value[i] = 0;
engine->Set(key, zero_filled);
auto SetOrGet = [&](size_t tid)
{
size_t n_iteration = (1ULL << 20);
if (tid % 2 == 0)
{
ProgressBar bar2{std::cout, "", n_iteration, (tid == 0)};
for (size_t i = 0; i < n_iteration; i++)
{
EXPECT_EQ(engine->Set(key, zero_filled), kvdk::Status::Ok);
EXPECT_EQ(engine->Set(key, value), kvdk::Status::Ok);
if ((i + 1) % (1ULL << 10) == 0 && (tid == 0))
bar2.Update(i + 1);
}
}
else
{
std::string value_got;
for (size_t i = 0; i < n_iteration; i++)
{
EXPECT_EQ(engine->Get(key, &value_got), kvdk::Status::Ok);
ASSERT_TRUE(value_got == zero_filled || value_got == value)
<< "Got invalid value: \n"
<< value_got << "\n"
<< "Expected: \n"
<< value << std::endl;
}
}
};
LaunchNThreads(4, SetOrGet);
_sink = system(cmd.data());
}
Build as Release and run
./build/dbstress_test --gtest_filter="*ABA*" --gtest_break_on_failure --gtest_catch_exceptions=1 --gtest_repeat=-1
Test case success
Test case fail
This test case is found by birthday attack.
get_timestamp() is fixed to return 0 to help reproduce the issue.
The problem/use-case that the feature addresses
One of Redis's key features is that it allows data structures to set an expire times. Currently KVDK do not provide the KV expire capability
Description of the feature
We expect KVDK to provide the option to set the expiry time when executing commands such as Set, Get, etc. This feature will also be used to implement the redis command such as command TTL, EXPIRE, etc.
Alternatives you've considered
No
Additional information
No
Branched from main:e13db52c
TEST_F(EngineBasicTest, TestBasicHashHotspot) {
int n_thread_reading = 16;
int n_thread_writing = 16;
configs.max_write_threads = n_thread_writing;
ASSERT_EQ(Engine::Open(db_path.c_str(), &engine, configs, stdout),
Status::Ok);
std::string key{"SuperHotspot"};
std::string val1(1024, 'a');
std::string val2(1024, 'b');
ASSERT_EQ(engine->Set(key, val1), Status::Ok);
engine->ReleaseWriteThread();
auto EvenWriteOddRead = [&](uint32_t id) {
for (size_t i = 0; i < 1000000; i++) {
if (id % 2 == 0) {
// Even Write
if (id % 4 == 0) {
ASSERT_EQ(engine->Set(key, val1), Status::Ok);
} else {
ASSERT_EQ(engine->Set(key, val2), Status::Ok);
}
} else {
// Odd Read
std::string got_val;
ASSERT_EQ(engine->Get(key, &got_val), Status::Ok);
bool match = false;
match = match || (got_val == val1);
match = match || (got_val == val2);
if (!match) {
std::string msg;
msg.append("Wrong value!\n");
msg.append("The value should be 1024 of a or 1024 of b.\n");
msg.append("Actual result is:\n");
msg.append(got_val);
msg.append("\n");
GlobalLogger.Error(msg.data());
}
ASSERT_TRUE(match);
}
}
};
LaunchNThreads(n_thread_reading + n_thread_writing, EvenWriteOddRead);
delete engine;
}
Running this test may print following message
[ RUN ] EngineBasicTest.TestBasicHashHotspot
[LOG] time 2 ms: Initializing PMem size 17179869184 in file /mnt/pmem0/data/data
[LOG] time 186 ms: Map pmem space done
[LOG] time 187 ms: In restoring: iterated 0 records
[ERROR] time 1138 ms: Wrong value!
The value should be 1024 of a or 1024 of b.
Actual result is:
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
/home/shizy/kvdk/tests/tests.cpp:175: Failure
Value of: match
Actual: false
Expected: true
[ERROR] time 1229 ms: Wrong value!
The value should be 1024 of a or 1024 of b.
Actual result is:
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
[ERROR] time 1229 ms: Wrong value!
The value should be 1024 of a or 1024 of b.
Actual result is:
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
[ERROR] time 1229 ms: Wrong value!
The value should be 1024 of a or 1024 of b.
Actual result is:
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
/home/shizy/kvdk/tests/tests.cpp:175: Failure
Value of: match
Actual: false
Expected: true
[ERROR] time 1229 ms: Wrong value!
The value should be 1024 of a or 1024 of b.
Actual result is:
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
/home/shizy/kvdk/tests/tests.cpp:175: Failure
Value of: match
Actual: false
Expected: true
[ERROR] time 1229 ms: Wrong value!
The value should be 1024 of a or 1024 of b.
Actual result is:
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
[ERROR] time 1229 ms: Wrong value!
The value should be 1024 of a or 1024 of b.
Actual result is:
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
/home/shizy/kvdk/tests/tests.cpp:175: Failure
Value of: match
Actual: false
Expected: true
[ERROR] time 1229 ms: Wrong value!
The value should be 1024 of a or 1024 of b.
Actual result is:
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
/home/shizy/kvdk/tests/tests.cpp:175: Failure
Value of: match
Actual: false
Expected: true
[ERROR] time 1229 ms: Wrong value!
The value should be 1024 of a or 1024 of b.
Actual result is:
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
/home/shizy/kvdk/tests/tests.cpp:175: Failure
Value of: match
Actual: false
Expected: true
[ERROR] time 1229 ms: Wrong value!
The value should be 1024 of a or 1024 of b.
Actual result is:
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
/home/shizy/kvdk/tests/tests.cpp:175: Failure
Value of: match
Actual: false
Expected: true
[ERROR] time 1229 ms: Wrong value!
The value should be 1024 of a or 1024 of b.
Actual result is:
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
/home/shizy/kvdk/tests/tests.cpp:175: Failure
Value of: match
Actual: false
Expected: true
[ERROR] time 1229 ms: Wrong value!
The value should be 1024 of a or 1024 of b.
Actual result is:
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
/home/shizy/kvdk/tests/tests.cpp:175: Failure
Value of: match
Actual: false
Expected: true
[ERROR] time 1229 ms: Wrong value!
The value should be 1024 of a or 1024 of b.
Actual result is:
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
/home/shizy/kvdk/tests/tests.cpp:175: Failure
Value of: match
Actual: false
Expected: true
/home/shizy/kvdk/tests/tests.cpp:175: Failure
Value of: match
Actual: false
Expected: true
/home/shizy/kvdk/tests/tests.cpp:175: Failure
Value of: match
Actual: false
Expected: true
[ERROR] time 1229 ms: Wrong value!
The value should be 1024 of a or 1024 of b.
Actual result is:
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
/home/shizy/kvdk/tests/tests.cpp:175: Failure
Value of: match
Actual: false
Expected: true
[ERROR] time 1229 ms: Wrong value!
The value should be 1024 of a or 1024 of b.
Actual result is:
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
/home/shizy/kvdk/tests/tests.cpp:175: Failure
Value of: match
Actual: false
Expected: true
[ERROR] time 1229 ms: Wrong value!
The value should be 1024 of a or 1024 of b.
Actual result is:
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
/home/shizy/kvdk/tests/tests.cpp:175: Failure
Value of: match
Actual: false
Expected: true
/home/shizy/kvdk/tests/tests.cpp:175: Failure
Value of: match
Actual: false
Expected: true
[LOG] time 54723 ms: Closing instance ...
[LOG] time 54723 ms: Waiting bg threads exit ...
[LOG] time 54723 ms: Instance closed
[ FAILED ] EngineBasicTest.TestBasicHashHotspot (54788 ms)
Branch: pmem/kvdk/main:19332d5
Binary is built under kvdk/build/
by calling cmake ..
and make -j
Then the old directory under pmem is cleared to create a new instance by calling
rm -rf /mnt/pmem0/debugging2
The new engine is created by
./bench -populate=1 -value_size=120 -threads=1 -time=10 -path=/mnt/pmem0/debugging2 -num=10485760 -space=2097152000 -max_write_threads=1 -collections=1 -type=sorted -fill=1
and then closed.
Then the engine is reopened, trying to be restored, but only PMem space is mapped, recovery seems to take infinitely long time.
./bench -populate=1 -value_size=120 -threads=1 -time=10 -path=/mnt/pmem0/debugging2 -num=10485760 -space=2097152000 -max_write_threads=1 -collections=1 -type=sorted -fill=0 -read_ratio=1
[root@NC03 build]# ./bench -fill=1 -value_size=4096 -threads=2 -path=/mnt/pmem0/kvdk -space=2748779069 -num=4000000 -max_write_threads=1 -type=string -populate=1
to fill 4000000 uniform keys
[LOG] time 0 ms: Initializing PMEM size 2748779069 in file /mnt/pmem0/kvdk/data
[LOG] time 380 ms: Map pmem space done
[LOG] time 1163 ms: In restoring: iterated 0 records
[LOG] time 1163 ms: Populating PMEM space ...
[LOG] time 2697 ms: Populating done
init 2 write threads
init 0 read threads
------- ops in seconds -----------
time (ms), read ops, not found, write ops, total read, total write
Segmentation fault (core dumped)
compile the debug version and use the gdb to see the offset of the thread cache entry is overflow:
Thread 20 "bench" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffafd7f8700 (LWP 4346)]
0x00007ffff7e613a5 in memmove_mov_sse2_clwb () from /lib64/libpmem.so.1
Missing separate debuginfos, use: dnf debuginfo-install gflags-2.2.2-5.fc32.x86_64 libgcc-10.3.1-1.fc32.x86_64 libpmem-1.8-2.fc32.x86_64 libstdc++-10.3.1-1.fc32.x86_64
(gdb) bt
#0 0x00007ffff7e613a5 in memmove_mov_sse2_clwb () from /lib64/libpmem.so.1
#1 0x00007ffff7e49da6 in memmove_nodrain_sse2_clwb () from /lib64/libpmem.so.1
#2 0x00007ffff7e496cd in pmem_memcpy_persist () from /lib64/libpmem.so.1
#3 0x00007ffff7f8560a in kvdk::PMEMAllocator::Allocate (this=0x43bca0, size=4128) at /home/dennis/kvdk/engine/pmem_allocator.cpp:334
#4 0x00007ffff7f4e472 in kvdk::KVEngine::HashSetImpl (this=0x43ba90, key=..., value=..., dt=1, batch_hint=0x0) at /home/dennis/kvdk/engine/kv_engine.cpp:806
#5 0x00007ffff7f4e97c in kvdk::KVEngine::Set (this=0x43ba90, key="I\023\000\000\000\000\000",
value="0yVu77DE49c8Rj9wD4D7pCWGv9DBdsi96M6QHZD3M64pnOX9FB7c8bLqj3si5DiyjMtX497gkFeLpiePIP8L5Xg5bw7cALT9gGYqjRSKz4SIsCYOjg5VLp94IdMZ3saQB1X475fOxuF3Vas5u04aIxKff25DzR05401cg2QClimC6FsL53o182S5plURj25TrkJBAZ0Y"...) at /home/dennis/kvdk/engine/kv_engine.cpp:871
#6 0x000000000041282d in DBWrite (id=1) at /home/dennis/kvdk/benchmark/bench.cpp:200
#7 0x000000000041bfda in std::__invoke_impl<void, void ()(int), int> (__f=@0x43bf70: 0x4126ab <DBWrite(int)>) at /usr/include/c++/10/bits/invoke.h:60
#8 0x000000000041bf35 in std::__invoke<void ()(int), int> (__fn=@0x43bf70: 0x4126ab <DBWrite(int)>) at /usr/include/c++/10/bits/invoke.h:95
#9 0x000000000041bea5 in std::thread::_Invoker<std::tuple<void ()(int), int> >::_M_invoke<0ul, 1ul> (this=0x43bf68) at /usr/include/c++/10/thread:264
#10 0x000000000041be44 in std::thread::_Invoker<std::tuple<void ()(int), int> >::operator() (this=0x43bf68) at /usr/include/c++/10/thread:271
#11 0x000000000041bd80 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(int), int> > >::M_run (this=0x43bf60)
at /usr/include/c++/10/thread:215
#12 0x00007ffff7cfd994 in execute_native_thread_routine () from /lib64/libstdc++.so.6
#13 0x00007ffff7e8e432 in start_thread () from /lib64/libpthread.so.0
#14 0x00007ffff79fa6d3 in clone () from /lib64/libc.so.6
(gdb) f 3
#3 0x00007ffff7f8560a in kvdk::PMEMAllocator::Allocate (this=0x43bca0, size=4128) at /home/dennis/kvdk/engine/pmem_allocator.cpp:334
334 pmem_memcpy_persist(
(gdb) list
329 // Padding remaining space
330 auto extra_space = thread_cache.free_entry.size - b_size;
331 // TODO optimize, do not write PMEM
332 if (extra_space >= FREE_SPACE_PADDING_BLOCK) {
333 DataHeader header(0, extra_space);
334 pmem_memcpy_persist(
335 offset2addr(thread_cache.free_entry.space_entry.offset + b_size),
336 &header, sizeof(DataHeader));
337 } else {
338 b_size = thread_cache.free_entry.size;
(gdb) p header
$1 = {checksum = 0, b_size = 4152360919}
(gdb) p thread_cache
$2 = (kvdk::PMEMAllocator::ThreadCache &) @0x43bc60: {segment_offset = 4294968816, segment_usable_blocks = 49, free_entry = {space_entry = {
offset = 140737352581152, hash_entry_reference = 0x43bc40, hash_entry_mutex = 0x7ffff7800000}, size = 4152360984}, freelist = {
offsets = std::vector of length -178956965, capacity 5863877454623 = {}
(gdb) p thread_cache.free_entry.space_entry.offset
$3 = 140737352581152
(gdb) p /x thread_cache.free_entry.space_entry.offset
$4 = 0x7ffff7e84020
The offset looks like a address, please take a look to see if there is something wrong.
It's possible to SSet()
and SGet()
element which contains only zeroes as key, but iterator for such element is invalid.
/* Create a string that contains 8 bytes from uint64_t. */
static inline std::string uint64_to_string(uint64_t &key) {
return std::string(reinterpret_cast<const char *>(&key), 8);
}
TEST_F(EngineBasicTest, TestSeekZeroedKey) {
const std::string collection = "col";
std::string val;
ASSERT_EQ(Engine::Open(db_path.c_str(), &engine, configs, stdout),
Status::Ok);
uint64_t z = 0;
auto zero_filled_str = uint64_to_string(z);
ASSERT_EQ(engine->SSet(collection, zero_filled_str, zero_filled_str), Status::Ok);
ASSERT_EQ(engine->SGet(collection, zero_filled_str, &val), Status::Ok);
auto iter = engine->NewSortedIterator(collection);
ASSERT_NE(iter, nullptr);
iter->Seek(zero_filled_str);
ASSERT_TRUE(iter->Valid());
}
root@3d4f55155e2e:/opt/workspace/kvdk/build# PMEM_IS_PMEM_FORCE=1 ./dbtest --gtest_filter=*TestSeekZeroedKey*
Note: Google Test filter = *TestSeekZeroedKey*
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from EngineBasicTest
[ RUN ] EngineBasicTest.TestSeekZeroedKey
[LOG] time 0 ms: Initializing PMem size 17179869184 in file /mnt/pmem0/data/data
[LOG] time 2399 ms: Map pmem space done
[LOG] time 2402 ms: In restoring: iterated 0 records
/opt/workspace/kvdk/tests/tests.cpp:91: Failure
Value of: iter->Valid()
Actual: false
Expected: true
[ FAILED ] EngineBasicTest.TestSeekZeroedKey (2457 ms)
[----------] 1 test from EngineBasicTest (2457 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (2457 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] EngineBasicTest.TestSeekZeroedKey
1 FAILED TEST
I'm not sure if this is currently possible but it seems to me that WriteBatch
is only supported for the anonymous collection. Would it be possible to support WriteBatch
for sorted collections?
2 * Intel(R) Xeon(R) Platinum 8269C CPU @ 2.50GHz
CPU: 2 * Intel(R) Xeon(R) Platinum 8269C CPU @ 2.50GHz
DDR: 12 * DDR4 16384 MB 2666 MT/s
PMEM: 4 * 129408 MB 2666 MT/s
Inject this code snippet between line 250 and line 251 in skiplist.cpp to simulate a thread pausing
{
using namespace std::chrono_literals;
std::this_thread::sleep_for(1000ms);
}
Put this code snippet in tests.cpp
TEST_F(EngineBasicTest, SortedCollectionIssue2) {
const std::string collection01 = "C1";
const std::string collection02 = "C2";
int num_threads = 2;
configs.max_write_threads = num_threads;
configs.pmem_segment_blocks = 16;
ASSERT_EQ(Engine::Open(db_path.c_str(), &engine, configs, stdout),
Status::Ok);
auto Operations = [&](size_t tid)
{
if (tid == 0)
{
for (size_t i = 0; i < 11; i++)
engine->Set(std::to_string(i), "Dummy");
engine->SSet(collection01, "a", "aa");
engine->SSet(collection01, "b", "bb");
engine->SSet(collection01, "c", "cc");
engine->SSet(collection01, "d", "dd");
// This SSet will sleep 1000ms right before acquiring locks
engine->SSet(collection01, "b1", "b1b1");
}
else
{
for (size_t i = 0; i < 15; i++)
engine->Set(std::to_string(i), "Dummy");
using namespace std::chrono_literals;
std::this_thread::sleep_for(500ms);
engine->SDelete(collection01, "b");
engine->SDelete(collection01, "c");
engine->SSet(collection02, "c", "cc");
engine->SSet(collection02, "b", "bb");
}
};
auto IterateThrough = [&](std::string collection)
{
size_t cnt = 0;
std::cout << "Contents in " << collection << std::endl;
auto iter = engine->NewSortedIterator(collection);
for (iter->SeekToFirst(); iter->Valid(); iter->Next())
++cnt;
if (collection == collection01)
{
EXPECT_EQ(cnt, 3)
<< R"(There should be three keys in C1, "a", "b1" and "d")";
}
else if (collection == collection02)
{
EXPECT_EQ(cnt, 2)
<< R"(There should be two keys in C2, "b" and "c")";
}
};
LaunchNThreads(num_threads, Operations);
IterateThrough(collection01);
IterateThrough(collection02);
delete engine;
}
Test case success
Test case fail
Malicious user may use this defect to intercept data from other users, or may even break data on PMem during recovery if the attack is carefully designed.
Why is the reading and writing speed of kvdk not as fast as leveldb? I think this is very strange. Is it the problem of my use or the problem of kvdb itself? If it is the problem of kvdb itself, do you have a good recommendation for kv database, it is best to use B+ tree as the engine, thank you.
The problem/use-case that the feature addresses
There are a series number of Redis commands such as INCR, DECR, etc, which increase/decrease the value atomically. To support such commands effectively, we need a Read-Modify-Write API.
Alternatives you've considered
We can implement such function outside, but it is not very efficient
Additional information
No
In the code below I create iterator to the collection before adding any element to it by SSet() method.
TEST_F(EngineBasicTest, TestSeekToFirst) {
const std::string empty_collection = "Empty";
ASSERT_EQ(Engine::Open(db_path.c_str(), &engine, configs, stdout),
Status::Ok);
auto iter = engine->NewSortedIterator(empty_collection);
iter->SeekToFirst();
ASSERT_FALSE(iter->Valid());
}
As Collection would be implicitly created on first SSet()
it do not exists during SeekToFirst()
call. In that case iter->Valid()
should gracefully return false, as such iterator obviously is invalid. Instead of that it cause segmentation fault.
Test output:
localhost/kvdk:build# PMEM_IS_PMEM_FORCE=1 ./dbtest --gtest_filter="*SeekToFirst*"
Note: Google Test filter = *SeekToFirst*
[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from EngineBasicTest
[ RUN ] EngineBasicTest.TestSeekToFirst
[LOG] time 0 ms: Initializing PMEM size 17179869184 in file /mnt/pmem0/data/data
[LOG] time 2087 ms: Map pmem space done
[LOG] time 2091 ms: In restoring: iterated 0 records
Segmentation fault (core dumped)
benchmark: scripts/basic_benchmarks.py
zipf read of sorted collection is significantly slower than random read, and miss some keys.
benchmark result:
[LOG] time 0 ms: Initializing PMem size 412316860416 in file /mnt/pmem0/kvdk/data
[LOG] time 894 ms: Map pmem space done
[LOG] time 49673 ms: In restoring: iterated 846691530 records
init 0 write threads
init 64 read threads
------- ops in seconds -----------
time (ms), read ops, not found, write ops, total read, total write
1000 58199000 0 0 58199000 0
2001 58351000 0 0 116550000 0
3002 58362000 0 0 174912000 0
4003 58273000 0 0 233185000 0
5004 58140000 0 0 291325000 0
6005 58131000 0 0 349456000 0
7006 58152000 0 0 407608000 0
8007 58152000 0 0 465760000 0
9008 58140000 0 0 523900000 0
10009 58140000 0 0 582040000 0
finish bench
------------ statistics ------------
read ops 58204000, write ops 0
[LOG] time 59690 ms: Closing instance ...
[LOG] time 59691 ms: Waiting bg threads exit ...
[LOG] time 59715 ms: Instance closed
[LOG] time 0 ms: Initializing PMem size 412316860416 in file /mnt/pmem0/kvdk/data
[LOG] time 865 ms: Map pmem space done
[LOG] time 49674 ms: In restoring: iterated 846579819 records
init 0 write threads
init 64 read threads
------- ops in seconds -----------
time (ms), read ops, not found, write ops, total read, total write
1000 14564000 589000 0 14564000 0
2001 14692000 629000 0 29256000 0
3002 14679000 628000 0 43935000 0
4003 14692000 633000 0 58627000 0
5004 14683000 626000 0 73310000 0
6005 14642000 628000 0 87952000 0
7006 14685000 626000 0 102637000 0
8007 14694000 632000 0 117331000 0
9008 14688000 629000 0 132019000 0
10009 14668000 624000 0 146687000 0
finish bench
------------ statistics ------------
read ops 14668800, write ops 0
[LOG] time 59692 ms: Closing instance ...
[LOG] time 59692 ms: Waiting bg threads exit ...
[LOG] time 59701 ms: Instance closed
numactl --cpunodebind=0 --membind=0 ./bench -latency=1 -populate=1 -value_size=4096 -threads=128 -time=10 -path=/mnt/pmem0/kvdk -num=26163299 -space=536870912000 -max_write_threads=128 -fill=1 -type=string > ./results/string_vs4096_fill_thread128
Write latency overflow: 10266872 us
sh: line 1: 45103 Segmentation fault (core dumped) numactl --cpunodebind=0 --membind=0 ./bench -latency=1 -populate=1 -value_size=4096 -threads=128 -time=10 -path=/mnt/pmem0/kvdk -num=26163299 -space=536870912000 -max_write_threads=128 -fill=1 -type=string > ./results/string_vs4096_fill_thread128
EMON collector successfully stopped.
Read string-type kv
EMON collector successfully stopped.
Insert new string-type kv
EMON collector successfully stopped.
Batch write string-type kv
EMON collector successfully stopped.
Update string-type kv
EMON collector successfully stopped.
Mixed read/update string-type kv
EMON collector successfully stopped.
Fill string-type kv
numactl --cpunodebind=0 --membind=0 ./bench -latency=1 -populate=1 -value_size=4096 -threads=256 -time=10 -path=/mnt/pmem0/kvdk -num=26163299 -space=536870912000 -max_write_threads=256 -fill=1 -type=string > ./results/string_vs4096_fill_thread256
Write latency overflow: 10097061 us
sh: line 1: 46877 Segmentation fault (core dumped) numactl --cpunodebind=0 --membind=0 ./bench -latency=1 -populate=1 -value_size=4096 -threads=256 -time=10 -path=/mnt/pmem0/kvdk -num=26163299 -space=536870912000 -max_write_threads=256 -fill=1 -type=string > ./results/string_vs4096_fill_thread256
EMON collector successfully stopped.
Read string-type kv
EMON collector successfully stopped.
Insert new string-type kv
Write latency overflow: 11150634 us
sh: line 1: 47928 Segmentation fault (core dumped) numactl --cpunodebind=0 --membind=0 ./bench -latency=1 -populate=1 -value_size=4096 -threads=256 -time=10 -path=/mnt/pmem0/kvdk -num=26163299 -space=536870912000 -max_write_threads=256 -fill=0 -type=string -read_ratio=0 -existing_keys_ratio=0 > ./results/string_vs4096_insert_thread256
EMON collector successfully stopped.
Batch write string-type kv
EMON collector successfully stopped.
Update string-type kv
Write latency overflow: 11173244 us
sh: line 1: 48963 Segmentation fault (core dumped) numactl --cpunodebind=0 --membind=0 ./bench -latency=1 -populate=1 -value_size=4096 -threads=256 -time=10 -path=/mnt/pmem0/kvdk -num=26163299 -space=536870912000 -max_write_threads=256 -fill=0 -type=string -read_ratio=0 > ./results/string_vs4096_update_thread256
EMON collector successfully stopped.
Mixed read/update string-type kv
EMON collector successfully stopped.
Branch: pmem/kvdk/main:19332d5
Binary is built under kvdk/build/
by calling cmake ..
and make -j
Then the old directory under pmem is cleared to create a new instance by calling
rm -rf /mnt/pmem0/debugging2
The new engine is created by
./bench -populate=1 -value_size=120 -threads=1 -time=10 -path=/mnt/pmem0/debugging2 -num=1048576 -space=209715200 -max_write_threads=1 -collections=1 -type=sorted -fill=1
and then closed.
Then the engine is reopened, trying to be restored, which will trigger segment error.
./bench -populate=1 -value_size=120 -threads=1 -time=10 -path=/mnt/pmem0/debugging2 -num=1048576 -space=209715200 -max_write_threads=1 -collections=1 -type=sorted -fill=0 -read_ratio=1
The problem/use-case that the feature addresses
There are a series of Redis commands, such as DUMP and RESTORE, which are used to dump the database as a snapshot and restore it somewhere. We hope such functions can be supported by KVDK.
Segmentation fault (core dumped) numactl --cpunodebind=0 --membind=0 ./bench -latency=1 -populate=1 -value_size=4096 -threads=64 -time=10 -path=/mnt/pmem0/kvdk -num=26163299 -space=536870912000 -max_write_threads=64 -fill=0 -type=sorted -read_ratio=0 -existing_keys_ratio=0 > ./results/sorted_vs4096_insert_thread64
Segmentation fault (core dumped) numactl --cpunodebind=0 --membind=0 ./bench -latency=1 -populate=1 -value_size=4096 -threads=64 -time=10 -path=/mnt/pmem0/kvdk -num=26163299 -space=536870912000 -max_write_threads=64 -fill=0 -type=sorted -read_ratio=0 > ./results/sorted_vs4096_update_thread64
please take a look.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.