oceanbase / miniob Goto Github PK

View Code? Open in Web Editor NEW

2.8K 21.0 996.0 27.36 MB

MiniOB is a compact database that assists developers in understanding the fundamental workings of a database.

Home Page: https://oceanbase.github.io/miniob/

License: Mulan Permissive Software License, Version 2

CMake 1.01% C 1.92% C++ 89.53% Lex 0.53% Yacc 1.48% Python 3.25% Dockerfile 0.12% Shell 0.50% Lua 1.65%

database learn-to-database mini-database classroom education training cplusplus

miniob's Introduction

English | 中文版

OceanBase Database is a distributed relational database. It is developed entirely by Ant Group. The OceanBase Database is built on a common server cluster. Based on the Paxos protocol and its distributed structure, the OceanBase Database provides high availability and linear scalability. The OceanBase Database is not dependent on specific hardware architectures.

Key features

Transparent Scalability

An OceanBase cluster can be scaled to 1,500 nodes transparently, handling petabytes of data and a trillion rows of records.
Ultra-fast Performance

The only distributed database that has refreshed both the TPC-C record, at 707 million tmpC, and the TPC-H record, at 15.26 million QphH @30000GB.
Real-time Operational Analytics

A unified system for both transactional and real-time operational analytics workloads.
Continuous Availability

OceanBase Database adopts the Paxos Consensus algorithm to achieve Zero RPO and less than 8 seconds of RTO. Supports intra-city/remote disaster recovery, enabling multi-activity in multiple locations and zero data loss.
MySQL Compatible

OceanBase Database is highly compatible with MySQL, which ensures that zero or a few modifications are needed for migration.
Cost Efficiency

The cutting-edge compression technology saves 70%–90% of storage costs without compromising performance. The multi-tenancy architecture achieves higher resource utilization.

Quick start

🔥 Start with all-in-one

You can quickly deploy a stand-alone OceanBase Database to experience with the following commands:

Note: Linux Only

# download and install all-in-one package (internet connection is required)
bash -c "$(curl -s https://obbusiness-private.oss-cn-shanghai.aliyuncs.com/download-center/opensource/oceanbase-all-in-one/installer.sh)"
source ~/.oceanbase-all-in-one/bin/env.sh

# quickly deploy OceanBase database
obd demo

🐳 Start with docker

Start an OceanBase Database instance:

# Deploy a mini standalone instance.
docker run -p 2881:2881 --name oceanbase-ce -e MODE=mini -d oceanbase/oceanbase-ce

Connect to the OceanBase Database instance:

docker exec -it oceanbase-ce ob-mysql sys # Connect to the root user of the sys tenant.

See also Quick experience or Quick Start (Simplified Chinese) for more details.

☸️ Start with Kubernetes

You can deploy and manage OceanBase Database instance in kubernetes cluster with ob-operator quickly. Refer to the document Quick Start for ob-operator to see details.

👨‍💻 Start developing

See OceanBase Developer Document to learn how to compile and deploy a manually compiled observer.

Roadmap

For future plans, see Product Iteration Progress. See also OceanBase Roadmap for more details.

Case study

OceanBase has been serving more than 1000 customers and upgraded their database from different industries, including Financial Services, Telecom, Retail, Internet, and more.

See also success stories and Who is using OceanBase for more details.

System architecture

Introduction to system architecture

Contributing

Contributions are highly appreciated. Read the development guide to get started.

License

OceanBase Database is licensed under the Mulan Public License, Version 2. See the LICENSE file for more info.

Community

Join the OceanBase community via:

Slack Workspace
Ask on Stack Overflow
Chinese User Forum
DingTalk Group: 33254054 (QR code)
WeChat Group (Add the assistant with WeChat ID: OBCE666)

miniob's People

Contributors

Stargazers

Watchers

Forkers

db2013x blablablawang lidongjiehiha jizhongzhiji zhiyang0310 tomsmspersonal i-m-so-sorry-for-come-to-zju overduewhite zenoyang bluefatt xishuaidelin zbtzbtzbt yrreallycute aijianchi hzmciu yrisreallycute bombia tangyiheng2021 escapekk xuyifangreeneyes humblehunger awfeequdng nietiezheng ingdex deng23yu maizesty wzj111 tanxujie charlie17li kaimary a920264845 rawmaterials223 lodisaq bxx-seu duxuchong leafgodgood sciengineer-97 xyl93 m-m-dot ciproud a1445605672 zyk1314 xuetongyao sanbeicha lemon007-kui reverie-dev aiceflower liuxiaocs7 httdty jiangnanmax luokeyue d821776892 wzr3016 licheng-xu lidwchn lightingmax caichenshu guopengju9239 cnlhc thinksoso wangwangteaminoceanbasecompetition wushengzuinonglie wedreamer cao-y wheel-maker rugege 1320911490 algo-goer baymax-lin ginahh sunnyboy-wyh hengtian xiongbi nicknyu beeter-yong lansaber mjy-hust liukun4515 dawnwalk blueskylxb zhuichen csu-wrz lundiwei chenjlcsdn jedmax ft1573734 ericdoug-qi gyming1849 hyj0 frank-gh longbozhan mengjiahao mark0078 nkzxw harryhare hubing2610 cococa hustxmy zhaoyushuo123 plsmaop

miniob's Issues

penggan666:

test penggan666 complex-sub-query

将单元测试模块设置为可选择依赖

stage线程分配问题

由于代码逻辑错误，将所有stage处理都分配给DefaultThreads线程池。

cannot find gdb in docker

I cannot find the gdb in docker image, but it is important

构建镜像的时候下载 cmake 一直会超时

我这边是直接 wget 拉对应版本的 cmake 的，看样子 ob 这边是把指定版本的 cmake 做成了一个 rpm 包放在 oss 上面了? 不知道这个服务器在哪，我这边本地访问还有使用 github action 访问都是不通的，这个链接是不是要换一下？

handler.h is unused

the file handler.h is not referenced by any file

asan doc is need

client 增加更便捷的操作模式

当前的client实现比较简单，并且不支持超长字符串输入，可以考虑使用readline模块实现

BanBuDu0

test BanBuDu0 date

需要增加.gitignore

增加.gitignore防止误提交一些文件或目录

pthread_setname_np 函数的引入导致在Mac上编译失败

请问这里是有bug吗？我是Mac环境，拉取main分支最新的编译出错了。

-- from @Joenhle

此BUG由 #79 引入

penggan666:

test penggan666 complex-sub-query

test OBserver-BUPT basic date update select-meta select-tables aggregation-func drop-table insert join-tables null unique simple-sub-query multi-index text expression complex-sub-query order-by group-by

duplicate codes in bplus_tree.h and tuple_cell.cpp

the codes for attribute compare is duplicate

int operator()(const char *v1, const char *v2) const {
    switch (attr_type_) {
    case INTS: {
      return *(int *)v1 - *(int *)v2;
    }
      break;
    case FLOATS: {
      float result = *(float *)v1 - *(float *)v2;
      if (-1e-6 < result && result < 1e-6) {
        return 0;
      }
      return result > 0 ? 1 : -1;
    }
    case CHARS: {
      return strncmp(v1, v2, attr_length_);
    }
    default:{
      LOG_ERROR("unknown attr type. %d", attr_type_);
      abort();
    }
    }
  }

the other

int TupleCell::compare(const TupleCell &other) const
{
  if (this->attr_type_ == other.attr_type_) {
    switch (this->attr_type_) {
    case INTS: return compare_int(this->data_, other.data_);
    case FLOATS: return compare_float(this->data_, other.data_);
    case CHARS: return compare_string(this->data_, this->length_, other.data_, other.length_);
    default: {
      LOG_WARN("unsupported type: %d", this->attr_type_);
    }
    }
  } else if (this->attr_type_ == INTS && other.attr_type_ == FLOATS) {
    float this_data = *(int *)data_;
    return compare_float(&this_data, other.data_);
  } else if (this->attr_type_ == FLOATS && other.attr_type_ == INTS) {
    float other_data = *(int *)other.data_;
    return compare_float(data_, &other_data);
  }
  LOG_WARN("not supported");
  return -1; // TODO return rc?
}

miniob 在 m1 芯片的 mac 上调试的问题

想问一下 miniob 应该是还没有适配 m1 的 air 对吧，目前我拉取官方的镜像显示的是只支持 linux/amd64 平台。

空跑CPU太高

空跑时，占用一个CPU。
环境：MacOS arm平台
占用CPU的栈信息：

(lldb) bt
* thread #11
  * frame #0: 0x0000000104fdc50c libcommon.1.0.0.dylib`std::__1::__tree<std::__1::__value_type<common::TimerToken, common::StageEvent*>, std::__1::__map_value_compare<common::TimerToken, std::__1::__value_type<common::TimerToken, common::StageEvent*>, bool (*)(common::TimerToken const&, common::TimerToken const&), false>, std::__1::allocator<std::__1::__value_type<common::TimerToken, common::StageEvent*> > >::erase(std::__1::__tree_const_iterator<std::__1::__value_type<common::TimerToken, common::StageEvent*>, std::__1::__tree_node<std::__1::__value_type<common::TimerToken, common::StageEvent*>, void*>*, long>, std::__1::__tree_const_iterator<std::__1::__value_type<common::TimerToken, common::StageEvent*>, std::__1::__tree_node<std::__1::__value_type<common::TimerToken, common::StageEvent*>, void*>*, long>) + 4
    frame #1: 0x0000000104fd3168 libcommon.1.0.0.dylib`std::__1::map<common::TimerToken, common::StageEvent*, bool (*)(common::TimerToken const&, common::TimerToken const&), std::__1::allocator<std::__1::pair<common::TimerToken const, common::StageEvent*> > >::erase(this=0x0000000112804ce0 size=1, __f=std::__1::map<common::TimerToken, common::StageEvent *, bool (*)(const common::TimerToken &, const common::TimerToken &), std::__1::allocator<std::__1::pair<const common::TimerToken, common::StageEvent *> > >::const_iterator @ 0x000000016b734d00, __l=std::__1::map<common::TimerToken, common::StageEvent *, bool (*)(const common::TimerToken &, const common::TimerToken &), std::__1::allocator<std::__1::pair<const common::TimerToken, common::StageEvent *> > >::const_iterator @ 0x000000016b734cf8) at map:1309:25
    frame #2: 0x0000000104fd246c libcommon.1.0.0.dylib`common::TimerStage::check_timer(this=0x0000000112804c00) at timer_stage.cpp:450:18
    frame #3: 0x0000000104fd0470 libcommon.1.0.0.dylib`common::TimerStage::start_timer_thread(arg=0x0000000112804c00) at timer_stage.cpp:423:11
    frame #4: 0x0000000183e9b878 libsystem_pthread.dylib`_pthread_start + 320

cmake TARGET_LINK_DIRECTORIES in version 3.13

Begin to build miniob
-- This is PROJECT_BINARY_DIR dir /root/miniob/build/src
-- This is PROJECT_SOURCE_DIR dir /root/miniob/src
Begin to build obclient
CMake Error at src/obclient/CMakeLists.txt:11 (TARGET_LINK_DIRECTORIES):
Unknown CMake command "TARGET_LINK_DIRECTORIES".

-- Configuring incomplete, errors occurred!
See also "/root/miniob/build/CMakeFiles/CMakeOutput.log".
[root@host-10-208-65-5 build]# cmake -version
cmake version 3.12.1

TARGET_LINK_DIRECTORIES can be use in version >=3.13

miniob-test2

大家找一块自己的地方，在一个comment里面，可以重复提交测试，每次comment变更，都可以检测到。
在一个时间段内，一个团队的任务提交，只会有一个任务执行。
使用方法：

# 执行测试
test <your-github-name> [case1] [case2] ...

# 查看失败原因
view <your-github-name> <case1> [case2] ...

FAQ:

查看失败原因长时间没有返回结果
如果返回的数据中包含二进制字符，即不是纯文本文件，那么执行结果无法反馈到github中，目前无法看到结果。
每个case的名称可以看这个链接
https://oceanbase-partner.github.io/lectures-on-dbms-implementation/miniob-topics
basic测试的名称就叫 basic

undefined reference to `typeinfo for testing::Test'

编译完成所有deps后，编译miniob时，报错：
95%] Built target path_test

[ 95%] Building CXX object unitest/CMakeFiles/md5_test.dir/md5_test.cpp.o

[ 96%] Linking CXX executable ../bin/md5_test

[ 96%] Built target md5_test

[ 96%] Building CXX object unitest/CMakeFiles/log_test.dir/log_test.cpp.o

[ 97%] Linking CXX executable ../bin/log_test

/usr/bin/ld: CMakeFiles/log_test.dir/log_test.cpp.o:(.data.rel.ro._ZTI35checkRotateTest_CheckRoateTest_Test[_ZTI35checkRotateTest_CheckRoateTest_Test]+0x10): undefined reference to `typeinfo for testing::Test'

/usr/bin/ld: CMakeFiles/log_test.dir/log_test.cpp.o:(.data.rel.ro._ZTI35testEnableTest_CheckEnableTest_Test[_ZTI35testEnableTest_CheckEnableTest_Test]+0x10): undefined reference to `typeinfo for testing::Test'

collect2: error: ld returned 1 exit status

make[2]: *** [unitest/CMakeFiles/log_test.dir/build.make:105: bin/log_test] Error 1

make[1]: *** [CMakeFiles/Makefile2:497: unitest/CMakeFiles/log_test.dir/all] Error 2

make: *** [Makefile:149: all] Error 2

'basic' test failed

branch: main

基本的比如insert,delete操作有问题

比如我此图，上面的查询是我更改了delete的逻辑，在dispose_page的时候如果把页面删除掉了就让pageCount--，才不会出现delete后select出错的情况。
还有，原生的insert和delete在执行后均不能在磁盘上持久化，miniob的事务代码中并没有此逻辑，并且也没有让我们实现事务的功能。

编译选项增加-Werror可以提前发现一些BUG

增加-Werror，可以帮助提前发现一些问题，比如函数没有返回值。
现在的CMake中增加了-Wall，但是还存在一些编译告警，消灭编译告警后，才可以加上-Werror

ASAN

miniob-test

# 执行测试
test <your-github-name> [case1] [case2] ...

# 查看失败原因
view <your-github-name> <case1> [case2] ...

FAQ:

查看失败原因长时间没有返回结果
如果返回的数据中包含二进制字符，即不是纯文本文件，那么执行结果无法反馈到github中，目前无法看到结果。
每个case的名称可以看这个链接
https://oceanbase-partner.github.io/lectures-on-dbms-implementation/miniob-topics
basic测试的名称就叫 basic

mac 上使用gcc 12.2编译失败

rewrite B+ tree

The B+ tree's implementaion isn't good, it had better be reimplemented. The speed is low, at the same time, there is no unit test for it.

[compile issue] Could NOT find OpenSSL

Issue Report

Mac OS, running cmake as it describes in this doc: how to build
At the 'libevent' step, got errors like this:

build git:(5df3037d) cmake .. -DEVENT_DISABLE_OPENSSL=ON                                
CMake Warning at CMakeLists.txt:663 (message):
  Cannot check if kqueue works with pipes when crosscompiling, use
  EVENT__FORCE_KQUEUE_CHECK to be sure (this requires manually running a test
  program on the cross compilation target)


CMake Error at /usr/local/Cellar/cmake/3.18.4/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
  Could NOT find OpenSSL, try to set the path to OpenSSL root folder in the
  system variable OPENSSL_ROOT_DIR (missing: OPENSSL_CRYPTO_LIBRARY
  OPENSSL_INCLUDE_DIR)
Call Stack (most recent call first):
  /usr/local/Cellar/cmake/3.18.4/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:458 (_FPHSA_FAILURE_MESSAGE)
  /usr/local/Cellar/cmake/3.18.4/share/cmake/Modules/FindOpenSSL.cmake:486 (find_package_handle_standard_args)
  CMakeLists.txt:844 (find_package)

Running ENV

Software:

    System Software Overview:

      System Version: macOS 12.4 (21F79)
      Kernel Version: Darwin 21.5.0
      Boot Volume: MacintoshHD
      Boot Mode: Normal
      Secure Virtual Memory: Enabled
      System Integrity Protection: Enabled
      Time since boot: 7 days 13 minutes

Hardware:

    Hardware Overview:

      Model Name: MacBook Pro
      Model Identifier: MacBookPro16,2
      Processor Name: Quad-Core Intel Core i5
      Processor Speed: 2 GHz
      Number of Processors: 1
      Total Number of Cores: 4
      L2 Cache (per Core): 512 KB
      L3 Cache: 6 MB
      Hyper-Threading Technology: Enabled
      Memory: 16 GB
      System Firmware Version: 1731.120.10.0.0 (iBridge: 19.16.15071.0.0,0)
      OS Loader Version: 540.120.3~6
      Activation Lock Status: Enabled

m1 系统 server 关闭后没有执行刷盘导致重启数据失效

m1 系统 kill server 之后会因为文件没有刷盘而导致重启获取不到数据
用 ls -l 查看可以发现在 kill 后文件没有修改

no prompt in client while without readline

build googletest failed

when execute make command according to build.md step by step, it failed like this:

make[2]: *** [googletest/CMakeFiles/gtest.dir/src/gtest-all.cc.o] Error 1
make[1]: *** [googletest/CMakeFiles/gtest.dir/all] Error 2

tried to update gcc from 4.8 to upper version, but it didn't work.

Solution: checkout to googletest release version instead of using master branch to build

build google test step

git submodule add https://github.com/google/googletest deps/googletest 
cd deps
cd googletest
git checkout release-1.8.1  #add this step to checkout to release-1.8.1: the lateast release version
mkdir build
cd build
cmake ..
make
sudo make install

event_set is a deprecated function

event_set is used in server.cpp, but it is deprecated

observer 不能运行

按照文档编译成功了，但是运行 observer 报错，如下：

➜  bin git:(main) ✗ ./observer 
Successfully load ../etc/observer.ini
Not set log file name, use default observer.log
Output configuration 
Begin dump configuration
[]

Finish dump configuration 

(Re)Starting State: Pid: 11171 Time: Sat Oct 16 20:55:43 2021

The process Name is observer
Successfully init_event_history, EventHistory:0, MaxEventHops:100
Configuration hasn't set ThreadPools
Failed to init thread pool

SedaConfig: unable to initialize seda stages
Failed to init seda configuration!
Shutdown due to failed to init!

test_bingandbing

test bingandbing drop-table select-meta

server.cpp:send中关闭连接后，应该终止会话

在server.cpp: send中，发送消息出现异常，会close_connection，但是没有终止会话，导致还可能再次通过连接发送消息，引起coredump

how_to_build文档不准确

how_to_build文档的描述步骤，不能完全按照描述来操作。
比如GCC，在MAC上不需要安装；libevent的git add submodule操作，需要在git clone miniob之后才能执行

lex_sql.l 格式混乱

lex_sql.l 格式比较乱

简化编译，将依赖模块以external project的方式引入miniob

test <a1445605672> [basic测试]

使用docker报错，无法启动server

使用docker拉取镜像后，执行build.sh
启动server后报错，找不到jsoncpp依赖等

CMakeLists.txt: “-DCMAKE_EXPORT_COMPILE_COMMANDS=1”不应该放在此处.

miniob的小伙伴们你们好，我使用vim搭配coc.nvim以及clangd实现c/c++的自动补全，因此对compile_commands.json有需求。

当前的CMakeLists.txt已经涉及compile_commands.json的生成，但是这个FLAG的位置应该摆错了。

miniob/CMakeLists.txt

Line 40 in c611d5f

 SET(CMAKE_COMMON_FLAGS "${CMAKE_COMMON_FLAGS} -Wall -DCMAKE_EXPORT_COMPILE_COMMANDS=1") 

虽然CMAKE_COMMON_FLAGS这个变量感觉上是设置了cmake的FLAGS，但是最后其实是赋值给了CMAKE_CXX_FLAGS，而-DCMAKE_EXPORT_COMPILE_COMMANDS=1这个参数是用在cmake上的，而不是gcc。

解决方法是：

1、在CMakeLists.txt开头位置附近加上set(CMAKE_EXPORT_COMPILE_COMMANDS ON)

2、或在运行cmake时加上参数，即cmake -DCMAKE_EXPORT_COMPILE_COMMANDS=1 ..

这样就能生成compile_commands.json：

实现resolver模块

在resolver模块中讲“字符串”语法树转换为可执行语法树，并对元数据做校验

test pistacio520

Add lecture

Add database lecture

[question]能编译成功但observer无法启动

问题：miniob在build文件夹下编译成功后未能成功启动observer

init.cpp文件中的第82行，

std::map<std::string, std::string> log_section = properties.get(log_section_name);
其中properties.get()函数似乎没有获取，log_section的size仍为0，想问下是什么原因呢

环境

Windows11，WSL2，Ubuntu22.04
gcc 11.2.0，cmake 3.24.1

运行

yulinux@yu:/mnt/e/github/miniob/build$ make
-- The C compiler identification is GNU 11.2.0
-- The CXX compiler identification is GNU 11.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- This is SOURCE dir 
-- This is BINARY dir 
-- This is Project source dir /mnt/e/github/miniob
-- This is PROJECT_BINARY_DIR dir /mnt/e/github/miniob/build
-- HOME dir: /home/yulinux
-- This is UNIX
Disable debug
CMAKE_CXX_FLAGS is  -Wall -DCMAKE_EXPORT_COMPILE_COMMANDS=1  -O2 -g
CMAKE_INSTALL_PREFIX has been set as /usr/local
Install target dir is /usr/local
Begin to build common
-- This is PROJECT_BINARY_DIR dir /mnt/e/github/miniob/build/deps/common
-- This is PROJECT_SOURCE_DIR dir /mnt/e/github/miniob/deps/common
Use /mnt/e/github/miniob/deps/common/conf/ini.cpp
Use /mnt/e/github/miniob/deps/common/io/io.cpp
Use /mnt/e/github/miniob/deps/common/io/roll_select_dir.cpp
Use /mnt/e/github/miniob/deps/common/lang/bitmap.cpp
Use /mnt/e/github/miniob/deps/common/lang/mutex.cpp
Use /mnt/e/github/miniob/deps/common/lang/string.cpp
Use /mnt/e/github/miniob/deps/common/log/log.cpp
Use /mnt/e/github/miniob/deps/common/math/md5.cpp
Use /mnt/e/github/miniob/deps/common/math/random_generator.cpp
Use /mnt/e/github/miniob/deps/common/math/regex.cpp
Use /mnt/e/github/miniob/deps/common/metrics/console_reporter.cpp
Use /mnt/e/github/miniob/deps/common/metrics/histogram_snapshot.cpp
Use /mnt/e/github/miniob/deps/common/metrics/log_reporter.cpp
Use /mnt/e/github/miniob/deps/common/metrics/metrics.cpp
Use /mnt/e/github/miniob/deps/common/metrics/metrics_registry.cpp
Use /mnt/e/github/miniob/deps/common/metrics/reporter.cpp
Use /mnt/e/github/miniob/deps/common/metrics/reservoir.cpp
Use /mnt/e/github/miniob/deps/common/metrics/sampler.cpp
Use /mnt/e/github/miniob/deps/common/metrics/timer_snapshot.cpp
Use /mnt/e/github/miniob/deps/common/metrics/uniform_reservoir.cpp
Use /mnt/e/github/miniob/deps/common/mm/mem_pool.cpp
Use /mnt/e/github/miniob/deps/common/os/os.cpp
Use /mnt/e/github/miniob/deps/common/os/path.cpp
Use /mnt/e/github/miniob/deps/common/os/pidfile.cpp
Use /mnt/e/github/miniob/deps/common/os/process.cpp
Use /mnt/e/github/miniob/deps/common/os/process_param.cpp
Use /mnt/e/github/miniob/deps/common/os/signal.cpp
Use /mnt/e/github/miniob/deps/common/seda/callback.cpp
Use /mnt/e/github/miniob/deps/common/seda/event_dispatcher.cpp
Use /mnt/e/github/miniob/deps/common/seda/example_stage.cpp
Use /mnt/e/github/miniob/deps/common/seda/init.cpp
Use /mnt/e/github/miniob/deps/common/seda/kill_thread.cpp
Use /mnt/e/github/miniob/deps/common/seda/metrics_report_event.cpp
Use /mnt/e/github/miniob/deps/common/seda/metrics_stage.cpp
Use /mnt/e/github/miniob/deps/common/seda/seda_config.cpp
Use /mnt/e/github/miniob/deps/common/seda/stage.cpp
Use /mnt/e/github/miniob/deps/common/seda/stage_event.cpp
Use /mnt/e/github/miniob/deps/common/seda/thread_pool.cpp
Use /mnt/e/github/miniob/deps/common/seda/timer_stage.cpp
Use /mnt/e/github/miniob/deps/common/time/datetime.cpp
Use /mnt/e/github/miniob/deps/common/time/timeout_info.cpp
-- This is the func_static OUTPUT_NAME:common
Install conf/ini.h to /usr/local/common/include/conf
Install defs.h to /usr/local/common/include/
Install io/io.h to /usr/local/common/include/io
Install io/roll_select_dir.h to /usr/local/common/include/io
Install io/select_dir.h to /usr/local/common/include/io
Install lang/bitmap.h to /usr/local/common/include/lang
Install lang/defer.h to /usr/local/common/include/lang
Install lang/mutex.h to /usr/local/common/include/lang
Install lang/serializable.h to /usr/local/common/include/lang
Install lang/string.h to /usr/local/common/include/lang
Install log/log.h to /usr/local/common/include/log
Install math/md5.h to /usr/local/common/include/math
Install math/random_generator.h to /usr/local/common/include/math
Install math/regex.h to /usr/local/common/include/math
Install metrics/console_reporter.h to /usr/local/common/include/metrics
Install metrics/histogram_snapshot.h to /usr/local/common/include/metrics
Install metrics/log_reporter.h to /usr/local/common/include/metrics
Install metrics/metric.h to /usr/local/common/include/metrics
Install metrics/metrics.h to /usr/local/common/include/metrics
Install metrics/metrics_registry.h to /usr/local/common/include/metrics
Install metrics/reporter.h to /usr/local/common/include/metrics
Install metrics/reservoir.h to /usr/local/common/include/metrics
Install metrics/sampler.h to /usr/local/common/include/metrics
Install metrics/snapshot.h to /usr/local/common/include/metrics
Install metrics/timer_snapshot.h to /usr/local/common/include/metrics
Install metrics/uniform_reservoir.h to /usr/local/common/include/metrics
Install mm/debug_new.h to /usr/local/common/include/mm
Install mm/mem.h to /usr/local/common/include/mm
Install mm/mem_pool.h to /usr/local/common/include/mm
Install os/os.h to /usr/local/common/include/os
Install os/path.h to /usr/local/common/include/os
Install os/pidfile.h to /usr/local/common/include/os
Install os/process.h to /usr/local/common/include/os
Install os/process_param.h to /usr/local/common/include/os
Install os/signal.h to /usr/local/common/include/os
Install seda/callback.h to /usr/local/common/include/seda
Install seda/class_factory.h to /usr/local/common/include/seda
Install seda/event_dispatcher.h to /usr/local/common/include/seda
Install seda/example_stage.h to /usr/local/common/include/seda
Install seda/init.h to /usr/local/common/include/seda
Install seda/kill_thread.h to /usr/local/common/include/seda
Install seda/metrics_report_event.h to /usr/local/common/include/seda
Install seda/metrics_stage.h to /usr/local/common/include/seda
Install seda/seda_config.h to /usr/local/common/include/seda
Install seda/seda_defs.h to /usr/local/common/include/seda
Install seda/stage.h to /usr/local/common/include/seda
Install seda/stage_event.h to /usr/local/common/include/seda
Install seda/stage_factory.h to /usr/local/common/include/seda
Install seda/thread_pool.h to /usr/local/common/include/seda
Install seda/timer_stage.h to /usr/local/common/include/seda
Install time/datetime.h to /usr/local/common/include/time
Install time/timeout_info.h to /usr/local/common/include/time
Install version.h to /usr/local/common/include/
Begin to build miniob
-- This is PROJECT_BINARY_DIR dir /mnt/e/github/miniob/build/src
-- This is PROJECT_SOURCE_DIR dir /mnt/e/github/miniob/src
Begin to build obclient
-- Looking for include files stdio.h, readline/readline.h
-- Looking for include files stdio.h, readline/readline.h - not found
readline is not found
Use /mnt/e/github/miniob/src/obclient/client.cpp
Binary directory:/mnt/e/github/miniob/build/src/../bin
Begin to build observer
-- This is PROJECT_BINARY_DIR dir /mnt/e/github/miniob/build/src/observer
-- This is PROJECT_SOURCE_DIR dir /mnt/e/github/miniob/src/observer
MAIN SRC: /mnt/e/github/miniob/src/observer/main.cpp
Use /mnt/e/github/miniob/src/observer/event/session_event.cpp
Use /mnt/e/github/miniob/src/observer/event/sql_event.cpp
Use /mnt/e/github/miniob/src/observer/init.cpp
Use /mnt/e/github/miniob/src/observer/main.cpp
Use /mnt/e/github/miniob/src/observer/net/server.cpp
Use /mnt/e/github/miniob/src/observer/rc.cpp
Use /mnt/e/github/miniob/src/observer/session/session.cpp
Use /mnt/e/github/miniob/src/observer/session/session_stage.cpp
Use /mnt/e/github/miniob/src/observer/sql/executor/execute_stage.cpp
Use /mnt/e/github/miniob/src/observer/sql/expr/expression.cpp
Use /mnt/e/github/miniob/src/observer/sql/expr/tuple_cell.cpp
Use /mnt/e/github/miniob/src/observer/sql/operator/delete_operator.cpp
Use /mnt/e/github/miniob/src/observer/sql/operator/index_scan_operator.cpp
Use /mnt/e/github/miniob/src/observer/sql/operator/insert_operator.cpp
Use /mnt/e/github/miniob/src/observer/sql/operator/predicate_operator.cpp
Use /mnt/e/github/miniob/src/observer/sql/operator/project_operator.cpp
Use /mnt/e/github/miniob/src/observer/sql/operator/table_scan_operator.cpp
Use /mnt/e/github/miniob/src/observer/sql/optimizer/optimize_stage.cpp
Use /mnt/e/github/miniob/src/observer/sql/parser/lex.yy.c
Use /mnt/e/github/miniob/src/observer/sql/parser/parse.cpp
Use /mnt/e/github/miniob/src/observer/sql/parser/parse_stage.cpp
Use /mnt/e/github/miniob/src/observer/sql/parser/resolve_stage.cpp
Use /mnt/e/github/miniob/src/observer/sql/parser/yacc_sql.tab.c
Use /mnt/e/github/miniob/src/observer/sql/plan_cache/plan_cache_stage.cpp
Use /mnt/e/github/miniob/src/observer/sql/query_cache/query_cache_stage.cpp
Use /mnt/e/github/miniob/src/observer/sql/stmt/delete_stmt.cpp
Use /mnt/e/github/miniob/src/observer/sql/stmt/filter_stmt.cpp
Use /mnt/e/github/miniob/src/observer/sql/stmt/insert_stmt.cpp
Use /mnt/e/github/miniob/src/observer/sql/stmt/select_stmt.cpp
Use /mnt/e/github/miniob/src/observer/sql/stmt/stmt.cpp
Use /mnt/e/github/miniob/src/observer/sql/stmt/update_stmt.cpp
Use /mnt/e/github/miniob/src/observer/storage/common/condition_filter.cpp
Use /mnt/e/github/miniob/src/observer/storage/common/db.cpp
Use /mnt/e/github/miniob/src/observer/storage/common/field_meta.cpp
Use /mnt/e/github/miniob/src/observer/storage/common/index_meta.cpp
Use /mnt/e/github/miniob/src/observer/storage/common/meta_util.cpp
Use /mnt/e/github/miniob/src/observer/storage/common/record_manager.cpp
Use /mnt/e/github/miniob/src/observer/storage/common/table.cpp
Use /mnt/e/github/miniob/src/observer/storage/common/table_meta.cpp
Use /mnt/e/github/miniob/src/observer/storage/default/default_handler.cpp
Use /mnt/e/github/miniob/src/observer/storage/default/default_storage_stage.cpp
Use /mnt/e/github/miniob/src/observer/storage/default/disk_buffer_pool.cpp
Use /mnt/e/github/miniob/src/observer/storage/index/bplus_tree.cpp
Use /mnt/e/github/miniob/src/observer/storage/index/bplus_tree_index.cpp
Use /mnt/e/github/miniob/src/observer/storage/index/index.cpp
Use /mnt/e/github/miniob/src/observer/storage/mem/mem_storage_stage.cpp
Use /mnt/e/github/miniob/src/observer/storage/trx/trx.cpp
Use /mnt/e/github/miniob/src/observer/util/comparator.cpp
Binary directory:/mnt/e/github/miniob/build/src/observer/../../bin
Archive directory:/mnt/e/github/miniob/build/src/observer/../../lib
Begin to build test
-- This is PROJECT_BINARY_DIR dir /mnt/e/github/miniob/build/test/perf
-- This is PROJECT_SOURCE_DIR dir /mnt/e/github/miniob/test/perf
 -Wall -DCMAKE_EXPORT_COMPILE_COMMANDS=1  -O2 -g
Build client_performance_test according to /mnt/e/github/miniob/test/perf/client_performance_test.cpp
Begin to build unitest
-- This is PROJECT_BINARY_DIR dir /mnt/e/github/miniob/build/unitest
-- This is PROJECT_SOURCE_DIR dir /mnt/e/github/miniob/unitest
 -Wall -DCMAKE_EXPORT_COMPILE_COMMANDS=1  -O2 -g
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
Build bitmap_test according to /mnt/e/github/miniob/unitest/bitmap_test.cpp
Build bp_manager_test according to /mnt/e/github/miniob/unitest/bp_manager_test.cpp
Build bplus_tree_test according to /mnt/e/github/miniob/unitest/bplus_tree_test.cpp
Build log_test according to /mnt/e/github/miniob/unitest/log_test.cpp
Build md5_test according to /mnt/e/github/miniob/unitest/md5_test.cpp
Build mem_pool_test according to /mnt/e/github/miniob/unitest/mem_pool_test.cpp
Build path_test according to /mnt/e/github/miniob/unitest/path_test.cpp
Build pidfile_test according to /mnt/e/github/miniob/unitest/pidfile_test.cpp
Build rc_test according to /mnt/e/github/miniob/unitest/rc_test.cpp
Build record_manager_test according to /mnt/e/github/miniob/unitest/record_manager_test.cpp
-- Configuring done
-- Generating done
-- Build files have been written to: /mnt/e/github/miniob/build
Consolidate compiler generated dependencies of target common
[ 21%] Built target common
Consolidate compiler generated dependencies of target common_static
[ 41%] Built target common_static
Consolidate compiler generated dependencies of target obclient
[ 42%] Built target obclient
Consolidate compiler generated dependencies of target observer
[ 66%] Built target observer
Consolidate compiler generated dependencies of target observer_static
[ 90%] Built target observer_static
Consolidate compiler generated dependencies of target client_performance_test
[ 91%] Built target client_performance_test
Consolidate compiler generated dependencies of target bitmap_test
[ 91%] Built target bitmap_test
Consolidate compiler generated dependencies of target bp_manager_test
[ 92%] Built target bp_manager_test
Consolidate compiler generated dependencies of target bplus_tree_test
[ 93%] Built target bplus_tree_test
Consolidate compiler generated dependencies of target log_test
[ 94%] Built target log_test
Consolidate compiler generated dependencies of target md5_test
[ 95%] Built target md5_test
Consolidate compiler generated dependencies of target mem_pool_test
[ 96%] Built target mem_pool_test
Consolidate compiler generated dependencies of target path_test
[ 97%] Built target path_test
Consolidate compiler generated dependencies of target pidfile_test
[ 98%] Built target pidfile_test
Consolidate compiler generated dependencies of target rc_test
[ 99%] Built target rc_test
Consolidate compiler generated dependencies of target record_manager_test
[100%] Built target record_manager_test
yulinux@yu:/mnt/e/github/miniob/build$ ./bin/observer -s miniob.sock -f observer.ini
Successfully load observer.ini
Not set log file name, use default observer.log
Output configuration
Begin dump configuration
[]

Finish dump configuration

Extend one pool, this->size:128, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:256, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:384, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:512, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:640, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:768, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:896, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:1024, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:1152, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:1280, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:1408, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:1536, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:1664, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:1792, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:1920, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:2048, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:2176, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:2304, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:2432, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:2560, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:2688, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:2816, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:2944, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:3072, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:3200, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:3328, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:3456, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:3584, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:3712, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:3840, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:3968, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:4096, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:4224, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:4352, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:4480, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:4608, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:4736, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:4864, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:4992, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:5120, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:5248, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:5376, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:5504, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:5632, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:5760, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:5888, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:6016, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:6144, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:6272, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:6400, item_num_per_pool:128, this->name:BufPool.
Extend one pool, this->size:6400, item_num_per_pool:128, this->name:BufPool.
(Re)Starting State: Pid: 271 Time: Tue Sep 20 10:24:52 2022

The process Name is observer
Successfully init_event_history, EventHistory:0, MaxEventHops:100
Configuration hasn't set ThreadPools
Failed to init thread pool

SedaConfig: unable to initialize seda stages
Failed to init seda configuration!
Shutdown due to failed to init!
Successfully do cleanup, this->name:BufPool.
Shutdown Cleanly!
yulinux@yu:/mnt/e/github/miniob/build$ ps -A
  PID TTY          TIME CMD
    1 ?        00:00:00 init
    9 tty1     00:00:00 init
   10 tty1     00:00:00 bash
  272 tty1     00:00:00 ps

Code format all code by clang-format

'client_performance_test' failed

What happened:
In the current version, client_performance_test.cpp uses DEFAULT_PORT as socket communication port, which is different from '*.ini' file provided by the community in ./etc directory. It leads to failed to connect error.
(And guidance commands are not suitable for this, cuz it uses *.sock file)

Should we just change the PORT used in client_performance_test.cpp to the port set by .ini file?

无法执行git submodule add

尝试根据 doc 添加 deps/libevent 和 deps/googletest 时均失败，报错"already exists in the index"

使用 "git ls-files deps/libevent" 可以看到相关信息，怀疑是cache 导致

执行如下清理之后可以正常执行git add submodule add

git rm --cached deps/libevent
git rm --cached deps/googletest

rm -rf deps/libevent
rm -rf deps/googletest

一键安装开发环境的脚本

如果不使用docker，希望可以提供一个更小白的，能够可以一键搭建起来开发环境的脚本。

Table::make_record 直接使用memcpy，可能导致内存越界访问

memcpy(record + field->offset(), value.data, field->len());

考虑这行代码，如果value.type 是CHARS，并且value.data 的实际长度比较小，比如2个字符，那么执行memcpy会出现越界访问内存的情况

load data 去掉了一些空白字符

按照load data 的原意，如果csv文件中包含了一些空白字符，那么也应该导入进来，比如:
1| a |1
那么中间的字符串应该是' a '，而不是'a'