Git Product home page Git Product logo

tiflash's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

tiflash's Issues

Consider adding timeout for wait index

In TiDB, users can set max_execution_time to limit on how long a statement is permitted to execute before the server terminates it.
If we happen to deadlocks for raft threads, then the coprocessor queries wait index may block forever.

I think we can add a timeout for wait index to mitigating the impact of infinite wait.

Refine the "stable" PageStorage

We introduced a new PageStorage which support multi-threads write in #464. It was a big change for PageStorage code.
I have supported backward compatibility for reading single page file of the older format, but I was not sure it can completely handle all old format data (including the checkpoint, legacy, formal page file data). To keep the stability of TiFlash at that time, I only use the new PageStorage for DeltaTree, while keeping the old code as "stable::PageStorage" for storing raft data.

Having two version codebase for PageStorage may make some trouble if we support encryption or other new features for PageStorage. We'd better add some test cases to ensure we can unify them and upgrade can performance smoothly.

Tombstone table make drop-table test with fault-injected lose effectiveness

In this pr #599, we tombstone table instead of instantly physical drop data on disk.
It means that in tests/fullstack-test/fault-inject/drop-table.test, failpoint exception (exception_between_drop_meta_and_data and exception_drop_table_during_remove_meta) is not triggered.

Those fail points are not triggered and may make trouble in later tests.
For example:

## Simulation of enable fail point but not triggered.
>> DBGInvoke __init_fail_point()
>> DBGInvoke __enable_fail_point(exception_between_drop_meta_and_data)

## Drop database will actually drop table data physically.
mysql> drop table if exists test_new.t2;
mysql> drop database if exists test_new;
mysql> create database if not exists test_new;
mysql> create table test_new.t2(a int, b int)
mysql> alter table test_new.t2 set tiflash replica 1
func> wait_table test_new t2
mysql> insert into test_new.t2 values (1, 1),(1, 2);
mysql> set session tidb_isolation_read_engines='tiflash'; select * from test_new.t2;
+------+------+
| a    | b    |
+------+------+
|    1 |    1 |
|    1 |    2 |
+------+------+

mysql> drop table if exists test_new.t2;
mysql> drop database if exists test_new;

## Refresh schema will trigger fail point and throw an exception
## It makes TiFlash's schema can not be synced with TiDB
>> DBGInvoke __refresh_schemas()  

[MPP coordinator] implement task generation and schedule.

After coordinator receive an ExecQueryRequest , it should record it and generate MPP computation tasks. In first developing phase, we suppose the resources are enough, and allocate tasks to different tiflash node.

Then schedule stage starts, coordinator sends ExecTaskRequest to corresponding node and track task status.

[MPP planner] embed shuffle hash join into current cbo framework.

Consider 'partition' property for current cbo framework and generate shuffled hash join plan for final physical plan.

Shuffled hash join should get less cost for huge tables but the concurrency is unknow right now, hence the cost model is complex and we should discuss carefully. Maybe we can drive it by hint at first.

Deadlocks between learner read and raft threads

In the case of #803, we found that one region's logging like "Start to remove region xxx", but can not find belonging "Remove region xxx done". About one hour later, all queries accessing to TiFlash stuck. And there is no new raft command log at that period of time.
After restarting TiFlash, that region did not appear in the restored region list. So it is believed that possibly there are deadlocks inside RegionTable::RemoveRegion.
But I can't find the deadlocks by reviewing those codes. Maybe we need to add more logging to find out what happens first.

[ 8 ] <Information> Region: [region 36252027, applied: term 7 index 7] execute admin command ChangePeer at [term: 7, index: 8]
[ 8 ] <Information> Region: [region 36252027] execute change peer type: RemoveNode
[ 8 ] <Information> KVStore: Start to remove [region 36252027]
...

[MPP coordinator] implement coordinator server and client.

When MPP function is enabled, TiDB should start a coordinator leader thread and write its server address to Etcd.
When it crashes, other TiDB should become leader instead.

TiDB and TiFlash should both implement coordinator client, find the server address from etcd and keep heartbeat.

[MPP worker] implement MPP worker service.

MPP worker service is driven by TiFlash, and serves MPP Task Request. After receiving ExecTaskRequest, should establish connection with upstream node. For Table Scan plan, we should retry for region error or resolve lock error. If there is any error, we should throw an exception to coordinator.

Rare seen proxy core dump and unable to recover when restart

We've seen a rare proxy core dump in a previous POC, we had no clue and neither was able to reproduce back that time so we didn't take further actions.

We saw a proxy core dump today in another customer's production env, looks alike the one above.

We'll take whatever it costs to investigate this issue, cause this is unacceptable to have such a core dump in production.

Mishandling of changing pk column name in DeltaTree

First, make sure TiDB is started with alter-primary-key = false.
Then execute following SQL statements through mysql-client.

create table t (a blob(34) null, b mediumint, primary key(b));
alter table t set tiflash replica 1;
alter table t change column b c mediumint;
insert into t values('1', 2);
select * from t;

In TiFlash's log, we can see error that:

2020.06.28 12:03:38.459711 [ 23 ] <Error> DB::TiFlashApplyRes DB::HandleWriteRaftCmd(const DB::TiFlashServer*, DB::WriteCmdsView, DB::RaftCmdHeader): Code: 10, e.displayText() = DB::Exception: Not found column b in block. There are only columns: a, c, _INTERNAL_VERSION, _INTERNAL_DELMARK, e.what() = DB::Exception, Stack trace:

0. /data1/jaysonhuang/nodes/28/tiflash/tiflash(StackTrace::StackTrace()+0x1c) [0x128a8fd4]
1. /data1/jaysonhuang/nodes/28/tiflash/tiflash(DB::Exception::Exception(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int)+0x4b) [0xce292f7]
2. /data1/jaysonhuang/nodes/28/tiflash/tiflash(DB::Block::getPositionByName(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) const+0x10a) [0x111078e2]
3. /data1/jaysonhuang/nodes/28/tiflash/tiflash(DB::DM::DeltaMergeStore::write(DB::Context const&, DB::Settings const&, DB::Block const&)+0x226) [0x11f6ae94]
4. /data1/jaysonhuang/nodes/28/tiflash/tiflash(DB::StorageDeltaMerge::write(DB::Block&&, DB::Settings const&)+0x18d) [0x11ebf565]
5. /data1/jaysonhuang/nodes/28/tiflash/tiflash() [0x121c8032]
6. /data1/jaysonhuang/nodes/28/tiflash/tiflash(DB::writeRegionDataToStorage(DB::Context&, std::shared_ptr<DB::Region> const&, std::vector<std::tuple<long, unsigned char, unsigned long, std::shared_ptr<DB::StringObject<false> const> >, std::allocator<std::tuple<long, unsigned char, unsigned long, std::shared_ptr<DB::StringObject<false> const> > > >&, Poco::Logger*)+0xd5) [0x121c857a]
7. /data1/jaysonhuang/nodes/28/tiflash/tiflash(DB::RegionTable::writeBlockByRegion(DB::Context&, std::shared_ptr<DB::Region> const&, std::vector<std::tuple<long, unsigned char, unsigned long, std::shared_ptr<DB::StringObject<false> const> >, std::allocator<std::tuple<long, unsigned char, unsigned long, std::shared_ptr<DB::StringObject<false> const> > > >&, Poco::Logger*, bool)+0x160) [0x121c8fea]
8. /data1/jaysonhuang/nodes/28/tiflash/tiflash(DB::Region::handleWriteRaftCmd(DB::WriteCmdsView const&, unsigned long, unsigned long, DB::TMTContext&)+0x58e) [0x121d930e]
9. /data1/jaysonhuang/nodes/28/tiflash/tiflash(DB::KVStore::handleWriteRaftCmd(DB::WriteCmdsView const&, unsigned long, unsigned long, unsigned long, DB::TMTContext&)+0x1bf) [0x121baf59]
10. /data1/jaysonhuang/nodes/28/tiflash/tiflash(DB::HandleWriteRaftCmd(DB::TiFlashServer const*, DB::WriteCmdsView, DB::RaftCmdHeader)+0x4e) [0x121d2e25]
11. /data1/jaysonhuang/nodes/28/tiflash/libtiflash_proxy.so(+0x11c9543) [0x7f600b76a543]
12. /data1/jaysonhuang/nodes/28/tiflash/libtiflash_proxy.so(+0x11b58d2) [0x7f600b7568d2]
13. /data1/jaysonhuang/nodes/28/tiflash/libtiflash_proxy.so(+0x11b8d24) [0x7f600b759d24]
14. /data1/jaysonhuang/nodes/28/tiflash/libtiflash_proxy.so(+0x350c86) [0x7f600a8f1c86]
15. /data1/jaysonhuang/nodes/28/tiflash/libtiflash_proxy.so(+0x38ba55) [0x7f600a92ca55]
16. /data1/jaysonhuang/nodes/28/tiflash/libtiflash_proxy.so(+0xc714bd) [0x7f600b2124bd]
17. /data1/jaysonhuang/nodes/28/tiflash/libtiflash_proxy.so(+0xc732c7) [0x7f600b2142c7]
18. /usr/lib64/libpthread.so.0(+0x7e64) [0x7f6009428e64]
19. /usr/lib64/libc.so.6(clone+0x6c) [0x7f60089df88c]

And the metadata of that table is

ATTACH TABLE t_58
(
    a Nullable(String),
    c Int32
)
ENGINE = DeltaMerge(b, ... )  <- the pk name(s) in create statement is not changed.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.