koordinator-sh / yarn-copilot Goto Github PK

What is your proposal:
case1:
slo-controller计算出batch total为12c，k8s requested=6c，yarn requested=4c
此时
batch unallocated=2c=12-6-4
需要让k8s和yarn的调度器知道还有2c的batch unallocated
k8s allocatable=2+6=8c=12-4
yarn allocatable=2+4=6c=12-6

case2:
slo-controller计算出batch total为9c，k8s requested=6c，yarn requested=4c
此时
batch unallocated=-1c=9-6-4
需要让k8s和yarn的调度器知道已经没有batch unallocated
k8s allocatable=-1+6=5c=9-4，< k8s requested
yarn allocatable=-1+4=3c=12-6，< yarn requested
这样yarn和k8s都不会再往上调度，符合调度模型

Why is this needed:

Is there a suggested solution, if so, please add it:

实现上，slo-controller将batch total放在annotation上，yarn operator计算batch_total-k8s_requested发送给rm，slo-controller计算batch_total-yarn_requested更新在node.status上

What is your proposal:

Why is this needed:

Is there a suggested solution, if so, please add it:

What is your proposal:

Why is this needed:

Is there a suggested solution, if so, please add it:

What is your proposal:

Why is this needed:

Is there a suggested solution, if so, please add it:

What happened:
yarn client occur errors
I1012 09:34:33.366808 1 client.go:196] error: dial tcp 10.219.135.80:8032: connect: connection refused
W1012 09:34:33.366831 1 client.go:151] Couldn't setup connection: dial tcp 10.219.135.80:8032: connect: connection refused
E1012 09:34:33.366840 1 Nodes.go:56] dial tcp 10.219.135.80:8032: connect: connection refused
I1012 09:34:34.367349 1 client.go:196] error: dial tcp 10.219.135.80:8032: connect: connection refused
W1012 09:34:34.367368 1 client.go:151] Couldn't setup connection: dial tcp 10.219.135.80:8032: connect: connection refused
E1012 09:34:34.367376 1 Nodes.go:56] dial tcp 10.219.135.80:8032: connect: connection refused
I1012 09:34:35.369831 1 client.go:196] error: dial tcp 10.219.135.80:8032: connect: connection refused
W1012 09:34:35.369848 1 client.go:151] Couldn't setup connection: dial tcp 10.219.135.80:8032: connect: connection refused
E1012 09:34:35.369854 1 Nodes.go:56] dial tcp 10.219.135.80:8032: connect: connection refused
What you expected to happen:
yarn client try to access other yarn node

What is your proposal:

Support deploying YARN in HA mode.

Why is this needed:

Is there a suggested solution, if so, please add it:

Fetch yarn RMs via some discovery methods (e.g., kubectl) in bootstrap.sh.

What is your proposal:
Now the reconciler only handles k8s.Node update event, yarn requested info may not be updated in time.

Why is this needed:
the yarn requested info on node maybe outdated if k8s.Node not update.

Is there a suggested solution, if so, please add it:
using an independent controller to handle yarn requested update from yarnNodeSyncer.

What is your proposal:
add EventHandler for yarn-operator during watch node event, ignore unnecessary node update event.

Why is this needed:
limit unnecessary requests to yarn RM.

Is there a suggested solution, if so, please add it: