Git Product home page Git Product logo

kratos's People

Contributors

180909 avatar casper-mars avatar ccheers avatar cluas avatar czyt avatar daemon365 avatar demomanito avatar dependabot[bot] avatar devxiaolan avatar faycheng avatar felixhao avatar kagaya85 avatar kvii avatar letian0805 avatar libi avatar lintanghui avatar longxboy avatar luckylsx avatar miyabyte avatar realityone avatar shenqidebaozi avatar songzhibin97 avatar terry-mao avatar tomwei7 avatar tonybase avatar windfarer avatar wuxingzhong avatar xyuwang avatar ymh199478 avatar zhaoshichen1 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

kratos's Issues

errors: Go 1.13 error chain 优化

https://godoc.org/golang.org/x/exp/errors

Go 1.13 会有大改进,期待跟进

从 pkg/errors 到 exp/errors

目前 go-common 主要试用 pkg/errors 来封装错误链(err's chain) 典型的用法如下

var ErrNotFound = errors.New("not found")

err := errors.Wrap(ErrNotFound, "user xxx")

fmt.Println(err.Error())
// user xxx: not found
fmt.Println(errors.Cause(err) == ErrNotFound)
// true

通过 Wrap 封装错误然后通过 Cause 取回原始的 err, 这样可以做到判断 err 的类型同事丰富错误信息。

同样的 exp/errors 也提供了同样的处理方法。

var ErrNotFound = errors.New("not found")

// NOTE: 必须要 : %w 接我才会被 wrapper
err := fmt.Errorf("user xxx: %w", ErrNotFound)

fmt.Println(err.Error())
// user xxx: not found
fmt.Println(errors.Is(err, ErrNotFound))
// true

exp/errors 其他功能

Formatter interface 更加友好的错误显示 https://godoc.org/golang.org/x/exp/errors#Formatter

kratos testing-infra 引入

考虑到长期建设开源kratos,尽快把基于k8s prow 的 CICD体系

  • OWNERS 目录级权限

  • Compile & Lint & Security Job

  • LGTM & Approve 组件

log: improvement

目前 log 存在的问题

  • format 方式不统一 agent 和 stdout, file 不一样

  • 缺少hook功能 好多人说想自己给 http 和 grpc log 加字段

  • 重复代码的问题,有些地方写了三份
    bug 类

  • agent 配置无效

  • vmodule 配置无效

log 改造

将 log 拆分为三块

  • logger 日志逻辑包括level 字段的处理 filter 之类的

  • writer 纯粹的把序列化好的 一条日志 []byte 写入某个地方

  • formater 序列化日志,json text 之类的

demo里面不能生成API文件夹

mod 开启了 是1.12 版本

build kratos-demoha/cmd: cannot load kratos-demoha/api: cannot find module providing package kratos-demoha/api

pkg/cache/memcache: refactor interface

代码结构调整

sturct,interface 迁移,目前 Memcache structclient.goConn interfacememcache.go 中,分布比较混乱。所以首先对代码结构进行调整

memcache.go // 公共结构,interface 定义

Memcache struct
Dial()
Item struct
Conn interface

// conn_xxxx.go 代表某一特定 Conn 实现
conn_ascii.go // 当前仓库的 Conn 实现及 conn struct 迁移到 conn_ascii.go
conn_trace.go // trace conn
conn_pool.go // 带有连接池管理的 conn

接口调整

目前 Conn 的 method 都没有传递 Context 不利于做超时控制,所以这里对 Conn interface 进行调整,调整不影响兼容性。对所有方法添加带 Context 的方法。

例如

Get(key string) (*Item, error)
// 添加 GetContext
GetContext(ctx context.Context, key string) (*Item, error)

在实际调用的过程中,Memcache 库会自动挑选最合适的 Method 进行调用,兼容修的 Conn 实现

kratos tool: 更友好的方式创建项目和运行

kratos tool

kratos new hello_world [-o owner] [-d directory] [-grpc]
kratos build
kratos run
  • 修改为更方便的方式创建项目;
  • 添加build、run指令可以在当前项目任意目录执行;

创建项目

$ kratos new kratos-demo -o test

Project: kratos-demo
Owner: test
WithGRPC: false
Directory: /data/github/kratos-demo

The application has been created.

编译项目

$ cd kratos-demo
$ kratos build

directory: /data/github/kratos-demo/cmd
kratos: v0.0.1
build success.

运行项目

$ cd kratos-demo
$ kratos run

[2019/04/27 14:31:16.445] [proc.go:200] kratos-demo start
2019/04/27 14:31:16 start watch filepath: /data/github/kratos-demo/configs
[2019/04/27 14:31:16.446] [http.go:33] blademaster: start http listen addr: 0.0.0.0:8000

render html

请问,如何支持渲染 HTML 呢?如gin中的ctx.HTML()

undefined: api.HelloReq

While installing and selecting the gPRC YES, I get this error:

# kratos-demo/internal/service
../internal/service/service.go:34:54: undefined: api.HelloReq

using this command:

cd kratos-demo/cmd <- this works
go build <- this is what throws the error
./cmd -conf ../configs <- not sure yet

If I select NO I have no issues and everything works... if I select the first build option, there are no issues.

I did have to convert Chinese into English, so some things may have translated incorrectly, but I think the terminal commands are pretty straight forward.

quick start并不quick, 需要自己go get 很多依赖包

go get -u github.com/golang/protobuf/proto
go get github.com/pkg/errors
go get github.com/prometheus/client_golang/prometheus/promhttp
go get gopkg.in/go-playground/validator.v9
go get github.com/fsnotify/fsnotify
go get github.com/BurntSushi/toml
go get github.com/dgryski/go-farm

go get -u google.golang.org/grpc/encoding
go get -u github.com/golang.org/x/net

stat/prom: prometheus监控模版

Background

现阶段下使用的prometheus监控,是由基础库统一定义的方式,比如:go_lib_client、go_rpc_client、go_http_client、go_http_server和go_business_info_count等

Suggestion

基础库创建prometheus针对每个组件创建独立的监控模版,比如:rpc模版、http模版、memcache模版、redis模版

Expected behaviour

监控的项目dashboard作为统一入口只展示
服务的关键图表、指标等常用数据,具体组件的监控可以跳转到组件的详细监控中,展示更多的详细的指标和图。比如:rpc监控、http监控、memcache监控和redis监控,以及缓存中的连接池监控。

接下来我们会推进Kratos 监控的模板化,把Kratos的所有组件,更加精细的覆盖普罗米修斯监控,之前为了强行整合到Dashboard,是使用的统一的Label.

现在做法是,把组件当做监控的一个模板,然后在额外制作Dashboard,在大盘里只包含每个模板里组件的部分关键指标,这样在每个组件还可以HyperLink到细节的组件监控大盘,感觉Root Cause就吊的飞起了。

support tracing: 支持接口级拦截,以及Context

老的cache/redis库,都是在协议代码级别拦截的tracing,而不是接口级别拦截,为了后续代码更优雅,可维护

考虑扩展cache 下的redis接口,增加WithContext,然后提供一个traceConn的实现类,在底层拦截实现tracing

同理在database/sql库在sql.Driver层面拦截。

  • library/cache/redis

  • library/database/sql

  • library/queue/databus

windows err

cd cmd / go build main.go ..\internal\service\service.go:34:54: undefined: api.HelloReq

cd ../api
go generate

运行 kprotoc 出错: exec: "bash": executable file not found in %PATH%
运行 kprotoc 出错: exec: "d:\www\bin\kprotoc": file does not exist

打开D:\www\src\github.com\bilibili\kratos\tool\kprotoc
./install_kprotoc.shln: failed to create symbolic link 'd/bin/kprotoc': No such file or directory

你们的,kprotoc.sh好像只是写的macos吧....windows 错误的

Redis 更加友好的 Pipeline API

背景

目前 go-common 的 Redis Pipeline 不是特别友好,使用 Redis Pipeline 需要自己计数

例如:

var n = 2
if mid > 0 {
	if err = conn.Send("HSET", keyMidServer(mid), key, server); err != nil {
		return
	}
	if err = conn.Send("EXPIRE", keyMidServer(mid), _redisExpire); err != nil {
		return
	}
	n += 2
}
if err = conn.Send("SET", keyKeyServer(key), server); err != nil {
	return
}
if err = conn.Send("EXPIRE", keyKeyServer(key), _redisExpire); err != nil {
	return
}
if err = conn.Flush(); err != nil {
	return
}
for i := 0; i < n; i++ {
	if _, err = conn.Receive(); err != nil {
		return
	}
}

参考 SQL 库 Rows 使用 Next 迭代

conn.Send("GET", "test1")
conn.Send("GET", "test2")
conn.Send("GET", "test3")
// 第一次 Next 可以自动 Flush,不需要开发手动 Flush
for conn.Next() {
    result, err := conn.Receive()
    // do something ...
}

2、对开发隐藏 Pool 的存在

背景

目前开发使用 Redis 时,需要手动从 Redis Pool 中取出连接进行操作,操作完成之后关闭连接既将连接放回 Pool

例如:

func (d *Dao) PopRedis(c context.Context, key string) (bs []byte, err error) {
	var conn = d.redis.Get(c)
	defer conn.Close()

	if bs, err = redis.Bytes(conn.Do("LPOP", key)); err != nil && err != redis.ErrNil {
		log.Error("redis.Bytes(conn.Do(LPOP, %s)) error(%v)", key, err)
	}
	return
}

这种方式有三个较为明显的问题,

  • 需要多写几行代码

  • 一旦忘记关闭将导致连接泄漏

  • 当操作中混有耗时操作时,会出现一个 Goroutine 长期占有连接不使用的问题,不利于提高连接的利用率

改进

将 Pool 放到内部对开发隐藏,执行命令时从 Pool 取出连接返回时放回 Pool

示例代码

// conn 为全局单例,可以安全的在 Goroutinue 间共享
conn := redis.New(cfg)

// 某个业务方法
func XXFoo(ctx context.Context) (xxx, error){
    reply, err := conn.Do(ctx, "GET", "xxxx")
    // 摸鱼...摸鱼...
    return xxx, err
}

为了适配这种 API 对上面的 Pipeline 对上面方案 2 的 Pipeline 会稍作调整

实例代码

conn := redis.New(cfg)

pipeline := redis.Pipeline(ctx)
pipeline.Send("GET", "test1")
pipeline.Send("GET", "test2")
pipeline.Send("GET", "test3")

// 当 pipeline 读取到最后一个 response 时,会自动回收连接到 Pool,开发无序手动操作
for pipeline.Next() {
   reply, err = pipeline.Receive()
}

好处:

  • 简化了代码

  • 提高了连接利用率

总览

type Pipeliner interface {
    Send(command string, args ...interface{})
    Next() bool
    Receive() (reply interface{}, err error)
}

type Conn interface {
   ... // 参考已有 Conn Interface
}

type Redis interface {
    Do(ctx context.Context) (reply interface{}, err error)
    Pipeline(ctx context.Context) Pipeliner
    Close() error
    Conn() (Conn) // 从当前 Pool 中获取一个原始的 Conn,并一直持有直到开发手动关闭 Conn
}

编译cmd的时候报错“..\internal\service\service.go:34:54: undefined: api.HelloReq”

说明:进入cmd目录运行go build的时候,报了错误“..\internal\service\service.go:34:54: undefined: api.HelloReq”,对比官方给出的kratos-demo,发现api目录下面的文件api.pb.go并没有

重现步骤:

  1. go get -u github.com/bilibili/kratos/tool/kratos
  2. kratos init,按照命令要求,依次输入项目名称,负责人,并选择有gRPC的,当前目录生成
  3. 进入cmd目录执行go build

环境:

  1. Win7+64位
  2. Go1.12.4

database-mysql.md文档什么时候会完善

使用kratos-demo与mysql连接,发现无论怎么更改mysql.toml中的内容,都没有效果,但是sql.NewMySQL(dc.Demo)也没有出现panic,所以想问问database-mysql.md文档什么时候会完善?

net/metadata: 提供Range方法

现在枚举metadata的方式:

if cmd, ok := metadata.FromContext(c); ok {
    for key, value := range cmd {
        if !metadata.IsOutgoingKey(key) {
        }
    }

比较麻烦,提供一个简易的方式:

func Range(rn func(key string, value interface{}), ...fn func (key string) bool)

metadata.Range(func(key string, value interface{}) {
   fmt.Println("%s: %v", key, value)
}, metadata.IsOutgoingkey)

pkg/cache/memcache: 性能优化

  • 大量使用了fmt.Fprintf ,可以使用底层bufio直接WriteString,避免多次反射

  • Multi等操作,是前置O(n)循环key来判断是否key合法,可以lazy处理,在bufio中写时判断

  • 解析Response中,使用了stirngs.Split会导致多次内存分配

  • 写入操作中,多创建了一个Item,可以走item值传递copy

go build 失败

go get -u github.com/bilibili/kratos/tool/kratos
cd $GOPATH/src
kratos new kratos-demo
cd $GOPATH/src/kratos-demo/cmd
go build

失败

../../github.com/bilibili/kratos/pkg/conf/paladin/toml.go:8:2: cannot find package "github.com/BurntSushi/toml" in any of:
	/usr/local/go/src/github.com/BurntSushi/toml (from $GOROOT)
	/root/go/src/github.com/BurntSushi/toml (from $GOPATH)
../../github.com/bilibili/kratos/pkg/conf/paladin/file.go:14:2: cannot find package "github.com/fsnotify/fsnotify" in any of:
	/usr/local/go/src/github.com/fsnotify/fsnotify (from $GOROOT)
	/root/go/src/github.com/fsnotify/fsnotify (from $GOPATH)
../../github.com/bilibili/kratos/pkg/database/sql/mysql.go:10:2: cannot find package "github.com/go-sql-driver/mysql" in any of:
	/usr/local/go/src/github.com/go-sql-driver/mysql (from $GOROOT)
	/root/go/src/github.com/go-sql-driver/mysql (from $GOPATH)
../../github.com/bilibili/kratos/pkg/net/http/blademaster/render/protobuf.go:6:2: cannot find package "github.com/gogo/protobuf/proto" in any of:
	/usr/local/go/src/github.com/gogo/protobuf/proto (from $GOROOT)
	/root/go/src/github.com/gogo/protobuf/proto (from $GOPATH)
../../github.com/bilibili/kratos/pkg/net/http/blademaster/render/render.pb.go:18:8: cannot find package "github.com/gogo/protobuf/types" in any of:
	/usr/local/go/src/github.com/gogo/protobuf/types (from $GOROOT)
	/root/go/src/github.com/gogo/protobuf/types (from $GOPATH)
../../github.com/bilibili/kratos/pkg/net/trace/proto/span.pb.go:6:8: cannot find package "github.com/golang/protobuf/proto" in any of:
	/usr/local/go/src/github.com/golang/protobuf/proto (from $GOROOT)
	/root/go/src/github.com/golang/protobuf/proto (from $GOPATH)
../../github.com/bilibili/kratos/pkg/ecode/status.go:9:2: cannot find package "github.com/golang/protobuf/ptypes" in any of:
	/usr/local/go/src/github.com/golang/protobuf/ptypes (from $GOROOT)
	/root/go/src/github.com/golang/protobuf/ptypes (from $GOPATH)
../../github.com/bilibili/kratos/pkg/ecode/types/status.pb.go:9:8: cannot find package "github.com/golang/protobuf/ptypes/any" in any of:
	/usr/local/go/src/github.com/golang/protobuf/ptypes/any (from $GOROOT)
	/root/go/src/github.com/golang/protobuf/ptypes/any (from $GOPATH)
../../github.com/bilibili/kratos/pkg/net/trace/proto/span.pb.go:9:8: cannot find package "github.com/golang/protobuf/ptypes/duration" in any of:
	/usr/local/go/src/github.com/golang/protobuf/ptypes/duration (from $GOROOT)
	/root/go/src/github.com/golang/protobuf/ptypes/duration (from $GOPATH)
../../github.com/bilibili/kratos/pkg/net/trace/proto/span.pb.go:10:8: cannot find package "github.com/golang/protobuf/ptypes/timestamp" in any of:
	/usr/local/go/src/github.com/golang/protobuf/ptypes/timestamp (from $GOROOT)
	/root/go/src/github.com/golang/protobuf/ptypes/timestamp (from $GOPATH)
../../github.com/bilibili/kratos/pkg/net/trace/config.go:9:2: cannot find package "github.com/pkg/errors" in any of:
	/usr/local/go/src/github.com/pkg/errors (from $GOROOT)
	/root/go/src/github.com/pkg/errors (from $GOPATH)
../../github.com/bilibili/kratos/pkg/stat/prom/prometheus.go:8:2: cannot find package "github.com/prometheus/client_golang/prometheus" in any of:
	/usr/local/go/src/github.com/prometheus/client_golang/prometheus (from $GOROOT)
	/root/go/src/github.com/prometheus/client_golang/prometheus (from $GOPATH)
../../github.com/bilibili/kratos/pkg/net/http/blademaster/prometheus.go:4:2: cannot find package "github.com/prometheus/client_golang/prometheus/promhttp" in any of:
	/usr/local/go/src/github.com/prometheus/client_golang/prometheus/promhttp (from $GOROOT)
	/root/go/src/github.com/prometheus/client_golang/prometheus/promhttp (from $GOPATH)
../../github.com/bilibili/kratos/pkg/net/trace/propagation.go:7:2: cannot find package "google.golang.org/grpc/metadata" in any of:
	/usr/local/go/src/google.golang.org/grpc/metadata (from $GOROOT)
	/root/go/src/google.golang.org/grpc/metadata (from $GOPATH)
../../github.com/bilibili/kratos/pkg/conf/dsn/dsn.go:8:2: cannot find package "gopkg.in/go-playground/validator.v9" in any of:
	/usr/local/go/src/gopkg.in/go-playground/validator.v9 (from $GOROOT)
	/root/go/src/gopkg.in/go-playground/validator.v9 (from $GOPATH)

go version 为1.12.4

API 设计: Options、Config 讨论

目前Kratos中大量都是依赖Config来初始化的,这是为了适配配置文件解析和加载,但是Kratos可能作为vendor第三方库给其他人使用,那么就需要他们自己的初始化解析配置的方式,因此API最好是提供两种,一种是全套使用kratos tool 工具链 利用toml或者将来yaml方式来启动,一种是把kratos当做第三方库,自己初始化组件,那么使用options会比较方便

net/http/blademaster 以 metadata 形式代替 Context.Keys 来传递上下文数据

Background

bm 中原来通过 Keys 字段在多个 handler 之间传递额外的数据,比如通过 token拿到的 userid。但是这种方式要求 http 层必须编写额外的代码来保证从 bm 的 Context 中获取了正确的 userid。

考虑到和 gRPC 的 metadata 模式与代码生成的一致性,考虑将 Keys 字段改成使用 valueContext 存储 metadata 来进行传递。

Suggestion

  1. 去除 bm 中 Context.Keys 改为使用类似 gRPC 中的 metadata 形式,metadata 存储与 valueContext 中,这样可以使 gRPC 的 service 逻辑与 bm 的 service 逻辑统一,也为之后网关全面迁移 gRPC 打下基础。

  2. metadata 的跨服务传递性?

Expected behaviour

bm Context 中的 Keys 字段被去除,并通过类似 gRPC 的 metadata 形式代替。

kratos tool: protoc生成grpc或者bm

google protoc:
google protobuf编译工具,可以指定生成工具 protoc-gen-{name}: --{name}_out=.

grpc:

go get github.com/gogo/protobuf/protoc-gen-{gogofast}

protoc --proto_path=$GOPATH/src --proto_path=/data/github/demo/api --{gogofast}_out=plugins=grpc:. api.proto

bm

go get github.com/bilibili/kratos/tool/protobuf/protoc-gen-{bm}
protoc --proto_path=$GOPATH/src --proto_path=/data/github/demo/api --{bm}_out=explicit_http=true:. api.proto

swagger

go get github.com/bilibili/kratos/tool/protobuf/protoc-gen-{bswagger}
protoc --proto_path=$GOPATH/src --proto_path=/data/github/demo/api --{bswagger}_out=explicit_http=true:. api.proto

kratos protoc

go get github.com/bilibili/kratos/tool/kratos-protoc
kratos took protoc api.proto  // generate all
kratos tool protoc --grpc api.proto // generate grpc

protobuf generator

kratos/tool/protobuf
-protoc-gen-bm
-protoc-gen-bswagger
-pkg/extensions

net/http/blademaster & net/rpc/warden(warden) client 支持透明降级(fallback)

背景

部分服务在后端 service 500 后需要做降级处理,目前降级代码大部分嵌入下业务代码中,编写比较繁琐。http 和 gRPC 的 client 可以提供透明降级功能,并提供更多的降级选项

功能(TBD)

  • 根据指定错误码进行降级

  • 降级粒度到单个 API

  • 多级降级支持

降级选项(TBD)

  • callbacb 在 client 收到指定错误码时调用预先设定的 callback 函数生成 response 内容

  • cache 功能,client 能够 cache 请求,收到指定错误码时返回上次成功的结果

功能实现(TBD)

gRPC

方案一: 代码生成

生成新的 fallback client wrapper 已有的 client interface 缺点: 需要代码生成

方案二: interceptor

缺点:需要业务方知道每个 RPC method 的名称

http

http 实现比较简单直接拦截

net/trace: trace SetTag 小对象问题

目前 trace SetTag 会分配一些小对象,导致 Golang GC 压力,所以需要 trace 暴露采样信息,让 SDK 能够完整的跳过 SetTag 的调用,解决分配过多小对象问题

无法找到相应想依赖包

go build main.go
main.go:11:2: cannot find package "kratos-demo/internal/server/grpc" in any of:
/usr/local/Cellar/go/1.12.4/libexec/src/kratos-demo/internal/server/grpc (from $GOROOT)
/Users/sunzhenya/Work/go/src/kratos-demo/internal/server/grpc (from $GOPATH)
main.go:12:2: cannot find package "kratos-demo/internal/server/http" in any of:
/usr/local/Cellar/go/1.12.4/libexec/src/kratos-demo/internal/server/http (from $GOROOT)
/Users/sunzhenya/Work/go/src/kratos-demo/internal/server/http (from $GOPATH)
main.go:13:2: cannot find package "kratos-demo/internal/service" in any of:
/usr/local/Cellar/go/1.12.4/libexec/src/kratos-demo/internal/service (from $GOROOT)
/Users/sunzhenya/Work/go/src/kratos-demo/internal/service (from $GOPATH)

net/rpc/warden: 多机房自动调度

现状

多机房部署的服务,只有在本机房服务全部挂掉的情况下才会调度到其他机房,可能导致本机房部分节点不可用的情况下,剩余健康节点也被打挂

解决方案

1.多机房节点根据负载自动调度,当本机房由于部分节点或其他原因导致负载飙升的时候,自动调度流量到其他机房空闲节点。
2.根据节点上报的负载指标进行自动调度。

net/rpc/warden、net/http/blademaster 支持重要性传递

  1. 不同服务间通过 net/metadata 来传递

  2. 原始重要性设置于 bm 的路由上,默认以 CRITICAL 形式存在,通过 Handler 的形式将优先级作用于每个请求中。

  3. 重要性分以下几级(重要等级递减):

    • CRITICAL_PLUS

    • CRITICAL

    • SHEDDABLE_PLUS

    • SHEDDABLE

  4. http service 之间通过 header 形式进行传递,rpc service 之间通过 rpc metadata 形式进行传递。

  5. 在 bm 处理请求时,请求中的重要性永远优先于预设于路由上的重要性。

warden.NewClient 初始化后 please config a default HOST错误

~/go/src/kratos-http-service/cmd$ ./cmd -conf ../configs/
[2019/04/30 09:53:08.474] [proc.go:200] kratos-http-service start
[2019/04/30 09:53:08.475] [discovery.go:164] disocvery: AddWatch(infra.discovery) already watch(false)
2019/04/30 09:53:08 start watch filepath: ../configs/
[2019/04/30 09:53:08.475] [discovery.go:536] discovery: client.Get(http://NOTE: please config a default HOST/discovery/polls?appid=infra.discovery&env=dev&hostname=Noah.local&latest_timestamp=0) error(parse http://NOTE: please config a default HOST/discovery/polls?appid=infra.discovery&env=dev&hostname=Noah.local&latest_timestamp=0: invalid character " " in host name
method:GET,uri:http://NOTE: please config a default HOST/discovery/polls
github.com/bilibili/kratos/pkg/net/http/blademaster.(*Client).NewRequest
/Users/noahwang/go/pkg/mod/github.com/bilibili/[email protected]/pkg/net/http/blademaster/client.go:145
github.com/bilibili/kratos/pkg/net/http/blademaster.(*Client).Get
/Users/noahwang/go/pkg/mod/github.com/bilibili/[email protected]/pkg/net/http/blademaster/client.go:165
github.com/bilibili/kratos/pkg/naming/discovery.(*Discovery).polls
/Users/noahwang/go/pkg/mod/github.com/bilibili/[email protected]/pkg/naming/discovery/discovery.go:614
github.com/bilibili/kratos/pkg/naming/discovery.(*Discovery).serverproc
/Users/noahwang/go/pkg/mod/github.com/bilibili/[email protected]/pkg/naming/discovery/discovery.go:536
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:1337)

rate/limit: implement bbr-like limiter

一、采样指标

a, cpu,最近 1s 的 CPU 均值,采样周期 250ms

cpu 使用一个独立的 goroutine 采样,每隔 250ms 触发一次(和 sys/stat 中的间隔保持一致)。在计算均值时,cpu 仅仅使用了简单滑动平均。

b, inflight,当前处理中的请求数量

c, pass,最近5s,每100ms内请求成功的数量

d, rt,最近5s,每100ms内请求的响应时间

e, prevDrop,最近一次丢弃请求的时间

二、限流算法
判断是否丢弃当前请求的算法如下:

cpu > 800 AND (Now - PrevDrop) < 1s AND (MaxPass * MinRt * windows / 1000) < InFlight

MaxPass 表示最近 5s 内,单个采样窗口中最大的请求数。

MinRt 表示最近 5s 内,单个采样窗口中最小的响应时间。

windows 表示一秒内采样窗口的数量,默认配置中是 5s 50 个采样,那么 windows 的值为 10。

三、其他事项

a,  保留了冷却时间,来判断当前是否超载

原因:

限流效果生效后,cpu 会在临界值(800)附近抖动,如果不使用冷却时间,那么一个短时间的 cpu 下降就可能导致大量请求被放行,严重时会打满 cpu

使用滑动平均无法解决 cpu 在高负载时下滑问题,即使使用平均,也会出现 cpu 下滑,cpu 打满的问题
image
b, 最小 rt 使用的是采样窗口的平均 rt(sentinal 使用的是最小点的 rt)

压测发现使用最小点的 rt 可能会造成限流过度,即窗口没有及时放大,导致较多流量被拒绝

c, 限流对部分场景无效

请求上升后,cpu 变化不大

非请求上升造成的 cpu 上升

prometheus监控模版化

接下来我们会推进Kratos 监控的模板化,把Kratos的所有组件,更加精细的覆盖普罗米修斯监控,之前为了强行整合到Dashboard,是使用的统一的Label。

现在做法是,把组件当做监控的一个模板,然后在额外制作Dashboard,在大盘里只包含每个模板里组件的部分关键指标,这样在每个组件还可以HyperLink到细节的组件监控大盘,感觉Root Cause就吊的飞起了。

cannot use p2cPicker literal (type *p2cPicker) as type balancer.Picker

kratos init 时选择使用grpc, 然后 go build 出错

go build
# github.com/bilibili/kratos/pkg/net/rpc/warden/internal/encoding/json
../../github.com/bilibili/kratos/pkg/net/rpc/warden/internal/encoding/json/json.go:14:2: undefined: encoding.RegisterCodec
# github.com/bilibili/kratos/pkg/net/rpc/warden/balancer/p2c
../../github.com/bilibili/kratos/pkg/net/rpc/warden/balancer/p2c/p2c.go:35:5: cannot use p2cPicker literal (type *p2cPicker) as type balancer.Picker in assignment:
	*p2cPicker does not implement balancer.Picker (wrong type for Pick method)
		have Pick("context".Context, balancer.PickOptions) (balancer.SubConn, func(balancer.DoneInfo), error)
		want Pick("golang.org/x/net/context".Context, balancer.PickOptions) (balancer.SubConn, func(balancer.DoneInfo), error)
../../github.com/bilibili/kratos/pkg/net/rpc/warden/balancer/p2c/p2c.go:130:2: cannot use p (type *p2cPicker) as type balancer.Picker in return argument:
	*p2cPicker does not implement balancer.Picker (wrong type for Pick method)
		have Pick("context".Context, balancer.PickOptions) (balancer.SubConn, func(balancer.DoneInfo), error)
		want Pick("golang.org/x/net/context".Context, balancer.PickOptions) (balancer.SubConn, func(balancer.DoneInfo), error)
../../github.com/bilibili/kratos/pkg/net/rpc/warden/balancer/p2c/p2c.go:232:16: di.Trailer undefined (type balancer.DoneInfo has no field or method Trailer)
# kratos-test/internal/service
../internal/service/service.go:34:54: undefined: api.HelloReq

env: go version go1.11 darwin/amd64

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.