trpc-group / trpc-go Goto Github PK

default log formt is like this:

2023-12-13 11:02:02.381	DEBUG	demo/test.go:47	start test...

then, how to add cutom fields into the formatter? for example, add trace_id

2023-12-13 11:02:02.381	[a5d7e7d8-dec7-437c-8a48-6f31891fe028]	DEBUG	demo/test.go:47	start test...

It seems that the zapcore.Encoder function cannot be overridden

func newEncoder(c *OutputConfig) zapcore.Encoder { 
	encoderCfg := zapcore.EncoderConfig{
		TimeKey:        GetLogEncoderKey("T", c.FormatConfig.TimeKey),
		LevelKey:       GetLogEncoderKey("L", c.FormatConfig.LevelKey),
		NameKey:        GetLogEncoderKey("N", c.FormatConfig.NameKey),
		CallerKey:      GetLogEncoderKey("C", c.FormatConfig.CallerKey),
		FunctionKey:    GetLogEncoderKey(zapcore.OmitKey, c.FormatConfig.FunctionKey),
		MessageKey:     GetLogEncoderKey("M", c.FormatConfig.MessageKey),
		StacktraceKey:  GetLogEncoderKey("S", c.FormatConfig.StacktraceKey),
		LineEnding:     zapcore.DefaultLineEnding,
		EncodeLevel:    zapcore.CapitalLevelEncoder,
		EncodeTime:     NewTimeEncoder(c.FormatConfig.TimeFmt),
		EncodeDuration: zapcore.StringDurationEncoder,
		EncodeCaller:   zapcore.ShortCallerEncoder,
	}
	if c.EnableColor {
		encoderCfg.EncodeLevel = zapcore.CapitalColorLevelEncoder
	}
	switch c.Formatter {
	case "console":
		return zapcore.NewConsoleEncoder(encoderCfg)
	case "json":
		return zapcore.NewJSONEncoder(encoderCfg)
	default:
		return zapcore.NewConsoleEncoder(encoderCfg)
	}
}

I would like to know if there are any key functional differences between the open-source trpc-go and internal network frameworks. For the parts that are not open-sourced, can these tools be replaced with other similar tools?

Currently, we have README_CN.md, README_zh_CN.md, README_zh_cn.md, and even README_ZH.md (which I observed in some trpc-eco repositories).

We need to reach a final decision and establish a rule to guide all repositories, not only those related to trpc-go, but possibly all trpc related repositories. Therefore, a voting mechanism might be involved to make this decision.

Additionally, are there two versions coexisting?

may be this link is right ?
https://github.com/trpc-ecosystem/go-filter/blob/main/slime/README_zh_CN.md

is README_zh_CN, not README.zh_CN

When I tried to save the project code push to my personal repository, git lfs failed to upload both files

Question

This./ Trpc_go.yaml refers to yaml that is already in the same level project as main？

It looks like there are no matches for trpc-go.

Preliminary Research

• I have read the project's documentation.
• I have searched existing issues for similar questions to see if my question has already been addressed.

Question

What's the reason why client side show error code :141, it show the connection is closed, and how to solve the problem?

When client request server, but the server will spent more than 1 minute to handle some data, the client side will occur error, and the log shows type:framework, code:141, msg:tcp client transport ReadFrame, cost:49.999809717s, caused by conn is closed when using tnet, and after change the transport method from tnet to go-net, the error code change to 171, and error message show : type:framework, code:171, msg:tcp client transport ReadFrame: EOF

Additional Information

I will show some code to reappear the question:

client side：

func main() {
	c := pb.NewGreeterClientProxy(client.WithTarget("ip://127.0.0.1:8000"))
	rsp, err := c.Hello(trpc.BackgroundContext(), &pb.HelloRequest{Msg: "world"})
	if err != nil {
		log.Error(err)
		return
	}
	log.Info(rsp.Msg)
}

server side:

func main() {
	s := trpc.NewServer()
	pb.RegisterGreeterService(s, &Greeter{})
	if err := s.Serve(); err != nil {
		log.Error(err)
	}
}

type Greeter struct{}

func (g Greeter) Hello(ctx context.Context, req *pb.HelloRequest) (*pb.HelloReply, error) {
	log.Infof("got hello request: %s", req.Msg)
	time.Sleep(1 * time.Minute)
	log.Info("sleep enough time")
	return &pb.HelloReply{Msg: "Hello " + req.Msg + "!"}, nil
}

pb file:

syntax = "proto3";

package trpc.hello;
option go_package="xagent/cmd/rpc/pb";

service Greeter {
  rpc Hello (HelloRequest) returns (HelloReply) {}
}

message HelloRequest {
  string msg = 1;
}

message HelloReply {
  string msg = 1;
}

PS D:\code\gocode\trpc-go-r1.0\examples\helloworld\server> go install
go: downloading trpc.group/trpc/trpc-protocol/pb/go/trpc v0.1.2-0.20230530025122-c44533fe44bd
......\errs\errs.go:15:2: trpc.group/trpc/trpc-protocol/pb/go/[email protected]: invalid version: unknown revision c44533fe44bd

Preliminary Research

• I have read the project's documentation.
• I have searched existing issues for similar questions to see if my question has already been addressed.

Question

I read the code about FormDataSerialization.Marshal and find that the Marshal member function of FormDataSerialization uses JSONPBSerialization.Marshal to implement.

From my point of view, the Marshal member function of FormDataSerialization needs to implement HTTP body serialization of Content-Type: multipart/form-data but not Content-Type: application/json.

This implementation can be confusing to users.

Additional Information

Related codes are shown below.

var (
	// tagJSON uses same tag with json.
	tagJSON = "json"
	// FormDataMarshalType the serialization method of the response data,
	// default is json serialization.
	FormDataMarshalType = codec.SerializationTypeJSON
)

func init() {
	codec.RegisterSerializer(
		codec.SerializationTypeFormData,
		NewFormDataSerialization(tagJSON),
	)
}

// Marshal serializes.
func (j *FormDataSerialization) Marshal(body interface{}) ([]byte, error) {
	serializer := codec.GetSerializer(FormDataMarshalType)
	if serializer == nil {
		return nil, errors.New("empty json serializer")
	}
	return serializer.Marshal(body)
}

Question

https://github.com/trpc-group/trpc-go/blob/main/restful/README.zh_CN.md
The example in this document contains google.protobuf.empty. When I use trpc create to generate code, powershell gives an error：
Execution err:
parser.Parse during pre run err: parse IDL[protobuf] test.proto error: parseProtoFile err: test.proto:26:5: field test.HelloRequest.oneof_empty: unknown type google.protobuf.Empty
Please run "trpc -h" or "trpc create -h" (or "trpc {some-other-subcommand} -h") for help messages.

Windows，Go1.19

Question

This./ Trpc_go.yaml refers to yaml that is already in the same level project as main？

Preliminary Research

• I have read the project's documentation.
• I have searched existing issues for similar questions to see if my question has already been addressed.

Question

Additional Information

The performance of official encoding/json. The saying that "official json is slow" is out-of-day. Besides, json-iterator use some low-level packages those may panic when the major version of go advances.

Json-iterator panic issues:

• it will panic fault when Marshal map (go version 1.18)
• 升级go1.18版本json-iterator coredump问题

Json-iterator performance issues:

• JSON-ITERATOR 使用要注意的大坑: "在使用json-iterator的过程中，没有发现有json-iterator有内存泄露问题，可以放心使用，但在Go1.10以后性能与标准库差异不大，这个第三方库的意义已经不大了"
• Go 语言原生的 json 包有什么问题？如何更好地处理 JSON 数据？: "jsoniter 并没有什么明显优势。即便是 jsoniter 作为卖点的大数据量解析，优势也是微乎其微"
• Go总结（二一）| JSON编解码: "用json-iterator意义不大，还是用标准库吧，真要提高性能，再想想别的办法，或者研究一下上面恐怖的sonic"
• 有篇文章说“在Go1.10以后性能与标准库差异不大，这个第三方库的意义已经不大了”，是真的吗？有大佬说一下吗？

If you really have performance concern, please use sonic, NOT json-iterator. If not, choose the official one.

Besides, less dependency, less safety issues.

Is your feature request related to a problem?

When I want to get configurations from default global trpc_go.yaml file, I had to create a type, implementing plugin.Factory.

For simple confiration, these codes is quite annoying an unnecessary.

What is the solution you'd like?

A simple function call, parsing a callback or simply a local address binding for yaml unmarshaling.

What alternatives have you considered?

Please refer to my implemtation: https://github.com/Andrew-M-C/trpc-go-utils/blob/master/config/config.go

Could you provide additional context?

Please review my implementation. I can also raise a PR.

Currently, we've got the following two kinds of styles:

# Title

English | 中文

English | 中文

# Title

They need to be unified.

部分文档引导到内部域名了

如：
https://github.com/trpc-group/trpc-go/blob/main/examples/features/http/README.md

Preliminary Research

• I have read the project's documentation.
• I have searched existing issues for similar questions to see if my question has already been addressed.

Question

Additional Information

hi the trpc team, I'm the maintainer of api-testing which aims to be an alternative solution to Postman. Instead of Postman, api-testing is totally open-source. And it has an extension (or plug-in) mechanism.

Add more background of api-testing. It already supports HTTP and gRPC protocols. For Docker users, you could simply type the following command to get started:

docker run --network host linuxsuren/api-testing:master

The web server local address is http://localhost:8080.

[xx](../../examples/features/stream)

v.s

[xx](/examples/features/stream)

Question

When non-Tencent internal learners study the trpc-go framework, they will inevitably encounter some problems. These problems are also likely to be encountered or asked by community learners. These issues can to some extent help community learners better understand the use of this framework and solve some confusing questions.

Question

When I wanted to quickly get started with the trpc-go project to write my own backend project (replacing gin), I found it a bit difficult. I wonder if there are any other relevant excellent code repositories that can provide some writing references. I'm very grateful

Preliminary Research

• I have read the project's documentation.
• I have searched existing issues for similar questions to see if my question has already been addressed.

Question

None

Additional Information

None

What version of tRPC-Go are you using?

v1.0.0

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

$ go env
GO111MODULE='on'
GOARCH='amd64'
GOBIN=''
GOCACHE='/root/.cache/go-build'
GOENV='/root/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/root/go/pkg/mod'
GONOPROXY='trpc.group,github.com'
GONOSUMDB='trpc.group,github.com'
GOOS='linux'
GOPATH='/root/go'
GOPRIVATE='trpc.group'
GOPROXY='https://wineguo:[email protected]'
GOROOT='/root/sdk/go1.21.2'
GOSUMDB='sum.woa.com+643d7a06+Ac5f5VOC4N8NUXdmhbm8pZSXIWfhek5JSmWdWrq7pLX4'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/root/sdk/go1.21.2/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.21.2'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/data/projects/github/trpc-go/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build1451082008=/tmp/go-build -gno-record-gcc-switches'

What did you do?

use helloworld.proto

run

trpc create -p helloworld.proto -o out

edit the out/trpc_go.yaml file, add transport: go-net to every service, then

$ cd out 
$ go run .
// ...
2023-10-18 10:50:05.779 DEBUG   maxprocs/maxprocs.go:47 maxprocs: Leaving GOMAXPROCS=16: CPU quota undefined
panic: HelloWorldServerStreamService register error:server StreamTransport is not implemented

goroutine 1 [running]:
trpc.group/examples/helloworld.RegisterHelloWorldServerStreamService({0xd4afb0?, 0xc0004564c0?}, {0xd44000?, 0x12d5480?})
        /data/projects/github/trpc-cmdline/testcase/create/1-without-import/out/stub/trpc.group/examples/helloworld/helloworld.trpc.go:133 +0x9f
main.main()
        /data/projects/github/trpc-cmdline/testcase/create/1-without-import/out/main.go:14 +0xb6
exit status 2

What did you expect to see?

no panic

What did you see instead?

panic

resolve url: [trpc.group/trpc/trpc-protocol/pb/go/trpc v0.1.2-0.20230530025122-c44533fe44bd] failed, module not found
unable to access 'https://github.com/trpc-group/trpc/'

trpc-go repo url path is github/trpc-group/trpc-go/trpc-go,but the go.mod module name is trpc.group/trpc-go/trpc-go

What version of tRPC-Go are you using?

master

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

$ go env

What did you do?

I have service on both restful and trpc, and needs the restful listen on all addresses, trpc listen on localhost(127.0.0.1)
it seems cannot work likes this.

1. test the examples/features/restful
2. update configure trpc_go.yaml to

global:                             # global config.
  namespace: Development            # environment type, two types: production and development.
  env_name: test                    # environment name, names of multiple environments in informal settings.

server:                                            # server configuration.
  app: test                                        # business application name.
  server: helloworld                               # service process name.
  bin_path: /usr/local/trpc/bin/                   # paths to binary executables and framework configuration files.
  conf_path: /usr/local/trpc/conf/                 # paths to business configuration files.
  data_path: /usr/local/trpc/data/                 # paths to business data files.
  service:                                         # business service configuration，can have multiple.
    - name: trpc.test.helloworld.Greeter           # the route name of the service.
      # ip: 127.0.0.1                                # the service listening ip address, can use the placeholder ${ip}, choose one of ip and nic, priority ip.
      port: 9092                                   # the service listening port, can use the placeholder ${port}.
      network: tcp                                 # the service listening network type,  tcp or udp.
      protocol: restful                            # application layer protocol. NOTE restful service this is restful.
      timeout: 1000                                # maximum request processing time in milliseconds.
      idletime: 300000                             # connection idle time in milliseconds.
    - name: trpc.test.helloworld.Greeter1           # the route name of the service.
      ip: 127.0.0.1                                # the service listening ip address, can use the placeholder ${ip}, choose one of ip and nic, priority ip.
      port: 9093                                   # the service listening port, can use the placeholder ${port}.
      network: tcp                                 # the service listening network type,  tcp or udp.
      protocol: trpc                            # application layer protocol. NOTE restful service this is restful.
      timeout: 1000                                # maximum request processing time in milliseconds.
      idletime: 300000                             # connection idle time in milliseconds.

plugins:                                          # plugin configuration.
  log:                                            # log configuration.
    default:                                      # default log configuration, support multiple outputs.
      - writer: console                           # console standard output default.
        level: debug                              # standard output log level.

4. update main.go to

package main

import (
   "context"
   "fmt"

   trpc "trpc.group/trpc-go/trpc-go"
   "trpc.group/trpc-go/trpc-go/examples/features/restful/pb"
   "trpc.group/trpc-go/trpc-go/log"
)

func main() {
   // init trpc server
   server := trpc.NewServer()
   // Register the greeter service with the server
   svc := new(greeterService)
   pb.RegisterGreeterService(server.Service("trpc.test.helloworld.Greeter"), svc)
   pb.RegisterGreeterService(server.Service("trpc.test.helloworld.Greeter1"), svc)
   // Run the server
   if err := server.Serve(); err != nil {
   	log.Fatal(err)
   }
}

// greeterService is used to implement pb.GreeterService.
type greeterService struct {
   // unimplementedGreeterServiceServer is the unimplemented greeter service server
   pb.UnimplementedGreeter
}

// SayHello implements pb.GreeterService.
func (g greeterService) SayHello(ctx context.Context, req *pb.HelloRequest) (*pb.HelloReply, error) {
   log.InfoContextf(ctx, "[restful] Received SayHello request with req: %v", req)
   // handle request
   rsp := &pb.HelloReply{
   	Message: "[restful] SayHello Hello " + req.Name,
   }
   return rsp, nil
}

// Message implements pb.GreeterService.
func (g greeterService) Message(ctx context.Context, req *pb.MessageRequest) (*pb.MessageInfo, error) {
   log.InfoContextf(ctx, "[restful] Received Message request with req: %v", req)
   // handle request
   rsp := &pb.MessageInfo{
   	Message: fmt.Sprintf("[restful] Message name:%s,subfield:%s",
   		req.GetName(), req.GetSub().GetSubfield()),
   }
   return rsp, nil
}

// UpdateMessage implements pb.GreeterService.
func (g greeterService) UpdateMessage(ctx context.Context, req *pb.UpdateMessageRequest) (*pb.MessageInfo, error) {
   log.InfoContextf(ctx, "[restful] Received UpdateMessage request with req: %v", req)
   // handle request
   rsp := &pb.MessageInfo{
   	Message: fmt.Sprintf("[restful] UpdateMessage message_id:%s,message:%s",
   		req.GetMessageId(), req.GetMessage().GetMessage()),
   }
   return rsp, nil
}

// UpdateMessageV2 implements pb.GreeterService.
func (g greeterService) UpdateMessageV2(ctx context.Context, req *pb.UpdateMessageV2Request) (*pb.MessageInfo, error) {
   log.InfoContextf(ctx, "[restful] Received UpdateMessageV2 request with req: %v", req)
   // handle request
   rsp := &pb.MessageInfo{
   	Message: fmt.Sprintf("[restful] UpdateMessageV2 message_id:%s,message:%s",
   		req.GetMessageId(), req.GetMessage()),
   }
   return rsp, nil
}

What did you expect to see?

trpc.test.helloworld.Greeter listen on :9092
trpc service:trpc.test.helloworld.Greeter1 listen on 127.0.0.1:9093

2023-12-11 21:11:51.967	DEBUG	maxprocs/maxprocs.go:47	maxprocs: Leaving GOMAXPROCS=8: CPU quota undefined
2023-12-11 21:12:03.947	INFO	server/service.go:167	process:7755, restful service:trpc.test.helloworld.Greeter launch success, tcp::9092, serving ...
2023-12-11 21:12:03.947	INFO	server/service.go:167	process:7755, trpc service:trpc.test.helloworld.Greeter1 launch success, tcp:127.0.0.1:9093, serving ...

What did you see instead?

2023-12-11 21:11:51.967	DEBUG	maxprocs/maxprocs.go:47	maxprocs: Leaving GOMAXPROCS=8: CPU quota undefined
2023-12-11 21:12:03.947	INFO	server/service.go:167	process:7755, restful service:trpc.test.helloworld.Greeter launch success, tcp:127.0.0.1:9092, serving ...
2023-12-11 21:12:03.947	INFO	server/service.go:167	process:7755, trpc service:trpc.test.helloworld.Greeter1 launch success, tcp:127.0.0.1:9093, serving ...

i clone the latest version of trpc-go
my system is darwin arm64

when i cd example to run hello as the doc say. i got the error as follow:
unrecognized import path "trpc.group/trpc-go/go_reuseport": https fetch: Get "https://trpc.group/trpc-go/go_reuseport?go-get=1":