Git Product home page Git Product logo

go2sky's Introduction

GO2Sky (Archived)

NOTICE, SkyWalking upstream began to build the auto-instrument agent on April 2023

This repository are going to be archived and replaced by the official agent once the upstream does the first release. Please consider immigrating to the better solution.

Build Coverage GoDoc

GO2Sky is an instrument SDK library, written in Go, by following Apache SkyWalking tracing and metrics formats.

Installation

  • Require Golang 1.16
$ go get -u github.com/SkyAPM/go2sky

The API of this project is still evolving. The use of vendoring tool is recommended.

Quickstart

By completing this quickstart, you will learn how to trace local methods. For more details, please view the example.

Configuration

GO2Sky can export traces to Apache SkyWalking OAP server or local logger. In the following example, we configure GO2Sky to export to OAP server, which is listening on oap-skywalking port 11800, and all the spans from this program will be associated with a service name example. reporter.GRPCReporter can also adjust the behavior through reporter.GRPCReporterOption, view all.

r, err := reporter.NewGRPCReporter("oap-skywalking:11800")
if err != nil {
    log.Fatalf("new reporter error %v \n", err)
}
defer r.Close()
tracer, err := go2sky.NewTracer("example", go2sky.WithReporter(r))

In some scenarios, we may need a filter to filter segments that do not need to be submitted, for example, to reduce the load of gRPC reporting, or only track the request of error.

r, err := reporter.NewGRPCReporter("oap-skywalking:11800", reporter.WithReportStrategy(func(s *v3.SegmentObject) bool {
	var isReport bool
	for _, span := s.GetSpans() {
		if span.GetIsError() {
			isReport = true
			break
		}
	}
	
	return isReport
}))

You can also create tracer with sampling rate. It supports decimals between 0-1 (two decimal places), representing the sampling percentage of trace.

tracer, err := go2sky.NewTracer("example", go2sky.WithReporter(r), go2sky.WithSampler(0.5))

Also could customize correlation context config.

tracer, err := go2sky.NewTracer("example", go2sky.WithReporter(r), go2sky.WithSampler(0.5), go2sky.WithCorrelation(3, 128))

Create span

To create a span in a trace, we used the Tracer to start a new span. We indicate this as the root span because of passing context.Background(). We must also be sure to end this span, which will be show in End span.

span, ctx, err := tracer.CreateLocalSpan(context.Background())

Create a sub span

A sub span created as the children of root span links to its parent with Context.

subSpan, newCtx, err := tracer.CreateLocalSpan(ctx)

Get correlation

Get custom data from tracing context.

value := go2sky.GetCorrelation(ctx, key)

Put correlation

Put custom data to tracing context.

success := go2sky.PutCorrelation(ctx, key, value)

End span

We must end the spans so they becomes available for sending to the backend by a reporter.

subSpan.End()
span.End()

Global Tracer

Set and get global Tracer

// new tracer
tr, err := go2sky.NewTracer("example")

// registers `tracer` as the global Tracer
go2sky.SetGlobalTracer(tr)

// returns the registered global Tracer
// if none is registered then an instance of `nil` is returned
tracer := go2sky.GetGlobalTracer()

Get Active Span

Get the activeSpan in the Context.

go2sky.ActiveSpan(ctx)

With Span

Save the activeSpan to Context

go2sky.WithSpan(ctx, activeSpan)

Get Global Service Name

Get the ServiceName of the activeSpan in the Context.

go2sky.ServiceName(ctx)

Get Global Service Instance Name

Get the ServiceInstanceName of the activeSpan in the Context.

go2sky.ServiceInstanceName(ctx)

Get Global TraceID

Get the TraceID of the activeSpan in the Context.

go2sky.TraceID(ctx)

Get Global TraceSegmentID

Get the TraceSegmentID of the activeSpan in the Context.

go2sky.TraceSegmentID(ctx)

Get Global SpanID

Get the SpanID of the activeSpan in the Context.

go2sky.SpanID(ctx)

Report Application Logs

go2sky provides a logger to transmit logs of application to the SkyWalking OAP server.

import (
	"context"
	"github.com/SkyAPM/go2sky"
	"github.com/SkyAPM/go2sky/reporter"
	"log"
)

func reportLogsTest(ctx context.Context)  {
	r, err := reporter.NewGRPCReporter("oap-skywalking:11800")
	if err != nil {
		log.Fatalf("new reporter error %v \n", err)
	}
	defer r.Close()

	logger, errOfLoggerCreation := go2sky.NewLogger(r)
	if errOfLoggerCreation != nil{
		log.Fatalf("new Logger error %v \n", skyapmError)
	}	
	logData := "your application log need to send to backend here..."
	logger.WriteLogWithContext(ctx,go2sky.LogLevelError, logData)
}

Periodically Report

Go2sky agent reports the segments periodically. It would not wait for all finished segments reported when the service exits.

Advanced Concepts

We cover some advanced topics about GO2Sky.

Context propagation

Trace links spans belong to it by using context propagation which varies between different scenario.

In process

We use context package to link spans. The root span usually pick context.Background(), and sub spans will inject the context generated by its parent.

//Create a new context
entrySpan, entryCtx, err := tracer.CreateEntrySpan(context.Background(), ...)

// Some operation
...

// Link two spans by injecting entrySpan context into exitSpan
exitSpan, err := tracer.CreateExitSpan(entryCtx, ...)

Crossing process

We use Entry span to extract context from downstream service, and use Exit span to inject context to upstream service.

Entry and Exit spans make sense to OAP analysis which generates topology map and service metrics.

//Extract context from HTTP request header `sw8`
span, ctx, err := tracer.CreateEntrySpan(r.Context(), "/api/login", func(key string) (string, error) {
		return r.Header.Get(key), nil
})

// Some operation
...

// Inject context into HTTP request header `sw8`
span, err := tracer.CreateExitSpan(req.Context(), "/service/validate", "tomcat-service:8080", func(key, value string) error {
		req.Header.Set(key, value)
		return nil
})

Tag

We set tags into a span which is stored in the backend, but some tags have special purpose. OAP server may use them to aggregate metrics, generate topology map and etc.

They are defined as constant in root package with prefix Tag.

Log x Trace context

Inject trace context into the log text. SkyWalking LAL(log analysis language) engine could extract the context from the text and correlate trace and logs.

// Get trace context data
import go2skylog "github.com/SkyAPM/go2sky/log"
logContext = go2skylog.FromContext(ctx)

// Build context data string
// Inject context string into log
// Context format string: [$serviceName,$instanceName,$traceId,$traceSegmentId,$spanId]
contextString := logContext.String()

Plugins

Go to go2sky-plugins repo to see all the plugins, click here.

Supported Environment Variables

Below is the full list of supported environment variables you can set to customize the agent behavior, please read the descriptions for what they can achieve.

Environment Variable Description Default
SW_AGENT_NAME The name of the Go service unset
SW_AGENT_LAYER Instance belong layer name which define in the backend unset
SW_AGENT_INSTANCE_NAME The name of the Go service instance Randomly generated
SW_AGENT_SAMPLE Sample rate, 1 setting it to 1 means full sampling 1
SW_AGENT_COLLECTOR_BACKEND_SERVICES The backend OAP server address unset
SW_AGENT_AUTHENTICATION The authentication token to verify that the agent is trusted by the backend OAP, as for how to configure the backend, refer to the yaml. unset
SW_AGENT_COLLECTOR_HEARTBEAT_PERIOD Agent heartbeat report period. Unit, second 20
SW_AGENT_COLLECTOR_GET_AGENT_DYNAMIC_CONFIG_INTERVAL Sniffer get agent dynamic config interval. Unit, second 20
SW_AGENT_COLLECTOR_MAX_SEND_QUEUE_SIZE Send span queue buffer length 30000
SW_AGENT_PROCESS_STATUS_HOOK_ENABLE Enable the Process Status Hook feature false
SW_AGENT_PROCESS_LABELS The labels of the process, multiple labels split by "," unset

CDS - Configuration Discovery Service

Configuration Discovery Service provides the dynamic configuration for the agent, defined in gRPC and stored in the backend.

Available key(s) and value(s) in Golang Agent.

Golang agent supports the following dynamic configurations.

Config Key Value Description Value Format Example
agent.sample_rate The percentage of trace when sampling. It's [0, 1], Same with WithSampler parameter. 0.1

Process Status Hook

This feature is used in cooperation with the skywalking-rover project.

When go2sky keeps alive with the backend, it would write a metadata file to the local (temporary directory) at the same time, which describes the information of the current process. The rover side scans all processes, find out which process contains this metadata file. Finally, the rover could collect, profiling, with this process.

Metadata File

The metadata file use to save metadata with current process, it save in: {TMPDIR}/apache_skywalking/process/{pid}/metadata.properties.

Also, when the go2sky keep alive with backend, modify and open time of the metadata file would be updated.

Key Type Description
layer string this process layer.
service_name string this process service name.
instance_name string this process instance name.
process_name string this process process name, it's same with the instance name.
properties json the properties in instance, the process labels also in the properties value.
labels string the process labels, multiple labels split by ",".
language string current process language, which is golang.

Please read the official documentation of rover to get more information.

License

Apache License 2.0. See LICENSE file for details.

go2sky's People

Contributors

arugal avatar chwjbn avatar easonyipj avatar fgksgf avatar hanahmily avatar humbertzhang avatar jaredtan95 avatar jj-jasmin avatar just-maple avatar kagaya85 avatar kehuili avatar kezhenxu94 avatar kuaikuai avatar limfriend avatar liweiv avatar lokichoggio avatar luckyboys avatar matianjun1 avatar mrproliu avatar nacx avatar withlin avatar wu-sheng avatar yaojingguo avatar zhucheer avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

go2sky's Issues

not work for go2sky.WithInstance(instanceName)

Describe the bug
A clear and concise description of what the bug is.
Do reporter.NewGRPCReporter with the go2sky.TracerOption go2sky.WithInstance("instance@hostname")

but can not found the instance name "instance@hostname" at http://127.0.0.1:8080/trace on dropdown menu instance

To Reproduce

  1. Install skywalking on windows 10
  2. run /path/to/apache-skywalking-apm-bin/bin/startup.bat
  3. type testtracer.go as follow:
    package main

import (
"context"
"log"
"time"

"github.com/tetratelabs/go2sky"
"github.com/tetratelabs/go2sky/reporter"

)

func main() {
report, err := reporter.NewGRPCReporter("127.0.0.1:11800")
if err != nil {
log.Fatalf("new reporter error %v \n", err)
}
tracer, err := go2sky.NewTracer("serverName", go2sky.WithReporter(report), go2sky.WithInstance("instance@hostname"))
if err != nil {
log.Fatalf("create tracer error %v \n", err)
}
tracer.WaitUntilRegister()
defer report.Close()

span, ctx, err := tracer.CreateLocalSpan(context.Background())
if err != nil {
	log.Fatalf("create new local span error %v \n", err)
}
span.SetOperationName("invoke data")
span.Tag("kind", "outer")
time.Sleep(time.Second)
subSpan, _, err := tracer.CreateLocalSpan(ctx)
if err != nil {
	log.Fatalf("create new sub local span error %v \n", err)
}
subSpan.SetOperationName("invoke inner")
subSpan.Log(time.Now(), "inner", "this is right")
time.Sleep(time.Second)
subSpan.End()
time.Sleep(500 * time.Millisecond)
span.End()
time.Sleep(time.Second)

}

go run testtracer.go
go2sky-gRPC2019/08/20 15:12:47 the id of service 'serverName' is 5
go2sky-gRPC2019/08/20 15:12:47 the id of instance 'instance@hostname' id is 6

  1. access to http://127.0.0.1:8080/trace
    select the service "serverName", then click down instance, only 2 items over there: All and serverName

Expected behavior
I can found the instance "instance@hostname" at http://127.0.0.1:8080/trace in the dropdown menu "instance"

Screenshots

Desktop (please complete the following information):

  • OS: [windows]
  • Version [10]

Additional context
Add any other context about the problem here.

Please do not delete releases or tags

Describe the bug
When user depends on your library, they may not vendor it. If a tag or a release was removed, then go build will fail.

To Reproduce
go.mod:

module whatever.com/ccc

go 1.12

require (
        github.com/SkyAPM/go2sky v0.3.1-0.20200329092408-8b3e4d536d8d
)

Expected behavior
go build success

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

panic: proto: duplicate enum registered: SpanType

Describe the bug

panic: proto: duplicate enum registered: SpanType

goroutine 1 [running]:
github.com/golang/protobuf/proto.RegisterEnum(0x2bb3c53, 0x8, 0xc000289080, 0xc0002890e0)
E:/workspace/go/src/pkg/mod/github.com/golang/[email protected]/proto/registry.go:104 +0x125
github.com/SkyAPM/go2sky/reporter/grpc/language-agent.init.0()
E:/workspace/go/src/pkg/mod/github.com/!sky!a!p!m/[email protected]/reporter/grpc/language-agent/Tracing.pb.go:520 +0x59

To Reproduce
Steps to reproduce the behavior:

with go micro and gin framework

Is Go2sky can build relationship between two process

I'm trying to monitor this, for example, Service A calling a method from Service B(just like RPC). And I create Tracer and Span in both of them. I can only build relationship between two spans from one process. How can I build the relationship between two spans from different process.
In one process, I build relationship like this:
`
span1, ctx, err := tracer.CreateEntrySpan(context.Background(), "/API/Login", func() (string, error) {
return "", nil
})

...

span2, err := tracer.CreateExitSpan(ctx, "Service/OnLogin", "client_1", func(header string) error {
return nil
})
`

I'm already read some documents, and I think it should be done by span's context. But how can I reach it, is it a possible thing? Expect for your reply!

Make span tag key/value pairs to be utf-8 encoded

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Our application shows the log like below due to the URL path contain non utf-8 encoded characters.
"go2sky-gRPC2020/12/28 18:49:46 send segment error rpc error: code = Internal desc = grpc: error while marshaling: proto: field "KeyStringValuePair.Value" contains invalid UTF-8"
Describe the solution you'd like
A clear and concise description of what you want to happen.
The sdk should always encode the key/value string for tags with utf-8.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

panic: send on closed channel

go2sky-gRPC2020/03/20 15:20:53 the id of service 'libra' is 2
go2sky-gRPC2020/03/20 15:20:53 register error fail to instance service
go2sky-gRPC2020/03/20 15:20:54 the id of instance '55a92af0-6a7b-11ea-aedd-c4b3018d0c5e' id is 10
go2sky-gRPC2020/03/20 15:20:54 pinging error rpc error: code = Unavailable desc = transport is closing

[GIN] 2020/03/20 - 15:23:55 | 200 | 11.69461ms | ::1 | POST /api/v1/ldapLogin
panic: send on closed channel

goroutine 106 [running]:
github.com/SkyAPM/go2sky/reporter.(*gRPCReporter).Send(0xc0001b4000, 0xc0002a1220, 0x1, 0xa)
/Users/changsq/go/pkg/mod/github.com/!sky!a!p!m/[email protected]/reporter/grpc.go:269 +0x751
github.com/SkyAPM/go2sky.newSegmentRoot.func1(0xc000432960, 0xc0000c7200)
/Users/changsq/go/pkg/mod/github.com/!sky!a!p!m/[email protected]/segment.go:237 +0x243
created by github.com/SkyAPM/go2sky.newSegmentRoot
/Users/changsq/go/pkg/mod/github.com/!sky!a!p!m/[email protected]/segment.go:222 +0x168
exit status 2

  • go version go1.13 darwin/amd64
  • go2sky v0.3.3
  • OS: MacOS Mojave
  • Skywalking 6.6

客户端协程泄漏

客户端协程未正常关闭,导致携程泄漏。
客户端上报发生错误时(send segment error EOF),会尝试关闭stream并重新开启stream进行后续上报,但是调用的是stream.CloseSend(),而不是CloseAndRecv(),导致创建stream过程中,开启的协程未能正常关闭。

go2sky 到 java

I want to use go2sky to generate the traceID and then call the backend java service so that I can complete the link monitoring.

go2sky conflict with etcdv3

import (
"context"
"github.com/SkyAPM/go2sky"
"github.com/SkyAPM/go2sky/reporter"
_ "github.com/coreos/etcd/clientv3"
"github.com/robfig/cron"
"log"
"time"
)

report error:

GOROOT=C:\Go #gosetup
GOPATH=D:\gopath #gosetup
C:\Go\bin\go.exe build -o C:\Users\admin\AppData\Local\Temp__go_build_main_go__1.exe -gcflags "all=-N -l" D:\projects\go-micro-demo\micro\crontab-service\main.go #gosetup

github.com/coreos/etcd/clientv3/balancer/resolver/endpoint

D:\gopath\pkg\mod\github.com\coreos\[email protected]+incompatible\clientv3\balancer\resolver\endpoint\endpoint.go:114:78: undefined: resolver.BuildOption
D:\gopath\pkg\mod\github.com\coreos\[email protected]+incompatible\clientv3\balancer\resolver\endpoint\endpoint.go:182:31: undefined: resolver.ResolveNowOption

github.com/coreos/etcd/clientv3/balancer/picker

D:\gopath\pkg\mod\github.com\coreos\[email protected]+incompatible\clientv3\balancer\picker\err.go:37:44: undefined: balancer.PickOptions
D:\gopath\pkg\mod\github.com\coreos\[email protected]+incompatible\clientv3\balancer\picker\roundrobin_balanced.go:55:54: undefined: balancer.PickOptions

Compilation finished with exit code 2

How to trace MQTT

In examples, we need to pass request context to extract or inject information, however, there's no such thing in MQTT when publish or subscribe messages.
I would like to know if there is a way to trace MQTT, thanks.

Provide cross-thread transfer and expand information field function

Describe the solution you want
Provides cross-thread link transfer extended information field

Describe the alternatives you considered
Similar function to Java agent:
TraceContext.putCorrelation("test-flow", testFlag);
Optional testFlowOpt = TraceContext.getCorrelation("test-flow");
Similar functions can be obtained from go2sky sdk to facilitate the transmission of field information links

Other content
Add any other context or screenshots about the feature request here.

Performance of report.

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
I find that function newSegmentRoot will create a coroutine for every request . At the end of each Segment collection, a message is sent immediately.I'm afraid this will affect performance.

Describe the solution you'd like
A clear and concise description of what you want to happen.

I'm wondering if go2sky can implement a collector to collect messages and send them based on time and quantity dimensions, so as to avoid creating a large number of coroutines in high concurrency scenarios.User can set report interval and number of segments.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

routeMap key is handler name, if different url paths route to same handler, only the last one is available to trace, consider using Method+Path as the key?

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

routeMap key is handler name, if different url paths route to same handler, only the last one is available to trace, consider using Path as the key to solve this? or any other solutions?

mm[r.Handler] = routeInfo{ operationName: fmt.Sprintf("/%s%s", r.Method, r.Path), }

described situation can be avoided by using different handlers for different urls, but it takes time to find why when it happens.

Additional context
Add any other context or screenshots about the feature request here.

hope reporter support log

Is your feature request related to a problem? Please describe.
I want to report some log with span, use skywalking Logging.proto.

The call chain is messed up

When I modified the code for plugins/http/example_http_test.go it looks like this:

// ServeHTTP implements http.Handler.
func (h handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	span, ctx, err := h.tracer.CreateEntrySpan(r.Context(), getOperationName(h.name, r), func() (string, error) {
		return r.Header.Get(propagation.Header), nil
	})
	if err != nil {
		h.next.ServeHTTP(w, r)
		return
	}
	span.SetComponent(httpServerComponentID)
	for k, v := range h.extraTags {
		span.Tag(go2sky.Tag(k), v)
	}
	span.Tag(go2sky.TagHTTPMethod, r.Method)
	span.Tag(go2sky.TagURL, fmt.Sprintf("%s%s", r.Host, r.URL.Path))
	span.SetSpanLayer(common.SpanLayer_Http)

	rww := &responseWriterWrapper{w: w, statusCode: 200}
	// defer func() {
	// 	code := rww.statusCode
	// 	if code >= 400 {
	// 		span.Error(time.Now(), "Error on handling request")
	// 	}
	// 	span.Tag(go2sky.TagStatusCode, strconv.Itoa(code))
	// 	span.End()
	// }()
	
	// Here, I called span.End() in advance. Because in some scenarios, I can't wait for the child span.End() method.
	span.End()
	h.next.ServeHTTP(rww, r.WithContext(ctx))
}

�After modifying the above code and run ExampleNewServerMiddleware() , the call chain becomes the following

image

But what I am looking for is this

image

EntrySpan and ExitSpan.

Describe the bug
A clear and concise description of what the bug is.
I have three services a, b and c. Interaction between services is user->a->b->c. I use function CreateEntrySpan in a and CreateExitSpan in b. The topology diagram in skywalking is not in line with my expectations which is user->a and b->c. But there is no line between a and b.
Could you show me how to solve this problem. Thanks

To Reproduce
Steps to reproduce the behavior:

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.
image

In this png user -> mersher-webapp(CreateEntrySpan) -> calculator(CreateExitSpan) -> 127.0.0.1:4500.
Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

hope reporter support logger interface{}

Is your feature request related to a problem? Please describe.
our application use self defined logger module , which is not log.Logger . but implements level logger interface{} .

but the reporter.grpc.WithLogger(logger *log.Logger) only support *log.Logger

type LevelLogger interface {
Debug(msg string)
Debugf(format string, v ...interface{})
Info(msg string)
Infof(format string, v ...interface{})
Warn(msg string)
Warnf(format string, v ...interface{})
Error(msg string)
Errorf(format string, v ...interface{})
}

Describe the solution you'd like
define a logger interface , so we can overwrite

Get Span from ctx

Sometime I need spanFromContext

such as go-redis-hook

type Hook interface {
	BeforeProcess(ctx context.Context, cmd Cmder) (context.Context, error)
	AfterProcess(ctx context.Context, cmd Cmder) error
}

only ctx can be propagation, I cant do span.End further

Maybe there is a easy way to solve, set ctxKeyInstance to be public. Then I can use this key to get activeSpan from ctx and do (.Span) to get a Span

Database is not getting registered and the database is not being displayed in the topology

Describe the bug
Database metrics are not getting generated when the agent is connected to OAP backend

To Reproduce

  • Ran Skywalking locally
  • Added go2Sky plugin to demo app and made connection to the collector(refer the screenshot for more info)
  • The database used for agent is mysql running on port 3306

Expected behavior

  • Database related metrics along with the database should be displayed on the UI
  • The database relation to be displayed in topology

Screenshots
Screen Shot 2020-11-19 at 17 12 45
Screen Shot 2020-11-19 at 17 15 44
Screen Shot 2020-11-19 at 17 15 13

Code
main file of demo app
`package main

import (
"encoding/json"
"fmt"
"io/ioutil"
"log"
"net/http"
"strconv"

"github.com/SkyAPM/go2sky"
httpPlugin "github.com/SkyAPM/go2sky/plugins/http"
"github.com/SkyAPM/go2sky/reporter"
"github.com/gorilla/mux"
"github.com/jinzhu/gorm"
_ "github.com/jinzhu/gorm/dialects/mysql"
_ "github.com/jinzhu/gorm/dialects/postgres"

)

var db *gorm.DB
var err error

type Booking struct {
Id int json:"id"
User string json:"user"
Members int json:"members"
}

func homePage(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Welcome to HomePage!")
fmt.Println("Endpoint Hit: HomePage")
}

func main() {

database, err := gorm.Open("mysql", "user:password@/test?charset=utf8&parseTime=True&loc=Local")
if err != nil {
	panic("Failed to connect to database!")
}

database.AutoMigrate(&Booking{})
db = database
re, err := reporter.NewGRPCReporter("localhost:11800")
if err != nil {
	log.Fatalf("new reporter error %v \n", err)
}
defer re.Close()
// re, err := reporter.NewLogReporter()
// if err != nil {
// 	log.Fatalf("new reporter error %v \n", err)
// }
// defer re.Close()

tracer, err := go2sky.NewTracer("go-http-client", go2sky.WithReporter(re))
if err != nil {
	log.Fatalf("create tracer error %v \n", err)
}

sm, err := httpPlugin.NewServerMiddleware(tracer)
if err != nil {
	log.Fatalf("create server middleware error %v \n", err)
}
// span, ctx, err := tracer.CreateLocalSpan(context.Background())
// go2sky.TraceID(ctx)
// subSpan, newCtx, err := tracer.CreateLocalSpan(ctx)
// fmt.Println(newCtx)
// subSpan.End()
// span.End()

log.Println("Starting development server at http://127.0.0.1:10000/")
log.Println("Quit the server with CONTROL-C.")
// creates a new instance of a mux router

myRouter := mux.NewRouter().StrictSlash(true)

myRouter.HandleFunc("/", homePage)
myRouter.HandleFunc("/new-booking", createNewBooking).Methods("POST")
myRouter.HandleFunc("/all-bookings", returnAllBookings)
myRouter.HandleFunc("/booking/{id}", returnSingleBooking)
log.Fatal(http.ListenAndServe(":10000", sm(myRouter)))

}
func createNewBooking(w http.ResponseWriter, r *http.Request) {
// get the body of our POST request
// return the string response containing the request body
reqBody, _ := ioutil.ReadAll(r.Body)
var booking Booking
json.Unmarshal(reqBody, &booking)
db.Create(&booking)
fmt.Println("Endpoint Hit: Creating New Booking")
json.NewEncoder(w).Encode(booking)
}
func returnAllBookings(w http.ResponseWriter, r *http.Request) {
bookings := []Booking{}
db.Find(&bookings)
fmt.Println("Endpoint Hit: returnAllBookings")
json.NewEncoder(w).Encode(bookings)
}
func returnSingleBooking(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
key := vars["id"]
bookings := []Booking{}
db.Find(&bookings)
for _, booking := range bookings {
// string to int
s, err := strconv.Atoi(key)
if err == nil {
if booking.Id == s {
fmt.Println(booking)
fmt.Println("Endpoint Hit: Booking No:", key)
json.NewEncoder(w).Encode(booking)
}
}
}
}
`

Desktop (please complete the following information):

  • OS: macOs
  • Version Skywalking version 8

Additional context
I'm fairly new to agent side connections ,please let me know if there is anomaly in the agent connection
i followed the steps shown on the go2Sky repo

release new feature

什么时候release 新特性,我目前项目迫切想使用 新特性
在创建CreateEntrySpan(r.Context(), getOperationName(h.name, r), func(key string)
非常想使用

Go-agent can register with the collector but cannot report the trace

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Build tracer with customized sampler object not just by sampling rate

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Currently, trace_opts.go provides a function to set sampler with signature func WithSampler(samplingRate float64) TracerOption . I hope it can provide another function like func WithSampler(sampler Sampler) TracerOption, then I can set my customized sampler.
At the same time, the RandomSampler has a little bug:

func (s *RandomSampler) IsSampled(operation string) bool {
	return s.threshold >= s.rand.Intn(100)
}

It should be s.threshold > s.rand.Intn(100).

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

When User NewGRPCReporter and the peer reset , program will log panic: send on closed channel and exit

version : 0.6.0

use helm install skywalking

skywalking version : 8.3.0

local connect to skywalking-oap use:
export POD_NAME=$(kubectl get pods --namespace default -l "app=skywalking,release=skywalking,component=oap" -o jsonpath="{.items[0].metadata.name}")
echo "Visit http://127.0.0.1:8050 to use your application"
kubectl port-forward $POD_NAME 8050:11800 --namespace default

program connect:
newReporter, err := reporter.NewGRPCReporter("127.0.0.1:8050")

when the connect is not normal

program will log :

panic: send on closed channel

goroutine 746 [running]:
github.com/SkyAPM/go2sky.(*segmentSpanImpl).End.func1(0xc00087aa20)
/Users/liuyungen/workspace/pkg/mod/github.com/!sky!a!p!m/[email protected]/segment.go:95 +0x4f
created by github.com/SkyAPM/go2sky.(*segmentSpanImpl).End
/Users/liuyungen/workspace/pkg/mod/github.com/!sky!a!p!m/[email protected]/segment.go:94 +0x6e
exit status 2

May be can change segment.go end function like below:

func (s *segmentSpanImpl) End() {
s.defaultSpan.End()
go func() {
defer func(){
err := recover()
if err != nil {
log.Printf("Segment error: %v",err)
}
}()
s.Context().collect <- s
}()
}

gin plugin bugfixed in master, but it still exists in v0.3.0, release a new version?

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.
using this still import the version with bug:
import "github.com/SkyAPM/go2sky/plugins/gin"

found the bug was fixed 2 months ago, the code(in plugin/gin.go):
from operationName: c.Request.Host + c.Request.URL.Path, to operationName: fmt.Sprintf("/%s%s", r.Method, r.Path),

release a new version?

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

Performance profiling in go2sky

It is wonderful to analyze performance issue using SkyWalking such as Java in this blog. Golang has pprof which can snapshot goroutine and heap allocation, is any plan to support analyzing performance bottleneck in go2sky?

THANK YOU 😄

register error fail to instance service

Describe the bug
Prompt the following information when I run

go2sky-gRPC2019/08/10 12:41:20 the id of service 'gin-server111' is 3
go2sky-gRPC2019/08/10 12:41:20 register error fail to instance service 
go2sky-gRPC2019/08/10 12:41:21 the id of instance '1948345b-bb29-11e9-853c-0a002700000c' id is 10

To Reproduce

	re, err := reporter.NewGRPCReporter("192.168.56.122:11800")
	if err != nil {
		log.Fatalf("new reporter error %v \n", err)
	}
	defer re.Close()
	tracer, err := go2sky.NewTracer("gin-server", go2sky.WithReporter(re))
	if err != nil {
		log.Fatalf("create tracer error %v \n", err)
	}
	tracer.WaitUntilRegister()
	gin.SetMode(gin.ReleaseMode)
	r := gin.New()
	r.GET("/user/:name", func(c *gin.Context) {
		name := c.Param("name")
		c.String(200, "Hello %s", name)
	})
	//Use go2sky middleware with tracing
	r.Use(Middleware(r, tracer))

Expected behavior
Register log reporter

Additional context

skywalking opa 6.2.0
ui 6.2.0
golang gin middleware
https://github.com/SkyAPM/go2sky/tree/master/plugins/gin

version: '3.3'
services:
  elasticsearch:
    #image: docker.elastic.co/elasticsearch/elasticsearch:6.4.3
    image: elasticsearch:6.4.3
    container_name: elasticsearch
    restart: always
    ports:
      - 9200:9200
      - 9300:9300
    environment:
      discovery.type: single-node
    ulimits:
      memlock:
        soft: -1
        hard: -1
  oap:
    image: apache/skywalking-oap-server:6.2.0
    container_name: oap
    depends_on:
      - elasticsearch
    links:
      - elasticsearch
    restart: always
    ports:
      - 11800:11800
      - 12800:12800
    environment:
      SW_STORAGE: elasticsearch
      SW_STORAGE_ES_CLUSTER_NODES: elasticsearch:9200
  ui:
    image: apache/skywalking-ui:6.2.0
    container_name: ui
    depends_on:
      - oap
    links:
      - oap
    restart: always
    ports:
      - 8080:8080
    environment:
      SW_OAP_ADDRESS: oap:12800

image

How to get the size and capacity send queue ?

场景:
动态调整上报采样率,在低上报时全采样,随着队列大小增加降低采样率。对于GRPCReporter可如下实现,LogReporter没有对应的实现。

`
// GetSendQueueSize return the size of send queue and the capacity of send queue
func GetSendQueueSize(reporter interface{}) (int, int) {

r, ok := (reporter).(gRPCReporter)
if !ok {
	return -1, -1
} else {
	return len(r.sendCh), cap(r.sendCh)
}

}
`

Support database query tracing using ocsql library

Is your feature request related to a problem? Please describe.
Currently go2sky doesn't automatically trace the database queries and its execution

Describe the solution you'd like
A wrapper which automatically traces the database query execution and produces the trace and links it to the parent span which can be a Http method/ any other

Additional context
ocsql warpper which provides tracing for tracing applications like zipkin and since skywalking uses different data format some code changes to the library will be required

Irregular use of go tag

Describe the bug

Upgrades that are not compatible in the go language should use the following forms: https://blog.golang.org/v2-go-modules instead of simply upgrading tags, this is because go get -u will upgrade the direct and indirect dependencies in the project. This will upgrade the indirect dependencies if possible Will cause code incompatibility, as shown below:

To Reproduce
Steps to reproduce the behavior:

I have 2 projects A, B, and B depends on A, A depends on [email protected]:

A: v1 (dep: go2sky)
B: v1 (dep: B)

when you use go get -u in B project. A(direct dependency) and go2sky (indirect dependencies) will upgrade to latest, If the v0.5.0 version of go2sky is not backward compatible, B project will get error

Expected behavior

Incompatible upgrades can follow golang version management specifications

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

register fail

go2sky-gRPC2019/06/19 17:57:22 register error fail to register service

Error while dialing dial tcp: lookup oap-skywalking: Temporary failure in name resolution

Describe the bug
A clear and concise description of what the bug is.

go2sky-gRPC2019/09/29 19:19:41 register error rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = "transport: Error while dialing dial tcp: lookup oap-skywalking: Temporary failure in name resolution"

To Reproduce
Steps to reproduce the behavior:

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

Is v0.5.0 not backward compatible?

Describe the bug

When I use [email protected] client to conection [email protected] server. I got this error:
report serviceInstance properties error rpc error: code = Unimplemented desc = Method not found ManagementService/reportInstanceProperties.

To Reproduce
Steps to reproduce the behavior:

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • go2sky client version: v0.5.0
  • skywalking server version: 6.6.0

Additional context
Add any other context about the problem here.

time out

go2sky-gRPC2019/06/11 17:27:39 register error rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: timed out waiting for server handshake

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.