subdivideGCPTypes only resolves kubernetes and GCE resource types.

This makes this library not usable from other environments such as GAE, or with a generic_task resource with additional lables.

Since the otel resource model doesn't have a special type field, perhaps a "special" label of "__resource_type" can use a literal value, instead of smart matching to a constant?

Current implementation of metric exporter doesn't consider the concurrent processes well and it is not as efficient as expected and may cause data race. The parts to check concurrency safety are:

• MetricDescriptor caching
• Batch conversion from Record to TimeSeries

Default span naming in the go exporter is Span.<kind>.<name>. According to @aabmass, it is just <name> for python and for javascript. Since the opentelemetry collector is written in golang, this means a python application's span naming will change if it sends to a collector vs if it sends directly to stackdriver.

Originally opened at open-telemetry/opentelemetry-go#460 by @shanna

Span events with string or bool attributes don't show in the stackdriver UI. I'm not 100% this is an open-telemetry stackdriver exporter issue or a stackdriver tracing issue. I don't have a standalone test case for you but it's easy to reproduce:

go.opentelemetry.io/otel v0.2.1
go.opentelemetry.io/otel/exporter/trace/stackdriver v0.2.1
Example:

// ...
span.AddEvent(ctx, "example-event",
core.Key("example-string").String("foo"),
core.Key("example-int").Int(123),
core.Key("example-bool").Bool(true),
)
All the keys appear in the stackdriver UI but only the value 123 is displayed, all other attributes are empty.
If I use the stdout exporter or just %+v the span I can see all the attributes exist.
The set of attributes appears to be valid according to Googles v2 documentation.

open-telemetry/opentelemetry-go#1303

Seems to have broken a bunch of imports

github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric imports
	go.opentelemetry.io/otel/api/global: module go.opentelemetry.io/otel@latest found (v0.15.0), but does not contain package go.opentelemetry.io/otel/api/global

For reasons (of my fault/ignorance) I can't seem to get pinned .14 v to work in my go.mod file

equivalent of stackdriver exporter for opencensus metrics.

I'm unable to get the Cloud Monitoring exporter working:

https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/blob/master/exporter/metric/README.md

There are appear to be a couple of issues in addition to my likely incompetence; I'm reasonable familiar with Cloud Monitoring and OpenCensus and this is my first dip into OpenTelemetry. Code at end based upon the README example. The code works with the stdout exporter.

I've exported:

• GOOGLE_APPLICATION_CREDENTIALS to a known working Service Account with roles/monitoring.admin
• PROJECT to a project with Stackdriver enabled

The module appears to have a bug using InstallNewPipeline(opts, nil):

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x972274]

goroutine 1 [running]:
go.opentelemetry.io/otel/sdk/metric/controller/push.New(0xbfa380, 0xc0001f08c0, 0xbebb60, 0xc000010740, 0xc00000fba0, 0x2, 0x2, 0x1)
	/home/dazwilkin/go/pkg/mod/go.opentelemetry.io/otel/[email protected]/metric/controller/push/push.go:56 +0x84
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric.NewExportPipeline(0xc0002dfc50, 0x1, 0x1, 0xc0002dfc88, 0x1, 0x1, 0x4d1c01, 0xc00000f800, 0xc0002dfb98)
	/home/dazwilkin/go/pkg/mod/github.com/!google!cloud!platform/opentelemetry-operations-go/exporter/[email protected]/metric.go:166 +0x1cf
github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric.InstallNewPipeline(0xc0002dfc50, 0x1, 0x1, 0xc0002dfc88, 0x1, 0x1, 0x7, 0x6, 0xc00a20)
	/home/dazwilkin/go/pkg/mod/github.com/!google!cloud!platform/opentelemetry-operations-go/exporter/[email protected]/metric.go:147 +0x6d
main.main()
	/home/dazwilkin/Projects/go-open-telemetry/main.go:25 +0xe1
exit status 2

This occurs in metric.go line 166 because the append results in len(opts)==2 with the second value being void:

pusher := push.New(
	checkpointer,
	exporter,
	append([]push.Option{
		push.WithPeriod(period),
	}, popts...)...,
)

Which then breaks push.go line 56 on the second apply:

for _, opt := range opts {
	opt.Apply(c)
}

I can fudge this by creating an non-nil popts:

opts := []cloudmonitoring.Option{
	cloudmonitoring.WithProjectID(projectID),
}
popts := []push.Option{}
pusher, err := cloudmonitoring.InstallNewPipeline(opts, popts...)

But this then results in a second failure (which bemuses me);

2020/09/09 14:49:49 [main] ProjectID: [[REDACTED]]
2020/09/09 14:49:49 Error during exporting TimeSeries: currently the aggregator is not supported: &{{0 0} 1 [1]}
2020/09/09 14:49:49 rpc error: code = InvalidArgument desc = Request was missing field timeSeries.

Code

package main

import (
	"context"
	"log"
	"math/rand"
	"os"
	"time"

	"go.opentelemetry.io/otel/api/metric"
	"go.opentelemetry.io/otel/label"
	"go.opentelemetry.io/otel/sdk/metric/controller/push"

	cloudmonitoring "github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric"
)

func main() {
	projectID := os.Getenv("PROJECT")
	if projectID == "" {
		log.Fatal("Environment variable `PROJECT` is required and was unset")
	}

	log.Printf("[main] ProjectID: %s", projectID)
	opts := []cloudmonitoring.Option{
		cloudmonitoring.WithProjectID(projectID),
	}
	popts := []push.Option{}
	pusher, err := cloudmonitoring.InstallNewPipeline(opts, popts...)
	if err != nil {
		log.Fatal(err)
	}
	defer pusher.Stop()

	meter := global.Meter("cloudmonitoring/example")

	counter := metric.Must(meter).NewInt64Counter("counter.foo")

	lemonsKey := label.Key("ex.com/lemons")
	labels := []label.KeyValue{
		lemonsKey.Int(10),
		label.Key("key").String("value"),
		label.String("dog", "Freddie"),
	}

	ctx := context.Background()

	s := rand.NewSource(time.Now().UnixNano())
	r := rand.New(s)

	counter.Add(ctx, r.Int63n(100), labels...)
}

The License boilerplate currently states that OpenTelemetry owns the code:

It should probably say that the code is owned by Google, e.g. see example.

Add trace exporter from opentelemetry-go

In order for developers to confirm easily whether the example is working as expected, add a script to create a Cloud Monitoring dashboard.

We tried to instrument http handlers using go.opentelemetry.io/contrib/instrumentation/net/http and got the following error.

2020/09/17 05:36:44 rpc error: code = InvalidArgument desc = Request was missing field timeSeries.
2020/09/17 05:37:44 rpc error: code = InvalidArgument desc = One or more TimeSeries could not be written: Field timeSeries[0].metric.labels had an invalid value of "http.scheme": Label key contains invalid characters.: timeSeries[0]; Field timeSeries[1].metric.labels had an invalid value of "http.server_name": Label key contains invalid characters.: timeSeries[1]; Field timeSeries[2].metric.labels had an invalid value of "http.host": Label key contains invalid characters.: timeSeries[2]

The version we use is 7f2025a.

Currently the second argument of Exporter.Export() is meant for the common resource among all records stored via a meter, and the resource is the counterpart of MonitoredResource in TimeSeries.

In OpenCensus, the interface for MonitoredResource is defined apart from the protobuf and controlled separately. Similarly to OpenCensus, OpenTelemetry exporter should have the extra care for MonitoredResource.

For now, it just returns "global" resource. (hard coded)

When running Stackdriver as an exporter for opentelemetry I'm seeing some latecy issues in HTTP requests.
In the example I'm running GET on https://google.com which normally takes 50-100ms. With Stackdriver exporter enabled the request averages between 5000-7000ms.

This is the code I'm running:

package main

import (
	"context"
	"log"
	"net/http"
	"os"

	texporter "github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace"
	"go.opentelemetry.io/otel/api/global"
	"go.opentelemetry.io/otel/plugin/httptrace"

	sdktrace "go.opentelemetry.io/otel/sdk/trace"
)

func main() {
	projectID := os.Getenv("GCP_PROJECT_ID")
	exporter, err := texporter.NewExporter(texporter.WithProjectID(projectID))
	if err != nil {
		log.Fatalf("texporter.NewExporter: %v", err)
	}
	tp, err := sdktrace.NewProvider(sdktrace.WithSyncer(exporter))
	if err != nil {
		log.Fatal(err)
	}
	global.SetTraceProvider(tp)

	tracer := global.TraceProvider().Tracer("example.com/trace")

	ctx := context.Background()
	ctx, span := tracer.Start(ctx, "Opentelemetry HTTP GET")

	client := http.DefaultClient

	req, _ := http.NewRequest("GET", "https://google.com", nil)
	ctx, req = httptrace.W3C(ctx, req)
	httptrace.Inject(ctx, req)

	_, err = client.Do(req)
	if err != nil {
		panic(err)
	}

	span.End()
}

My go.mod file:

module foo.com

go 1.14

require (
	github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace v0.2.0
	go.opentelemetry.io/otel v0.6.0
)

Trace overview:

See recent change open-telemetry/opentelemetry-go#1166 (issue: open-telemetry/opentelemetry-go#1072)

See comment: #105 (review)

Problem: Currently it exports all spans using a single go subroutine. In a high QPS environment you can increase the batch size but eventually it will result into rpc timeout.

Solution:

• Create multiple workers (go subroutines) and let each worker take spans from the queue to send it to the backend. See census-ecosystem/opencensus-go-exporter-stackdriver#246 and census-ecosystem/opencensus-go-exporter-stackdriver#213
• Make the number of workers configurable.
• Add tests to verify with N workers (N > 1).

Presently, trace context via HTTP header X-Cloud-Trace-Context (or the equivalent gRPC header) doesn't appear to be extracted from inbound requests.

OpenCensus equivalent: https://pkg.go.dev/go.opencensus.io/exporter/stackdriver/propagation

https://pkg.go.dev/go.opentelemetry.io/otel/propagators

We should add a checklicense target to validate licenses are appended to source files going forward. We can do something similar to Otel Go - see: https://github.com/open-telemetry/opentelemetry-go/blob/master/Makefile#L145

Note however that some code in this repo was authored by contributors who signed the OpenTelemetry CLA so we need to retain this different license info for some existing files.

Tag for exporter/trace/v0.2.1 is missing:

GOPROXY=direct go get github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/[email protected] results in invalid version: unknown revision exporter/trace/v0.2.1

Hello,

Is it possible to see span events on the google cloud trace UI? I have a span and I add events on it but I can't find this on cloud trace, Is it that cloud trace doesn't support viewing spans with events, or am I doing something wrong?

Readme should contain

• steps to install
• Usage Example,
• Link to full Example.

Hi! I'm trying to create a demo using OpenTelemetry with GCP monitoring backend.

When instrumenting with OpenTelemetry's async metric instuments, only ValueObserver seems to work, while SumObserver and UpDownSumObserver both give me the following error:

Metrics Accumulator error: rpc error: code = InvalidArgument desc = Request was missing field metricDescriptor.metricKind: The descriptor does not have the metric kind set.

Is there any plan on fixing this? I'd be happy to contribute.

Readme should contain.

• steps to install
• Usage Example,
• Link to full Example.

The sample as-is appears to not work:
https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/blob/master/exporter/metric/README.md

• See #94
• There is no (longer) a kv package (now label?)
• meter := pusher.Provider().Meter("...") vs. global.Meter("...") as used by other examples

Recommend: ensuring this sample is part of the package's sources so that it can be tested when the exporter is built.

I'm trying to use the metrics exporter, but getting an import error:

package go.opentelemetry.io/otel/sdk/export/metric/aggregator: cannot find package "go.opentelemetry.io/otel/sdk/export/metric/aggregator" in any of:
	/usr/local/Cellar/go/1.14.5/libexec/src/go.opentelemetry.io/otel/sdk/export/metric/aggregator (from $GOROOT)
	/Users/gloriazhao/go/src/go.opentelemetry.io/otel/sdk/export/metric/aggregator (from $GOPATH)
	/Users/gloriazhao/stackdriver-sandbox/src/go.opentelemetry.io/otel/sdk/export/metric/aggregator

Not 100% sure what the issue is, but at quick glance it seems like it's trying to import aggregator package but it should be aggregation package. Is there something I can do to fix it?

Oh I see - it looks like it was called aggregator in v0.4.0 but was changed to aggregation recently.

Update CheckpointSet iteration to use the exporter as KindSelector instead of explicitly passing CUMULATIVE. See this comment thread for more details: #73 (comment). Also note this is somewhat dependent on future changes to the SDK.

In addition to this, we should add integration tests to validate DELTAs are exported correctly as described here: #73 (review)

I tried installing a pipeline like this to export my metrics

b := ioutil.ReadFile(...)
pusher, err := mexporter.InstallNewPipeline(
        []mexporter.Option{
            mexporter.WithMetricDescriptorTypeFormatter(formatter),
            mexporter.WithInterval(time.Minute * 3),
            mexporter.WithMonitoringClientOptions(option.WithCredentialsJSON(b)),
        },
    )

However it failed because it could not find any credentials in the typical locations as described here. However, I would expect this to work as well and when I made some modifications to the code to not check using FindDefaultCredentials, I was able to export my metrics without issue.

Expected:
Support to pass credentials using option.WithCredentialsJSON(b), and similarly the family of option.WithCredentials* options.

Actual:
Application Default Credentials are required even if they are going to be overridden by MonitoringClientOptions.

https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/blob/master/exporter/trace/go.mod should be at https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/blob/master/go.mod instead

Currently the process to call CreateMetricDescriptor is synchronous and should block if the number of MetricDescriptors is large. (Note: calling CreateMetricDescriptor to existing MD is allowed, so not checking existing MD in Cloud Monitoring itself is fine.)

It should launch a new goroutine when the new MD is found and call the request asynchronously in the goroutine.

The current manual process is error prone: we need to create additional tags for Go module subdirectories but often forget about that (see #38, #69).

See #38 (comment) with links to existing opentelemetry-go scripts which we can reuse to address this.

I observe that the exporter has an opinion about span names, and I feel that it shouldn't.

Note that exporter/trace/trace_proto.go computes the span display name, which makes it impossible to use short, application-specific span names as enjoyed by other Cloud Tracing applications (e.g. a method name).

Suggestion for improvement:

1. introduce a formatter in the exporter options to generate the span display name.
2. Use the existing logic as the default formatter.

Need an analog of this convenience function for the GCP specific header X-Cloud-Trace-Context

Use the version number in trace/version.go, which is automatically updated by the release scripts, to set the g.co/agent/component attribute for traces to the value opentelemetry-go <ot_version>; google-cloud-<trace|metric>-exporter <exporter_version>

Use the version number in metric/version.go, which is automatically updated by the release scripts, to set the user agent string supplied to Cloud Monitoring to the same value.

For reference, see requirements doc

In either case, the user should be able to override this value by passing in a UserAgent string explicitly

Can you create a 0.2.0 tag for exporter/trace as well?

It looks like only "global" monitored resource is supported right now.

In the OpenCensus metric exporter, one can specify "generic_task" metric type.

Update metric and trace exporter to opentelemetry-go v0.6.0 compat

• Metric
• Trace

In current spec and implementation of opentelemetry-go, it doesn't return timestamps in aggregated data. (ref. https://pkg.go.dev/go.opentelemetry.io/[email protected]/sdk/export/metric/aggregator)

But, for Cloud Monitoring, timestamps (TimeInterval) are required for Point

In the current implementation (#24), I leave this with tentative implementation but this should be handled in the upstream or workarounds in metric exporter.

I'm trying to install both this and
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc
to trace gRPC calls in google cloud. Unfortunately, this module is based on otel v0.11 and that module requires v0.12.

I can get this module to work if I add this to my mod.go:
replace (
go.opentelemetry.io/otel v0.12.0 => go.opentelemetry.io/otel v0.11.0
)

That breaks the other module, though. When I try to go to v0.12.0, this module won't compile and I see the following errors:
# github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace vendor/github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace/cloudtrace.go:236:49: undefined: "go.opentelemetry.io/otel/sdk/trace".ProviderOption vendor/github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace/cloudtrace.go:236:75: undefined: "go.opentelemetry.io/otel/api/trace".Provider vendor/github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace/cloudtrace.go:241:2: undefined: global.SetTraceProvider vendor/github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace/cloudtrace.go:247:48: undefined: "go.opentelemetry.io/otel/sdk/trace".ProviderOption vendor/github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace/cloudtrace.go:247:74: undefined: "go.opentelemetry.io/otel/api/trace".Provider vendor/github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace/cloudtrace.go:252:13: undefined: "go.opentelemetry.io/otel/sdk/trace".NewProvider vendor/github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace/cloudtrace.go:252:67: cannot use exporter (type *Exporter) as type "go.opentelemetry.io/otel/sdk/export/trace".SpanExporter in argument to "go.opentelemetry.io/otel/sdk/trace".WithSyncer: *Exporter does not implement "go.opentelemetry.io/otel/sdk/export/trace".SpanExporter (missing ExportSpans method)

Create single readme or one for each example in example directory.
Readme should contain

• Brief description about the example.
• how to run
• Expected output.

See full description here: https://github.com/GoogleCloudPlatform/opentelemetry-operations-go/pull/84/files

I am having issues with otel version.
can you please update to go.opentelemetry.io/otel v0.10.0

I tried with go.mod replace

replace github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/trace v0.2.1 => github.com/gzhao408/opentelemetry-operations-go/exporter/trace v0.2.2-0.20200727182237-86360c493900

replace github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric v0.2.1 => github.com/gzhao408/opentelemetry-operations-go/exporter/metric v0.2.2-0.20200727182237-86360c493900

but getting error

go: finding module for package go.opentelemetry.io/otel/api/kv/value
../../../../go/pkg/mod/github.com/gzhao408/opentelemetry-operations-go/exporter/[email protected]/trace_proto.go:31:2: module go.opentelemetry.io/otel@latest found (v0.10.0), but does not contain package go.opentelemetry.io/otel/api/kv/value

Currently the metric exporter doesn't support MeasureKind (Distribution, Histgram, et al.) at all, which is the counterpart of DISTRIBUTION.

I have tried simple Counter example, it doesn't show up properly in metrics dashboard.
Both rate/delta aligners show constantly growing counter instead of rate.
I had a simple loop, incrementing counter every 100ms and it supposed to show rate of 10requests/sec, instead it seems to show something constantly growing (not a raw value though as well).

Change agent label to follow the GCP exporter spec:

Example of correct version:

opentelemetry-go 0.5.0; google-cloud-trace-exporter 0.1.0

Frequently when attempting to collect metrics I encounter this error

2020/10/19 15:37:59 rpc error: code = Internal desc = Internal error encountered. Please retry after a few seconds. If internal errors persist, contact support at https://cloud.google.com/support/docs.

and then in metric explorer the metrics are temporarily unavailable until the next time I run my program and don't see this error.

I believe this issue is similar to googleapis/python-monitoring#7 as the issue seems to be related to whenever I try to write time series that have different labels one run after the other, and metric descriptor creation requests are always sent despite them existing already.

This comment googleapis/python-monitoring#7 (comment) mentions that appropriate metric descriptors are created on creation of time series, and that it is not necessary to create metric descriptors with a separate call. The fix to the issue I linked has a different fix in which they check whether the metric descriptor exists first and if it does not then they create it.

I've tried not making the call to exportMetricDescriptors and that worked for me, so I believe that is the issue.

Steps to reproduce:
Change example/metric/example.go to use different labels at runtime depending on os.Args as such

	of := newObservedFloat(12.34)
	i, _ := strconv.ParseUint(os.Args[1], 10, 32)
	callback := func(_ context.Context, result metric.Float64ObserverResult) {
		v := of.get()
		result.Observe(v, olabels[i])
	}

	metric.Must(meter).NewFloat64ValueObserver("observer-a", callback)

	// Add measurement once an every 10 second.
	timer := time.NewTicker(10 * time.Second)
	for range timer.C {
		rand.Seed(time.Now().UnixNano())

		r := rand.Int63n(100)
		cv := 100 + r
		counter.Add(ctx, cv, clabels[i])

		r2 := rand.Int63n(100)
		ov := 12.34 + float64(r2)/20.0
		of.set(ov)
		log.Printf("Most recent data: counter %v, observer %v", cv, ov)
	}

Run go run example.go 1 && go run example.go 0
An error will appear after a couple of runs if not on the first

Convert Otel Resources to span attributes. Similar to jaeger exporter converts to tags.