influxcommunity / influxdb3-go Goto Github PK

Use Case

The Method on QueryIterator named AsPoints returns a pointer to an object of type PointValues. There is also the type Point.

Intuitively a developer could on first view expect a method named AsPoints to return an object of type or pointer to an object of type Point or a container holding objects of this type. When quickly coding Intellij IDE (And I imagine other IDEs as well) offers the Method AsPoints. Inattention to the actual return value can lead to this assumption. There are situations in which this misunderstood usage can compile, but at runtime leads to an NPE being thrown.

Expected behavior

Expect that a method returning a pointer or object of type PointValues to have a more intuitive name, when the return value can be confused with another type.

e.g.

1. func (i *QueryIterator) AsPointValues() *PointValues{...}
2. func (i *QueryIterator) AsValues() *PointValues{...}

Actual behavior

See Use Case section

Additional info

Changing the name of this method could of course break existing implementations. Adding this request for future discussion or review.

Use Case

Users who want to query data from InfluxDB Cloud, transform it, and then write it back into InfluxDB. For example users who want to perform downsampling need a way to query the data, downsample it, convert it back into line protocol, and then write it.

Expected behavior

Convert the query results into line protocol.

Actual behavior

Feature request, doesn't exist yet. Thank you!

Additional info

No response

Use Case

• When the type is explanatory, allow simple addition
  • see #42 (comment)

Expected behavior

I can easily add strings, integers, and floats. example

func (m *Point) AddStringField(k string, v string) *Point {
	m.AddField(k, v)
	return m
}

Actual behavior

It is not here.

Additional info

• relates #43

Use Case

In testing and inspection it's often handy to get a lineprotocol representation of how a Point will be written to Influxdb. This currently can be done by creating a string from the results of the method MarshalBinary. However this is not a friendly or intuitive solution.

Expected behavior

A method with a name like ToLineprotocol would be more intuitive.
e.g.

func (p *Point) ToLineprotocol() string {
	binArr, err := p.MarshalBinary(lineprotocol.Millisecond)
	if err != nil {
		panic(err)
	}
	return string(binArr)
}

Actual behavior

Currently I need to write a similar function in a local project, though ideally it should be a method for the Point type.

Additional info

No response

Because the client uses the (IOx native) Flight RPC API, and not FlightSQL, shouldn't the QueryType be SQL? It makes explaining the distinctions even more confusing--for example:

• "native" Flight API and FlightSQL are different APIs built on the same protocol
• you can use SQL with this client
• you're not using FlightSQL, although the default QueryType here suggests that you are:

Use Case

It seems that a common idiom in some implementations is translating from a local type to Point and vice versa.

e.g.

func (w *Widget)ToPoint(measurement string) *influxdb3.Point {...}
...
func WidgetFromPoint(point *influxdb3.Point) *Widget {...}

And this would apply as well to containers or streams of local objects. The Downsampling example encourages the use of PointValues when dealing with query results, however sticking to translations between just Point and LocalType is semantically simpler and cleaner. Since downsampling involves writing data back to the server and the write API uses Points, it would be handy to work directly with Points from query results, without exposing underlying PointValues.

Granted such a method could entail a slight performance hit, so this is not to suggest that the current approach should be entirely replaced.

Expected behavior

Since the Point type is preferred in the Write API it should also be easy to work with in the QueryAPI and this would include containers or streams of Point.

Actual behavior

The Query API's QueryIterator has a method AsPoints that returns *PointValues. This of course is not a Point container or stream. Furthermore it gives precedent to the PointValue type, which in the WriteAPI is hidden within Point

Additional info

No response

Use Case

• I want asserure that AddField() without panicking.
• avoiding reflect.
• Please show me the error while I am editing in VSCode.

Expected behavior

• use unsupported types, then compile error.

func example() {
	p := NewPoint("measurement", map[string]string{}, map[string]interface{}{"measurement": "air"}, time.Now())
        // unsupported type.
	p.AddFieldFromValue("✅ Compile error.", cmplx.Inf())
        // supported type.
	p.AddFieldFromValue("✅ This is safe.", NewValueFromInt(255))
}

Actual behavior

• use unsupported types, then panic.

func example() {
	p := NewPoint("measurement", map[string]string{}, map[string]interface{}{"measurement": "air"}, time.Now())
        // unsupported type.
	p.AddField("❌ Will panic.", cmplx.Inf())
        // supported type.
	p.AddField("❓This is OK, but really?", 255)
}

Additional info

• #42
• Which API do you prefer(NewValueFrom VS NewFieldFrom)?
  • NewValueFrom wins.

NewValueFrom

func (m *Point) AddFieldFromValue(k string, v lineprotocol.Value) *Point {}
func NewValueFromInt[I Integer](v I) lineprotocol.Value {}

(Retracted)NewFieldFrom

func (m *Point) AddFieldRaw(k string, v Field) *Point {}
func NewFieldFromInt[I Integer](v I) Field {}

Why not line-protocol?

• line-protocol v2 dont support generics yet.

Specifications

• Package Version: 0.6.0
• InfluxDB Version: Cloud

Code sample to reproduce problem

When result contains unsupported data type, the following calls may cause panic.

ok := iterator.Next()
// or
pv := iterator.AsPoints()

Expected behavior

Next() and AsPoints() should not panic, but somehow return error.

Next methods could be changed according to these iterator guidelines and AsPoints similarly. ie

func (i *QueryIterator) Next() (map[string]interface{}, error) // how about adding a new type (or alias) for the value?
func (i *QueryIterator) AsPoints() (*PointValues, error)

This renders Value method useless, btw.

So perhaps adding a new iterator type designed according to the guidelines, and deprecating the existing one would be a better solution.

Actual behavior

panic may occur

Additional info

No response

Specifications

Following the readme instructions, importing the module fails

Code sample to reproduce problem

go get github.com/influxcommunity/influxdb3-go

import (
  "context"
  "encoding/json"
  "fmt"
  "os"

  "github.com/InfluxCommunity/influxdb3-go/influx"
)

// actual code

go mod tidy

Expected behavior

"github.com/InfluxCommunity/influxdb3-go/influx" should get installed according to the readme instructions

Actual behavior

go: finding module for package github.com/InfluxCommunity/influxdb3-go/influx
go: found github.com/InfluxCommunity/influxdb3-go/influx in github.com/InfluxCommunity/influxdb3-go v0.0.0-20230607144859-c57855ac97f4
go: github.com/metrico/node-flight imports
github.com/InfluxCommunity/influxdb3-go/influx: github.com/InfluxCommunity/[email protected]: parsing go.mod:
module declares its path as: github.com/influxcommunity/influxdb3-go
but was required as: github.com/InfluxCommunity/influxdb3-go

Additional info

No response

Code coverage by unit tests is very low, and coverage checks for PRs from forks fails (some code is only tested by e2e test).

The query "helper" should iterate over all endpoints not only over first:

https://github.com/bonitoo-io/influxdb3-go/blob/93ccdcd8d9283bef43b3dc333501d458c2dba5d1/influx/query.go#L52

Use Case

Give the user option to process the result as deems best without any processing by the package itself. Such option exists in other client libs.

Expected behavior

Return "raw" query result.

Actual behavior

Only iterator over array(s) created by the implementation from flightsql reader is returned.

Additional info

No response

The new V3 management api lets the user manage databases and database tokens. Is it already in the roadmap to implement this?

https://app.circleci.com/pipelines/github/InfluxCommunity/influxdb3-go/84/workflows/202426f0-d44f-4b78-b0d1-caa315d60904/jobs/90

Use Case

Currently the Write API has the methods.

func (c *Client) WriteData(ctx context.Context, points ...interface{}) error { ... }

and

func (c *Client) WriteDataWithOptions(ctx context.Context, options *WriteOptions, points ...interface{}) error {...}

It would also be useful to be able to write batches/collections of a generic interface type.

Expected behavior

When I've a collection of objects of a local type, I would like to write them in one call using the points ...inteface{} idiom.

Actual behavior

Currently this requires repeated calls, in a loop to one of the methods mentioned under Use Case

Additional info

I've made a simple sequential implementation of this locally.

func WriteBatchDataSeq[T any](client *influxdb3.Client,
	ctx context.Context,
	opts *influxdb3.WriteOptions,
	ifaces []T) error {

	for _, iface := range ifaces {
		fmt.Printf("%s %v\n", reflect.TypeOf(iface).Kind().String(), iface)
		err := client.WriteDataWithOptions(ctx, opts, iface)
		if err != nil {
			return err
		}
	}
	return nil
}

If its not specified use "global" value from ClientConfigs.

Change order of arguments:

func (c *Client) Query(ctx context.Context, query string, database string, queryParams ...string) (*QueryIterator, error) {
	return c.queryWithType(ctx, database, query, "sql", queryParams...)
}

Rename queryParams to metadata - the query params will be used for parameters in sql query (prepared statement):

func (c *Client) Query(ctx context.Context, query string, database string, metadata ...string) (*QueryIterator, error) {
	return c.queryWithType(ctx, database, query, "sql", queryParams...)
}

Use Case

time.Time is Golang native type for time/timestamp. Now user has to do this:

	value := iterator.Value()
	timestamp := value["time"].(arrow.Timestamp).ToTime(arrow.Nanosecond)

Expected behavior

Golang type for timestamp should be time.Time

Actual behavior

Time is of arrow.Timestamp

Additional info

No response

Specifications

• Package Version: v0.6.0
• InfluxDB Version: V3
• Platform:

Code sample to reproduce problem

WriteData(context.Background(), []any{measurement})

Some measurement has empty tags and some don't. The Write method should use a logger to warn for the empty values but instead it prints the warning which we cannot control over. This will result in lot of logs.

Expected behavior

Should only print the empty tag as warning

Actual behavior

2024-04-02T18:53:19.013+02:00	Empty tags has no effect, tag [*****], measurement [*****]
2024-04-02T18:53:19.013+02:00	Empty tags has no effect, tag [*****], measurement [*****]
2024-04-02T18:53:19.013+02:00	Empty tags has no effect, tag [*****], measurement [*****]
2024-04-02T18:53:19.013+02:00	Empty tags has no effect, tag [*****], measurement [*****]
2024-04-02T18:53:19.013+02:00	Empty tags has no effect, tag [*****], measurement [*****]

Additional info

Code block https://github.com/InfluxCommunity/influxdb3-go/blob/main/influxdb3/point_values.go#L73
No response

Use Case

Comparing the iterator with Java iterators I was bothered that there is now HasNext() method to peek as to whether it contained an empty result. This lead to inspecting the underlying ipc.Reader which has a Read() method returning rec, err and error can be EOF. It would be nice if QueryIterator simply wrapped this method in its own Read() method, without having to go directly to Raw().

Expected behavior

Would like to be able to detect empty response from the iterator before attempting iteration. For example in Java an iterator has a HasNext() method.

Actual behavior

Without using Raw(), currently just start with iterator.Next() without peeking first to see that the iterator response has any next record to get.

Using Raw().Read() returns rec, err and the err can be EOF which kind of provides the desired check. It would be nicer though to have Read()(record, err) as a direct Method of QueryIterator.

Additional info

No response

Specifications

• Package Version: v0.5.0
• InfluxDB Version: 2.0 Serverless
• Platform: Mac

Code sample to reproduce problem

INFLUXDB_URL, INFLUXDB_TOKEN, INFLUXDB_DATABASE exported in bash before run the code.

package main

import (
	"context"
	"fmt"
	"os"

	"github.com/InfluxCommunity/influxdb3-go/influxdb3"
)

func main() {
	// Use env variables to initialize client
	url := os.Getenv("INFLUXDB_URL")
	token := os.Getenv("INFLUXDB_TOKEN")
	database := os.Getenv("INFLUXDB_DATABASE")

	// Create a new client using an InfluxDB server base URL and an authentication token
	influxdb3Client, err := influxdb3.New(influxdb3.ClientConfig{
		Host:     url,
		Token:    token,
		Database: database,
	})

	if err != nil {
		panic(err)
	}

	// Close influxdb3Client at the end and escalate error if present
	defer func(influxdb3Client *influxdb3.Client) {
		err := influxdb3Client.Close()
		if err != nil {
			panic(err)
		}
	}(influxdb3Client)

	// Prepare FlightSQL query
	query := `
			SELECT *
			FROM "BUCKET_NAME"
			WHERE
			time >= now() - interval '7 day'
		`

	iterator, err := influxdb3Client.Query(context.Background(), query)
	fmt.Printf("\niterator: %v\n", iterator)

	if err != nil {
		panic(err)
	}
}

ERROR:

iterator: <nil>
panic: flight reader: rpc error: code = NotFound desc = Database '984e6e8db0df2d3a_b9b50c56286d6cf2' not found

goroutine 1 [running]:
main.main()
        /Users/rogeriocassares/Git/OpenDataTelemetry/timeseries-api/cmd/app/main.go:73 +0x340
exit status 2

Expected behavior

Query the data

Actual behavior

iterator:

[Error ](panic: flight reader: rpc error: code = NotFound desc = Database '984e6e8db0df2d3a_b9b50c56286d6cf2' not found)

Additional info

No response

Specifications

• Package Version: 2.0.0
• InfluxDB Version: latest
• Platform: N/A

"github.com/InfluxCommunity/influxdb3-go/influxdb3" v2.0.0 introduces changes that are not reflected in the current Getting Started documentation on the influx cloud under /new-user-setup/golang

Code sample to reproduce problem

Expected behavior

Influx Cloud should be updated to use the updated names and ClientConfig, ie:

package main

import (
	"github.com/InfluxCommunity/influxdb3-go/influxdb3"
	"os"
)

func main() {
	// Create client
	url := "https://us-east-1-1.aws.cloud2.influxdata.com"
	token := os.Getenv("INFLUXDB_TOKEN")

	// Create a new client using an InfluxDB server base URL and an authentication token
	client, err := influxdb3.New(influxdb3.ClientConfig{
		Host:  url,
		Token: token,
	})

	if err != nil {
		panic(err)
	}
	// Close client at the end and escalate error if present
	defer func(client *influxdb3.Client) {
		err := client.Close()
		if err != nil {
			panic(err)
		}
	}(client)

	database := "events"
        ...
}

Actual behavior

Current code examples do not work with latest golang library for influx-go

Additional info

No response

Specifications

InfluxCloud 3.0

Code sample to reproduce problem

If my URL is
--build-arg INFLUXDB_URL=us-east-1-1.aws.cloud2.influxdata.com:443
Then the following script works

package main

import (
	"context"
	"fmt"
	"os"

	"github.com/InfluxCommunity/influxdb3-go/influx"
)

type Value map[string]interface{}

var lines []string // Array to store lines in influx line protocol format

func main() {
	// Use env variables to initialize client
	url := os.Getenv("INFLUXDB_URL")
	token := os.Getenv("INFLUXDB_TOKEN")
	database := os.Getenv("INFLUXDB_DATABASE")
	fmt.Printf("url", url)

	// Create a new client using an InfluxDB server base URL and an authentication token
	client, err := influx.New(influx.Configs{
		HostURL:   url,
		AuthToken: token,
	})

	if err != nil {
		panic(err)
	}
	// Close client at the end and escalate error if present
	defer func(client *influx.Client) {
		err := client.Close()
		if err != nil {
			panic(err)
		}
	}(client)
	
	// Prepare FlightSQL query
		query := `
	    SELECT *
	    FROM "cpu"
	  `

		iterator, err := client.Query(context.Background(), database, query)

		if err != nil {
			panic(err)
		}

		// Specify your configuration here
		measurement := "cpu_test"
		timestamp := "time"
		tags := []string{"host", "cpu"}
		fields := []string{"usage_user"}

		for iterator.Next() {
			value := iterator.Value()

			// Collect tag set
			var tagSet []string
			for _, tag := range tags {
				tagSet = append(tagSet, fmt.Sprintf("%s=%v", tag, value[tag]))
			}

			// Collect field set
			var fieldSet []string
			for _, field := range fields {
				fieldSet = append(fieldSet, fmt.Sprintf("%s=%v", field, value[field]))
			}

			// Here we convert each value into InfluxDB line protocol format.
			line := fmt.Sprintf("%s,%s %s %v",
				measurement,
				strings.Join(tagSet, ","),
				strings.Join(fieldSet, ","),
				value[timestamp])

			fmt.Println(line)
			lines = append(lines, line) // Append the line to lines array
		}

However I can't perform:

	line := fmt.Sprintf("stat,unit=temperature avg=%f,max=%f", 23.5, 45.0)
	err = client.Write(context.Background(), database, []byte(line))
	if err != nil {
		panic(err)
	}

I get the following error:
panic: error calling us-east-1-1.aws.cloud2.influxdata.com:///api/v2/write?bucket=demo&org=&precision=ns: Post "us-east-1-1.aws.cloud2.influxdata.com:///api/v2/write?bucket=demo&org=&precision=ns": unsupported protocol scheme "us-east-1-1.aws.cloud2.influxdata.com"
I need to change the url to
https://us-east-1-1.aws.cloud2.influxdata.com for writes.
It would be great if on url could be provided.
Thank you!

Expected behavior

Expect to be able to provide a single url:
us-east-1-1.aws.cloud2.influxdata.com

Actual behavior

The write and query method require different urls.
Query requires: us-east-1-1.aws.cloud2.influxdata.com:443
Write requires: https://us-east-1-1.aws.cloud2.influxdata.com

Additional info

Thank you!!!