Git Product home page Git Product logo

gogm's People

Contributors

erictg avatar fbiville avatar ggrossetie avatar mjhatamy avatar nikitawootten avatar serafdev avatar shihai1991 avatar tobikris avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

gogm's Issues

Improve how properties are represented, add validation

Currently, in order to represent properties in GoGM, the field must have the type map[string]interface{}

This is to simplify the process of reading/saving the properties, but being able to specify the type of the map fully (e.g. map[string]string) would be quicker, cleaner, and safer.

Many to many of the same type not deserializing properly

De-serialization of Nodes with incoming and outgoing of the same type not working properly.

type F Struct {
  Parents []*F `gogm:"relationship=test;direction=outgoing"`
  Children []*F `gogm:"relationship=test;direction=incoming"`
}

F1->F2->F3 for example.

F1 should have parent to F2
F2 should have child to F1 and Parent to F3
F3 should have child to F2

Instead F2 has Parent to F1 and F2.

Essentially the bug is that it doesn't write to the correct field in a M2M of the same type

Gogm does not handle embedded structs

Add functionality to support embedded struct processing for mapped structs.

This includes:

  • processing config on struct
  • converting struct to query
  • converting query result back to struct

load config not found for node [dc994ea0-d5b6-4cbc-b597-53d335b409b9]

I created some node event with [ff733301-4d3c-4892-8bfe-7d2c440edb43] by err = sess.Save(&event).
Then I created another node action [dc994ea0-d5b6-4cbc-b597-53d335b409b9] which is connected with event by some relation and saved branch event--HAS->Action by err = sess.SaveDepth(&event, 2) - all is ok.

But when I tried to create another pair (with different UUIDs) event--HAS->Action and saved it by err = sess.SaveDepth(&event, 2) I got

load config not found for node [dc994ea0-d5b6-4cbc-b597-53d335b409b9]

... but [dc994ea0-d5b6-4cbc-b597-53d335b409b9] is UUID from previous pair.

What should I check in this case?

Cypher v4 compatibility: remove usages of {param} parameters

{param} parameters are deprecated in Neo4j 3.x and have been removed entirely as of Neo4j 4.0 (see https://neo4j.com/docs/cypher-manual/current/deprecations-additions-removals-compatibility/#cypher-deprecations-additions-removals-4.0). There are usages of the old {param} syntax throughout the project that should be replaced with the new $param syntax.

Some concrete examples that cause failures (probably not all of them):

This issue also affects any usages of parameters from within the DSL (mindstand/go-cypherdsl):

Connectivity test query should take configured target DBs into account

Currently, ""return 1"" is sent against the server to test the connectivity and verify the server version.

The session used to run execute this statement targets the default database, which may not be accessible by the configured user.

A possible fix could be to try executing the test query against every configured target database until a successful execution occurs.

Pagination validation contradict with ordering validation

I assume there is an error in Validate function for pagination or in the logic of LoadAllDepthFilterPagination. To build query you need to pass parameters for ordering and paging (LoadAllDepthFilterPagination), but Validate function has such implementation

func (p *Pagination) Validate() error {
	if p.PageNumber >= 0 && p.LimitPerPage > 1 && p.OrderByField != "" && p.OrderByVarName != "" {
		return errors.New("pagination configuration invalid, please double check")
	}

	return nil
}

You can pass parameters for ordering or paging, but not both. In other case you should use LimitPerPage = 0 to pass validation.
As a result you can get only an error or empty result in such implementation:

//if the query requires pagination, set that up
	if pagination != nil {
		err := pagination.Validate()
		if err != nil {
			return err
		}

		query = query.
			OrderBy(dsl.OrderByConfig{
				Name:   pagination.OrderByVarName,
				Member: pagination.OrderByField,
				Desc:   pagination.OrderByDesc,
			}).
			Skip(pagination.LimitPerPage * pagination.PageNumber).
			Limit(pagination.LimitPerPage)
	}

Results:

  • Without pagination: sess.LoadAllDepthFilterPagination(&sensors, 1, cond, nil, &gogm.Pagination{OrderByField: "name", OrderByVarName: "n"}) - query works but has LIMIT and SKIP equal 0 - MATCH p=(n)-[*0..1]-() WHERE ... RETURN p ORDER BY n.name SKIP 0 LIMIT 0
  • Without ordering: sess.LoadAllDepthFilterPagination(&sensors, 1, cond, nil, &gogm.Pagination{LimitPerPage: 100, PageNumber: 0}) - OrderBy gives error - errors found: name and member have to be defined -- total errors (1)
  • With ordering and paging - sess.LoadAllDepthFilterPagination(&sensors, 1, cond, nil, &gogm.Pagination{LimitPerPage: 100, PageNumber: 0, OrderByField: "name", OrderByVarName: "n"} - pagination validation gives error - pagination configuration invalid, please double check

Bad errors

Ensure errors are wrapped adding more information. This should make debugging easier and potentially save people hours of development time :).

As an example, GoGM returns ErrNotFound even when the issue is invalid cypher syntax.

Ensure typedef properties works

Ensure the type def properties work correctly, and provide gogm typdefs for properties and any other strictly enforced type.

gogm save depth fails to create nodes connected by slice to slice (many to many) relationships

When saving a node to a certain depth, connected nodes with slice to slice relationships will not be created.

For example:

Consider the node types:

type A struct {
	Id       int64   `json:"-" gogm:"name=id"`
	UUID     string  `json:"uuid" gogm:"pk;name=uuid"`
	//relationship
	B_REL    []B     `gogm:"direction=outgoing;relationship=SOME_REL"`
}

type B struct {
	Id       int64   `json:"-" gogm:"name=id"`
	UUID     string  `json:"uuid" gogm:"pk;name=uuid"`
	//relationship
	A_REL    []A     `gogm:"direction=incoming;relationship=SOME_REL"`
}

When creating a simple subgraph:

nodeA1 := A{
	B_REL: []B{},
}

nodeB1 := B{
	A_REL: []A{nodeA1},
}

nodeB2 := B{
	A_REL: []A{nodeA1},
}

nodeA1.B_REL = []B{nodeB1, nodeB2}

gogm_session.SaveDepth(&nodeB1, 3)

The expected behavior would be to create nodes like this: nodeB1 <- nodeA1 -> nodeB2. Instead gogm stops once it reaches node b1: nodeB1 <- nodeA1.

Error in readme that creates bug

Hi, just tested this library and found an error in the provided readme that is a bit crucial.

type VertexA struct {
	// provides required node fields
	gogm.BaseNode

	TestField         string     `gogm:"name=test_field"`
	TestTypeDefString tdString   `gogm:"name=test_type_def_string"`
	TestTypeDefInt    tdInt      `gogm:"name=test_type_def_int"`
	SingleA           *VertexB   `gogm:"direction=incoming;relationship=test_rel"`
	ManyA             []*VertexB `gogm:"direction=incoming;relationship=testm2o"`
	MultiA            []*VertexB `gogm:"direction=incoming;relationship=multib"`
	SingleSpecA       *EdgeC     `gogm:"direction=outgoing;relationship=special_single"`
	MultiSpecA        []*EdgeC   `gogm:"direction=outgoing;relationship=special_multi"`
}
type VertexB struct {
	// provides required node fields
	gogm.BaseNode

	TestField  string     `gogm:"name=test_field"`
	TestTime   time.Time  `gogm:"name=test_time"`
	Single     *VertexA   `gogm:"direction=outgoing;relationship=test_rel"`
	ManyB      *VertexA   `gogm:"direction=incoming;relationship=testm2o"`
	Multi      []*VertexA `gogm:"direction=outgoing;relationship=multib"`
	SingleSpec *EdgeC     `gogm:"direction=incoming;relationship=special_single"`
	MultiSpec  []*EdgeC   `gogm:"direction=incoming;relationship=special_multi"`
}

In the type VertexB the line: ManyB *VertexA gogm:"direction=incoming;relationship=testm2o"
should be changed to ManyB *VertexA gogm:"direction=outgoing;relationship=testm2o"

Thanks for a great library that is maturing!

Support mapping IDs other than UUID

Currently, GoGM requires entities to include a UUID property, required to identify them.
This can be an issue in read-only scenarios e.g., where the configured user does not have the proper permissions to alter the graph.

Object loading (by id) performance - neo4j indexing of properties

There is some performance impact when loading an object with own key policy (non neo4j id).
The Load functions loads objects by id and creates such query:

 MATCH p=(n)-[*0..1]-() WHERE n.seId = $idprm RETURN p

seId is id from key policy.
Indexes for properties in neo4j include label.
To filter by index you need to set a label in a query.
For example, my test graph contains 2821689 nodes:

sess.Load(context.Background(), sensor, sensor.SeId)
// Generates MATCH p=(n)-[*0..1]-() WHERE n.seId = $idprm RETURN p 
// it takes about 2 seconds -  index is not used

whereFilter = gocypherdsl.C(&gocypherdsl.ConditionConfig{Label: "Sensor", Name: "n"})
b := whereFilter.(*gocypherdsl.ConditionBuilder)
sess.LoadDepthFilter(context.Background(), sensor, sensor.SeId, 1, b, nil) 
// Generates 'MATCH p=(n)-[*0..1]-() WHERE n:Sensor AND n.seId = $idprm RETURN p' 
// it takes about 5-10 ms - index is used 

In current implementation the Load function is useful for very small graphs (when you use own key).
Why does not GOGM add label based on struct name? Is there a problem with this?

Add context support

Calling New and NewSession, do not allow for contexts to be passed in, from what I can see. It should probably be added since these are blocking calls, and probably be apart of a v3 release.

Maybe a NewContext to avoid a new release.

Different types for filter parameters

Function for Load... and LoadAll... have different types for the filter parameter (dsl.ConditionBuilder/dsl.ConditionOperator):

//load with depth and filter
LoadDepthFilter(ctx context.Context, respObj, id interface{}, depth int, filter *dsl.ConditionBuilder, params map[string]interface{}) error
//load all of type with depth and filter
LoadAllDepthFilter(ctx context.Context, respObj interface{}, depth int, filter dsl.ConditionOperator, params map[string]interface{}) error

I need to cast to dsl.ConditionBuilder in a code:

whereFilter = gocypherdsl.C(&gocypherdsl.ConditionConfig{Label: "...", Name: "n"}).And(...).And(...)
sess.LoadAllDepthFilter(context.Background(), &sensors, 1, whereFilter, nil) // OK - no need to cast
b := whereFilter.(*gocypherdsl.ConditionBuilder)
sess.LoadDepthFilter(context.Background(), sensor, sensor.SeId, 1, b, nil) // Need to cast

I think the parameters should be of the same type, and most likely it should be dsl.ConditionOperator.

Pooled driver not recycling dead connections

Pooled driver is not recycling dead connections, causing queries to fail with err driver bad connection.

Not sure if this is in both the normal pooled driver and the routing driver. This needs more investigation to determine a solution.

Session.QueryRaw() result not serializable with new driver

Session.QueryRaw() returns a 2D array of empty interfaces. With the old driver (go-bolt), nodes would cast from type go-bolt/structures/graph.Node, with serializable NodeIdentity, Labels, and Properties members.

Using the new official driver, nodes are cast from neo4j.node, a private type that does not expose the underlying Id, Labels, and Props members (which are analogous to the old driver's members), resulting in an empty object when marshaling the raw result to JSON. All data is still accessible through accessor methods from the neo4j.Node interface.

This behavior breaks previous workflows expecting QueryRaw() to return data serializable by the JSON library.

invalid memory address or nil pointer dereference

Hi, I'm getting constant panics with specified error message. By looking at stack trace it's obvious that it's not on my side, because I do check everything to not be nil, before calling any gogm methods. It happens only on session.QueryRaw or session.Query functions. As a note I also check session for nil value before calling any of this functions, but still panics are happening. Please take a look at it, as it's critical bug.

runtime error: invalid memory address or nil pointer dereference

goroutine 155 [running]:
runtime/debug.Stack(0x1, 0x0, 0x0)
	/usr/local/go/src/runtime/debug/stack.go:24 +0x9f
runtime/debug.PrintStack()
	/usr/local/go/src/runtime/debug/stack.go:16 +0x25
panic(0xac42c0, 0xfd92b0)
	/usr/local/go/src/runtime/panic.go:969 +0x1b9
github.com/neo4j/neo4j-go-driver/neo4j.(*result).doFetch(0xc00060fdd0, 0xae28a0)
	/home/bezhan/Programming/go/pkg/mod/github.com/neo4j/neo4j-go-driver@v1.8.1-0.20200622090208-4295b59525c9/neo4j/result.go:73 +0x32
github.com/neo4j/neo4j-go-driver/neo4j.(*result).Next(0xc00060fdd0, 0x1b0)
	/home/bezhan/Programming/go/pkg/mod/github.com/neo4j/neo4j-go-driver@v1.8.1-0.20200622090208-4295b59525c9/neo4j/result.go:96 +0x146
github.com/mindstand/gogm.(*Session).QueryRaw(0xc00008f590, 0xc00071f500, 0x1b0, 0xc000325320, 0x1b0, 0xb2a940, 0xc00032bba0, 0x0, 0x0)

...

This is how I call QueryRaw function and panics happen:

if sess == nil {
    return
}
sess.QueryRaw("MATCH ...", map[string]interface{}{}) // panic happens on this line

Missing Documentation

I'm interested in this package i would like to learn more about it but there is no readme yet.

clean up for wider community usage

Clean up for wider community usage

This is from awesome-go contributing:

Quality standards

To be on the list, project repositories should adhere to these quality standards (https://goreportcard.com/report/github.com/ github_user / github_repo):

  • Code functions as documented and expected
  • Generally useful to the wider community of Go programmers
  • Actively maintained
    • Regular, recent commits
    • Or, for finished projects, issues and pull requests are responded to
  • Stable or progressing toward stable
  • Thoroughly documented (README, godoc comments, etc.) in english language, so everyone is able to understand the project's intention and how it works
  • Tests, where practical. If the library/program is testable, then coverage should be >= 80% for non-data-related packages and >=90% for data related packages. Notice: the tests will be reviewed too. We will check your coverage manually if your package's coverage is just a benchmark results.

Array properties in nodes

Good day. Can you please tell me how you can organize a property in the form of an array using GOGM? Like this

image

Parameter syntax not supported?

Got this error when trying to initialize and save some basic options (following example)

ERRO[0011] Server error: [Neo.ClientError.Statement.SyntaxError] The old parameter syntax `{param}` is no longer supported. Please use `$param` instead (line 1, column 8 (offset: 7))
"UNWIND {rows} as row MERGE (n:`AddressObject`{uuid:row.uuid}) SET n += row RETURN row.uuid AS uuid, ID(n) AS id"

I have only one object type, and no relationships yet.

type AddressObject struct {
	gogm.BaseNode
	Type string `gogm:"name=type`
	Name string `gogm:"name=name`
	Description string `gogm:"name=description`
	Value string `gogm:"name=value`
}

This was tested with the latest v2-dev branch, as I had other issues with the master branch.

test decoder in sad path scenarios

Decoder works great when it gets its edges, nodes, and start nodes. Iv'e run into issues where if we're fetching a single node, it freaks out. This needs to be tested and fixed.

Error using BaseNode on default primary key loading strategy

Error when using the default primary key loading strategy:

panic: failed to init gogm instance, struct defined type (int64) different than register pk type (int)

Changing pk type to be an int only causes the same error to happen when decoding a query result.

Object fields not saved?

I have defined an object as shown

type AddressObject struct {
	gogm.BaseNode
	Type string `gogm:"name=type`
	Name string `gogm:"name=name`
	Description string `gogm:"name=description`
	Value string `gogm:"name=value`
}

I create the object like this...

destAddrObj := &AddressObject{
    Name: destinationAddress,
    Type: "AddressGroup",
    Value: destinationAddress,
}

and when I save with sess.Save(destAddrObj) - the object is created with a UUID, but it has no fields. I'm not sure what I'm doing wrong here or if I've found a bug. Would love a pointer, this library looks like exactly what I need.

obscure errors

Hi, recently I started working with gogm package and got stuck with some obscure errors that pop out on some of the actions and not on others. So, this are list of the errors that pop out and aren't very well documented and don't report the issue well:

  1. Invalid handle (comes from Query function)
  2. Invalid struct size: 191 (comes from SaveDepth function, size number changes from time to time)
  3. read tcp 10.8.0.58:47700->128.199.56.38:6687: read: connection reset by peer (comes from Query function, I know that it's related to neo4j connection, but not sure why it's happening)
  4. Unconsumed data (comes from Query function)
  5. EOF (comes from Query function)
  6. Map key is not string type: int64 (comes from Query functions, from BaseNode.LoadMap field)

After doing some research, I've found out that most of the errors are just being redirected from neo4j-go-driver package, so my enhancement proposal is to add more context related info to the errors (like with errors.Wrapf or errors.WithMessagef), because it's really hard to understand the meaning of errors or why they are happening.

This are the queries that I run:

errors: 1, 3, 4, 5, 6)

MATCH (n:ReferenceInstantValue {name:"{{.Name}}", beginReferencePeriod:"{{.BeginReferencePeriod}}", endReferencePeriod:"{{.EndReferencePeriod}}", uniqId:"{{.UniqID}}"})-[:PROVIDED_BY]->(sensor) WHERE sensor.name="{{.Sensor.Name}}" RETURN n.uniqId, n.endReferencePeriod, n.beginReferencePeriod, n.startVersion, n.name, n.endVersion, n.canonicalValuesNames, n.fieldsNames, n.tableName

errors: 2

MATCH (n:ReferenceInstantValue {name:"{{.Name}}", uniqId:"{{.UniqID}}"}),(m:Sensor {name:"{{.Sensor.Name}}"}) WHERE n.beginReferencePeriod="{{.BeginReferencePeriod}}" AND n.endReferencePeriod="{{.EndReferencePeriod}}" AND m.uniqId="{{.Sensor.UniqID}}" CREATE (n)-[r:PROVIDED_BY]->(m) RETURN type(r)

If this explanation doesn't help a lot, please comment on this issue, and I will post more information.

Default omited fields to zero value

When serializing params from map into node type, default omitted fields to zero value... Current implementation returns "unknown field [x]"

Update conn.go file to support Special Characters in password

If the password contains special characters, especially' #', the url.Parse will fail in Bolt Driver, because,
golang/ net/url/url.go code for Parse function uses this character for splitting purposes.

--- gogm/config.go
From line: 78
// ConnectionString builds the neo4j bolt/bolt+routing connection string
func (c *Config) ConnectionString() string {
var protocol string
if c.IsCluster {
protocol = "bolt+routing"
} else {
protocol = "bolt"
}
password := url.QueryEscape(c.Password)
return fmt.Sprintf("%s://%s:%s@%s:%v", protocol, c.Username, password, c.Host, c.Port)
}

TypeDef primitives as value of maps cause failure

Having a type deffed int for example causes panic. Probably need to check in the save function to see if it is type deffed. Not sure how to handle this from the perspective of loading. This might also need to be fixed in the driver. @nikitawootten what do you think?

Internal Error(*errors.errorString):driver: bad connection

call Save with large data panic

neo4j_test.go:14: delErr=(Couldn't read expected bytes for message length. Read: 0 Expected: 2.
    Internal Error(*errors.errorString):driver: bad connection



     Stack Trace:

    goroutine 52 [running]:
    runtime/debug.Stack(0x917ce3, 0x46, 0xc0002177d0)
            D:/sdk/go1.13amd64/go/src/runtime/debug/stack.go:24 +0xa4
    github.com/mindstand/golang-neo4j-bolt-driver/errors.Wrap(0x99d580, 0xc00004c3a0, 0x917ce3, 0x46, 0xc0002177d0,

0x1, 0x1, 0x99d580)
D:/sdk/gopath/pkg/mod/github.com/mindstand/[email protected]/e
rrors/errors.go:38 +0x18d
github.com/mindstand/golang-neo4j-bolt-driver/encoding.Decoder.read(0x99d7c0, 0xc00011cc30, 0xc0007aa210, 0x40d0
6f, 0x30, 0x8aa500)
D:/sdk/gopath/pkg/mod/github.com/mindstand/[email protected]/e
ncoding/decoder.go:45 +0x3d9
github.com/mindstand/golang-neo4j-bolt-driver/encoding.Decoder.Decode(0x99d7c0, 0xc00011cc30, 0xc0007aa210, 0xc8
b00c, 0x2, 0xc00037e004, 0x3006b0)
D:/sdk/gopath/pkg/mod/github.com/mindstand/[email protected]/e
ncoding/decoder.go:93 +0x4a
github.com/mindstand/golang-neo4j-bolt-driver.(*BoltConn).consume(0xc00011cc30, 0xc00079f230, 0x877940, 0xc0007a
a030, 0x0)
D:/sdk/gopath/pkg/mod/github.com/mindstand/[email protected]/c
onn.go:515 +0xa4
github.com/mindstand/golang-neo4j-bolt-driver.(*BoltConn).consumeAll(0xc00011cc30, 0x877940, 0xc0007aa030, 0x0,
0x0, 0x0, 0x6d, 0x6d)
D:/sdk/gopath/pkg/mod/github.com/mindstand/[email protected]/c
onn.go:541 +0xa3
github.com/mindstand/golang-neo4j-bolt-driver.(*BoltConn).sendRunPullAllConsumeAll(0xc00011cc30, 0xc00019e230, 0
x6c, 0xc0005c63f0, 0x6d, 0x1, 0x6d, 0x1, 0x2, 0x1, ...)
D:/sdk/gopath/pkg/mod/github.com/mindstand/[email protected]/c
onn.go:667 +0xef
github.com/mindstand/golang-neo4j-bolt-driver.(*boltStmt).ExecNeo(0xc0002e9b40, 0xc0005c63f0, 0x85ab00, 0x1, 0xc
000406801, 0x8)
D:/sdk/gopath/pkg/mod/github.com/mindstand/[email protected]/s
tmt.go:99 +0x7f
github.com/mindstand/golang-neo4j-bolt-driver.(*BoltConn).ExecNeo(0xc00011cc30, 0xc00019e230, 0x6c, 0xc0005c63f0
, 0x0, 0x0, 0x0, 0x0)
D:/sdk/gopath/pkg/mod/github.com/mindstand/[email protected]/c
onn.go:814 +0x117
github.com/mindstand/go-cypherdsl.(*QueryBuilder).Exec(0xc0000ae730, 0xc0005c63f0, 0x8fc05b, 0x4, 0xc00019ae08,
0x9acde0)
D:/sdk/gopath/pkg/mod/github.com/mindstand/[email protected]/cypher.go:324
+0x15c
github.com/mindstand/gogm.removeRelations(0xc00011cc30, 0xc00079ec30, 0x0, 0x0)
D:/sdk/gopath/pkg/mod/github.com/mindstand/[email protected]/save.go:271 +0xa63
github.com/mindstand/gogm.saveDepth.func2(0xc00038f330, 0xc00079ec30, 0xc00011cc30, 0xc000132cd0)
D:/sdk/gopath/pkg/mod/github.com/mindstand/[email protected]/save.go:155 +0x3c
created by github.com/mindstand/gogm.saveDepth
D:/sdk/gopath/pkg/mod/github.com/mindstand/[email protected]/save.go:154 +0x841
, gogm: internal error) | relErr=(An error occurred reading message data
An error occurred reading from stream
Internal Error(*errors.errorString):driver: bad connection

     Stack Trace:

    goroutine 53 [running]:
    runtime/debug.Stack(0x90dbda, 0x25, 0x0)
            D:/sdk/go1.13amd64/go/src/runtime/debug/stack.go:24 +0xa4
    github.com/mindstand/golang-neo4j-bolt-driver/errors.Wrap(0x99d580, 0xc00004c3a0, 0x90dbda, 0x25, 0x0, 0x0, 0x0,

0x46ff3e)
D:/sdk/gopath/pkg/mod/github.com/mindstand/[email protected]/e
rrors/errors.go:38 +0x18d
github.com/mindstand/golang-neo4j-bolt-driver/encoding.Decoder.readData(0x99d7c0, 0xc00011cc30, 0xc0005c65a0, 0x
b170, 0x2, 0x2, 0x2, 0x0, 0x0)
D:/sdk/gopath/pkg/mod/github.com/mindstand/[email protected]/e
ncoding/decoder.go:76 +0x26f
github.com/mindstand/golang-neo4j-bolt-driver/encoding.Decoder.read(0x99d7c0, 0xc00011cc30, 0xc0005c65a0, 0x40d0
6f, 0x30, 0x8aa500)
D:/sdk/gopath/pkg/mod/github.com/mindstand/[email protected]/e
ncoding/decoder.go:55 +0x119
github.com/mindstand/golang-neo4j-bolt-driver/encoding.Decoder.Decode(0x99d7c0, 0xc00011cc30, 0xc0005c65a0, 0xc8
b00c, 0x2, 0xc0004068a0, 0x3006b0)
D:/sdk/gopath/pkg/mod/github.com/mindstand/[email protected]/e
ncoding/decoder.go:93 +0x4a
github.com/mindstand/golang-neo4j-bolt-driver.(*BoltConn).consume(0xc00011cc30, 0xc00079f170, 0x8a5b60, 0xc0005c
64b0, 0x0)
D:/sdk/gopath/pkg/mod/github.com/mindstand/[email protected]/c
onn.go:515 +0xa4
github.com/mindstand/golang-neo4j-bolt-driver.(*BoltConn).consumeAll(0xc00011cc30, 0x8a5b60, 0xc0005c64b0, 0x0,
0x0, 0x0, 0xcf, 0xcf)
D:/sdk/gopath/pkg/mod/github.com/mindstand/[email protected]/c
onn.go:541 +0xa3
github.com/mindstand/golang-neo4j-bolt-driver.(*BoltConn).sendRunPullAllConsumeAll(0xc00011cc30, 0xc00020e000, 0
xce, 0xc00079f0e0, 0xcf, 0x1, 0xcf, 0x1, 0x2, 0x1, ...)
D:/sdk/gopath/pkg/mod/github.com/mindstand/[email protected]/c
onn.go:667 +0xef
github.com/mindstand/golang-neo4j-bolt-driver.(*boltStmt).ExecNeo(0xc0002eda10, 0xc00079f0e0, 0x85ab00, 0x1, 0x9
04101, 0x8)
D:/sdk/gopath/pkg/mod/github.com/mindstand/[email protected]/s
tmt.go:99 +0x7f
github.com/mindstand/golang-neo4j-bolt-driver.(*BoltConn).ExecNeo(0xc00011cc30, 0xc00020e000, 0xce, 0xc00079f0e0
, 0x0, 0x0, 0x0, 0x0)
D:/sdk/gopath/pkg/mod/github.com/mindstand/[email protected]/c
onn.go:814 +0x117
github.com/mindstand/go-cypherdsl.(*QueryBuilder).Exec(0xc000206050, 0xc00079f0e0, 0x8fc05b, 0x4, 0xc0001a9288,
0xc0001d4201)
D:/sdk/gopath/pkg/mod/github.com/mindstand/[email protected]/cypher.go:324
+0x15c
github.com/mindstand/gogm.relateNodes(0xc00011cc30, 0xc0001983f0, 0xc000533440, 0x91b858, 0x85adc0)
D:/sdk/gopath/pkg/mod/github.com/mindstand/[email protected]/save.go:473 +0xf8b
github.com/mindstand/gogm.saveDepth.func3(0xc00038f330, 0xc00011cc30, 0xc0001983f0, 0xc000533440, 0xc000132ce0)
D:/sdk/gopath/pkg/mod/github.com/mindstand/[email protected]/save.go:166 +0x4a
created by github.com/mindstand/gogm.saveDepth
D:/sdk/gopath/pkg/mod/github.com/mindstand/[email protected]/save.go:165 +0x7cf
) | reallocErr=()

`gogm: data not found` with intense loading

Hello.
I got a strange situation with GOGM and intense loading of objects. Sometimes GOGM returns "gogm: data not found" instead of loaded objects or timeout-error.

Here's some code example.
It creates go-routines that loads objects and outputs the result (error or not).

func gogmTest() {
	config := &gogm.Config{
		Host:          "127.0.0.1",
		Port:          7687,
		Username:      "neo4j",
		Password:      "neo4j",
		PoolSize:      100,
		IndexStrategy: gogm.IGNORE_INDEX,
		DefaultTransactionTimeout: time.Second * 10,
	}

	_gogm, _ := gogm.New(config, IdPolicy(), globalTypes...)
	var wg sync.WaitGroup
	for i := 1; i < 70; i++ {
		wg.Add(1)
		go func(i int) {
                      	sensors := []Sensor{}
                        sess, _ := _gogm.NewSessionV2(gogm.SessionConfig{AccessMode: gogm.AccessModeRead})
			defer wg.Done()
			defer sess.Close()
			whereFilter := gocypherdsl.C(&gocypherdsl.ConditionConfig{Name: "n", Field: "name", ConditionOperator: gocypherdsl.EqualToOperator, Check: "CupWndSpdMeasVirtual"})
			err := sess.LoadAllDepthFilter(context.Background(), &sensors, 0, whereFilter, nil)

			if err != nil && errors.Is(err, gogm.ErrNotFound) {
				fmt.Printf("%d - Not found - %s\n", i, err.Error())
			} else  if err != nil {
				fmt.Printf("%d - Other error - %s\n", i, err.Error())
			} else {
				fmt.Printf("%d - OK - %s\n", i, sensors[0].Name)
			}
		}(i)
	}
	wg.Wait()
	fmt.Printf("DONE\n")
}

And the output.
Timeout errors are OK, but why does GOGM return gogm: data not found?

42 - OK - CupWndSpdMeasVirtual
23 - OK - CupWndSpdMeasVirtual
15 - Not found - failed auto read tx, no primary nodes to return, gogm: data not found
33 - Other error - failed auto read tx, Neo4jError: Neo.ClientError.Transaction.TransactionTimedOut (...)
3 - Not found - failed auto read tx, no primary nodes to return, gogm: data not found
13 - Not found - failed auto read tx, no primary nodes to return, gogm: data not found
12 - Not found - failed auto read tx, no primary nodes to return, gogm: data not found
21 - Other error - failed auto read tx, Neo4jError: Neo.ClientError.Transaction.TransactionTimedOut (...)
...
53 - Not found - failed auto read tx, no primary nodes to return, gogm: data not found
56 - Not found - failed auto read tx, no primary nodes to return, gogm: data not found
57 - Not found - failed auto read tx, no primary nodes to return, gogm: data not found
69 - Not found - failed auto read tx, no primary nodes to return, gogm: data not found
52 - Not found - failed auto read tx, no primary nodes to return, gogm: data not found
19 - Not found - failed auto read tx, no primary nodes to return, gogm: data not found
...
61 - OK - CupWndSpdMeasVirtual
63 - OK - CupWndSpdMeasVirtual
67 - OK - CupWndSpdMeasVirtual
DONE

Error handling typedef'ed primitives

GoGM treats typedeff'ed primitives unexpectedly... There must be some way to check if a variable is typedeff'ed, and cast it to its pure type.

Implement schema load strategy

Implement schema load strategy similar to the spring data ogm load strategy. This strategy should generate queries based off of schema to improve speed instead of using generic queries like path load strategy. Please reference the java ogm for this

Support multiple labels / default label

Do you have any plans of supporting multiple labels for nodes (structures) or some default(main) label for struct?

GOGM uses first label of node.
This leads to difficulties if additional labels are used.
For example:

n1:Sensor
n2:Active:Sensor

The main label/entity is Sensor. In both cases it will be useful to load struct by GOGM (without any raw queries). But currently for the second case you need to have Active struct that is same as Sensor (with same links and etc).

Some config for structs with main labels or with ignoring labels by GOGM will be really useful.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.