qasaur / gremgo Goto Github PK

View Code? Open in Web Editor NEW

98.0 6.0 49.0 132 KB

A fast, efficient, and easy-to-use Go client for the Apache TinkerPop graph database stack

License: MIT License

Go 100.00%

tinkerpop golang

gremgo's People

Contributors

Stargazers

Watchers

gremgo's Issues

Make a distinction between language driver and language variant.

http://tinkerpop.apache.org/docs/3.2.1-SNAPSHOT/tutorials/gremlin-language-variants/

driver: the machinery to connect to Gremlin Server (Websockets, etc.)
variant: the machinery to write Gremlin in native Go.

Request: return struct instead of interface{}

For example the function: ExecuteStruct(...) = same as Execute(...) but a result as a struct rather than an interface.

Reason for request: sometimes the structure is known and this makes it easier to handle.

Full example:

type Edges [][]struct {
	ID         int    `json:"id"`
	InV        int    `json:"inV"`
	InVLabel   string `json:"inVLabel"`
	Label      string `json:"label"`
	OutV       int    `json:"outV"`
	OutVLabel  string `json:"outVLabel"`
	Properties struct {
		LocationURL string `json:"locationUrl"`
		Type        string `json:"type"`
	} `json:"properties"`
	Type string `json:"type"`
}

var edges *Edges

err := f.client.ExecuteStruct(
	         "SOME-TINKERPOP-QUERY",
	        &edges,
		map[string]string{"uuid": UUID.String()},
		map[string]string{},
	)

Gremgo should provide a language variant representation.

http://tinkerpop.apache.org/docs/current/reference/#gremlin-variants
http://tinkerpop.apache.org/docs/current/tutorials/gremlin-language-variants/

http://tinkerpop.apache.org/gremlin.html (see host language embedding)
http://tinkerpop.apache.org/providers.html (see query language providers)

[SCRIPT EVALUATION ERROR] on setting UUID

The following results in the evaluation error: g.addV(" + thing.AtClass + ").property(T.id, uuid)

The issue lies in T.id but I'm not sure how to set this value.

thanks

readme example is not working

expected result: resp including a vertice id
but the real result: [SERVER SERIALIZATION ERROR]

// replace map[string]string{"x": "1234"} with this:
map[string]string{"x": "'1234'"},
// but got 
// [<nil>]

Currently if the connection breaks the library logs 2018/03/29 16:28:06 websocket: close 1006 (abnormal closure): unexpected EOF but no errors or events are bubbled up to the user of the library. This makes it impossible for users of the library to build in reconnection logic.

Error due API change on "github.com/satori/go.uuid"

# github.com/qasaur/gremgo
../../qasaur/gremgo/request.go:31:17: multiple-value uuid.NewV4() in single-value context
# github.com/qasaur/gremgo
../../qasaur/gremgo/request.go:31:17: multiple-value uuid.NewV4() in single-value context
# github.com/qasaur/gremgo
../../qasaur/gremgo/request.go:31:17: multiple-value uuid.NewV4() in single-value context
Makefile:30: recipe for target 'compile' failed
make: *** [compile] Error 2

isConnected is not an public

Hi Team,

As we are using the gremgo for our application development the gremgo.Ws struct is having the method isConnected() which is not public method if you can make this as an public method which will help us in understand whether our application connected to server or not

How to use gremgo to create a database

Hi,
I am just getting to understand gremlin. I am using neo4j plugin within gremlin server and trying to use gremgo. However, what i didn't understand is how to create a graph using gremgo? For eg, on gremlin console, we can do

gr = Neo4j.open("trial")

Whats the equivalent of this in gremgo? or is gremgo only for query.

readme connection example up to date?

Hi,

When I try to connect as in the readme example:

dialer := gremgo.NewDialer("127.0.0.1") // Returns a WebSocket dialer to connect to Gremlin Server
g, err := gremgo.Dial(dialer) // Returns a gremgo client to interact with
if err != nil {
fmt.Println(err)
return
}

I get the following compilation error:
cannot use dialer (type gremgo.Ws) as type gremgo.dialer in argument to gremgo.Dial:
gremgo.Ws does not implement gremgo.dialer (gremgo.connect method has pointer receiver)

Is this connection example uptodate?

Thanks!

Want to be an contributor

I have to contribute to this library can you please add my key in the repo as an contributor and also I wanna give one pull request

How does this compare to go-gremlin/gremlin?

@qasaur I'm looking into using Gremlin. How does this package compare to https://github.com/go-gremlin/gremlin? Did you consider using that before. Curious regardless, but if you did, what drew you towards writing gremgo?

How to use result

Hi - I am new to both Gremlin and to Go. Having spent the afternoon with the debugger, I now have the following code (after g.addV()). If this is wrong, maybe you could show the correct usage in the README file. Thanks.

idFromGremlin := int64(res.([]interface{})[0].([]interface{})[0].(map[string]interface{})["id"].(float64))
labelFromGremlin := res.([]interface{})[0].([]interface{})[0].(map[string]interface{})["label"].(string)
typeFromGremlin := res.([]interface{})[0].([]interface{})[0].(map[string]interface{})["type"].(string)
properties := res.([]interface{})[0].([]interface{})[0].(map[string]interface{})["properties"].(map[string]interface{})
nameFromGremlin := properties["name"].([]interface{})[0].(map[string]interface{})["value"]

Custom loader issue with DSE Graph

I have written a loader using this package and it works with Gremlin Server, but I cannot get it work against DSE Graph. The group thread below has some additional information. One of the responses to my question leads me to believe that gremgo must set a graph alias. Using the console, the setting looks like this:
:remote config alias g followTopic.g

The command given to gremgo is graph.addVertex(label, "part", "name", PART) with suitable bindings.

Whereas in the console you can set this once, I think I may have to send with each "transaction" with gremgo.

Of course, this may not even be the reason it is hanging, but I'll need the "g" alias once I start adding edges.

Here is the thread:
https://groups.google.com/forum/#!topic/gremlin-users/sjlzWPBJZ0k

Thanks in advance!
Cecil

websocket message length too small

Currently websocket message is limited to 8192 bytes:

gremgo/connection.go

Lines 51 to 52 in f9215ff

 WriteBufferSize: 8192, 

 ReadBufferSize: 8192,

this is definitely not enough for any serious management system and causes nasty errors on gremlin side:

gremlin_1  | [WARN] AbstractGraphSONMessageSerializerV2d0 - Request [PooledUnsafeDirectByteBuf(ridx: 8175, widx: 8175, cap: 8192)] could not be deserialized by org.apache.tinkerpop.gremlin.driver.ser.AbstractGraphSONMessageSerializerV2d0.
gremlin_1  | org.apache.tinkerpop.shaded.jackson.core.io.JsonEOFException: Unexpected end-of-input in character escape sequence
gremlin_1  |  at [Source: (byte[])"{"requestId":"44a7bc13-f7f4-4731-bd68-df6cab6b988b","op":"eval","processor":"","args":{"bindings":{"_0":"Grimis-long-label","_1":"long","_2":"\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u00"[truncated 7675 bytes]; line: 1, column: 16351]

Increasing this settings fixes the error, so we're preparing a MR making gremgo.NewDialer() more flexible with the possibility to set more websocket.Dialer options

Bad uuid.NewV4 call

$ go get github.com/qasaur/gremgo
# github.com/qasaur/gremgo
../../github.com/qasaur/gremgo/request.go:32:17: multiple-value uuid.NewV4() in single-value context

Concurrency issue on responseNotifyer map

Hi @qasaur,

First, thanks for your hard work on providing a Go Gremlin client. Really appreciate it. Let me know if i've misinterpreted the intended use of the package in any way... On to the issue.

Background: I first encountered the issue when performance test my API using ab (Apache Bench) with concurrency set to "2". I was not able to recreate using the existing test suite and therefore had to simulate concurrent "Execute" calls with a new test (see below).

tested on: go 1.7.3

Reproduce:

add the following code to client_test.go

func TestClient(t *testing.T) {
	dialer := NewDialer("127.0.0.1:8182")
	g, err := Dial(dialer)
	if err != nil {
		t.Fatal("Dial Error: ", err)
	}

	ch := make(chan bool)
	go func() {
		// first access
		g.Execute(
			"system.graph('foo').ifNotExists().create()",
			map[string]string{}, map[string]string{},
		)
		ch <- true
	}()

	// second access
	g.Execute(
		"system.graph('foo').ifNotExists().create()",
		map[string]string{}, map[string]string{},
	)
	<-ch
}

run go test -race -v

output:

=== RUN   TestClient
==================
WARNING: DATA RACE
Write at 0x00c420019080 by goroutine 9:
  runtime.mapassign1()
      /Users/skud/.gimme/versions/go1.7.3.darwin.amd64/src/runtime/hashmap.go:442 +0x0
  github.com/qasaur/gremgo.(*Client).executeRequest()
      /Users/skud/projects/src/github.com/qasaur/gremgo/client.go:61 +0x2e8
  github.com/qasaur/gremgo.(*Client).Execute()
      /Users/skud/projects/src/github.com/qasaur/gremgo/client.go:69 +0x70
  github.com/qasaur/gremgo.TestClient.func1()
      /Users/skud/projects/src/github.com/qasaur/gremgo/client_test.go:18 +0xc7

Previous write at 0x00c420019080 by goroutine 6:
  runtime.mapassign1()
      /Users/skud/.gimme/versions/go1.7.3.darwin.amd64/src/runtime/hashmap.go:442 +0x0
  github.com/qasaur/gremgo.(*Client).executeRequest()
      /Users/skud/projects/src/github.com/qasaur/gremgo/client.go:61 +0x2e8
  github.com/qasaur/gremgo.(*Client).Execute()
...

As indicated, the issue lies at: https://github.com/qasaur/gremgo/blob/master/client.go#L61

I believe you will also have issues at: https://github.com/qasaur/gremgo/blob/master/response.go#L65 and https://github.com/qasaur/gremgo/blob/master/response.go#L69.

A simple solution would be to add locks around those map ops, (perhaps consider RWMutex) but performance obviously may be affected.

Where is NewSecureDialer?

Seems to be missing?

Breaking change in Gremlin Server 3.2.2

Info is here:
https://groups.google.com/forum/#!topic/gremlin-users/x4hiHsmTsHM

Since your dialer takes an IP address and port instead of URL, this might impact the method. I will not have 3.2.2 any time soon to test this myself.

Just wanted awareness.

Thanks,
Cecil

Client hangs on backend error

If you submit a query and hit an exception in the database due to invalid input or something, currently the client just hangs forever. For example, if I make a query to add an id property to a vertex that isn't an integer the backend will throw an exception:

Query

tag.property(id, "mytaghashstring");

Database

[WARN] AbstractEvalOpProcessor - Exception processing a script on request [RequestMessage{, requestId=b5ee53a3-d52c-493b-89bb-5fdb73f3b69e, op='eval', processor='', args={gremlin=
tag.property(id, "mytaghashstring");
...
java.lang.NumberFormatException: For input string: "mytaghashstring"
	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
	at java.lang.Long.parseLong(Long.java:589)
	at java.lang.Long.parseLong(Long.java:631)
	at org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph$DefaultIdManager$1.convert(TinkerGraph.java:585)
	at org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph.lambda$createElementIterator$4(TinkerGraph.java:316)
	at org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils$3.next(IteratorUtils.java:247)
	at org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils$4.advance(IteratorUtils.java:299)
	at org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils$4.hasNext(IteratorUtils.java:269)
	at org.apache.tinkerpop.gremlin.tinkergraph.process.traversal.step.sideEffect.TinkerGraphStep.iteratorList(TinkerGraphStep.java:114)
	at org.apache.tinkerpop.gremlin.tinkergraph.process.traversal.step.sideEffect.TinkerGraphStep.vertices(TinkerGraphStep.java:85)
	at org.apache.tinkerpop.gremlin.tinkergraph.process.traversal.step.sideEffect.TinkerGraphStep.lambda$new$0(TinkerGraphStep.java:59)
	at org.apache.tinkerpop.gremlin.process.traversal.step.map.GraphStep.processNextStart(GraphStep.java:142)
	at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143)
	at org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal.hasNext(DefaultTraversal.java:192)
	at java_util_Iterator$hasNext.call(Unknown Source)
	at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
	at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117)
	at Script11.run(Script11.groovy:2)
	at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:690)
	at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:395)
	at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:233)
	at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$0(GremlinExecutor.java:263)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

This renders the caller useless since there seem to be no way out of the hung state. If you try to print the error or response after sending the query over you'll see that the hang happens before either of these points (somewhere inside gremgo):

		res, err := e.Db.Query(query, bindings)
		if err != nil {
			fmt.Println(err.Error()) // You'll never get here
			return nil, err
		}
		fmt.Println(res) // You'll never get here

100% cpu usage?

Things have been working well with this package, but noticed something strange. I think it's taking 100% CPU after setting up a client, even when idle.

dialer := gremgo.NewDialer("localhost:8182") 

g, err := gremgo.Dial(dialer) // Returns a gremgo client to interact with

if err != nil {

    fmt.Println(err)

    return

}

   //Do something to idly wait

It appears to take 100% cpu. Are you seeing the same thing?

Size limits on queries?

Seeing some size limitations on reading and writing, wondering if there are any limits that are configurable on client side?

For reading:

{

    for i := 0; i < 2000; i++ {
        res, err := g.Execute(
            fmt.Sprintf("g.addV('label').property('testString','%v')", i),
            nil)
        if err != nil {
            panic(err)
        }
        fmt.Println(res)
    }

    {
        res, err := g.Execute(
            "g.V().values('testString')",
            nil) 
        if err != nil {
            panic(err)
        }
        fmt.Println(res)   //<---- Expecting 2000 rows, only seeing around 60.  Works via java driver, so don't think this is a setting with my gremlin server
    }
}

For writing:

{
    longString := ""
    for i:=0 ;i<3900;i++{ //3800 works, but 3900 doesn't and returns error.  Didn't test this via java driver, can do so if it helps
        longString += "1"
        //fmt.Println("longString = ", longString)
    }
    fmt.Println("len(longString) = ", len(longString))

    szLongInsert := fmt.Sprintf("g.addV('longStringInsert').property('testString','%v')", longString)
    fmt.Println("len(szLongInsert) = ", len(szLongInsert))
    res, err := g.Execute(
        szLongInsert,
        nil)
    if err != nil { 
        panic(err)
    }
    fmt.Println(res)
}

How to use session?

Hi, I am new to gremlin and janusgraph,
Does gremgo support 'session' ?
How to do like script :remote connect tinkerpop.server conf/remote.conf session\n :remote console.
I want to do mgmt=graph.openManagement() and initial some label s,properties, and indexes by gremgo.

Execute(query string, bindings map) function can take nil bindings?

Can this function take nil for bindings?

I'm trying to run alot of addVertex() statements by calling Execute() funciton with bindings as nil.

When execute and try to print the response, I see just empty array "[]", but I see via my gremlin console the vertices were really added. Am I allowed to pass in nil for this? Please note there error was nil from Execute() call, also.

Hang on server failure

I'm seeing a timing dependent hang when the gremlin server goes away for some reason. It looks like the response handler sits on a channel waiting for a response with no timeout. If we get to this blocking call as the server goes down, there is never a response and so this call never unblocks.

https://github.com/qasaur/gremgo/blob/master/response.go#L73

Maybe I missed some setting that would avoid this? Otherwise, my temporary fix is simply to introduce a select/case style timeout, which I've hard coded to a second but I guess could come from some config option.

If you agree I'd be happy to do a PR.

Purpose of rebindings in Execute method

I've looked through Gremgo and Gremlin docs and i dont see a reference to "rebindings" and what their purpose is. Closest i could find is aliases for Graph and TraversalSource objects.

go get github.com/qasaur/gremgo returns an error

Thank you for creating a go client. When I received an error when running the 'go get' command. Thank you for taking a look at it.

$ go get github.com/qasaur/gremgo

github.com/qasaur/gremgo

../github.com/qasaur/gremgo/request.go:32:17: multiple-value uuid.NewV4() in single-value context

Replace satori/go.uuid

Replace with https://github.com/gofrs/uuid since it's no longer maintained and has security vulnerability WS-2018-0594 + create a new release.

satori/go.uuid#84

A vulnerability was found in github.com/satori/go.uuid through version v1.2.0. The UUID random generator uses Read() function which has fewer bytes than asked and might cause potential non-random UUIDs.

Publish Date: 2019-06-25

URL: WS-2018-0594

broken code in request.go

I'm getting an error when I try to get this library.

$ go get github.com/qasaur/gremgo
# github.com/qasaur/gremgo
../../go/src/github.com/qasaur/gremgo/request.go:31:17: multiple-value uuid.NewV4() in single-value context

Looks like request.go is not using the uuid library correctly. Here's the current code:

func prepareRequest(query string, bindings, rebindings map[string]string) (req request, id string) {
	id = uuid.NewV4().String()

	req.Requestid = id
	req.Op = "eval"
	req.Processor = ""

	req.Args = make(map[string]interface{})
	req.Args["language"] = "gremlin-groovy"
	req.Args["gremlin"] = query
	req.Args["bindings"] = bindings
	req.Args["rebindings"] = rebindings

	return
}

The problem is that uuid.NewV4() returns a (UUID, error) (https://godoc.org/github.com/satori/go.uuid#NewV4) but the calling code expects a single value and we are not doing anything the the error value.

Authentication

How can I authenticate to a gremlin server?

Documentation clean-up

I feel that README.md is currently a little too complex for someone who is using gremgo for the first time, particularly the structure and the code examples. If anybody feels like it, I think the project would benefit greatly from a rewrite of the documentation.

Unable to do a bulk request on database, number of documents returned is too low?

for the query
g.V('user1').inE('follows').outV().valueMap().range(0, 100)

The amount of document returned is around 60.

qasaur / gremgo Goto Github PK

gremgo's People

Contributors

Stargazers

Watchers

Forkers

gremgo's Issues

github.com/qasaur/gremgo

Recommend Projects

Recommend Topics

Recommend Org