code.google.com/p/snappy-go/snappy moved to github.com/google/snappy about sangrenel HOT 18 CLOSED

Hi! This is due to the deprecation of Google code. I'll get a fix in place shortly. Thanks for noticing!

from sangrenel.

Thanks a lot Jamie.

from sangrenel.

Haven't tested yet, but this builds - currently pulled in latest packages that are locally managed. I'll have to do some cleanup and testing: https://github.com/jamiealquiza/sangrenel/tree/updates

from sangrenel.

I updated more questions regarding the new update to your email [email protected] Would you like to check it? Thanks.

from sangrenel.

With more investigation, I think the snappy might be moved to 
https://github.com/golang/snappy

I modified sangrenel/vendor/github.com/Shopify/sarama/snappy.go as below.

import (
        "bytes"
        "github.com/golang/snappy"
        "encoding/binary"
)

when I go install the project, error was shown as below...
$ go install github.com/jamiealquiza/sangrenel
# github.com/jamiealquiza/sangrenel/vendor/github.com/Shopify/sarama
github.com/jamiealquiza/sangrenel/vendor/github.com/Shopify/sarama/snappy.go:13: 
not enough arguments to return

from sangrenel.

Hello,

Sorry for the trouble, due to some limitations on the way the 'go' command works and me vendoring packages, the following should work:

• mkdir -p $GOPATH/src/github.com/jamiealquiza
• cd $GOPATH/src/github.com/jamiealquiza
• git clone https://github.com/jamiealquiza/sangrenel.git
• cd sangrenel
• git checkout updates
• go get
• go build /github.com/jamiealquiza/sangrenel

This should result in a binary located at $GOPATH/bin/sangrenel. Let me know if you have any trouble!

from sangrenel.

With your instruction. I generated the sangrenel binary file. Thanks a lot. The tool is so wonderful!

I still have some questions on your tool.

1. -graphite-ip="": Destination Graphite IP address
   Is this IP is private ip or public ip address? Is the ip could be any ip or any machine?

When I run the sangrenel on AWS with AWS instance private IP for Graphite IP Address. It shows "Graphite unreachable: dial tcp .._._:2013: connection refused" no matter I use Kafka broker private IP or client private IP. Should I use any IP address other than broker and client?

1. Might I have more instructions on how to open those graphs?
2. I run command below and get result below. The result is so beautiful!

$ ./sangrenel -brokers=$myBrokerPrivateIp -size=10000 -rate=40000 -topic=load -clients=2 -graphite-ip=$myBrokerPrivateIp -graphite-port="2013" -producers=1

::: Sangrenel :::

Starting 2 client workers, 1 producers per worker
Message size 10000 bytes, 0 message limit per batch
Compression: none

2015/08/19 21:48:07 Graphite unreachable: dial tcp **.**.*.***:2013: connection refused
2015/08/19 21:48:07 client_2 connected
2015/08/19 21:48:07 client_1 connected
2015/08/19 21:48:12 Generating 159Mb/sec @ 2085 messages/sec | topic: load | 0.91ms 90%ile latency
2015/08/19 21:48:17 Generating 162Mb/sec @ 2117 messages/sec | topic: load | 0.87ms 90%ile latency
2015/08/19 21:48:22 Generating 162Mb/sec @ 2125 messages/sec | topic: load | 0.88ms 90%ile latency
2015/08/19 21:48:27 Generating 162Mb/sec @ 2127 messages/sec | topic: load | 0.84ms 90%ile latency
2015/08/19 21:48:32 Generating 160Mb/sec @ 2101 messages/sec | topic: load | 0.97ms 90%ile latency

I have some questions about those numbers.
Generating 1) 159Mb/sec @ 2) 2085 messages/sec | topic: load | 3) 0.91ms 4) 90%ile latency

1. the value (DATA_SIZE_PER_SEC) is global data size per second, right?
2. the value (MESSAGE_NUM_PER_SEC) is number message per second global or per client or per producer?
3. the value is the latency for average for each message, right?
4. 90%lie what is the value meaning?

How is the MESSAGE_NUM_PER_SEC related to rate from command line?
How is DATA_SIZE_PER_SEC related to MESSAGE_NUM_PER_SEC and size from command line?

from sangrenel.

I cannot seem to build your tool. I tried following the instructions above and continue to get this message:

package code.google.com/p/snappy-go/snappy: unable to detect version control system for code.google.com/ path

I have tried running the go get command with version 1.4-1.6 in go. Any help would be greatly appreciated.

from sangrenel.

Yeah, this is that issue I need to resolve with the code.google repo disappearing. I'll review and get this fixed as soon as I can.

--ja

On Feb 19, 2016, at 4:11 PM, Philip Alexander Etling [email protected] wrote:

I cannot seem to build your tool. I tried following the instructions above and continue to get this message:

package code.google.com/p/snappy-go/snappy: unable to detect version control system for code.google.com/ path
I have tried running the go get command with version 1.4-1.6 in go. Any help would be greatly appreciated.

—
Reply to this email directly or view it on GitHub.

from sangrenel.

I appreciate it :). This tools looks awesome!

from sangrenel.

So I just pulled in the latest vendored packages and updated the import schema to the Go 1.6 standards in the updates branch. I can do a clean build using Go 1.6 with the following:

• go get github.com/jamiealquiza/sangrenel
• cd $GOPATH/src/github.com/jamiealquiza/sangrenel
• git checkout updates
• go install github.com/jamiealquiza/sangrenel

Take note that I also just updated to the latest Sarama (the Kafka go client) release and can't ensure that the API hasn't changed, breaking Sangrenel. I'll be thoroughly testing this at some point soon.

from sangrenel.

I will give this a try monday!

from sangrenel.

It worked like a charm. Thank you so much @jamiealquiza !

from sangrenel.

awesome!

from sangrenel.

I followed this:
mkdir -p $GOPATH/src/github.com/jamiealquiza
cd $GOPATH/src/github.com/jamiealquiza
git clone https://github.com/jamiealquiza/sangrenel.git
cd sangrenel
git checkout updates
go get
go build /github.com/jamiealquiza/sangrenel)

got errors when executed goget:
# github.com/Shopify/sarama
../../Shopify/sarama/broker.go:72:28: error: reference to undefined identifier ‘tls.DialWithDialer’
b.conn, b.connErr = tls.DialWithDialer(&dialer, "tcp", b.addr, conf.Net.TLS.Config)
^
../../Shopify/sarama/snappy.go:7:26: error: import file ‘github.com/golang/snappy’ not found
"github.com/golang/snappy"
^
../../Shopify/sarama/broker.go:68:4: error: unknown field ‘KeepAlive’ in ‘net.Dialer’
KeepAlive: conf.Net.KeepAlive,
^
../../Shopify/sarama/snappy.go:14:9: error: reference to undefined name ‘snappy’
return snappy.Encode(nil, src)
^
../../Shopify/sarama/snappy.go:31:17: error: reference to undefined name ‘snappy’
chunk, err = snappy.Decode(chunk, src[pos:pos+size])
^
../../Shopify/sarama/snappy.go:40:9: error: reference to undefined name ‘snappy’
return snappy.Decode(nil, src)
^
../../Shopify/sarama/snappy.go:40:2: error: not enough arguments to return
return snappy.Decode(nil, src)

from sangrenel.

Hi,

I've just reran your commands using Go 1.6 on OSX 10.11 in a clean GOPATH and was not able to repeat it. What version of Go / operating system are you using? I'll see if I can repeat this.

from sangrenel.

go version is returning "go version xgcc (Ubuntu 4.9.3-0ubuntu4) 4.9.3
linux/amd64"
on Ubuntu 14.04.2 LTS (GNU/Linux 3.13.0-44-generic x86_64)

On Wed, Feb 24, 2016 at 7:57 PM, Jamie Alquiza [email protected]
wrote:

Hi,

I've just reran your commands using Go 1.6 on OSX 10.11 in a clean GOPATH
and was not able to repeat it. What version of Go / operating system are
you using? I'll see if I can repeat this.

—
Reply to this email directly or view it on GitHub
#14 (comment)
.

Saurabh Gupta
+918377943044

from sangrenel.

Ok, if I recall, that package has version 1.2 of the Go command. Also, you'll want to use the gc compiler. I'd suggest uninstalling the Ubuntu package and installing the latest Go 1.6 using the instructions here: https://golang.org/doc/install - this is what Sangrenel is current built/tested against.

from sangrenel.