I make hibernate-rediscala using rediscala.
I using rediscala github repo like this

resolvers += "rediscala" at "https://github.com/etaty/rediscala-mvn/raw/master/releases/"

libraryDependencies ++= Seq("com.etaty.rediscala" %% "rediscala" % "1.3")

compile and test is success in my local computer.
but travis ci fail by can't download rediscala (handshake error with github)

am i wrong?

Can rediscala guarantee correct message order when publishing to a Redis channel from multiple threads ? If using different RedisClient instances ? If using the same shared RedisClient and a lock for synchronization ?

onAddressChanged ~> tcpWorker ! ConfirmedClose

onConnectionClosed ~> if current connection then reconnect

+add tests

Hi,

I am evaluating the options for a redis client.

I noticed that there is no example using sentinel that has been put up..

I would like to use sentinel and have client pooling as well.

Is it possible?

Any tips would be useful!

In the UI I do not see how to label this as a question. Neither do I see any other way to communicate with the authors.

https://groups.google.com/forum/#!topic/redis-db/37xgI268Nh8

https://raw.github.com/antirez/redis/2.8/00-RELEASENOTES

Hi,

I'm curious how this client supports timeouts on non blocking ops to the redis server. For example, if the redis server is being extremely extremely slow, what is the behavior, and when will the client timeout?

Thanks

Hi,

How can I have this client maintain a persistent connection (via a connection pool) to a redis server? Is this possible? I looked at the RedisClientPool but that looks like it round robins requests to a set of servers.

Can we publish this to maven central, I want to use it but our policies dont allow bintray.

Redis’ BLPOP timeout option takes an integer.

If a timeout of < 1 second is specified for the RedisBlockingClient the Future is never returned.

It should rather require that an implicit Timeout be in scope like the rest of Akka.

Hello,
I deployed your rediscala client in Akka microkernel and I tested pub/sub functionality, which worked great.
I was wondering if it was possible to achieve queue behavior using rediscala? So that once subscriber actor consumed the message no other actor would get it?
Cheers,

Marcin

I am attempting to set a keyspace notification for regex key pattern on HM* events
The publisher in this case will be redis keyspace notifier

Step1:

Modify redis.conf to support keyspace notification

Add Line
notify-keyspace-events KEhx

(Refer http://redis.io/topics/notifications)

Step2:

Restart Redis with revised conf - "./src/redis-server redis.conf"

Step3

Create Subscriber using PubSub example indicated in redisscala

class SubscribeActor(channels: Seq[String] = Nil, patterns: Seq[String] = Nil)
extends RedisSubscriberActor(new InetSocketAddress("localhost", 6379), channels, patterns) {

def onMessage(message: Message) {
println(s" message received: $message")
}

def onPMessage(pmessage: PMessage) {
println(s"pattern message received: $pmessage")
}
}

implicit val akkaSystem = akka.actor.ActorSystem()

val redis = RedisClient()

// Pattern is keyspace@0:foo.*

val channels = Seq("keyspace@0:foo.*")
val patterns = Nil
akkaSystem.actorOf(Props(classOf[SubscribeActor], channels, patterns).withDispatcher("rediscala.rediscala-client-worker-dispatcher"))

Step4

Using Redis CLI, initiate an HMSET operation

127.0.0.1:6379> HMSET foo.x f1 "a" f2 "b"

I am not receiving or noticing event capture in the scalaredis subscriber console.

If I set up a subscription via redis-cli , I am able to see the notification captured as expected

redis 127.0.0.1:6379> psubscribe 'key*:foo_'
Reading messages... (press Ctrl-C to quit)

1. "psubscribe"
2. "key**:foo"
3. (integer) 1
4. "pmessage"
5. "_key**:foo*"
6. "keyspace@0:foobi"
7. "hset"

Am I missing anything?

I ran sbt bench:test which looks like it runs a bunch of tests, but there's no html output with the performance results. I then tried uncommenting out the benchTestSettings in the buildfile, and got a bunch of compilation errors. Is there a simple way to run the benchmarks? Thanks

Implemented in pull: #69

What are these parameters used for?

example:

def zrevrank[V: ByteStringSerializer](key: String, member: V): Future[Option[Long]]
def zrank[V: ByteStringSerializer](key: String, member: V): Future[Option[Long]]

What is expected to be used as the value for member?

I ran brpop, blpop operation to Slave Client, occur not use in Readonly server.

but brpop, blpop not setting "isMasterOnly = true"

In Master-Slave env, connect to slave server, and send blpop, brpop
READONLY *************

when download libs via sbt update, it shows:
[warn] [FAILED ] com.etaty.rediscala#rediscala_2.11;1.4.0!rediscala_2.11.jar: invalid sha1: expected=f7a7af8d640426897704864c91e267e6768e24a2 computed=8543f4b8b18a83884b5b0749c0f0db93f30eac96 (17812ms)

whatever it's downloaded from repo.typesafe.com or dl.bintray.com

I can do redis.set("key", MyCustomType(42)) as long as I have a valid RedisValueConverter in implicit scope for my custom type.

But when I do redis.get("key") all I get is a raw Option[ByteString] and I have to convert it back to my custom type myself.

Why is there not a get[T: RedisValueDeserializer](key: String): Future[Option[T]] with some basic support for common types (Strings, Ints, Longs, Booleans, Lists, Sets, Maps, etc.)? Then I can have my own RedisValueDeserializer in scope to deserialize my custom types.

Are there any plans to support connection pooling?

If connection to Redis was lost and then reestablished sometimes there are unhandled exceptions in all instances of RedisReplyDecoder at once:

20:57:11.004 [kka.actor.default-dispatcher-2] ERROR akka.actor.OneForOneStrategy - rediscala.rediscala-client-worker-dispatcher-90:akka://****/user/****-client-$G/$l - null
java.util.NoSuchElementException: null
    at scala.collection.mutable.MutableList.head(MutableList.scala:54) ~[scala-library-2.10.2.jar:na]
    at scala.collection.mutable.Queue.front(Queue.scala:168) ~[scala-library-2.10.2.jar:na]
    at redis.actors.RedisReplyDecoder.decodeRepliesRecur(RedisReplyDecoder.scala:47) ~[rediscala_2.10-1.1.jar:1.1]
    at redis.actors.RedisReplyDecoder.decodeReplies(RedisReplyDecoder.scala:42) ~[rediscala_2.10-1.1.jar:1.1]
    at redis.actors.RedisReplyDecoder$$anonfun$receive$1.applyOrElse(RedisReplyDecoder.scala:29) ~[rediscala_2.10-1.1.jar:1.1]
    at akka.actor.ActorCell.receiveMessage(ActorCell.scala:498) ~[akka-actor_2.10-2.2.1.jar:2.2.1]
    at akka.actor.ActorCell.invoke(ActorCell.scala:456) ~[akka-actor_2.10-2.2.1.jar:2.2.1]
    at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:237) ~[akka-actor_2.10-2.2.1.jar:2.2.1]
    at akka.dispatch.Mailbox.run(Mailbox.scala:219) ~[akka-actor_2.10-2.2.1.jar:2.2.1]
    at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:386) ~[akka-actor_2.10-2.2.1.jar:2.2.1]
    at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260) ~[scala-library-2.10.2.jar:na]
    at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339) ~[scala-library-2.10.2.jar:na]
    at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) ~[scala-library-2.10.2.jar:na]
    at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) ~[scala-library-2.10.2.jar:na]

Consider wrapping address: InetSocketAddress and reconnectDuration: FiniteDuration in a Config class. Should be useful in future when additional configuration parameters are required.

In our project we use password-protected Redis instance for writing.

Right after RedisClient creation it is more or less trivial to AUTH to it (and SELECT if it is needed), but if connection was lost afterwards, Rediscala do not provide any useful info about reconnection and also do not store AUTH data in connector actor to authentificate itself automatically.

Thus if we want to re-authenticate ourselves we need to either check Future's failures for "ERR Operation not permitted" messages or parse logs.

I think it would be very useful to provide some form of callback or listener to connection state changes. Also, it could be useful (but not too secure) to store last used AUTH and SELECT data to auto exec them right after reconnection.

Hi,

I'm trying to find the best way to use redis from a play framework application, to support a great number of connections and have a good request/second and response time ratio.

I made a few gatling tests : redisscala with async play features, redis-scala in blocking mode (with 'Await') and a bigger play thread pool, and finally Jedis/Sedis (synchronous and blocking).

My results are surprising : I reach the best scalability with the Jedis driver.
With Jedis I can easily reach 500 connected users (500 threads) but with redisscala if I use more than 100 threads to stress the app, it becomes very slow.

My code is very simple :

def asyncRedisQuery(q: String) = Action.async {
    val redis = RedisClient()
    redis.get(q).map(_.get).map(_.utf8String).map( result =>   
        Ok(result)
      )
  }

N.B : I've tried several execution context and thread pool configuration.

Did I miss something?

Thanks

Hi.

I am trying to use RedisPubSub and noticed that Message case class have data String, which is strange (and encode it to utf8). It looks more sense will be if it will be ByteString because message is just bytes (and i cannot use now java serialization to decode case classes).
Or maybe i miss something - how can i use data String to deserialize case class?

Th following Transaction will never commit, promises deadlock! That 's absolutely not acceptable.

object ExampleTransaction extends App {
  implicit val akkaSystem = akka.actor.ActorSystem()
  val redis = RedisClient("192.168.0.8", 6379)

  var keyX = "x"
  var keyY = "y"
  var keySum = "sum"
  //init values of x and y
  var setX = redis.set(keyX, 101)
  var setY = redis.set(keyY, 103)

  for {
     setXStatus <- setX
     setYStatus <- setY
  } yield {
    assert(setXStatus)
    assert(setYStatus)

    val redisTransaction = redis.transaction()
    redisTransaction.watch(keyX)
    redisTransaction.watch(keyY)
    var keyXFuture = redisTransaction.get(keyX);
    var keyYFuture = redisTransaction.get(keyY);
    var r = for {
        x <- keyXFuture
        y <- keyYFuture
    } yield {
        var sum = ParseNumber.parseInt(x.getOrElse(ByteString("-1"))) + ParseNumber.parseInt(y.getOrElse(ByteString("-1")))
        println("set(sum, x + y)|(x+y)=" + sum)
        redisTransaction.set(keySum, sum)
        redisTransaction.exec()
    }
     Await.result(r, 10 seconds)
  }

  Thread.sleep(1000*10)
  akkaSystem.shutdown()
}

Th following Transaction should not commit since watched keys has been changed. But it commit normally. That 's absolutely not acceptable.

object ExampleTransaction extends App {
  implicit val akkaSystem = akka.actor.ActorSystem()
  val redis = RedisClient("192.168.0.8", 6379)
  val anotherRedis = RedisClient("192.168.0.8", 6379)

  var keyX = "x"
  var keyY = "y"
  var keySum = "sum"
  //init value of x and y
  var setX = redis.set(keyX, 101)
  var setY = redis.set(keyY, 103)

  for {
     setXStatus <- setX
     setYStatus <- setY
  } yield {
    assert(setXStatus)
    assert(setYStatus)

    val redisTransaction = redis.transaction()
    redisTransaction.watch(keyX)
    redisTransaction.watch(keyY)
    var keyXFuture = redis.get(keyX);
    var keyYFuture = redis.get(keyY);

    //x and y modified by anotherRedis client
    anotherRedis.set(keyX, -101)
    anotherRedis.set(keyY, -103)
    Thread.sleep(1000)

    var r = for {
        x <- keyXFuture
        y <- keyYFuture
    } yield {
        var sum = ParseNumber.parseInt(x.getOrElse(ByteString("-1"))) + ParseNumber.parseInt(y.getOrElse(ByteString("-1")))
        println("set(sum, x + y)|(x+y)=" + sum)
        redisTransaction.set(keySum, sum)
        redisTransaction.exec()
    }

     Await.result(r, 10 seconds)
  }

  Thread.sleep(1000*10)
  akkaSystem.shutdown()
}

Hope that Transaction should be fully supported. I think Transaction should be implemented just like implemention of the blocking commands on the Lists.

Thanks & Best Regards!

Source : https://github.com/debop/redis-client-benchmark

rediscala : amazing speed

but retrieve large multiple value case, poor performance.
improve using 3rd party value formatter - 3rd party serializer or compressor, I guess.

Thanks for rediscala !
My Service success by redis and rediscala

Master is a RedisClientLike
Slaves is a RedisClientPoolLike

When sort key in Master-Slaves
I got error like this "READONLY You can't write against a read only slave."

need set isMasterOnly = true in Sort class

I am using rediscala @ 1.4.2. If I add a member to a sorted set with positive or negative infinity as the score, running the zscore command for that member of the sorted set results in an exception due to parseDouble.

In the example below, we get inf and -inf as the resulting scores from redis.

$ redis-cli
127.0.0.1:6379> zadd test:zset Inf "test"
(integer) 1
127.0.0.1:6379> zscore test:zset "test"
"inf"
127.0.0.1:6379> zadd test:zset -Inf "test1"
(integer) 1
127.0.0.1:6379> zscore test:zset "test1"
"-inf"
127.0.0.1:6379> exit

However, java.lang.Double.parseDouble expects Infinity or -Infinity. Also below is an example trying to parse inf.

$ scala
Welcome to Scala version 2.11.7 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_31).
Type in expressions to have them evaluated.
Type :help for more information.

scala> Double.PositiveInfinity
res0: Double = Infinity

scala> Double.PositiveInfinity.toString
res1: String = Infinity

scala> java.lang.Double.parseDouble(res1)
res3: Double = Infinity

scala> Double.NegativeInfinity.toString
res4: String = -Infinity

scala> java.lang.Double.parseDouble(res4)
res5: Double = -Infinity

scala> java.lang.Double.parseDouble("inf")
java.lang.NumberFormatException: For input string: "inf"
  at sun.misc.FloatingDecimal.readJavaFormatString(FloatingDecimal.java:2043)
  at sun.misc.FloatingDecimal.parseDouble(FloatingDecimal.java:110)
  at java.lang.Double.parseDouble(Double.java:538)
  ... 33 elided

scala> :q

Is anyone working on a rediscala version that works with scala-2.11, sbt-0.13.2, akka-2.3.2?

Current Rediscala 1.3 binaries somehow incompatible with Akka 2.3.0 binaries. During runtime following exception occurs:

java.lang.NoSuchMethodError: akka.actor.ActorSystem.dispatcher()Lscala/concurrent/ExecutionContext;
    at redis.RedisClientActorLike.<init>(Redis.scala:31)
    at redis.RedisClient.<init>(Redis.scala:69)

After repackaging of Rediscala with new Akka jar's problem disappears.

method is incrrect^^

BLists#brpopplush => BLists#brpoplpush

undertow-io/undertow@94cd888

https://github.com/etaty/rediscala/blob/master/src/main/scala/redis/protocol/ParseNumber.scala

Hi

I've a problem with SentinelMonitoredRedisClient when one of my sentinel instance is down, the sentinelClient Instance is removed from sentinelClients list but the actor try to reconnect to sentinel each 2 seconds.

In case when one or more sentinel instances will stay down, i'll have multiple actors tryin to reconnect to non-existing sentinel instances.

after search, i find that redisscala schedule reconnect when connection is closed (class RedisWorkerIO)

I'd like to not have this behavior in case when i use sentinelClient.

Thanks

http://www.redis.io/commands#server

I've tried to do the following with disastrously unpredictable behavior. It took a while to understand the problem was with overriding function receive. I would like to dynamically add and withdraw channels to a single subscriber instead of creating many subscribers and passing a final channel list to their constructor. Do you advise against this ? Do you have a suggestion on how to override receive properly to fit your design ?

case class RedisUuidSubscriberAdd(uuid: UUID)
case class RedisUuidSubscriberRemove(uuid: UUID)

// A simple Redis channel subscriber which will forward any Message to its parent
class RedisUuidSubscriber()
  extends RedisSubscriberActor(new InetSocketAddress("localhost", 6379), Seq(), Seq()) {

    def onMessage(message: Message) {
      context.parent ! message
    }

    def onPMessage(pmessage: PMessage) = Unit

    override def receive = {

      case RedisUuidSubscriberAdd(uuid: UUID) =>
        subscribe("uuid:" + uuid)

      case RedisUuidSubscriberRemove(uuid: UUID) =>
        unsubscribe("uuid:" + uuid)

      case other =>
        super.receive(other)
    }
}

Best regards,

In the following piece of code will the future have data even if the hash containing the data is deleted in the transaction ?

val tranBuilder = redisClient.transaction()
val data: Future[Map[String, ByteString]] = tranBuilder.hgetall(inputTableKey)
tranBuilder.del(inputTableKey)
tranBuilder.exec()

I'm trying to run a simple test by putting 1 million keys-values into Redis. For 100,000 keys it is really fast. However, the performance degrades a lot when I bump the number of operations to 1 million. The max. heapspace is 12G and I'm running this on a Macbook pro. As you can see the network write drops significantly after sometime. Not sure what's going on here. Any help would be really appreciated.

I'm using the following versions:
"com.etaty.rediscala" %% "rediscala" % "1.4.0"
scalaVersion := "2.11.4"

package redisbenchmark

import akka.util.ByteString

import scala.concurrent.{Future}
import redis.RedisClient

import java.util.UUID

object RedisLocalPerf {

def main(args:Array[String]) = {
implicit val akkaSystem = akka.actor.ActorSystem()

var numberRuns = 1000 //default to 100
var size = 1
if( args.length == 1 )
  numberRuns = Integer.parseInt(args(0))


val s = """How to explain ZeroMQ? Some of us start by saying all the wonderful things it does. It's sockets on steroids. It's like mailboxes with routing. It's fast! Others try to share their moment of enlightenment, that zap-pow-kaboom satori paradigm-shift moment when it all became obvious. Things just become simpler. Complexity goes away. It opens the mind. Others try to explain by comparison. It's smaller, simpler, but still looks familiar. Personally, I like to remember why we made ZeroMQ at all, because that's most likely where you, the reader, still are today.How to explain ZeroMQ? Some of us start by saying all the wonderful things it does. It's sockets on steroids. It's like mailboxes with routing. It's fast! Others try to share their moment of enlightenment, that zap-pow-kaboom satori paradigm-shift moment when it all became obvious. Things just become simpler. Complexity goes away. It opens the mind. Others try to explain by comparison. It's smaller, simpler, but still looks familiar. Personally, I like to remember why we made ZeroMQ at all, because that's most likely where"""

val msgSize = s.getBytes.size
val redis = RedisClient()
implicit val ec = redis.executionContext

val futurePong = redis.ping()
println("Ping sent!")
futurePong.map(pong => {
  println(s"Redis replied with a $pong")
})

val random = UUID.randomUUID().toString
val start = System.currentTimeMillis()
val result: Seq[Future[Boolean]] = for {i <- 1 to numberRuns} yield {
  redis.set(random + i.toString, ByteString(s))
}
val res: Future[List[Boolean]] = Future.sequence(result.toList)
val end = System.currentTimeMillis()
val diff = (end - start)
println(s"for msgSize $msgSize and numOfRuns [$numberRuns] time is $diff ms ")
akkaSystem.shutdown()

}

}

First of all, thank you for your efforts and contributions towards this package - it is very intuitive and useful.

However, the logging inside of RedisWorkerIO is a problem. Currently it binds to the implicit ActorSystem's LoggingAdapter (context.system.log) and I haven't found a reasonable way to silence connection/disconnection logging. It does not use getLogger like elsewhere in the code to provide for slf4j or log4j contextual logging. The loglevel in the reference.conf is not applied to Redis actors at all.

Disconnections are set to WARNING level, and reconnections are are INFO level. This is very annoying and chatty to run against a production Redis instance which frequently disconnects long-lived redis clients (reconnection handled by the driver).

Could you please update the logging to mute the particularly egregious WARN levels or apply a specific LoggingAdapter (log4j or slf4j style)? I have forks for both - can open a pull.

Thank you!

Sorry if this is obvious but I could not find any example in the transaction spec or other documentation that shows how to handle futures for transactions that need to read in order to decide what to do inside the transaction.

Let's say I want a transaction that read the value of A, checks for a condition on it and if it is fulfilled modifies a value for B. In plain redis I would use WATCH, GET A, MULTI, SET B and EXEC.

In rediscala watch is part of the transaction, and It does not return its own future. So I'm unsure how to proceed. I need to know when the watch has been received before doing GET and I need to keep watching and I can't execute the whole transaction since I'm still building it. I'm not sure if I should have two separate transactions, but the session semantics are not present, so I'm unsure if the watches will still be valid then.

So overall, how do you do this kind of thing?

Another example: http://stackoverflow.com/questions/10750626/transactions-and-watch-statement-in-redis

Correct me if I’m wrong, but it seems there is no way to connect to a server that requires Basic Auth, e.g. RedisCloud.

Hi, I can't download the jar with sbt without set proxy, and I don't know how to set the proxy in sbt and let it work.
Can you upload the jar into maven cental repo?

Thanks
Tiger

this
https://raw.githubusercontent.com/etaty/rediscala-mvn/master/releases/com/etaty/rediscala/rediscala_2.10/1.3/rediscala_2.10-1.3.pom
seems to be rendered as invalid xml for some reason. It's weird since it used to work

[warn] ==== rediscala: tried
[warn]   https://raw.githubusercontent.com/etaty/rediscala-mvn/master/releases/com/etaty/rediscala/rediscala_2.10/1.3/rediscala_2.10-1.3.pom
[info] Resolving org.fusesource.jansi#jansi;1.4 ...
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::          UNRESOLVED DEPENDENCIES         ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: com.etaty.rediscala#rediscala_2.10;1.3: not found
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
sbt.ResolveException: unresolved dependency: com.etaty.rediscala#rediscala_2.10;1.3: not found

I created a mirror of the mvn repo here http://pk11-scratch.googlecode.com/svn/trunk/com/etaty/ if that helps.

I am using rediscala 1.4.0 in my play application and when i run sbt to download sources(i need them to use debugger in idea 14) with idea with-sources=yes or idea sbt-classifiers i get this error

[warn]  [FAILED     ] xalan#serializer;2.7.1!serializer.jar(src):  (0ms)
[warn] ==== local: tried
[warn]   /home/lovesh/.ivy2/local/xalan/serializer/2.7.1/srcs/serializer-sources.jar
[warn] ==== public: tried
[warn]   http://repo1.maven.org/maven2/xalan/serializer/2.7.1/serializer-2.7.1-sources.jar
[warn] ==== Typesafe Releases Repository: tried
[warn]   http://repo.typesafe.com/typesafe/releases/xalan/serializer/2.7.1/serializer-2.7.1-sources.jar
[warn] ==== Typesafe Releases: tried
[warn]   http://repo.typesafe.com/typesafe/releases/xalan/serializer/2.7.1/serializer-2.7.1-sources.jar
[warn] ==== rediscala: tried
[warn]   http://dl.bintray.com/etaty/maven/xalan/serializer/2.7.1/serializer-2.7.1-sources.jar
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  ::              FAILED DOWNLOADS            ::
[warn]  :: ^ see resolution messages for details  ^ ::
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::
[warn]  :: xalan#serializer;2.7.1!serializer.jar(src)
[warn]  ::::::::::::::::::::::::::::::::::::::::::::::```

I beleive this error occurs because rediscala depends on this source. How can i resolve this?

In the Operation class, line 10, if you replace:

completeSuccess(r.get._1)

with

if (r.isDefined) completeSuccess(r.get._1)

then it won't crash when the redis response comes in more than one chunk (bytestring)