It seems that connection.exec() returns an empty array if a watched key is modified before exec() is called. According to Redis documentation, "if at least one watched key is modified before the EXEC command, the whole transaction aborts, and EXEC returns a Null multi-bulk reply to notify that the transaction failed." (see http://redis.io/topics/transactions, check-and-set section).

I work on the Spring Data Redis project and noticed this behavior when testing our RedisAtomicInteger.compareAndSet method, which expects connection.exec() to return null if the value was modified by another thread while watched (https://github.com/SpringSource/spring-data-redis/blob/master/src/main/java/org/springframework/data/redis/support/atomic/RedisAtomicInteger.java#L163).

Here's a test that seems to reproduce the behavior. You should see at least one empty array printed to the console (you may need to run it a few times):

@Test
public void testCompareSet() throws Exception {
    final CountDownLatch latch = new CountDownLatch(50);
    final RedisClient client = new RedisClient("localhost");
    RedisConnection<String, String> connection = client.connect();
    connection.set("mykey", "1");

    for (int i = 0; i < 50; i++) {
        new Thread(new Runnable() {
            public void run() {
                try {
                    RedisConnection<String, String> connection = client.connect();
                    connection.watch("mykey");
                    if ("1".equals(connection.get("mykey"))) {
                        connection.multi();
                        connection.set("mykey", "2");
                        List ops = connection.exec();
                        System.out.println(ops);
                    }
                } finally {
                    latch.countDown();
                }
            }
        }).start();
    }
    latch.await();
}

We've had a request to support the new time command in Spring Data (http://redis.io/commands/time). It doesn't seem to be implemented yet in RedisConnection.

It's kind of obvious that https://github.com/lettuce-io/lettuce-core superceded this repository. However it's not immediately clear on the 1st look if it's so - we can even see rather new issue report in this repo.

Maybe you want to add a prominent disclaimer pointing to the new repo and put this repo into something like maintenance mode if possible?

Admittedly this is an edge case (why would someone care about results of something that happened in a discarded transaction?). But it's possible at the moment in our API if someone "closes the pipeline" containing results of a discarded transaction. This makes a call to connection.awaitAll() with all the Futures queued since the pipeline was "opened". We don't have the context to know that some are related to discarded transactions (https://github.com/SpringSource/spring-data-redis/blob/master/src/main/java/org/springframework/data/redis/connection/lettuce/LettuceConnection.java#L274).

Wondering if anything can be done to avoid us having to track whether or not the Futures are part of a discarded transaction? Maybe an error could be set on the CommandOutputs for all the ops that are part of the discarded transaction?

public class ClientTests {

    private RedisClient client;

    @Before
    public void setUp() {
        this.client =  new RedisClient("localhost", 6379);
    }

    @Test
    public void testMultiDiscard() throws Exception {
        RedisAsyncConnection<byte[], byte[]> conn = client.connectAsync(LettuceUtils.CODEC);
        conn.multi();
        Future<String> results = conn.set("testitnow2".getBytes(), "notok".getBytes());
        conn.discard();
       // This hangs...
        results.get();
    }


}

Is there any plan for a streaming-based API for things like smembers, lrange or zrange?

There can be a huge number of items there, and it'd be great if a streaming or callback API was available that avoided loading the whole set/list/range into memory when retrieving.

Admittedly a bit of a corner case, but I ran into this recently in an integration test. The following code times out after a minute, where I would expect an immediate Exception, as when running "exec" without "multi" from redis-cli.

public class TxTest {

    private com.lambdaworks.redis.RedisConnection<byte[], byte[]> txConn;

    @Before
    public void setUp() {
        RedisClient client = new RedisClient("localhost", 6379);
        txConn = new com.lambdaworks.redis.RedisConnection<byte[], byte[]>(client.connectAsync(LettuceUtils.CODEC));
    }

    @Test
    public void testExecWithoutMulti() {
        txConn.exec();
    }

}

I know netty 4.0.0 is still baking, but is there any work towards supporting the new API?

Hi all, I'm using spring mvc and spring-data-redis. Sometimes my redis node goes down and ConnectionWatchdog throws a lot of exception "Connection refused". In this case I need to have a dns lookup and change the ip of my redis node and ConnectionWatchdog.reconnect should use the new ip.
Is there a way to override the remoteAddress or reconnect method?

Thanks
F

The 'mvn test' hang while running com.lambdaworks.redis.ClientTest.
x86-64 CentOS 5.6 with JDK 1.6.0_29.

any clues?

I'm very interested in using this library for some Redis integration. Is it still actively supported? I see there was a pull request accepted just 24 days ago so this is more of a double check?

Also - Do/Will you support Netty V5?

How to handle hashmap in jedis with following structure:

HashMap<String,List<RoleDetailsDTO>>

Shouldn't Command check the error status of CommandOutput, and if getError() is not null, actually return an ExecutionException on get()? Doesn't look like it's happening.

I'm using Lettuce in my Vert.x 2.0 project.
Vert.x 2.0 is on Netty 4. So now I have two major versions of Netty in my dependencies. Not sure if this will cause problems.. However, Is there any plans to upgrade to Netty 4.0 in Lettuce ?

-Simone

Hi, we store serialized objects inside Redis and - as strings - they typically have special characters inside.

Apparently it's just a parsing error.

We currently use Jedis extensively, but wanted to move to Lettuce for the Async connection feature.

Test case:

• Install redis 2.6.x
• redis-cli set key "\n\xab\a\n\xf9\x06\n\x0c\b\x01\x12\x06117390\x18\x02"
  OK
• redis-cli get key
  "\n\xab\a\n\xf9\x06\n\x0c\b\x01\x12\x06117390\x18\
• Write this sample Java program (either sync or async)

        RedisAsyncConnection<String,String> async = client.connectAsync();
        Future<String> val = async.get("key");
        if (async.awaitAll(val) == true) {
            try {
                System.out.println(val.get());
            } catch (Throwable e) {
                e.printStackTrace();
            }
        }
        client.shutdown();

Observed Output

The value printed is blank. Consistently with different keys containing other serialized objects and using other methods like hmget.

When acting as a pubsub channel and receiving a large number of messages the decoder finds itself in the wrong state. I attached a stack trace of the error (I added the stack trace to the handler)

To test, subscribe to a channel, then publish as quickly as possible (tight loop). The connection will be dropped, and resubscribe, and messages will be lost.

java.lang.IllegalArgumentException: No enum const class com.lambdaworks.redis.pubsub.PubSubOutput$Type.my_full_message_here

at java.lang.Enum.valueOf(Enum.java:196)
at com.lambdaworks.redis.pubsub.PubSubOutput$Type.valueOf(PubSubOutput.java:1)
at com.lambdaworks.redis.pubsub.PubSubOutput.set(PubSubOutput.java:50)
at com.lambdaworks.redis.protocol.RedisStateMachine.decode(RedisStateMachine.java:113)
at com.lambdaworks.redis.pubsub.PubSubCommandHandler.decode(PubSubCommandHandler.java:51)
at com.lambdaworks.redis.protocol.CommandHandler.messageReceived(CommandHandler.java:54)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:94)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:372)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:246)
at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:38)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)

Any thoughts?

If I run the following test, maybe 4 out of 5 times it fails with "java.lang.RuntimeException: ERR EXEC without MULTI" when running against Redis 2.4.

Against Redis 2.6, I get these failures intermittently also, as well as occasional assertion failures against the result of exec (sometimes it's an empty list and sometimes it contains "OK").

Perhaps I am doing something wrong? Seems to work OK if I eliminate the set done by the 2nd connection.

import java.util.List;
import java.util.concurrent.Future;
import static org.junit.Assert.assertTrue;
import org.junit.Test;

import com.lambdaworks.redis.RedisAsyncConnection;
import com.lambdaworks.redis.RedisClient;
import com.lambdaworks.redis.RedisConnection;
import com.lambdaworks.redis.protocol.Command;

public class LettuceTest {

@Test
public void testWatch() throws Exception {
    RedisClient client = new RedisClient("localhost", 6379);
    RedisAsyncConnection<byte[], byte[]> conn1 = client.connectAsync(LettuceUtils.CODEC);
    RedisAsyncConnection<byte[], byte[]> conn2 = client.connectAsync(LettuceUtils.CODEC);
    RedisConnection<byte[],byte[]> syncConn = new com.lambdaworks.redis.RedisConnection<byte[], byte[]>(conn2);

    // Watch with conn 1
    Future<String> watch = conn1.watch("testitnow".getBytes());

    // Synchronously set the value with conn 2
    syncConn.set("testitnow".getBytes(), "something".getBytes());

    // Start conn 1 tx
    Future<String> mul = conn1.multi();

    // Attempt to change watched variable value
    Future<String> set = conn1.set("testitnow".getBytes(), "somethingelse".getBytes());

    // Exec tx
    Future<List<Object>> f = conn1.exec();
    List<Object> results = f.get();

    // Results should be empty since watched var modified by other connection
    System.out.println(results);
    assertTrue(results.isEmpty());

    Command<?,?,?>[] ppline = new Command<?,?,?>[] {(Command<?,?,?>)watch, (Command<?,?,?>)mul, (Command<?,?,?>)set};
    conn1.awaitAll(ppline);
    for (Command<?, ?, ?> cmd : ppline) {
        if (cmd.getOutput().hasError()) {
            // Processing the "set" future often results in "ERR EXEC without MULTI" here
            throw new RuntimeException(cmd.getOutput().getError());
        }else {
            System.out.println(cmd.getOutput().get());
        }
    }
}
}

Does lettuce support specifying callbacks on completion of operations? Java Futures are not good enough, even though the operation is async, getting the value is synchronous, blocking. Using Java futures has all kinds of issues when desiging completely async applications and performing async computation composition

For example of better futures please see Scala futures, Guava ListenableFuture or Java 8 CompletableFuture, even custom callback is better then old Java Future

this issue caused by netty. When RedisClient.connect() call the Method DefaultChannelFuture.await(),netty will call the deadLockCheck() when the attr useDeadLockChecker is true. then because this process is in a undone future,the checker return false and cause the connection faild.

I tried changing the future.await() to add a ChannelFutureListener to fix this.But it seems will take some new problem, such as a unfunctional connection will be returned.Is there some good idea to fit this?

I am using the following to create the connection.
client = new RedisClient(config.getRedisHost());
async = client.connectAsync();
async.set("test", "123");
async.get("test");
I am not seeing the keys in the Redis. I also used 'async.lpush' after the set, still not seeing it. Is there anything I am missing here that I should be doing? Please help me with this.

It would be nice to have some sample code showing how to use transactions with multi and exec.

This would support use of JDK8 async utils.

java.nio.BufferOverflowException
at java.nio.HeapByteBuffer.put(Unknown Source)
at com.lambdaworks.redis.protocol.CommandArgs.realloc(CommandArgs.java:175)
at com.lambdaworks.redis.protocol.CommandArgs.add(CommandArgs.java:63)
at com.lambdaworks.redis.RedisAsyncConnection.hmset(RedisAsyncConnection.java:318)

This is my code, output "use 70 ms", when i remove "redisConnection.select(8);"
output "use 35 ms".
so is there any way to execute method select once?
and excute method select when it reconnects automatically

@OverRide
public void reportCount(int value) {
long start = System.currentTimeMillis();
redisConnection.select(8);
redisConnection.incrby("count", value);
long time = System.currentTimeMillis() - start;
System.out.println("use "+time+"ms");
}

hi, thanks for writing lettuce. it's very easy to use. i had a question when using it.How to pass multi type KEYS for eval?

I am just wondering is there any fall back with lettuce driver. My scenario is when App started with Redis pool and it died after few minutes. I am seeing connection time out issues with lettuce driver. I'd like to know if someone handle this scenario and would like to see the implementation ?

It would be nice to provide a password to RedisClient and have it call auth automatically on connect (ala this custom extension: https://github.com/SpringSource/spring-data-redis/blob/master/src/main/java/org/springframework/data/redis/connection/lettuce/AuthenticatingRedisClient.java)

The Netty threads created by Lettuce are not daemon threads, so they hold the process open. Is it possible to make these daemon threads?

I've seen this thread stack trace:

    at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:623)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:310)
    at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
    at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
    at java.lang.Thread.run(Thread.java:745)

I am using the same RedicClient(as recommended) for couple of connections types. and I am having huge latency issues while using them in parallel:

using biz.paluch.redis:lettuce:4.0.Beta2

    private RedisClient client;
    private StatefulRedisConnection<String, String> connection;
    private RedisCommands syncCommands;
    private RedisGeoAsyncCommands<String, String> geoAsyncCommand;
    private RedisAsyncCommands<String, String> asyncCommands;
    private RedisCommands<String, String> redisSyncConnection;

  @PostConstruct
    public void init() {
        log.debug("RedisRepo, Init.");
        client = new RedisClient(REDIS_HOST, REDIS_PORT);
        connection = client.connect();
        connection.setTimeout(10, TimeUnit.SECONDS);
        syncCommands = connection.sync();
        syncCommands.setTimeout(4, TimeUnit.SECONDS);
        geoAsyncCommand = client.connect().async();
        //  RedisClusterClient clusterClient = new RedisClusterClient(new RedisURI(REDIS_HOST,REDIS_PORT,60, TimeUnit.SECONDS));
        asyncCommands = client.connect().async();
        redisSyncConnection = client.connect().sync();
    }

and I have separate methods being invoked with diff Redis commands in parallel

Latency report:
]$ /opt/redis-unstable/src/redis-cli --intrinsic-latency 100
Max latency so far: 1 microseconds.
Max latency so far: 32 microseconds.
Max latency so far: 8147 microseconds.
Max latency so far: 8217 microseconds.
Max latency so far: 12169 microseconds.
Max latency so far: 12182 microseconds.
Max latency so far: 12199 microseconds.
Max latency so far: 12223 microseconds.
Max latency so far: 12255 microseconds.
Max latency so far: 12298 microseconds.
Max latency so far: 12303 microseconds.
Max latency so far: 12310 microseconds.
Max latency so far: 12330 microseconds.

253109899 total runs (avg latency: 0.3951 microseconds / 3950.85 nanoseconds per run).
Worst run took 31208x longer than the average latency.

1. Am I using the Redis Client properly?
2. Is it the right way to use multiple connection types with the same RedisClient?

Thanks.

This is an issue I've experienced with another Java based Redis client before (namely Jedis), where in effect the subscription dies silently. It doesn't call the unsubscribe method, and it doesn't cause any exceptions - it seemingly just stops listening for certain channels.

I'm just calling the subscribe method as shown in the README example, and while it will work reliably for a short period of time, it'll just stop later.

Lettuce 2.1.0 with Netty 3.3.0 Final.

when i test multi and the looptimes is 100000 , the looptimes is 10000 is ok
connection.multi();
for (int i = 0; i < 100000; i++) {
connection.set("t" + i, "t" + i);
}
connection.exec();

io.netty.channel.AbstractChannelHandlerContext notifyOutboundHandlerException
warning: Failed to fail the promise because it's done already: DefaultChannelPromise@f1f2cc(failure: java.util.ConcurrentModificationException)
java.util.ConcurrentModificationException
at java.util.ArrayDeque.delete(ArrayDeque.java:512)
at java.util.ArrayDeque.removeFirstOccurrence(ArrayDeque.java:333)
at java.util.ArrayDeque.remove(ArrayDeque.java:694)
at com.lambdaworks.redis.protocol.CommandHandler.queueCommand(CommandHandler.java:428)
at com.lambdaworks.redis.protocol.CommandHandler.writeSingleCommand(CommandHandler.java:372)
at com.lambdaworks.redis.protocol.CommandHandler.write(CommandHandler.java:355)
at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:718)
at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:710)
at io.netty.channel.AbstractChannelHandlerContext.access$1900(AbstractChannelHandlerContext.java:33)
at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.write(AbstractChannelHandlerContext.java:1037)
at io.netty.channel.AbstractChannelHandlerContext$WriteAndFlushTask.write(AbstractChannelHandlerContext.java:1089)
at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.run(AbstractChannelHandlerContext.java:1022)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:358)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:394)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:145)
at java.lang.Thread.run(Thread.java:619)

I have this exception every 10 Minutes on my server, I know this is ok, but is there a way do change the loglevel or something else so I didn't see this Exception every 10 Minutes in my log.

java.util.concurrent.RejectedExecutionException: Worker has already been shutdown
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.registerTask(AbstractNioSelector.java:115)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.executeInIoThread(AbstractNioWorker.java:73)
at org.jboss.netty.channel.socket.nio.NioWorker.executeInIoThread(NioWorker.java:36)
at org.jboss.netty.channel.socket.nio.AbstractNioWorker.executeInIoThread(AbstractNioWorker.java:57)
at org.jboss.netty.channel.socket.nio.NioWorker.executeInIoThread(NioWorker.java:36)
at org.jboss.netty.channel.socket.nio.AbstractNioChannelSink.execute(AbstractNioChannelSink.java:34)
at org.jboss.netty.channel.Channels.fireExceptionCaughtLater(Channels.java:496)
at org.jboss.netty.channel.AbstractChannelSink.exceptionCaught(AbstractChannelSink.java:46)
at org.jboss.netty.channel.Channels.close(Channels.java:812)
at org.jboss.netty.channel.AbstractChannel.close(AbstractChannel.java:197)
at org.jboss.netty.channel.ChannelFutureListener$2.operationComplete(ChannelFutureListener.java:52)
at org.jboss.netty.channel.DefaultChannelFuture.notifyListener(DefaultChannelFuture.java:427)
at org.jboss.netty.channel.DefaultChannelFuture.notifyListeners(DefaultChannelFuture.java:413)
at org.jboss.netty.channel.DefaultChannelFuture.setFailure(DefaultChannelFuture.java:380)
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.connect(NioClientSocketPipelineSink.java:130)
at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink.eventSunk(NioClientSocketPipelineSink.java:70)
at org.jboss.netty.channel.Channels.connect(Channels.java:634)
at org.jboss.netty.channel.AbstractChannel.connect(AbstractChannel.java:207)
at com.lambdaworks.redis.protocol.ConnectionWatchdog.run(ConnectionWatchdog.java:87)
at org.jboss.netty.util.HashedWheelTimer$HashedWheelTimeout.expire(HashedWheelTimer.java:561)
at org.jboss.netty.util.HashedWheelTimer$Worker.notifyExpiredTimeouts(HashedWheelTimer.java:465)
at org.jboss.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:404)
at java.lang.Thread.run(Thread.java:724)

I use lettuce in the version 2.3.3.