Example code not working about dyno HOT 12 CLOSED

@gonzajg I see why this is happening. My example was insufficient for the oss community.
Here is what's happening ...

When Dyno starts up, it needs the token map (topology) from the Dynomite servers.
Currently we have an internal mechanism inside Netflix where we can get this information from and we haven't open sourced that as of yet.

This is where you are getting it from
http://localhost:8080
and the class is TokenMapSupplierImpl.

But you can unblock yourself from this. You need to provide a simple TokenMapSupplier impl that just gives the tokens along with the host names to Dyno, and you will get past this error.

I'll start adding an example on how to do this. Thanks for your patience.

from dyno.

[
{"token":"3051939411","hostname":"foo11" , "zone":"us-east-1c"},
{"token":"188627880"," hostname":"foo12" , "zone":"us-east-1c"},
{"token":"3051939411","hostname":"foo21" , "zone":"us-east-1d"},
{"token":"188627880"," hostname":"foo22" , "zone":"us-east-1d"},
{"token":"3051939411","hostname":"foo31" , "zone":"us-east-1e"},
{"token":"188627880"," hostname":"foo32" , "zone":"us-east-1e"}
]

Here is the example of simple json that TokenMapSupplierImpl uses for 6 nodes in 3 racks, where we have 2 nodes per rack, hence RF = 3.

You just need to add a dummy TokenMapSupplier implementation to the config object for Dyno that uses information about your dynomite servers along with their tokens.

I'm working on adding a class that will do this if you give it the right json. But in the meantime you can construct this yourself.
Hope this is helpful.

from dyno.

Thank you opuneet! I'll try this ASAP and get back

from dyno.

I got it to work using my own TokenMapSupplier.

BTW there are some limitations I found like Pipeline not supporting more than one key. Will keep investigating! Thanks for your help!

from dyno.

Yes, I've documented that about Pipelining. Pipeline works well when all keys go to the same server.
Since Dynomite is distributed Redis, this functionality is hard to support.

There will be improvements to pipelining in the near future.

from dyno.

@gonzajg thanks for your patience. I just uploaded a default class that will make it easier for you to implement your own.

Here is the wiki guide
https://github.com/Netflix/dyno/wiki/Getting-started-with-Redis-client#adding-in-your-own-tokenmapsupplier

from dyno.

Hello Active Dyno Contributers:

@opuneet and @gonzajg and @ipapapa @quidryan ,

I am following same pattern for calling Dynomite and Redis running on an EC2 instance. I can use Redis CLI and can see Get and Set is working fine in EC2 which means Dynomite and Redis configuration is fine.

In Java Layer, To build the DynoJedisClient, I am providing Host Supplier and Token Map supplier as suggested above. I am verifying this through fetching ConnectionPool object from already built Dyno Client. All supplied values are coming properly. But i execute it i get the same error as @gonzajg got above.
Post debugging, i found that once it tries to create all configuration like Connection Pool, Selection Strategy..etc, everything comes fines except ConcurrentHashMap<Host, HostToken>. This map comes with null value hence there is no host token available error.

Need Immediate help:

Code Reference:

public class TestTokenMapSupplier extends AbstractTokenMapSupplier {

final String json = "[{\"token\":\"12345678\",\"hostname\":\"MyHost Name\",\"zone\":\"us-east-1c\"}]\"";


@Override
public String getTopologyJsonPayload(String hostname) {
    return json;
}

@Override
public String getTopologyJsonPayload(Set<Host> hosts){
    return json;
}

****************************************************************************************************


public class CustomHostSupplier implements HostSupplier {

final List<Host> hosts = new ArrayList<Host>();

@Override
public Collection<Host> getHosts() {

    hosts.add(new Host("myHost", 8102,
            Status.Up).setRack("rack"));
    return hosts;
}

Client Code:

// ConnectionPoolConfigurationImpl class to supply Token
dynoClient = new DynoJedisClient.Builder()
.withApplicationName("APP")
.withDynomiteClusterName("CLUSTER")
.withHostSupplier(customHostSupplierClass)
.withCPConfig(new ConnectionPoolConfigurationImpl("APP").withTokenSupplier(Object of TestTokenMapSupplier))
.build();
}

Error:
data::com.netflix.dyno.connectionpool.exception.NoAvailableHostsException: NoAvailableHostsException: [host=Host [name=UNKNOWN, port=0, dc: null, status: Down], latency=0(0), attempts=0]Token not found for key hash: 339455788
at com.netflix.dyno.connectionpool.impl.hash.BinarySearchTokenMapper.getToken(BinarySearchTokenMapper.java:68)
at com.netflix.dyno.connectionpool.impl.lb.TokenAwareSelection.getTokenForKey(TokenAwareSelection.java:110)
at com.netflix.dyno.connectionpool.impl.lb.TokenAwareSelection.getPoolForOperation(TokenAwareSelection.java:73)
at com.netflix.dyno.connectionpool.impl.lb.HostSelectionWithFallback.getConnection(HostSelectionWithFallback.java:111)
at com.netflix.dyno.connectionpool.impl.lb.HostSelectionWithFallback.getConnection(HostSelectionWithFallback.java:100)
at com.netflix.dyno.connectionpool.impl.ConnectionPoolImpl.executeWithFailover(ConnectionPoolImpl.java:291)
at com.netflix.dyno.jedis.DynoJedisClient.d_get(DynoJedisClient.java:302)
at com.netflix.dyno.jedis.DynoJedisClient.get(DynoJedisClient.java:296)

from dyno.

Reopening the issue.

from dyno.

@goru1984 I resurrected a simple working example of using a custom token supplier against redis running on the localhost. See CustomTokenSupplierExample.

The proper way to fix this, however, is not to rely on a sidecar for this information. Dynomite itself should publish this and we are actively working on that. Once that is the case I'll modify the Dyno client appropriately and you will not need to supply your own implementation.

Alternatively until then, you can easily change the configuration of the client to use a round-robin load balancing strategy if that is an option, which would also alleviate providing a custom token supplier implementation.

from dyno.

@jcacciatore $ @ipapapa

Thanks for your quick turn around. I tried above example in my local, it works fine. but when i try it in an EC2 instance, it fails with below error. Please note, I am able to access Redis through Dynomite in EC2 instance using Redis-CLI so can safely say that configuration should be fine on EC2 side. Please help me with the resolution.

Also, do you have any ETA on releasing the Dynomite support as said in your previous comment?

Error

[main] INFO com.netflix.dyno.connectionpool.impl.ConnectionPoolImpl - registered mbean com.netflix.dyno.connectionpool.impl:type=MonitorConsole
com.netflix.dyno.connectionpool.exception.NoAvailableHostsException: NoAvailableHostsException: [host=Host [name=UNKNOWN, port=0, dc: null, status: Down], latency=0(0), attempts=0]Could not find host connection pool for key: ABC, hash: 899048765
at com.netflix.dyno.connectionpool.impl.lb.TokenAwareSelection.getPoolForOperation(TokenAwareSelection.java:81)
at com.netflix.dyno.connectionpool.impl.lb.HostSelectionWithFallback.getConnection(HostSelectionWithFallback.java:111)
at com.netflix.dyno.connectionpool.impl.lb.HostSelectionWithFallback.getConnection(HostSelectionWithFallback.java:100)
at com.netflix.dyno.connectionpool.impl.ConnectionPoolImpl.executeWithFailover(ConnectionPoolImpl.java:291)
at com.netflix.dyno.jedis.DynoJedisClient.d_set(DynoJedisClient.java:1226)
at com.netflix.dyno.jedis.DynoJedisClient.set(DynoJedisClient.java:1216)

from dyno.

I'll get an ETA regarding the Dynomite support.

Can you post your full log with debug logging enabled for the com.netflix.dyno package ?

from dyno.

Closing this because of inactivity. We can always reopen it if needed.

from dyno.