Voldemort when connected to remote server it searches the data in the localhost

[Samyums]

Hi,

I have server A(x.y.z.a) where voldemort 0.80.1 is set up

I have server B where voldemort 0.80.1 is set up

When i tried to connect to server A from voldemort client at B 

with the following command

bin/voldemort-shell.sh datastore tcp://x.y.z.a:6666

It establishes connection with the server

But when i try to do a get operation 

it throws the following exception

voldemort.store.UnreachableStoreException: Failure while checking out socket for localhost:6666(vp1):

I dont understand why it is checking the socket for localhost:6666 when i am connected to x.y.z.a

Please suggest

Thanks,

Samyuktha. 

[Samyums]

Complete stack trace is as follows

Exception thrown during operation.

voldemort.store.InsufficientOperationalNodesException: 1 reads required, but 0 succeeded.

        at voldemort.store.routed.RoutedStore.get(RoutedStore.java:562)

        at voldemort.store.routed.RoutedStore.get(RoutedStore.java:454)

        at voldemort.store.routed.RoutedStore.get(RoutedStore.java:72)

        at voldemort.store.DelegatingStore.get(DelegatingStore.java:59)

        at voldemort.store.stats.StatTrackingStore.get(StatTrackingStore.java:64)

        at voldemort.store.serialized.SerializingStore.get(SerializingStore.java:79)

        at voldemort.store.DelegatingStore.get(DelegatingStore.java:59)

        at voldemort.store.versioned.InconsistencyResolvingStore.get(InconsistencyResolvingStore.java:50)

        at voldemort.client.DefaultStoreClient.get(DefaultStoreClient.java:121)

        at voldemort.client.DefaultStoreClient.get(DefaultStoreClient.java:154)

        at voldemort.VoldemortClientShell.processCommands(VoldemortClientShell.java:126)

        at voldemort.VoldemortClientShell.main(VoldemortClientShell.java:89)

        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

        at java.lang.reflect.Method.invoke(Method.java:616)

        at jline.ConsoleRunner.main(ConsoleRunner.java:69)

Caused by: voldemort.store.UnreachableStoreException: Failure while checking out socket for localhost:6666(vp1):

        at voldemort.store.socket.SocketPool.checkout(SocketPool.java:92)

        at voldemort.store.socket.SocketStore.get(SocketStore.java:121)

        at voldemort.store.socket.SocketStore.get(SocketStore.java:47)

        at voldemort.store.logging.LoggingStore.get(LoggingStore.java:105)

        at voldemort.store.routed.RoutedStore$1.execute(RoutedStore.java:79)

        at voldemort.store.routed.RoutedStore$GetCallable.call(RoutedStore.java:900)

        at voldemort.store.routed.RoutedStore$GetCallable.call(RoutedStore.java:880)

        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)

        at java.util.concurrent.FutureTask.run(FutureTask.java:166)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)

        at java.lang.Thread.run(Thread.java:636)

Caused by: java.net.ConnectException: Connection refused

        at java.net.PlainSocketImpl.socketConnect(Native Method)

        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:310)

        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:176)

        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:163)

        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:384)

        at java.net.Socket.connect(Socket.java:542)

        at voldemort.store.socket.SocketResourceFactory.create(SocketResourceFactory.java:84)

        at voldemort.store.socket.SocketResourceFactory.create(SocketResourceFactory.java:41)

        at voldemort.utils.pool.KeyedResourcePool.attemptGrow(KeyedResourcePool.java:153)

        at voldemort.utils.pool.KeyedResourcePool.checkoutOrCreateResource(KeyedResourcePool.java:133)

        at voldemort.utils.pool.KeyedResourcePool.checkout(KeyedResourcePool.java:106)

        at voldemort.store.socket.SocketPool.checkout(SocketPool.java:87)

        ... 11 more

[Brendan Harris]

Hi Samyums,

I have seen people run into this problem before when the cluster.xml on server A has "localhost" as it's host name. The way it works is, the client connects to server A at IP x.y.z.a:p and then you pull down the cluster.xml from it and parse it. Then the client gets the host name for server A from cluster.xml and uses that for the routing of its requests.

Brendan

[Samyums]

Hi Brendan,

Thanks for the info. One more doubt. In the server we may have multiple cluster.xml files right. But when we start the server we give which config to run say single_node_cluster. How does client identify to know that which cluster.xml does it need to refer to?

Do you have any doc or reference which describes the client and server interaction of voldemort.

[Brendan Harris]

Hi Samyuthka,

On Wednesday, January 16, 2013 11:11:34 PM UTC-8, Samyuktha M S wrote:

Thanks for the info. One more doubt. In the server we may have multiple cluster.xml files right. But when we start the server we give which config to run say single_node_cluster. How does client identify to know that which cluster.xml does it need to refer to?

The server starts with the cluster.xml that you start it up with. So, there is only one actively running cluster.xml per server. When you point the client to a bootstrap URL (tcp://hostname:port), it "bootstraps" off of the target host, reading in the server's running cluster.xml.

 

Do you have any doc or reference which describes the client and server interaction of voldemort.

I searched around and it looks like this process is not documented, unfortunately. This is one of those things that you learn either from experience or reading the code. I'll talk to the team about getting it properly documented on project-voldemort.com and the github wiki.

Brendan

[Vinoth Chandar]

Added a note in http://www.project-voldemort.com/voldemort/quickstart.html mentioning this.

[Samyuktha M S]

Thanks Brendan and Vinoth!!