ElephantDB client for Java?

299 views
Skip to first unread message

Tushar Deshpande

unread,
Apr 16, 2013, 2:08:21 PM4/16/13
to elephan...@googlegroups.com
Hi,

I am trying to use ElephantDB's Thrift interface for Java to read the data.
Does the class elephantdb.generated.keyval.ElephantDB represent this
interface?  In that case, I believe that I would need to first get an instance
of the class ElephantDB.Client, so that I can invoke the "get" method.
The constructor for ElephantDB.Client accepts an object of type
org.apache.thrift.protocol.TProtocol.  Would someone please tell me how
to get a TProtocol instance?  I also wonder if I am going the right way.


Best regards,

Tushar

Soren Macbeth

unread,
Apr 16, 2013, 3:25:17 PM4/16/13
to elephan...@googlegroups.com
Hi Tushar,

Yes, client interfaces are still lacking. There is a python example in the elephantdb-thrift module. I suggest checking out the thrift documentation at http://thrift.apache.org/ for further information. There is an also an internal client implementation in elephantdb-server in clojure. checkout the "with-kv-connection" macro in elephantdb.keyval.core.



--
You received this message because you are subscribed to the Google Groups "elephantdb-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elephantdb-us...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 



--
http://about.me/soren

Tushar Deshpande

unread,
Apr 16, 2013, 3:39:58 PM4/16/13
to elephan...@googlegroups.com
Hi Soren,

> There is a python example in the elephantdb-thrift module.
Do you mean the elephantdbclient.py file? In that case, is
there any example/test case illustrating its usage?

You have also asked me to take a look at the internal client
from the elephantdb-server. However, my understanding of
clojure is very limited. So, it is going to be quite a chalenge
for me.

I also found a clojure client for the elephantdb while doing a
web search. The file can be accessed at
http://searchco.de/codesearch/raw/2453158.
Is this the client that is referred in the tutorial.

Would you please tell me how do you read the data from
elephantdb?


Best regards,

Tushar Deshpande
> You received this message because you are subscribed to a topic in the
> Google Groups "elephantdb-user" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/elephantdb-user/3_vm4TZSR2U/unsubscribe?hl=en.
> To unsubscribe from this group and all its topics, send an email to

Jeroen van Dijk

unread,
Apr 17, 2013, 6:24:17 AM4/17/13
to elephan...@googlegroups.com
Hi Tushar,

Here is some example Clojure code I pulled from one of my projects https://gist.github.com/jeroenvandijk/b69180a733ba4aa0d9b0

HTH,

Jeroen

Tushar Deshpande

unread,
Apr 18, 2013, 10:53:03 AM4/18/13
to elephan...@googlegroups.com
Thanks Jeroen.

I have some questions about using your client.

1) To use your client, I would first need define an environment variable
called ELEPHANTDB_IP.  In my case ELEPHANTDB_IP would be
set to localhost. 

2) The function "fetch-key*" accepts name of a domain and a key
and it returns the value corresponding to the key.  Similarly, the
function "fetch-keys*" return accepts multiple keys and returns
values corresponding to those keys.  Am I right?

3) I have created a uberjar for elephantdb server.  So, I believe that
to use your client, I would simply need to include this uberjar in the
classpath. 

Would you please tell me if I am right?


Thank you

Tushar

Jeroen van Dijk

unread,
Apr 18, 2013, 3:54:38 PM4/18/13
to elephan...@googlegroups.com
Hi Tushar,

Sorry I'm actually not sure if my code works for the latest Elephantdb. I might have been to quick assuming that the new version (0.4.2) works as the older version which I'm using (0.2.0-wip4). I tried the latest version today and I wasn't able to connect the posted code. I'll need to make this work sooner or later. When I find how to get this working I'll post it again.

I have tried to answer your questions anyway


1) To use your client, I would first need define an environment variable
called ELEPHANTDB_IP.  In my case ELEPHANTDB_IP would be
set to localhost. 

Actually I have never been able to use a hostname. Instead I had to use the local ip address (often 10.x.x.x) or the public ip address to get the key lookup working (key lookup uses the hosts information).


2) The function "fetch-key*" accepts name of a domain and a key
and it returns the value corresponding to the key.  Similarly, the
function "fetch-keys*" return accepts multiple keys and returns
values corresponding to those keys.  Am I right?

Yeah correct. I have only been able to use Strings as key. Note that the value is a Thrift object. Also note that if you haven't generate your shards properly you might experience problems when fetching keys.


3) I have created a uberjar for elephantdb server.  So, I believe that
to use your client, I would simply need to include this uberjar in the
classpath. 

Yeah that should work. The command would be something like

java -cp path-to-jar elephantdb.keyval.core hdfs-path-to-global-config.clj local_config.clj

Also make sure you give enough memory to the java process.

I would recommend looking here for some extra info and examples:

Both links are not up to date with the latest Elephantdb yet, but I haven't seen resources for the newer version yet.

HTH, 
Jeroen



Thank you

Tushar


On Wednesday, April 17, 2013 6:24:17 AM UTC-4, Jeroen van Dijk wrote:
Hi Tushar,

Here is some example Clojure code I pulled from one of my projects https://gist.github.com/jeroenvandijk/b69180a733ba4aa0d9b0

HTH,

Jeroen

Soren Macbeth

unread,
Apr 18, 2013, 4:01:06 PM4/18/13
to elephan...@googlegroups.com
The reason it doesn't work with the newer version is because getString and multiGetString calls have been removed. >= 0.4.0 only speaks byte[] and only supports get and multiGet. It's up to the use to decide how to serialize/deserialize the keys and value on their way in/out of edb.

Jeroen van Dijk

unread,
Apr 18, 2013, 5:37:42 PM4/18/13
to elephan...@googlegroups.com
Thanks Soren, that helps. I've updated the gist (https://gist.github.com/jeroenvandijk/b69180a733ba4aa0d9b0) to use the new API. Are old EDB shards supposed to work with >= 0.4.0? So far I only get #<Value Value()> using shards generated for 0.2.0-wip4.

Soren Macbeth

unread,
Apr 18, 2013, 6:02:18 PM4/18/13
to elephan...@googlegroups.com
Old shards would only work if you were already using byte arrays for keys and values, such as serialized thrift structs or something like that. 0.2.0 used kryo under the covers to serialize other types. in >= 0.4.0, kryo was removed.

So if you have old shards that aren't compatible, you have to do a bit of work with cascading/cascalog to read your old shards out using 0.2.0, convert the keys and values to byte arrays and write out new domains. After that, they should load up fine in >= 0.4.0.

I'm working on a blog post covering 0.4.0, updating the old 0.1.0 tutorial. I can't promise a date it will be done, but since activity has picked up here on the mailing list, the pressure is on ;)
Reply all
Reply to author
Forward
0 new messages