REST Async

[David Smith]

Has anyone looked into creating an async version of the REST API?  It may be possible quite easily with something that has an API based on clj-http such as https://github.com/ztellman/aleph, personally I am a fan of aleph as it uses Manifold which aims to provide some interop between different Clojure async libraries.

[Michael Klishin]

We haven't heard of such experiments.

If you are looking for higher throughput, native client offers x5 better
results than the REST one on write-heavy workloads.

For us to consider implementing an async REST client, it should be an entirely
optional thing — no extra dependencies for "regular" REST and native client users.

Honestly, I think the native client will still be more efficient *and* it already uses
asynchronous I/O (Netty-based). 
--
@michaelklishin, github.com/michaelklishin

[David Smith]

The problem is that many people (including myself) shy away from the native client as you have to put more effort in when upgrading either the ES cluster or the client library.  The REST API is generally a lot more stable than the native API.

I imagined an async api being optional although it would still require a dependency (that could be excluded).  Alternatively using Aleph or http-kit for the async client and just making a synchronous version by simply dereffing the responses, you could get rid of the clj-http dependency.  Aleph has a few less dependencies than clj-http as well which is nice.

[Michael Klishin]

 On 17 March 2015 at 14:50:11, David Smith (shmi...@gmail.com) wrote:
> The problem is that many people (including myself) shy away
> from the native client as you have to put more effort in when upgrading
> either the ES cluster or the client library.

That was true before ES 1.0 but not any more.
--
@michaelklishin, github.com/michaelklishin

[David Smith]

Hmm, interesting.  We've also had some issues way in the past due to the client being an actual node in the cluster but that was ES 0.19 something and I am sure it isn't a problem anymore.

If versions aren't a problem any more then what are the reasons for using REST if you are on the JVM?  We may move to native if we are confident that we won't regret it in 12 months time :)

[Michael Klishin]

 On 17 March 2015 at 15:24:50, David Smith (shmi...@gmail.com) wrote:
> Hmm, interesting. We've also had some issues way in the past
> due to the client being an actual node in the cluster but that was
> ES 0.19 something and I am sure it isn't a problem anymore.

ES nodes can decide whether they will have shards allocated to them (and thus store data),
see http://www.elastic.co/guide/en/elasticsearch/client/java-api/current/client.html
(search for "hold data").

You should be able to pass any node configuration options you want with Elastisch.

> If versions aren't a problem any more then what are the reasons
> for using REST if you are on the JVM? 

Some environments only allow inbound requests on common HTTP ports.

> We may move to native if we are
> confident that we won't regret it in 12 months time :)

Most heavy Elastisch users (at least those who contribute regularly) use native client.
--
@michaelklishin, github.com/michaelklishin