How to upgrade several clusters to ES 2.0 with Elastica?

[Olivier]

Hi!

Here is my first message on this user group. :)

First of all, let me introduce myself. I'm Olivier Dolbeau and I work at
BlaBlaCar since a few years.
I used to be developer and I'm know on the Ops side.

Anyway, I would like to explain our problematic!
We run several elasticsearch cluster for different needs. Trips search
on the website, member search on the backoffice, etc. All those clusters
use ES 1.7.x.
As you can guess we use Elastica each time we need to access to those
clusters, otherwise, I won't be here. ;)

We would like to update all our clusters to ES 2.x. Obviously, it's
quite a huge task as we're still using facets everywhere for example.
BTW we will update our clusters one by one.

Anyway, my question is pretty simple! :)
Which version of elastica should we use to achieve this migration?

Elastica 2.3.1 seems to be a good choice. It's supposed to be compatible
with both ES 1.x & 2.x right?

If you have any question regarding our ES usage or anything, feel free
to ask. :)

Olivier

[ruflin]

Some facts to get started: Officially Elastica 3.x only supports Elasticsearch 2.x and Elastica 2.x only Elasticsearch 1.x

The main reason for the above is that ES 2.0 has BC breaks which lead to BC breaks in Elastica. If Elastica would have logic to deal with the different Elasticsearch versions, it would have to know before sending a request, which ES version it is (which it doesn't).

That being said, lets see what we could to to make this migration possible. Here some initial points:

* If you still use facets, you must get rid of them anyways in your code before upgrading.

* In general I would "try" to switch to Elastica 3.x to see what all breaks with elasticsearch 1.x. The good thing about Elastica 3.x is that it will also tell you, which of the code parts will probably not work anymore with elasticsearch 2.x and you will have to change it in advance

Obviously it would be best, if you could do the migration cluster by cluster, also to gain experience with the migration. So are there some applications that only talk to one cluster or is it a monolit that talks to different clusters? If not, you could for example first migrate the logging cluster (if there is one) etc.

[Nikolay Nikolov]

There are many changes especially on the filter side :-( We also move Elastica to 3.1 and there is a lot of work to make everything running.

On Friday, February 12, 2016 at 3:39:29 PM UTC+1, Olivier wrote:

[ruflin]

@Nikolay: When you upgrade, make sure you read http://elastica.io/2016/01/04/release-3-dot-0-0/ and also understand the changes on the elasticsearch side. Because most of the changes you have to make on the Elastica side are because of BC changes in elasticsearch 2.0.

[Nikolay Nikolov]

Hi ruflin, yes you have right! The changes are coming from ElasticSearch. I have manage the most of them. The only one are the nested aggregations. can you help me there? I have posted an example in one other post.