Death rows in Elasticsearch indicies

[NikMV]

Hello everyone!

I have one problem with this awesome plugin...

I wont to realize two cases: 1) searching data in postgres and 2)searching data straight in Elastic.

In first case I found a lot of docs and examples, it's great and clearly, but when i tried work with Elastic, then in result hits i get useful docs and docs, which generated from dead rows in tables (after update, for example).

When i use  function "vacuum", indicies in ES came normal. I don't want use "vacuum" more than one time at day. How disable in ZB function of storage old records/death rows? 

Or may be reconstruct ES request to aggregate only actual docs in pull of hits? 

[Eric Ridge]

You can't. This isn't a supported use case. ZomboDB tracks Postgres MVCC visibility information in Elasticsearch so that it can provide transactionally-correct results (via Postgres) for all queries, including those that are wholly answered by Elasticsearch (the various aggregate functions).

While it is possible, as you've found, to query ES directly, you'll get these "dead" rows in your results. Assuming you're using the latest version of ZDB against ES 6.x, you might can figure out a scheme using ES' "field collapsing" feature (https://www.elastic.co/guide/en/elasticsearch/reference/6.4/search-request-collapse.html), but you'll risk getting back an old row instead of what is the most current version.

Again, this really isn't a supported use case and is briefly discussed here: https://github.com/zombodb/zombodb/blob/master/THINGS-TO-KNOW.md#external-tools-like-kibana-are-supported

eric

> --
> You received this message because you are subscribed to the Google Groups "ZomboDB" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to zombodb+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/zombodb/fc9ea957-fcb0-4764-99c8-0a54fd809ef8%40googlegroups.com.

[NikMV]

Thank you very much for the detailed answer and the time spent.

вторник, 12 ноября 2019 г., 20:48:37 UTC+3 пользователь Eric Ridge написал:

> To unsubscribe from this group and stop receiving emails from it, send an email to zom...@googlegroups.com.