Hi,
we store "tag like" data in multi value fields and need to update those frequently, which is a performance problem. At the moment we have about 5 million documents and an update might affect 50% of those documents, which is already causing trouble. In the future, we want to do this with more than 20 million documents. As far as I understand, SOLR/HS is rewriting the whole document if we update a single field. Is that true? If yes: This is obviously not efficient in our use case. Is this a limitation of SOLR/HS and would it be realistic to optimize our use case by writing some Java code? Or is it a limitation of deep internals of Lucene, which are hard to change? It is still hard to grasp for me, what's going on which abstraction level.
Any hint regarding such a frequent high volume update scenario would be very appreciated.
kind regards,
Achim