[VuFind-General] Unexpected side effects of `DeleteRecordIfFieldNotEmpty` on SolrMARC indexing (and a possible fix?)

[Demian Katz]

Peter,

 I’m copying the solrmarc-tech mailing list into this reply in case anyone there has additional insight.

 I haven’t heard of this problem before, but I also do not use this feature of SolrMarc, so I have not had an opportunity to experience it firsthand. Doesn’t seem like intended behavior, in any case!

 It’s been a couple of years since the last SolrMarc release (2023), so it may also be a good idea to start another conversation about the long-term future of the tool. Maybe we should at least do some dependency checks and put together a refreshed version.

 - Demian

 From: Peter Murray <pe...@indexdata.com>
Sent: Tuesday, January 20, 2026 3:27 PM
To: VuFind-General <vufind-...@lists.sourceforge.net>
Subject: [EXTERNAL] [VuFind-General] Unexpected side effects of `DeleteRecordIfFieldNotEmpty` on SolrMARC indexing (and a possible fix?)

 I'm working with libraries that use FOLIO as the integrated library system. When a record is suppressed in FOLIO, it comes through the OAI-PMH feed as a 999‡t of "1". I've been using this addition to `marc_local.properties` to delete the record from the Solr index (in case it had been previously loaded).

 # Delete suppressed records

delete_suppressed_str = 999t ? ($t == 1), DeleteRecordIfFieldNotEmpty

 And, for a while, that seemed to be working. But I heard from a library that said they were seeing the suppressed record in VuFind. Looking at the log from `import-marc.sh` I was seeing this:

 DEBUG [MarcReader-Thread] (MarcReaderThread.java:35) - record read : b3164499

DEBUG [MarcReader-Thread] (MarcReaderThread.java:35) - record read : b3164638

DEBUG [MarcReader-Thread] (MarcReaderThread.java:35) - record read : b2138853

DEBUG [MarcReader-Thread] (MarcReaderThread.java:35) - record read : in00000000235

DEBUG [MarcReader-Thread] (MarcReaderThread.java:35) - record read : in00000010045

DEBUG [MarcReader-Thread] (MarcReaderThread.java:35) - record read : in00000000462

DEBUG [MarcReader-Thread] (MarcReaderThread.java:35) - record read : b1283034

DEBUG [MarcReader-Thread] (MarcReaderThread.java:35) - record read : b3168472

INFO [RecordIndexer-Thread-0] (Indexer.java:258) - Record will be Deleted b3164499  (record count 1)

INFO [RecordIndexer-Thread-0] (Indexer.java:258) - Record will be Deleted b3164638  (record count 2)

INFO [RecordIndexer-Thread-0] (Indexer.java:258) - Record will be Deleted b2138853  (record count 3)

INFO [RecordIndexer-Thread-0] (Indexer.java:258) - Record will be Deleted in00000000235  (record count 4)

INFO [RecordIndexer-Thread-0] (Indexer.java:258) - Record will be Deleted in00000000462  (record count 6)

INFO [RecordIndexer-Thread-0] (Indexer.java:258) - Record will be Deleted b3168472  (record count 8)

INFO [main] (ThreadedIndexer.java:259) - Done with all indexing, finishing writing records to solr

INFO [main] (ThreadedIndexer.java:272) - Done writing records to solr

INFO [main] (Indexer.java:576) - Deleting 6 records

DEBUG [main] (Indexer.java:581) - Deleting record b3164499

DEBUG [main] (Indexer.java:581) - Deleting record b3164638

DEBUG [main] (Indexer.java:581) - Deleting record b2138853

DEBUG [main] (Indexer.java:581) - Deleting record in00000000235

DEBUG [main] (Indexer.java:581) - Deleting record in00000000462

DEBUG [main] (Indexer.java:581) - Deleting record b3168472

INFO [main] (Indexer.java:595) - Commmiting updates to Solr

INFO [main] (IndexDriver.java:391) - 8 records read

INFO [main] (IndexDriver.java:392) - 2 records indexed  and

INFO [main] (IndexDriver.java:399) - 2 records sent to Solr in 1.14 seconds

 The record in question is `b3168472`, so this looks right — marking that record for deletion (along with 5 others). I knew I had a problem when all 6 records marked "delete" were appearing in the Solr index. The interesting bit came from prepping logs of Solr itself for the record identifier:

solr.log.6:2026-01-20 15:32:06.577 INFO  (qtp616207929-164339) [ x:biblio] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={wt=javabin&version=2}{delete=[b3168472 (-1854850336373604352)]} 0 0

solr.log.6:2026-01-20 15:32:08.791 INFO  (qtp616207929-164297) [ x:biblio] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr path=/update params={wt=javabin&version=2}{add=[b3164499 (1854850338683617280), b3164638 (1854850338689908736), b2138853 (1854850338689908737), in00000000235 (1854850338692005888), in00000010045 (1854850338693054464), in00000000462 (1854850338694103040), b1283034 (1854850338694103041), b3168472 (1854850338695151616)]} 0 10

 Hold up — I'm deleting the record, then two seconds later adding it back again?!?

 So what I think is happening is that the "DeleteRecordIfFieldNotEmpty" is indeed marking the record to be deleted, but it is not stopping the indexing pipeline from sending an "add" for the record when it finishes processing the rules. What seems to work is adding another rule that explicitly skips the record:

 # Delete suppressed records

delete_suppressed_str = 999t ? ($t == 1), DeleteRecordIfFieldNotEmpty

skip_suppressed_str = 999t ? ($t == 1), SkipRecordIfFieldNotEmpty

 Does this match anyone else's expectations or experience?

  Peter

--

Peter Murray

Open Source Community Advocate

Index Data, LLC