digital object load task - incomplete indexing

[Tim Hutchinson]

We've run into a problem with the digitalobject:load task ... which I haven't used before, so I'm not sure if we're using the wrong combination of options, or if a couple false starts have affected anything.

We first ran it with no options, which seemed fine but obviously the records weren't re-indexed.

After running it with the --index option, the thumbnails weren't appearing in the search results, so we tried it again with both --replace and --index.

That fixed the missing thumbnails, but now the search results for updated records show "untitled" for both the title and the parent collection. Dates and level of description have not been affected. (Screenshot attached.) The full records still have the correct title information.

Any suggestions? This is currently being run on our test box, so we don't necessarily need to fix these records, rather we'd like to figure out how to run the task correctly. I'd rather avoid re-indexing the full database if we can help it.

Thanks

Tim

[Steve Breker]

Hi Tim

That is odd - I will try to replicate your findings locally here...

Can you please forward to me:

- the full command string you used to initiate the import (php symfony digitalobject:load ..... )
- the CSV header row, plus the row that resulted in the image from your screenshot.

Thanks,

Steve

[Hutchinson, Tim]

Hi Steve,

 

Thanks for following up. We ran it again with a different test file (using records without a current digital object), and got the same results even without the replace option.

 

php symfony digitalobject:load --index /path/digital_object_load_test.csv
>> digital-object Load digital objects from /path/digital_object_load_test.csv...
(Mar 01, 12:50:26 PM) Loading 'A-10289.jpg' (1 of 6 remaining)
(Mar 01, 12:50:27 PM) Loading 'A-10290.jpg' (2 of 6 remaining)
(Mar 01, 12:50:27 PM) Loading 'A-10291.jpg' (3 of 6 remaining)
(Mar 01, 12:50:27 PM) Loading 'A-10292.jpg' (4 of 6 remaining)
(Mar 01, 12:50:28 PM) Loading 'A-10293.jpg' (5 of 6 remaining)
(Mar 01, 12:50:28 PM) Loading 'A-10308.jpg' (6 of 6 remaining)
>> digital-object Successfully Loaded 6 digital objects.

 

I’ve attached the CSV and a corresponding screenshot. The objects were loaded successfully but we have the same results with “Untitled”, for all records.

 

Tim

 

Tim Hutchinson
Head, University Archives and Special Collections

 

University of Saskatchewan

University Library

Ph: 306-966-6028 (voicemail)

 

 

From: ica-ato...@googlegroups.com <ica-ato...@googlegroups.com> On Behalf Of Steve Breker
Sent: Monday, March 01, 2021 2:14 PM
To: AtoM Users <ica-ato...@googlegroups.com>
Subject: [atom-users] Re: digital object load task - incomplete indexing

 

CAUTION: External to USask. Verify sender and use caution with links and attachments. Forward suspicious emails to phis...@usask.ca

 

--
You received this message because you are subscribed to a topic in the Google Groups "AtoM Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ica-atom-users/TTsFjUjAWcE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/a8997c63-753d-4d5c-b845-3aab5b1be297n%40googlegroups.com.

[Steve Breker]

Hi Tim

I am able to replicate this import behaviour you're seeing. It looks like adding the --index option is not causing the associated information object to be reindexed when the digital object is saved. 

Try running 'php symfony search:populate' to force the information objects to be re-indexed - this restored the titles on the Description browse page for me.

When loading the CSV, I used the same options as in the command you sent earlier: php symfony digitalobject:load --index /path/digital_object_load_test.csv

Let me know if that corrects the issue for you.

Steve

[Hutchinson, Tim]

Hi Steve,

 

Thanks for confirming the bug. We’re looking into whether it makes sense to rebuild the search index at this point (for only 60 new digital objects), but I’m sure that will work. I was able to manually force the re-indexing by resaving an individual record.

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/4a086f19-5e15-43ef-9d34-39d11510d975n%40googlegroups.com.

[Steve Breker]

Hi Tim - Thanks for getting back. I will be creating a ticket today for this. 

Regarding rebuilding the index: There are a few options to limit the effects of a full search:populate:

- use the --update option. This option is the same as a full re-index, except that the index is not deleted first. This leaves AtoM and search fully functional while the indexing task is running.

- use the --slug option to limit re-indexing to a specific slug and its descendants. This may not be feasible if there are too many slugs to individually run this task for.

- use the --exclude-types option to exclude any object type that is not an information object. This will probably have limited success since most objects in a given database are descriptions.

Steve

[Hutchinson, Tim]

Hi Steve,

 

Thanks for pointing out those options. Combining –slug and –update did the trick nicely, and in almost no time – all the affected records are children of the same collection.

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/38e108ba-7d4d-44df-87ba-5a62c7189953n%40googlegroups.com.