digital object load task - incomplete indexing

63 views
Skip to first unread message

Tim Hutchinson

unread,
Feb 25, 2021, 7:18:42 PM2/25/21
to AtoM Users
We've run into a problem with the digitalobject:load task ... which I haven't used before, so I'm not sure if we're using the wrong combination of options, or if a couple false starts have affected anything.

We first ran it with no options, which seemed fine but obviously the records weren't re-indexed.

After running it with the --index option, the thumbnails weren't appearing in the search results, so we tried it again with both --replace and --index.

That fixed the missing thumbnails, but now the search results for updated records show "untitled" for both the title and the parent collection. Dates and level of description have not been affected. (Screenshot attached.) The full records still have the correct title information.

Any suggestions? This is currently being run on our test box, so we don't necessarily need to fix these records, rather we'd like to figure out how to run the task correctly. I'd rather avoid re-indexing the full database if we can help it.

Thanks
Tim
screenshot-test.memorysask.ca-2021.02.25-18_13_08.png
Message has been deleted

Steve Breker

unread,
Mar 1, 2021, 3:13:34 PM3/1/21
to AtoM Users
Hi Tim

That is odd - I will try to replicate your findings locally here...

Can you please forward to me:
- the full command string you used to initiate the import (php symfony digitalobject:load ..... )
- the CSV header row, plus the row that resulted in the image from your screenshot.

Thanks,
Steve

Hutchinson, Tim

unread,
Mar 1, 2021, 4:08:24 PM3/1/21
to ica-ato...@googlegroups.com

Hi Steve,

 

Thanks for following up. We ran it again with a different test file (using records without a current digital object), and got the same results even without the replace option.

 

php symfony digitalobject:load --index /path/digital_object_load_test.csv
>> digital-object Load digital objects from /path/digital_object_load_test.csv...
(Mar 01, 12:50:26 PM) Loading 'A-10289.jpg' (1 of 6 remaining)
(Mar 01, 12:50:27 PM) Loading 'A-10290.jpg' (2 of 6 remaining)
(Mar 01, 12:50:27 PM) Loading 'A-10291.jpg' (3 of 6 remaining)
(Mar 01, 12:50:27 PM) Loading 'A-10292.jpg' (4 of 6 remaining)
(Mar 01, 12:50:28 PM) Loading 'A-10293.jpg' (5 of 6 remaining)
(Mar 01, 12:50:28 PM) Loading 'A-10308.jpg' (6 of 6 remaining)
>> digital-object Successfully Loaded 6 digital objects.

 

I’ve attached the CSV and a corresponding screenshot. The objects were loaded successfully but we have the same results with “Untitled”, for all records.

 

Tim

 

Tim Hutchinson
Head, University Archives and Special Collections

 

University of Saskatchewan

University Library

Ph: 306-966-6028 (voicemail)

 

 

From: ica-ato...@googlegroups.com <ica-ato...@googlegroups.com> On Behalf Of Steve Breker
Sent: Monday, March 01, 2021 2:14 PM
To: AtoM Users <ica-ato...@googlegroups.com>
Subject: [atom-users] Re: digital object load task - incomplete indexing

 

CAUTION: External to USask. Verify sender and use caution with links and attachments. Forward suspicious emails to phis...@usask.ca

 

--
You received this message because you are subscribed to a topic in the Google Groups "AtoM Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ica-atom-users/TTsFjUjAWcE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/a8997c63-753d-4d5c-b845-3aab5b1be297n%40googlegroups.com.

digital_object_load_test.csv
screenshot-test.memorysask.ca-2021.03.01-15_02_38.png

Steve Breker

unread,
Mar 2, 2021, 6:53:45 PM3/2/21
to AtoM Users
Hi Tim

I am able to replicate this import behaviour you're seeing. It looks like adding the --index option is not causing the associated information object to be reindexed when the digital object is saved. 

Try running 'php symfony search:populate' to force the information objects to be re-indexed - this restored the titles on the Description browse page for me.

When loading the CSV, I used the same options as in the command you sent earlier: php symfony digitalobject:load --index /path/digital_object_load_test.csv

Let me know if that corrects the issue for you.

Steve

Hutchinson, Tim

unread,
Mar 4, 2021, 5:03:18 PM3/4/21
to ica-ato...@googlegroups.com

Hi Steve,

 

Thanks for confirming the bug. We’re looking into whether it makes sense to rebuild the search index at this point (for only 60 new digital objects), but I’m sure that will work. I was able to manually force the re-indexing by resaving an individual record.

Steve Breker

unread,
Mar 5, 2021, 3:53:22 PM3/5/21
to AtoM Users
Hi Tim - Thanks for getting back. I will be creating a ticket today for this. 

Regarding rebuilding the index: There are a few options to limit the effects of a full search:populate:

- use the --update option. This option is the same as a full re-index, except that the index is not deleted first. This leaves AtoM and search fully functional while the indexing task is running.
- use the --slug option to limit re-indexing to a specific slug and its descendants. This may not be feasible if there are too many slugs to individually run this task for.
- use the --exclude-types option to exclude any object type that is not an information object. This will probably have limited success since most objects in a given database are descriptions.


Steve

Hutchinson, Tim

unread,
Mar 5, 2021, 4:48:26 PM3/5/21
to ica-ato...@googlegroups.com

Hi Steve,

 

Thanks for pointing out those options. Combining –slug and –update did the trick nicely, and in almost no time – all the affected records are children of the same collection.

Reply all
Reply to author
Forward
0 new messages