Skip to first unread message
Brandon Uhlman
Jul 27, 2018, 11:34:04 AM7/27/18
to AtoM Users
Hi, AtoM community.
Thanks to some typos in a CSV import file and some judicious misuse of the CSV importing utility, we've ended up with some archival descriptions that are in some kind of limbo state. Partially deleted? Partially created? We're not sure.
Essentially, having missed Steve Breker's excellent June 21 explanation of why our roundtrip CSV imports were failing to match on legacy ID [1], we continued working away anyway. One day, we attempted to roundtrip import a set of CSV records where the identifiers stayed the same in the roundtrip, but the titles (the other matchpoint on the fallback matching mechanism) of some objects changed. Some other information was probably changed in many of the records as well, but it is not pertinent to this problem.
While creating the 'delete and replace' update job, we did not choose the 'Skip unmatched records' option, as suggested in the documentation [2]. As a result, when this job ran, most of the incoming records deleted the matching record already in AtoM, but the records with updated titles created new information_objects. The unexpected behavior is that the new information_object ends up in the nested set where the old object was, and the old information_object loses its parent_id and becomes a top-level object -- possibly related to [3], though we didn't see that note in the job log.
If that were all that was happening, we would probably be fine, because we could identify the objects and delete them from the GUI. But, for some reason, the old (now top-level) information object isn't retrievable by the GUI ('Sorry, page not found') even though the slug still exists and points at the right place. And I think I've found the problem -- these new information_objects have very broken representations in the nested set. Specifically, they all have the same lft and rgt values, those values are each correctly used by another object, and the rgt > lft, which should be impossible.
Since we can identify them all this way, it seems like the easiest way to get rid of them will be to delete them using this [4] script and the erroneous lft and rgt values as matchpoints. Am I missing something obvious?
Brandon
[1] https://groups.google.com/d/msg/ica-atom-users/6gZjcgBl6q0/BB7nCUMLBgAJ
[2] https://www.accesstomemory.org/en/docs/2.4/user-manual/import-export/csv-import/#delete-matches-and-replace-with-imported-records
[3] https://www.accesstomemory.org/en/docs/2.4/user-manual/import-export/csv-import/#legacyid-and-parentid
[4] https://groups.google.com/d/msg/ica-atom-users/KJKSAvFm15I/QcxXNEAWAAAJ
Thanks to some typos in a CSV import file and some judicious misuse of the CSV importing utility, we've ended up with some archival descriptions that are in some kind of limbo state. Partially deleted? Partially created? We're not sure.
Essentially, having missed Steve Breker's excellent June 21 explanation of why our roundtrip CSV imports were failing to match on legacy ID [1], we continued working away anyway. One day, we attempted to roundtrip import a set of CSV records where the identifiers stayed the same in the roundtrip, but the titles (the other matchpoint on the fallback matching mechanism) of some objects changed. Some other information was probably changed in many of the records as well, but it is not pertinent to this problem.
While creating the 'delete and replace' update job, we did not choose the 'Skip unmatched records' option, as suggested in the documentation [2]. As a result, when this job ran, most of the incoming records deleted the matching record already in AtoM, but the records with updated titles created new information_objects. The unexpected behavior is that the new information_object ends up in the nested set where the old object was, and the old information_object loses its parent_id and becomes a top-level object -- possibly related to [3], though we didn't see that note in the job log.
If that were all that was happening, we would probably be fine, because we could identify the objects and delete them from the GUI. But, for some reason, the old (now top-level) information object isn't retrievable by the GUI ('Sorry, page not found') even though the slug still exists and points at the right place. And I think I've found the problem -- these new information_objects have very broken representations in the nested set. Specifically, they all have the same lft and rgt values, those values are each correctly used by another object, and the rgt > lft, which should be impossible.
Since we can identify them all this way, it seems like the easiest way to get rid of them will be to delete them using this [4] script and the erroneous lft and rgt values as matchpoints. Am I missing something obvious?
Brandon
[1] https://groups.google.com/d/msg/ica-atom-users/6gZjcgBl6q0/BB7nCUMLBgAJ
[2] https://www.accesstomemory.org/en/docs/2.4/user-manual/import-export/csv-import/#delete-matches-and-replace-with-imported-records
[3] https://www.accesstomemory.org/en/docs/2.4/user-manual/import-export/csv-import/#legacyid-and-parentid
[4] https://groups.google.com/d/msg/ica-atom-users/KJKSAvFm15I/QcxXNEAWAAAJ
Dan Gillean
Jul 30, 2018, 4:05:12 PM7/30/18
to ICA-AtoM Users
Hi Brandon,
There are a few steps you can try first to see what issues you can resolve with the corrupted data. Even if you choose to use the script shared in the forum in note [4] of your previous post, it may help AtoM find those descriptions if there are not other issues.
There are tasks to rebuild the nested set, and regenerate slugs, for example. You can find links to these tasks, as well as other suggestions for resolving data corruption issues, in this section of our Troubleshooting page:
- Regenerating slugs:
- https://www.accesstomemory.org/docs/2.5/admin-manual/maintenance/troubleshooting/#generating-slugs
- https://www.accesstomemory.org/docs/latest/admin-manual/maintenance/cli-tools/#cli-generate-slugs
- Rebuilding the nested set:
You will probably want to restart services and possibly repopulate the search index after as well.
This should hopefully allow you to access the records via the GUI. If you choose to try out the script shared note that:
- It is UNTESTED - you should definitely back up your data and proceed at your own risk
- It is designed to delete BLANK descriptions - those with no title, and no other data. If your records are partially created and include some data, then they might not be affected by that script
- The target descriptions in that case had not children. If your imports have ended up as part of an existing hierarchy, then the delete cascade may also delete any child records!
I'm hoping that rebuilding the nested set and regenerating slugs will help to correct some of these issues for you. Let us know how it goes.
Regards,
--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/07a57552-1345-4647-9185-5c8e7917988b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages