The digitalobject:load command does not perform any action

[Portales Municipales]

Good afternoon, I've been using this command so far without problems to make loads between 500 and 1300 images to different descriptions already created. For two days, when executing a command how this is:

php symfony digitalobject:load --index --path=/ /home/archivo/publico/CSV/listado_839355.csv

Where listado_839355.csv has this pint:

839355,/home/archivo/publico/MUNICIPALES_IMS/CORONIL/actas_pleno/LIB_14/Pagina_0001.jpg
839355,/home/archivo/publico/MUNICIPALES_IMS/CORONIL/actas_pleno/LIB_14/Pagina_0002.jpg
......

The different files that are being charged at the Linux exit appear but when checking it in Atom the images are not climbed to the description, I also do not see them by any other site.

I have tried to use Search: Populate in case they had not been indexed but they still do not appear.

We have recently updated version 2.6.1 although I could upload images successfully so far. I hope you can give me a solution. Thanks in advance, regards.

[Portales Municipales]

Hi Dan, I just made another execution test of digitalobject: load, it seems that it rises well the 1025 images and also check that the server is decreasing capacity, so I deduct that the images are importing correctly but are not associated with the description with the object_id that I indicated.

[Portales Municipales]

Hello, at least I know what the problem is. The images are climbed and indexed but do not nest under their description. Once you finish uploading, I run the propel:build-nested-set --exclude-tables = "term, menu" and appear on your site. Why can this happen? Greetings.

[Dan Gillean]

Hi there, 

Without seeing your whole process, it's difficult to determine what's happening. However, from the sounds of it, I think you are attempting to link more than one digital object to the same description (e.g. Pagina_0001, Pagina_0002). Is this correct?

We have a "notes on use" section at the end of the task documentation that describes what can happen in this case: 

NOTES ON USE

• If a single CSV row points to a description that already has a digital object, then the row will be skipped and reported in the console

• If the CSV contains multiple rows pointing at a description that already has a digital object, then new stub child descriptions will be created below the target, and digital objects will be attached there. If child descriptions already exist, they will be ignored (meaning, running the task more than once will result in duplicate child descriptions).

• Note that the --attach-only and --replace options change the above default behaviors when multiple rows point to one description. Read the option descriptions above for more information.

• Remember to repopulate the search index afterwards if you haven’t used the --index option! For more information, see: Populate search index.

• Additionally, if you use the --skip-nested-set-build option, you will need to manually rebuild the nested set after the task has completed. See: Rebuild the nested set.

I believe that the second bullet point above might apply to your case. Some of the digital objects you are attaching are being added to new child-level descriptions. These descriptions are being created as stubs, meaning they are new inserts into the hierarchy.

AtoM currently uses a nested set model as a way of maintaining hierarchical information in the flat, table-based structure of a relational database. If the above assumption is correct, then it seems likely that the "build nested set" option may not be running at the end of the task as expected - either that, or you are using the --skip-nested-set-build option when running the task, and therefore need to do so manually after the task completes. 

Similarly, by default the task will NOT update the search index as it progresses - this allows the task to run more quickly, but expects you to manually run the search:populate task at the end of the operation. You can use the --index option with the task if you would like the index to be updated progressively - this will make the task run slower overall, but also means you don't need to run a full re-index at the end. If your import is small, I would say you can try using the --index option, but if it's fairly big, it's likely better to run it manually at the end. 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/a59e613b-6241-4801-b712-53c7e7cf05f6n%40googlegroups.com.

[Portales Municipales]

Good afternoon, indeed, the problem is that the images are not nested at the end, although they are uploaded correctly. I don't use the --skip-nested-set-build option, I just use --index to get it indexed. The funny thing is that a few days ago it worked perfectly: image uploading, nesting and indexing and now it doesn't. Is it possible that it is the fault of a large number of images that are uploaded at the same time? Maybe you run out of memory? Greetings.

[Dan Gillean]

Hi again, 

Yes, running out of available memory is one possible reason this could be happening. It's also possible that you are hitting memory limits defined by the PHP execution limits. You can read more about this, and how to change some parameters, in this documentation page: 

• https://www.accesstomemory.org/docs/latest/admin-manual/installation/execution-limits/
  

You can also run the command with no default memory limit in place, like so: 

• php -d memory_limit=-1 symfony digitalobject:load /path/to/import-file.csv

If it is actually exhausting all available memory on your system, I would probably expect to see an out of memory error message in the terminal, from the task output. Do you see anything like that?

Another thing you could try is running the task again, while using htop to monitor your active processes and see how the system memory is being used. See: 

• https://www.accesstomemory.org/docs/latest/admin-manual/maintenance/troubleshooting/#monitoring-active-processes-with-htop
  

You'll probably need to open 2 separate terminal windows so you can execute the load task command in one, and monitor in the other one. 

Finally, it could also be that there are some unexpected results because of the matching criteria used - for example, if you are using the description identifier in your load CSV for matching, but your identifiers are not unique throughout the application, this could cause some digital objects to end up in unexpected places. If you are using 2.6 or later, I strongly recommend that you use the target description slug as your matching row in the CSV, as this is guaranteed to be unique. Ideally, you will not try to add more than 1 digital object to a single description either - while there is fallback behavior when this happens, the purpose of the task is to link digital objects to existing descriptions (with a 1:1 relationship!), not create new descriptions on the fly. A description CSV import, using the digitalObjectPath column to link your objects, would be better for adding new child descriptions with digital objects. 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/645f8aed-709d-4c5e-9a20-9a8648b9ce65n%40googlegroups.com.