No connection error

[ToniS]

Dear  all,

I'm facing following error when exporting a CSV (Screenshot).
Elasticsearch an Gearmand have already been restarted.

What else could cause trouble.

Thankful for any hint.

Toni

[Dan Gillean]

Hi Toni, 

There's not a lot of information here to go on, but when we've seen this message before, it's usually about not being able to connect to Elasticsearch - and that has sometimes been due to not enough memory. How much RAM does your AtoM installation have available?

That said, I can't think why a connection to Elasticsearh would be important for a CSV export job, so it might also be about connecting to the MySQL database. Is there any chance you've tried a unique installation pattern - for example, installing the job scheduler on a different server?

Is this an archival description CSV export? How many rows, approximately, were you expecting (i.e. is this a big export, or just a small one)?

Let's see if we can get a more detailed error message. The first place to check would be the atom-worker logs. Can you see if there's anything relevant in the worker logs, with the following command?

• sudo journalctl -f -u atom-worker
  

There may not be anything more in there than is shown in the Job output in the user interface but it's worth checking. 

Otherwise, do you have access to the command-line? If yes, I would suggest trying to run a command-line CSV export. That way the console should provide more information on any errors encountered. See: 

• https://www.accesstomemory.org/docs/latest/admin-manual/maintenance/cli-import-export/#export-archival-descriptions-in-csv-from-the-command-line
  

Hopefully with more information we can recommend some next steps.

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/ffb1ee72-ecc1-4ab7-a3ac-b9b17e400332n%40googlegroups.com.

[ToniS]

Hi Dan,

thank you so much. After a lot of checking logs restarting percona, elastic, everything.

How much RAM does your AtoM installation have available?

got me on the right track. After adding memory to the VM the export is working again . Yeah!

May I ask you one more question. I've tried to export all description of a institution like

php symfony csv:export --criteria="i.repository_id = (SELECT object_id FROM slug WHERE slug='orient')" /tmp/orient.csv

How would I add all drafts and descent records as well in the export?

Best 

Toni

[ToniS]

descent = descendant ;) (Like when using the clipboard export option)

[Dan Gillean]

Hi Toni, 

Avoiding a bunch of technical details, there are a couple things going on that make it so that code changes would be required to be able to support including descendants in the query when using the criteria option on this CLI. Additionally, the repository relationship is typically inherited programmatically at lower levels rather than actually present as a relation in the database, which is part of the reason why those descendants aren't included by default when using the criteria option as you have. 

However: in the CSV we do programmatically add the repository name on export, regardless of whether it was added directly by the user or inherited by a higher level. 

Therefore: honestly, I think the easiest way to get all the data you want would be to: 

• Run a command-line CSV export of all data
• Open the CSV in a spreadsheet application like LibreOffice Calc, etc
• Sort by the repository column
• Delete all rows that don't relate to your target repository

Not ideal, but I hope it will meet your needs! 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/f8367f05-c778-46dd-a453-1a20ac33dc61n%40googlegroups.com.

[ToniS]

As always thanks a lot for your extensive help, Dave.

The full data export is a way to go for me. Even the resulting file will be huge, I can load and filter it in postgres or openrefine afterwards to get what I need.

Have a sunny rest of the week and thanks,

Toni