Bulk import fails - can't open log file

[jthi...@bethelks.edu]

I'm trying to bulk import a few EAD files and I get the following error:

Unable to open the log file "/usr/share/nginx/atom/log/qubit_cli.log" for writing.

This is the command I used: php symfony import:bulk --verbose --output="/home/ubuntu/testbatch1.csv" /home/ubuntu/testbatch1

It also output an html file for a 500 Internal Server Error.

I'm using a 2.2.0 version of Atom

[Dan Gillean]

Hi John, 

First of all, I strongly recommend upgrading to the latest 2.5.x version if at all possible. There have been a number of performance enhancements to the import and export process since 2.2, as well as EAD mapping updates, bug fixes, security patches, and other new features. 

In the meantime, I would first suggest that you try resetting the filesystem permissions for your installation - it's possible something is not currently configured correctly. From AtoM's root installation directory (generally, /usr/share/nginx/atom), run the following: 

• sudo chown -R www-data:www-data /usr/share/nginx/atom
  

I would recommend that you clear the application cache, and restart PHP-FPM (and Memcached if installed) before re-testing as well. For 2.2, the commands if you have followed our recommended installation instructions should be: 

• php symfony cc
• sudo service php5-fpm restart
  
• sudo service memcached restart

Finally, I would also check the webserver error logs, to see if there is more specific information available about the error. With Nginx, you can typically use the following command to see the latest logged error: 

• sudo tail -f /var/log/nginx/error.log
  

If the above has not helped to resolve the issue and the error log isn't useful to you, feel free to share its contents here - hopefully that might offer us some further insight so our team can suggest other possible solutions for you. 

Let us know how it goes! 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/18df0661-c566-4640-b07c-347b6a44173a%40googlegroups.com.

[John Thiesen]

I was able to get it to work using sudo -u www-data and then the
import command string. So I guess it had to do with permissions on
files.

The context is this: We're part of a group of 4 archives that have a
shared Archon installation (originally my own Archon installation).
We're now planning on migrating to AtoM or ArchivesSpace. I've been
advocating AtoM. Some time ago I had installed AtoM on an Amazon EC2
server to play around with (I think this was right before version 2.2
was replaced by the following version). I've played around with it off
and on experimenting with various aspects of AtoM, but I hadn't ever
tried a bulk EAD upload.

The place where our Archon is currently hosted (libraryhost) said that
there was no clear migration path from Archon to AtoM and seemed to
favor ArchivesSpace (although there isn't a good migration path there
either). I wondered why not just bulk download and upload EAD files.
So I was trying that out now to see how it went. And it seemed to work
reasonably well. I didn't want to put in the effort of upgrading my
AtoM instance just to do some additional experimenting. Assuming we go
to a new shared AtoM, it would be hosted and would be the current
version.

John D. Thiesen

Archivist, Co-director of Libraries

Mennonite Library and Archives

Bethel College

North Newton, KS

> You received this message because you are subscribed to a topic in the Google Groups "AtoM Users" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/ica-atom-users/M9e6P6Muqy0/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to ica-atom-user...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/CAC1FhZLfXbvtVG9MqEArzL3wg7sdKox5oWekU47VZSs98ouLCA%40mail.gmail.com.

[Dan Gillean]

Hi John, 

Thank you for sharing what worked for you! 

We've been doing a lot of Archon --> AtoM migrations recently, and while an Archon EAD XML file will import into AtoM, there are some significant differences between AtoM and Archon, and between their EAD mappings, that can lead to some unexpected results, so make sure you examine some test inputs against the source EAD files carefully. 

I'd recommend searching for Archon in this forum and checking out some of the threads that come up. Here are a couple relevant ones: 

• https://groups.google.com/d/msg/ica-atom-users/Benf-urh9B4/fqerjKxUDwAJ
  
• https://groups.google.com/d/msg/ica-atom-users/E-NWLH15bUA/03fncNn1DwAJ
  

We have also learned that there are bugs in Archon that can affect exports. One of the export options (I *think* it was the administrator's bulk export available via the user interface) will append every single subject access point in Archon to any collection-level record with a subject associated, for example. Archon also doesn't always allow titles and identifiers to be added to folder and item level descriptions apparently, and the EAD export from Archon will include the Archon internal database ID in the <unitid> element, which AtoM imports as a description identifier. That's just a couple things off the top of my head. 

We do the Archon to AtoM migrations starting with the Archon EAD, but our developers usually prepare a few transformation scripts to clean up the EAD files prior to import. Just some things to watch out for! 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/CAOEgMjfmnydB%2B-oqB%3Dn6QUazvj2-R_CNxQx4sGC6x-xMci2GkQ%40mail.gmail.com.