Restricting XML import jobs by configuring user permissions

[Maciej Dudek]

Hello Everyone,

I have been wondering if user permissions might somehow affect the behaviour of XML import jobs?

For example: I would like to import an EAD XML file (attached to this message) that holds the definition of an archival description and authority record, but I would like the import job to fail if that authority record does not exist in the AtoM database. I have noticed that running a job as an administrator, but with revoked "create" authority record permissions (as shown on the screenshots below) does not affect the job's behaviour (i.e. both records are created).

I had a quick look into the arFileImportJob and QubitXmlImport files, and I could not find any permissions checks at that stage, so am I right saying that this cannot be done in AtoM 2.6?

Many thanks in advance for your replies.

Kind regards,
Maciej

[Dan Gillean]

Hi Maciej,

The short answer: You're right. AtoM doesn't have that level of support for permissions checks. But there is a way to configure which user groups can import. 

The longer response: 

Unfortunately, AtoM's permissions are not currently that deep. There are some hidden permissions controlling who can and cannot import files, but to use a metaphor, the permissions code only checks your ID at the door, it doesn't monitor what you do once inside. 

By this I mean: the check is to see who can get to the import page and launch an import, but there is no process that monitors all operations performed while running an import and checking to see if the original user has permissions to create the related records. If you can import a description but can't create an authority, AtoM will still create an authority if there is an unrecognized name in the import file. I think that technically once the import is approved and handed off, it's actually the job scheduler (running as the www-data user) who is performing the import. 

Maintaining checks throughout an import so every operation is associated with the user permissions would not only have significant impacts on performance and scalability, it would also introduce some complicated use cases to resolve. For example, if an EAD description import with thousands of descendants gets to the last 100 import records, but then finds some authority records or subject terms the user is not allowed to create.... what should happen? 

• Should the whole import fail and roll back when it hits the authorities? That would likely be computationally expensive, and frustrating for users
• Should the entire file be scanned before the import starts, and then fail with a message if there are some authorities the user can't import? Better, but.... now we need a lot of memory and other resources to scan the entire import file first
• Should the authority record just be skipped and import completed?  If so, what if the user doesn't know this happened? You've now given them an incomplete import that would be very difficult to manually check
• Should the import track what it doesn't import, and tell the user? Sounds great, but this is getting very complicated now, as you need a way to store decisions throughout a long-running process and then summarize them all for the user... and what if there are 100 descendants called "Correspondence", some of which are associated with different authority records - how will you tell the user which ones had authorities that were skipped? You can potentially use row numbers from the XML to report what is skipped, but again, we have now made every import much slower, less scalable, and much more resource intensive. If something fails along the way, it will be very difficult to debug. 
• Even if you tell the user exactly what was skipped... if you didn't know that you didn't have permissions to create some authorities, then now you either need to 1) delete the entire import and start again, or 2) manually create and link all the authorities you thought you were importing the first time. 2 is very onerous, and 1 is frustrating - and if you have permission to create place terms but not subjects for example, then deleting the archival description hierarchy would not automatically delete all the associated place terms you were allowed to create, so now you have a cleanup problem on your hands...

Etc.

There are certainly solutions and workarounds to the issues I raised above - but they are all going to require some careful analysis, a lot of development, and rigorous testing before they can exist. For now, it's easier to say: don't let users import files if they don't have permissions to do everything. 

That's part of the reason why by default imports are restricted to Administrators: they are ideally supposed to be reviewing import files before importing (and possibly even importing them into a test site before proceeding in a production instance). Additionally, for any major operations (such as important a lot of records) in a production system, we always recommend making a backup first, so you can easily roll back if the outcome is unexpected. 

However, if you do want to change what user groups can access the imports, there is a workaround. It's possible in earlier versions, but I have only recently documented it in the 2.7 docs - there are security configuration files that a system administrator can edit that affect some entities and modules not shown in the Permissions module user interface. See for example: 

• https://www.accesstomemory.org/docs/2.7/admin-manual/customization/config-files/#security-configuration-files
  

Notice one important thing at the end of the first subsection on customizing the import permissions: 

Access to the import Import menu is restricted by default to the administrator group in the application code. An administrator can use the Manage Menus module to add a link to another menu if access is required for other groups.

I couldn't figure out a way to allow other user groups to see and access the Admin menu. So, if you wanted to give users in the Editor group access to imports, then you can use the Manage menus module to add a new link in another menu, like the Manage menu. See Manage menus for more information on customizing the main menus in AtoM: 

• https://www.accesstomemory.org/docs/latest/user-manual/administer/manage-menus/#manage-menus
  

Additionally, while it doesn't directly solve your issue, it may help you vet imports before proceeding: the 2.7 release will include a new method to validate CSV import files for issues before importing. See: 

• https://www.accesstomemory.org/docs/2.7/user-manual/import-export/csv-validation/#csv-validation
  

Unfortunately, we don't have anything like this for XML imports at this time. 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/0a853049-a708-4e85-914a-2811b8e93cfbn%40googlegroups.com.

[Maciej Dudek]

Hi Dan,

Many thanks for your thorough and detailed answer - much appreciated! It all makes sense and I can definitely see how complex the business logic would have to be if the users' permissions were checked on the job run. Also, it's great to see that the CSV validation is coming in version 2.7!

Again, thank you for your time and all the best,
Maciej