REQUEST FOR FEEDBACK: Potential permissions change

[Sarah Romkey]

Dear AtoM Community,

We are seeking your feedback on a potential change to our code regarding read permissions for published archival descriptions.

A number of users have been reporting performance issues, especially in multi-repository instance with complex user permission scenarios. Upcoming improvements to how AtoM handles permissions will help ease this performance problem.

One change to AtoM's permissions that we are considering is always making published descriptions readable by all users, whether authenticated or not. In AtoM 2.0.x, it is possible to set the permissions so that even published descriptions cannot be read by any unauthenticated user. If we remove this permission option, the obvious work-around would be to leave descriptions in draft if you do not want public users to see them. Alternatively, if your entire database needs to be protected from public eyes, it would be advisable to put in place http authentication.

We cannot internally think of a use-case where there would be a desire to publish descriptions, yet make them inaccessible to the public. However, if this is a functionality you use, or believe you would use, we would like to hear from you! Your questions or comments are welcome in this thread.

Cheers,

Sarah

[Jordan Bass]

I think this is a good idea, especially if it improves performance. I also cannot think of any case where we have published a description and then restricted public access to it. 

J. 

[Dan Gillean]

Thanks for the feedback, Jordan!

Does anyone out there have a strong use case in favour of keeping this permission setting in place?

Cheers,

Dan Gillean, MAS, MLIS
AtoM Product Manager / Systems Analyst,
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

--
You received this message because you are subscribed to the Google Groups "ICA-AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.
Visit this group at http://groups.google.com/group/ica-atom-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/91e934c6-aa9f-4807-aafd-f312b99a0c4a%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Lise Summers]

Hi Dan

With our system, we have descriptions at your file level (our item level) that may be restricted access records, and where the context of the information would cause privacy concerns. Examples would be major crime files, or name based file titles for remand and juvenile prisons. We need to know that the information is there, so we can help users access the files through our freedom of information system. Currrently, we hold the data as draft, but we would also like authorised users from the relevant government agencies to be able to access the information, too - a sort of median level between fully published and draft.

We have also identified some fields that we want to show to administrators and contributors, but not to authenticated researchers - including the physical storage location of items. Would your proposed changes mean that we would not be able to do this anymore?

It's not so much a problem with our data, but I think you could also make a case for having different levels of access for Aboriginal and Torres Strait Islander records in some collections.  The access would be determined by skin group and gender, and possibly also age, both of the person viewing the data and the age of the person the information is about if they are deceased ('Sorry' time).

Lise Summers
State Records Office of Western Australia

[Sarah Romkey]

Hello Lise,

This change would not affect anything which is left in "draft" mode. You would still be able to apply very specific permissions to collections, and collections held by archival institutions, as long as they are held in "draft." Would it be acceptable in your use case for the government agencies, or Aboriginal group as the case may be, to need to have a login to access the descriptions, and have the records appear as "draft"?

If anything, our hope is that this change will make complex permission structures easier for the database to manage and therefore scale to larger collections. It is one of several ways that the database behind AtoM can be more efficient, and hopefully the cumulative effect will be more efficient processing speeds.

Cheers,

Sarah Romkey, MAS,MLIS

Systems Archivist
Artefactual Systems
604-527-2056

@accesstomemory / @ArchivesSarah

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/8475dbae-18af-4d7d-9d52-069c287bcec0%40googlegroups.com.

[Dan Gillean]

Hi Lise,

Thanks again for participating in this thread, and sharing your use case. As Sarah has said, if you are fine with setting up user accounts and asking users to log in, then there are many strategies you could follow with the existing permissions. Since you have called them "authenticated researchers," I am hopeful that this means that some kind of authentication/registration does or could take place, which would include the creation of a user account for the authenticated researchers.

If this is the case, then my recommendation would be to create new user groups. AtoM includes several default user groups (e.g. Administrator, Editor, Contributor, Translator, etc), but administrators can also create new user groups with custom permissions. As such, you could set the records in draft mode, and then create a Researcher user group where the only change in permission is to view drafts. This would also allow you, if desired, to make the permissions more granular - for example, you could add permissions only for a specific fonds or collection that is in draft mode - so your researchers could see what they have been granted access to, but not all draft records in the system.

• More information on creating groups in AtoM can be found here: https://www.accesstomemory.org/docs/2.0/user-manual/administer/manage-user-accounts/#add-group
• Information on editing the permissions for a specific user or group is here: https://www.accesstomemory.org/docs/2.0/user-manual/administer/edit-permissions/#edit-user-permissions

The problem of the physical storage information is perhaps the only catch, and I am glad that you have brought it to our attention. At the moment, physical storage information can be hidden from public users via the Visible elements module, but it is not currently part of the Permissions module - meaning that using groups to manage access, and giving users accounts, may make the physical storage information visible to those users.

As I understand it from your explanation (please correct me if I'm wrong!), the current permissions module, granular as it may be, does not give you the ability to meet your exact use case anyways - either the records are hidden from public users (you mark them as published but use the permissions module to hide access), or they are visible - and if your users have to log in to see records currently marked as draft, then the Visible elements module is already not hiding the physical storage location. Ultimately, while the changes we are proposing will unfortunately not improve your situation and allow you to meet your use case needs exactly, neither will it hinder it further - though the overall performance of the application will be improved by these changes for all users, including yours. Have I misunderstood how you are currently using the system?

Our goal is to simplify some of the complexity of the permissions so that we can gain performance improvements throughout the application, so adding further permissions to meet this need may not be viable at present - but to ensure we are exploring all options, Sarah and I will take your use case up with our developers here, and try to determine how much they feel performance would be affected if either a) physical storage access was added to the permissions module in the future, or b) custom groups could be associated with the Visible elements module, so these settings would extend to members of assigned groups. In either case, this would likely require significant development, and Artefactual would need to find a supporting development partner - and if they will undo our efforts to improve performance, we may have to find other solutions, or propose workarounds to you. Regardless, it has been very useful to hear from you, and my hope is that the changes we are proposing will offer you performance improvements without actually making your use case harder to achieve than it already is.

Again, I would like to invite others to add their perspectives to this thread!

Cheers,

Dan Gillean, MAS, MLIS
AtoM Product Manager / Systems Analyst,
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/CAAr2Qts_RE4GS2h7JV6h_JWP5LjbdCVTjt3gfj3tmZrGCtCueg%40mail.gmail.com.

[Lise Summers]

Hi Dan and Sarah,

Yes, our authenticated researchers will be required to provide some documentation before they can log in to the system.  Currently, that will still only allow them to see published material, and to use our new loans module being developed by GAIA Resources, which has a few different permission layers. 

We could certainly look to provide limited access to some draft descriptions, particularly if the level of access could be refined by accession as well, although it could mean creating a number of different user groups (basically one per Authority with those particular access requirements - good thing I can only think of three or four). At some stage, of course, we would look to make the records published. Is there some way of converting a range of descriptions from draft to published; currently, we are having to do it on a description by description, file by file basis.

The real question, though, is how the changes will affect our 'hidden' fields in published descriptions.

Lise

[Sarah Romkey]

Hello Lise,

Aside from Physical Storage, which is not currently possible to hide from any authenticated user, which fields are you hoping to hide? The permission groups in AtoM are fairly broad-  they are grouped by Archival descriptions, Authority records, Taxonomies, and Archival institutions. So if you wanted to "hide" a field from a group of users, it would involve hiding the entirety of one of those entities I just listed. alternatively, you can change the permission by specific archival institution or archival description. For example, you could say "This group can see all archival descriptions, but not descriptions X, Y, and Z."  There are some more examples outlined in our documentation: https://www.accesstomemory.org/en/docs/2.0/user-manual/administer/edit-permissions/#description-permissions

The only other way to publish an archival description that is in draft is through the command line- there are instructions here: https://www.accesstomemory.org/en/docs/2.0/admin-manual/maintenance/cli-tools/#update-the-publication-status-of-a-description

The only other suggestion I can think of to gain some efficiency in publishing drafts is when possible to do it at a higher level- so if you have an entire series of files that need to be published, publish at the series level and it will apply to all of the children (files) below.

Thank you for keeping this conversation going! Aside from our immediate need to make a decision on this particular change, it's always interesting to hear about different use-cases from the community.

Cheers,

 

Sarah Romkey, MAS,MLIS

Systems Archivist
Artefactual Systems
604-527-2056

@ArchivesSarah

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/a8677a27-70ab-4715-ae2f-7076ffc33949%40googlegroups.com.

[Dan Gillean]

Hi Lise,

There are some command line tools and instructions we've included to help publish a bulk set of records - Sarah has linked you to the command line task, but we have also included some sample database queries that can accomplish something similar. If you have a system administrator who is familiar with SQL, you should be able to use these examples to further refine the target, if needed. At the moment, there is no way to do so through the user interface. With the coming introduction of our job scheduler for asynchronous tasks (to be included in our 2.2 release, currently *tentatively* expected for late in Q3 or Q4 of 2014), the initial building blocks will be there for bulk actions development in the user interface, however. We hope that we can find development sponsors to help us implement such features in the future.

In the meantime, here are the sample SQL queries:

• https://www.accesstomemory.org/docs/2.0/admin-manual/maintenance/cli-tools/#update-all-draft-archival-descriptions-to-published
  
• https://www.accesstomemory.org/docs/2.0/admin-manual/maintenance/cli-tools/#sql-update-publication-status-repo

Ultimately Lise, you have raised a very valuable consideration - the use of the Visible Elements module to hide fields is only available on published records. So yes, if users must log in to see records, you will no longer be able to hide specific fields. However, in the current paradigm, if the record is published, *everyone* sees it - not just a subset of authorized researchers or community members. 

The changes we are proposing will not alter this current scenario - if the record was published but access was denied to public users, then whatever fields might have been hidden via Visible Elements would be irrelevant anyway, since public users could not access any of the record, and logged in users would still see all fields.

Your use case is extremely valuable and worth further consideration, and it is something we should consider for future development. My guess would be that the only way to manage the association of the Visible Elements module to permissions would be to keep it extremely simple - e.g. adding an option so that only administrators see all fields, while other user groups have fields restricted from view just as if the record were being viewed by a public user. Even that may have heavy consequences for performance - the complex and granular permissions allowed in AtoM are one of the most "expensive" parts of the application, and adding further granularity may prevent the application from scaling or performing well with large data sets, or in mulit-repository settings.

I would like to discuss this further, both internally with Artefactual (to garner thoughts from our developers about what might be possible in the future) and here on the list, so we can hear more from other users with similar use cases and try to determine an ideal approach, as well as an achievable one.

My sense is still that our original proposal - to ask users to make use of the draft setting, rather than publishing and then restricting access to specific public records - won't actually worsen your current scenario, though it will improve the performance of the application overall. Lise, would you agree?

As ever, I invite further thoughts, both from Lise and others!

Cheers,

Dan Gillean, MAS, MLIS
AtoM Product Manager / Systems Analyst,
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/a8677a27-70ab-4715-ae2f-7076ffc33949%40googlegroups.com.

[Lise Summers]

Thanks, Dan and Sarah.  Certainly, the draft option is one way to go - it just requires us to change the way we think about 'access' and 'published' data.  I'm aware of the permissions, and it would be great if we could filter by accession to refine what could and could not be seen. The same would also be true, I think, for the batch updates. Certainly, in the Australian context, being able to change, restrict or otherwise edit descriptions, on an accession by accession (consignment by consignment) basis would be extremely useful. It seems to me that accessions are aa somewhat underutilised tool in the AtoM arsenal. I'd love to have the repository code tied to the accession rather than the series description, for example, because the accession is what tells me what an archive really has, while a series or fonds description could be a purely intellectual exercise. 

Lise

[Janice Kerfoot]

If I understand it correctly, this change would affect our use of AtoM in at least one important way. We’d hoped to eventually use AtoM to create a directory of our institutional archives, viewable internally, as well as the catalogue for the collections and archives we want the public to see. I was thinking this could be accomplished by making a separate repository for those internal records and granting the anonymous user access to every repository but that one. 

The use of DRAFT status as a workaround is not very appealing due to the important role DRAFT is already going to have in our workflow: a lot of data entry will be done by volunteers, stage students, admin assistants and others whose entries will need to be verified or completed by an archivist or director before going public. This scenario applies equally to internal and for-the-public descriptions. If the DRAFT status has the dual meaning of “waiting to be published” and “never to be published”, we’ll have to be careful when we update the publication status of many entries all at once via command line, and we won’t be able to use the “Description updates” view (under Admin menu), filtered to drafts only, as a simple way to check on what recent additions are waiting for attention and publication.   

I wonder if there could be some way to make a distinction between the two kinds of unpublished description, even if they are effectively the same. 

[Edgar Rodríguez Silva]

Our system have a complex workflow in the same way that Janice. We had to develop a third status, we have called "private". We used it to prevent records not "draft" can be read by personal with not enough level of rights to modify but yes for reading.

[Suzanne Dubeau]

Greetings all...

I have been following this thread with interest.

I can think of many situations, including my own institution, where there could be a need for a finding aid for public access, and a more detailed one for 'internal' consumption.

I think to resort to using 'DRAFT' status for anything more than an actual 'DRAFT' would be a mis-use of the term that will come back to bite us later, no differently than using a given tag for a purpose other than what was intended.

Best wishes,
Suzanne

-- 

Suzanne Dubeau, MISt 
Assistant Head, Clara Thomas Archives & Special Collections
York University, 305 Scott Library, 4700 Keele Street
Toronto, ON M3J 1P3 
Tel:  416.736.5442	Fax:  416.650.8039 

--
You received this message because you are subscribed to the Google Groups "ICA-AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.
Visit this group at http://groups.google.com/group/ica-atom-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/29c9890d-6b34-4aa6-a3ab-4aea32217c8a%40googlegroups.com.

[Andreas Nef]

I also think that two different aspects are being mixed here. One is the usage of "publishing" in the sense of finalizing a description, i.e. moving it from draft/working into a state where actual usage should begin. The other is – mostly for published/finalized descriptions – to decide whether they should be publicly accessible or not.

For us this is definitely a use case to have parts of the descriptions "published" (i.e. finalized), but not be publicly available. Using the "draft" status for this might solve the problem technically, but it will lead to misunderstandings and is – agreeing with Suzanne – overloading the term.

Cheers,

Andreas

--

Andreas Nef, lic. phil.

Docuteam GmbH

Informationsmanagement und Archivdienstleistungen

Im Langacker 16, Postfach, CH-5405 Baden-Dättwil

Tel. +41 56 511 11 77

www.docuteam.ch

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/53D66A0D.9060207%40yorku.ca.

[Dan Gillean]

Hi all,

This has been an extremely useful discussion thus far, and I want to thank everyone for participating, and encourage continued thoughts. Permissions management is one of the greatest challenges to performance in AtoM as it currently is, and we need to find ways to improve and simplify some of that complexity for the application to be able to scale and remain performant; at the same time, there have been some valuable points and use cases raised here, and we don't want to remove existing functionality if it is in use by many members of our community.

A couple of thoughts/questions:

Edgar: the private status that you've developed - can you tell us more? Is it tied into the permissions module? Is the code available for viewing on Github or any other public repository? If it is tied to the ACL, have you seen any performance issues result from adding another publication status?

Do others feel that having another publication status of Private, where public users could not see the records but Editors could, would help clarify the distinction?

Suzanne and Andreas: you have both mentioned the desire to have "parts" of a finding aid published, but not all. The scenario we originally raised (publishing, but then denying public access) would not address that - however, the Visible Elements module might. Have you seen if that would meet your needs? I understand that this might not still clarify the different potential applications of the term "draft," however.

Is anyone making use of the Status and Level of Detail fields in the Description control area? These terms are managed in the taxonomies (and are therefore editable), and the Status field's default terms include Final, Revised, and Draft. I wonder if there are ways we might solve the "confusion of meanings" issue by leveraging these fields? E.g. making them part of the information viewable in the Description updates module; adding a flag for them in one of the command line tools so one could update draft descriptions based on Status? These features would require development, and Artefactual would hope to find someone willing to sponsor the work, but my thought is that it would be development we could do without putting further strain on the permissions module, which is our goal at the moment.

Thanks again for your thoughts, all. Both myself and Sarah Romkey, our other AtoM analyst, have been away this last week. I hope to discuss this feedback with our developers in the near future, and see what other options we may have.

Regards,

Dan Gillean, MAS, MLIS
AtoM Product Manager / Systems Analyst,
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/D8B24363-67BA-4E6B-BFB9-AFE05DCA4771%40docuteam.ch.

[Robert Dirig]

Hi everyone, 

I'm a bit late on this thread, but it's an issue that has come up with us.  Yes, the "Private" status would be very useful for the exact reasons that are discussed here.  Have there been any developments? 

Thanks, 

Bob

[Dan Gillean]

Hi Bob,

Unfortunately not. Edgar never got back to this thread with more information about the "Private" status he had developed.

Additionally, much of the revisions to the permissions module we hoped to accomplish by piggybacking it off of other related client work, did not in fact happen in the way we thought it might. We benchmarked some tests moving permissions to the search index, and found that we lost some of the options, gained little to no performance, but added a lot of complexity to the code. So for now, the permissions module is actually unchanged in AtoM 2.2 - users will have the same options as before. Nothing taken away, but nothing specific to permissions and performance improved either, however : /

Cheers,

Dan Gillean, MAS, MLIS
AtoM Product Manager / Systems Analyst,
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/aa2138f7-7d8b-4af3-9263-3509bfe080df%40googlegroups.com.

[Edgar Rodríguez Silva]

Hi;

I am sorry for late answer, we have made some changes in code to add one "Private" status, and It is our intention share it with comunity, but at the moment we have no time to prepare the fork to publish.

Answering to Dan; it works at the same way that draf status, we add a new state in the publication status.

Regards

[Dan Gillean]

Thanks for the update, Edgar!

Cheers,

Dan Gillean, MAS, MLIS
AtoM Product Manager / Systems Analyst,
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/72f3159c-8105-4055-ad3f-dbbe59d6fdeb%40googlegroups.com.