Create new functionality in AtoM

107 views
Skip to first unread message

Isabel Martín

unread,
Aug 14, 2020, 5:17:45 AM8/14/20
to AtoM Users
Hello, I need to create an administration zone where AtoM 2.5 administrators can directly upload CSV
with archive descriptions and upload them from the web, without accessing the command line.
Also create the possibility of uploading CSV with the digital objects of said descriptions
and uploading them also from the AtoM itself. I have thought about creating a new module or a plugin,
although it is not very clear what to use because I do not see much information about it,
what would you recommend? I would also like to know if there is any tutorial or example of
how to create your own functionality in AtoM. Thank you very much.

Dan Gillean

unread,
Aug 14, 2020, 4:05:31 PM8/14/20
to ICA-AtoM Users
Hi Isabel, 

In 2.5 and later, it is already possible to upload CSVs directly from the user interface without using the command-line. See: 
However, you are correct that bulk uploading of digital objects typically involves server access - you can include a filepath to a related digital object in the CSV, but the files themselves must be placed on the server first in the corresponding location, and this can't be done via the UI now. 

Some general thoughts on development ideas to solve this issue:

First, AtoM currently has a multi-upload widget that is Flash based, and therefore needs to be replaced very soon (Flash is being fully deprecated by December 20, 2020 in all major browsers). It is used when linking multiple digital objects as children to a current description: 
Artefactual has already begun research for a related library to replace this component, and we have wanted to find something that could be reused to solve other problems in AtoM, such as support for bulk digital object upload. Many of our team, myself included, quite like the uppy library - it's lightweight, and supports the chunking of files so that large files (and bulk uploads) can be handled via a web interface without timing out the browser. See: 
While we are hoping to undertake this library upgrade for the 2.7 release, the work is currently not sponsored. Because of this, there's currently no guarantee that we will have the time and resources to get to it - or extend the use of the library to new areas, such as supporting bulk digital object import with CSV metadata. 

If you're intending to do development work in this direction, I'd encourage you to investigate uppy, to see if it might meet your needs. That could align well with our plans, and/or make it easier for us to incorporate a pull request from you in a future release, if you decide to share your work back with the public project. 

For now, I can point you to some general development resources that might help orient you. I would suggest you start by checking out these resources, as well as studying our code to see how imports and digital objects are currently handled, and how you might extend the existing methods. If you have specific questions as you progress, feel free to post them here, and hopefully one of our developers can help point you in the right direction. 

General resources

We do maintain some basic development resources - you'll find some initial ones here: 



You will find an overview of the elements that make up AtoM, and a tour of the AtoM code, in the following resources:



AtoM was originally developed using the Symfony 1.x framework, and the Propel 1.x ORM. You might want to familiarize yourself with Symfony before beginning:



AtoM also currently uses Elasticsearch 5.6 (in the 2.5 version and later - if you are developing new features we strongly recommend you work against our qa/2.6.x branch) for its search index, Twitter Bootstrap 2.3.2 for theming, and several other libraries. MySQL (5.7 in 2.5; 8.0 in 2.6) is used for data storage. We keep all our code in a git repository, so being comfortable with git or other distributed version control systems will also help you. There are a lot of resources out there on all of these!


We maintain a Vagrant box that can be used for local development - see:

 


In this user forum thread thread, I explain how to upgrade the Vagrant box to ensure you are working against the latest code: 



We also have a slide deck that introduces developers to major feature development, here:



In terms of understanding AtoM's database and working with SQL, we keep copies of Entity Relationship Diagrams for AtoM's database on the wiki here: 



And we have a slide deck that introduces users to performing SQL queries against an AtoM database here: 



In terms of working with the Symfony 1.x framework, we recommend that developers adding features to AtoM follow Symfony's plugin development model whenever possible. Most of the existing modules in AtoM and ICA-AtoM have been built as plugins - developing this way makes them easier to maintain in the long-term. Here are some of the best Symfony 1.x resources on Plugin development: 



There is also a Symfony plugins page that lists "1458 symfony plugins developed by 609 contributors." This can be a great reference, to see how others using Symfony have developed their plugins for other projects. See: 



For reference all of AtoM's current plugins can be studied here: 



If you are considering development that you wish to share back with the public project, please be sure to review this page: 



As I suggested above, I would recommend you start by exploring these resources, and then studying AtoM's code to learn how we are currently handling import and digital object management, so you can try to trace what files are involved, what functions are already available, etc. Hopefully you still start to detect a pattern, and you can figure out where it makes sense to extend available methods, etc. 

Good luck! 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory
he / him


--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/571e74c9-d12c-4f44-bd1d-35d7f8487cd9o%40googlegroups.com.

Isabel Martín

unread,
Aug 17, 2020, 5:51:59 AM8/17/20
to AtoM Users
Hi Dan, thank you very much for the answer, I find it very useful.

I will look at the tool you recommend (Uppy) in case it can serve my purpose.

I have another question, should I update AtoM to version 2.6? What improvements does it bring with respect to the 2.5 I have now?

Thanks for everything.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-ato...@googlegroups.com.

Dan Gillean

unread,
Aug 17, 2020, 10:49:07 AM8/17/20
to ICA-AtoM Users
Hi Isabel, 

You can learn more about all the changes and new features included in the 2.6 release in the 2.6 Release page on the wiki - I also included a summary of the main points in the announcement posted to the user forum. 

If you are doing development work, I strongly suggest that you upgrade your development environment and work with 2.6 at least - ideally, I suggest working against our qa/2.x development branch so you can regularly pull in recent changes slated for the next major release, rebase your work, and ensure that your code will integrate well with upcoming changes. 

The 2.6 release includes a number of dependency changes (most prominently, an upgrade from MySQL 5.7 to 8.0, but also an upgrade to AtoM's media player, which could impact your testing for digital object management), as well as a number of code optimizations for performance, several of which touch the import process.

Our developers have also recently completed initial work to ensure that AtoM 2.7 will support at least PHP 7.3, and we hope to create a new Vagrant development environment soon that uses v7.3, so we can be regularly testing for any regressions as we continue work on this release. The Docker development environment has already been upgraded, as per this ticket:
Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory
he / him

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/7d2397e6-4248-4f2c-8eb4-d9e39be38c0co%40googlegroups.com.

ism...@gmail.com

unread,
Aug 25, 2020, 3:03:30 AM8/25/20
to AtoM Users
Good morning and thanks as always for the answer. Importing descriptions from CSV helps me. As for uploading digital objects, in principle there would be no problem uploading the images to the server via FTP. I would like to know if there is any way, as with the descriptions, to upload the CSV with the id and path of the image directly from the interface, without having to launch the command in Ubuntu. The idea is that the user himself uploads the images to the server, generates the CSV and uploads it from atom.
A greeting.

Dan Gillean

unread,
Aug 25, 2020, 1:22:20 PM8/25/20
to ICA-AtoM Users
Hi Isabel, 

In regards to your question: 

I would like to know if there is any way, as with the descriptions, to upload the CSV with the id and path of the image directly from the interface, without having to launch the command in Ubuntu. The idea is that the user himself uploads the images to the server, generates the CSV and uploads it from atom.
A greeting.

Yes, technically this already exists - the AtoM description CSV templates include 2 possible columns for linking a digital object to an incoming description: 
  • a digitalObjectURI column is used to link to external digital objects available on the public web, via URI, or
  • a digitalObjectPath column can be added to add a local server filepath to the related digital object instead. 


Note: if both columns are present and populated for a single row, AtoM will favor the URI over the local path. See: 
With the above workflow, you are creating new archival descriptions via CSV import, and attaching your digital objects to these new descriptions at the same time. 

If you want to be able to upload a CSV that contains ONLY paths to digital objects, to link them to existing descriptions, then AtoM does support this, but only from the command-line currently, so development would need to be done to support the same workflow via the user interface. See: 
Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory
he / him

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/4ee61933-0b62-4195-86df-eefe9baba923n%40googlegroups.com.

ism...@gmail.com

unread,
Aug 26, 2020, 2:42:18 AM8/26/20
to AtoM Users
Good morning, question resolved, thank you very much.
Reply all
Reply to author
Forward
0 new messages