Searching dates from eventDates field

[Debra Leigo]

Hi Dan,

I am an Archivist, not a developer, wanting to learn more about AtoM for using in establishing a very small archives (no existing systems/databases) which do not have in-house IT or a server. At the moment I am trying to customise the CSV templates - identify which fields are essential and which wont be required - so that the CSV can be used for the first year or so. The aim is to eventually move to AtoM.

I have been spending quite a lot of time on the AtoM online manuals and Demo. Can you please confirm that metadata entered into the eventDates field is not searchable at all? Is it possible to change the eventStartDates eventEndDates fields to accommodate DD-MM-YYYY format and be searchable?

I am also looking for a map (visual would be great) that clearly shows which fields are required to link different levels of descriptions E.g. authority record to accession, accession to series  series to file, files to items.  

Thanks in advance,

Deb

[Karl Goetz]

On Tue, 26 Nov 2019 00:37:07 -0800 (PST)
Debra Leigo <allun...@gmail.com> wrote:

> Hi Dan,
>
> I am an Archivist, not a developer, wanting to learn more about AtoM for
> using in establishing a very small archives (no existing systems/databases)
> which do not have in-house IT or a server. At the moment I am trying to
> customise the CSV templates - identify which fields are essential and which
> wont be required - so that the CSV can be used for the first year or so.
> The aim is to eventually move to AtoM.

We asked several projects the minimum they provide were the values which (approximately) matched the DC elements; gives a starting 15 which
should be acceptable even to more recalcentrent data suppliers.

> I have been spending quite a lot of time on the AtoM online manuals and
> Demo. Can you please confirm that metadata entered into the eventDates
> field is not searchable at all? Is it possible to change the
> eventStartDates eventEndDates fields to accommodate DD-MM-YYYY format and
> be searchable?

For the record, I assume you're referencing this doco?
https://www.accesstomemory.org/en/docs/2.5/user-manual/import-export/csv-import/#creator-related-import-columns-actors-and-events

Its true that eventDatse is a text field, and not indexed. I did investigate processing that data and using it to populate eventStartDates
and eventEndDates (for the added searchability) when they weren't provided but never finished the project.

Regarding the format of eventStart/EndDates; no you can't configure them (though if you find a programmer they'd be able to customise it if
you really really wanted it).

> I am also looking for a map (visual would be great) that clearly shows
> which fields are required to link different levels of descriptions E.g.
> authority record to accession, accession to series series to file, files
> to items.

I'm not aware of a map like that; perhaps Dan can provide some feedback here.

thanks,

--
Karl Goetz
Technical Services Officer - eResearch, Information Technology Services
University of Tasmania & Tasmanian Partnership for Advanced Computing

Mail: University of Tasmania, Private Bag 69, Hobart, Tasmania 7001
Delivery: TT Flynn Street, Sandy Bay, Tasmania 7005



University of Tasmania Electronic Communications Policy (December, 2014).
This email is confidential, and is for the intended recipient only. Access, disclosure, copying, distribution, or reliance on any of it by anyone outside the intended recipient organisation is prohibited and may be a criminal offence. Please delete if obtained in error and email confirmation to the sender. The views expressed in this email are not necessarily the views of the University of Tasmania, unless clearly intended otherwise.

[Dan Gillean]

Hi Debra, 

Some initial responses to your questions below: 

At the moment I am trying to customise the CSV templates - identify which fields are essential and which wont be required - so that the CSV can be used for the first year or so.

You might find the following slide deck useful, in addition to the documentation, which explains many of the key CSV fields in more detail: 

•   https://www.slideshare.net/accesstomemory/csv-import-in-atom

It's hard to say what is "essential" - AtoM is very permissive and will allow you to create entirely blank records. Do you mean essential based on the relevant standard? In ISAD(G) for example, the following fields are considered required or mandatory (below I will list the ISAD(G) field name in AtoM, and the corresponding CSV field. As you can see, AtoM's database structure for descriptions was originally built around ISAD(G), so the CSV field names match very closely, written in camelCase):

• Identifier - identifier
• Title - title
• Dates - eventDates, eventStartDates, eventEndDates, eventTypes
• See also eventActors, below
• Level of description - levelOfDescription
• Extent and medium - extentAndMedium
• Name of creator(s) - eventActors (and eventActorHistories for biographical / administrative histories, that will be stored as the history on the related authority record)
• Note that these are related to the date rows as well - see the slides and documentation
• Repository - repository

A few others to consider adding: 

• scopeAndContent - scope and content is not considered mandatory/required in ISAD(G), but your users will appreciate a description of the records
• legacyID and parentID - these are used to control the hierarchical relationships inside the CSV - so to nest a Series level row beneath a Collection record, you would put the legacyID value assigned to the collection in the parentID cell of the series. See the docs and slides for more information
• culture: defaults to en, but considered wise to add to all your rows, especially in a multilingual installation
• publicationStatus: expects Draft or Published as values. Relates to the visibility of descriptions to public (i.e. not logged in) users. 

Can you please confirm that metadata entered into the eventDates field is not searchable at all? Is it possible to change the eventStartDates eventEndDates fields to accommodate DD-MM-YYYY format and be searchable?

I have consulted with the developers, and yes, it seems that currently the free-text display date field (i.e. eventDates) is not currently searchable in AtoM. 

If you have added controlled start and end dates (eventStartDates and eventEndDates values) in ISO 8601 format (that is, YYYY-MM-DD, YYYY-MM, or simply YYYY), then you can use AtoM's date range search in the Advanced search panel to search for records based on a date range input. See: 

• https://www.accesstomemory.org/docs/latest/user-manual/access-content/advanced-search/#date-range-search-filter   

Apparently the reason the display date (and separately, using ES field names, the controlled start and end dates) is not searchable via the user interface at the moment is because it uses something that Elasticsearch calls a nested query, to keep the values grouped together. This is important for date range searching, so that start and end dates remain grouped together, and you don't end up searching against one start date and a different end date when a description has more than one event associated. 

It would be possible in the future for us to move the display date out of the nesting (or else duplicate it in the index), so we can make it searchable via the global search box. This would require some development for us to be able to implement, however. 

It's not possible without development to change the controlled start and end date fields to use DD-MM-YYYY instead of the ISO 8601 standard YYYY-MM-DD. That might be a bigger development project I think (since we are likely using existing methods and functions that follow the ISO standard), and in any case, it's not a change we would likely want to implement in a public release - AtoM is designed to follow national and international standards whenever possible, and we have implemented the international date format standard. You can of course structure your dates any way you wish in the free-text eventDates field. 

If your institution might be interested in sponsoring work to make the display date (i.e. eventDates) field searchable, feel free to contact me off-list and our team can prepare some estimates for you. We depend on our community to help implement enhancements and new features in AtoM  - to learn more about how we maintain and develop the application, please see: 

• https://wiki.accesstomemory.org/Development/Philosophy   

I am also looking for a map (visual would be great) that clearly shows which fields are required to link different levels of descriptions E.g. authority record to accession, accession to series  series to file, files to items.  

I don't have anything quite so visual unfortunately, but I can at least address the examples you provided. 

Authority record to accession: 

• The Creators field in the accession record edit template can link an authority record to an accession
• In the accession record CSV template, it's the creators column
• It's not currently possible to link an accession to an authority record via the authority record interface or CSV
• It's also not currently possible to add name access points to an accession

Accession to series:

• An accession record can be linked to a description at any level, by using the "Archival description" autocomplete field in the accession record edit template
• To link via CSV, add an accessionNumber column to the description csv, and add the related accession number in the corresponding description row
• It's not currently possible to create relations to descriptions from the accession record CSV template. 

Series to file; file to item:

• In the user interface, there are two ways of doing this: 
• In the edit page of the Series description, there is an "Add new children" widget, that allows you to quickly sketch out lower level records. It includes just a couple fields (Identifier, Title, Level of description, Display date), so you would likely want to navigate to the file-level record after and supplement it, but it can be useful when processing, to quickly create the structure of your archival hierarchy from a single edit page
• After the series is saved, click the "Add new" button in the button block at the bottom of the view page for the related archival description. This will open a new edit template, that is creating a new record as a child of the Series. 
• There are also two ways of doing this via the CSV:
• the legacyID and parentID columns are used to manage hierarchical arrangements of new records found in the CSV. So if both your series and file are in the CSV as new data being imported, this is what you'd use
• the qubitParentSlug column is used to relate a new record in the CSV to an existing record in AtoM. So if your series already exists in AtoM and you wanted to import a new file-level description as a child, this is what you'd use
• See the slides and docs I linked above for more information. 

Happy to answer further questions! 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/9de1e060-53b7-4167-b59d-b6269328073f%40googlegroups.com.

[Debra Leigo]

Hi Karl,

Thanks for your reply. I have been looking at https://www.accesstomemory.org/en/docs/2.4/user-manual/add-edit-content/archival-descriptions/#archival-descriptions but will now direct my ready at the link you have kindly provided.

Deb

[Debra Leigo]

Hi Dan,

Thank you for your detailed response. I will be working through all the information you have provide and have no doubt that I will have more questions in the future.

Deb

To unsubscribe from this group and stop receiving emails from it, send an email to ica-ato...@googlegroups.com.