Full List of Searchable Fields

[Grant McNulty (500 Year Archive)]

Hi Guys,

We have an installation of AtoM up and running and find it to be a great application. Thanks and well done! 

One thing we were not sure of - and I have scratched around on the documentation on the website and this forum - is the list of searchable fields. I found this ICA-Atom list: https://wiki.ica-atom.org/Search_fields but am wondering where I can find information about all of the searchable AtoM fields?

Thank you,

Grant

[Dan Gillean]

Hi Grant,

I've been meaning to put this together for a while, and still haven't had the time to formally test everything. However, based on some initial experimentation, I'm attaching a list of search fields. I'd recommend testing them, rather than taking them as gospel at this point. My hope is to do this testing myself at some point, and add this to the documentation... but in the meantime, I will share my working notes. Attached you'll find a text file with my current notes. Some fields are tested, some aren't. It's also likely not fully comprehensive - there may be more fields we index that I haven't found.

I prepared this list by installing Elasticsearch HEAD and exploring the output - and this was a while ago, so I haven't accounted for new fields added in 2.2 or later, for example. Note that I am NOT a developer, and who knows what I have misinterpreted - hence the need for testing.

If you want to see the full mapping of fields in Elasticsearch, you could try running the following command:

• curl -XGET 'http://localhost:9200/atom/_mapping/QubitInformationObject?pretty'

That will focus on information objects, which accounts for most major entity types in AtoM. To see the full index, try:

• curl -XGET 'http://localhost:9200/atom/_mapping?pretty'

Note that nested elements - e.g. different i18n cultures - are separated by periods when broken out into the searchable term in AtoM - for example, General notes can be translated, so are nested in an i18n element - and then you need to specify what culture you want to search in when using the field name for targeted search. In the index mapping, this means it is nested - we can write out the final searchable field name like so:

• generalNotes.i18n.%LANG%.content

...where %LANG% indicates that user will input the desired ISO language code (ie. generalNotes.i18n.en.content to search English notes)

In most cases, terms might appear in HEAD or in the ES mapping using underscores - for example,  donors.contact_informations.contact_person - but if I recall correctly, you have to change these to camelCase to be able to use them in the appropriate search box - e.g. donors.contactInformations.contactPerson. As you'll see in the attached document, I've already converted them to the proper search term for all but the section labelled "Accessions" - this area is still untouched in my testing, and I can no longer recall if all those fields relate to what is searchable via the accessions search box, or if the list continues onto other related entities (e.g. donors) that must be searched in a different dedicated search box.

THIS would be a great way for the AtoM community to help contribute to improving AtoM's documentation! To this end, I've put my notes into a publicly accessible Google doc. Anyone with this link can now view and edit the document. I've added some initial notes on the syntax used in AtoM to search on these fields, and some tricks I've learned along the way (e.g. how to search for records that have no data in a specific field, for example).

I'm hoping that if you or anyone else interested in seeing this added to the AtoM documentation has some time, we can collaboratively test and update this document. Mark those fields that you have tested and work in green. Use strikethrough (Alt+Shift+5) to indicate that a field (or any other variation you can think of) does not work at all. If you find things that need changing based on your tests (e.g. changing the case on the accessions fields), please feel free to update the field as needed.

I've set the permissions on the document as open, but set editing to "Suggesting" so I can track what changes people make. I will try to re-test and confirm fields as I go through any changes submitted, and copy them to an internal master list, to be added to the documentation.

The document with the notes can be found at: https://docs.google.com/document/d/1luhU9GRXnKO5Pr91s_epCMl5zn-IfGfmtKyXosFWGHo/edit?usp=sharing

No pressure - I'm not putting this all on you, Grant! But I figured it would be better to share my notes and hope for some collaboration than just have them sitting around waiting for me to have more time.

Cheers,

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

--
You received this message because you are subscribed to the Google Groups "ICA-AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/78577184-542c-4429-b687-8a0741616e8f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Grant McNulty (500 Year Archive)]

Thanks a lot, Dan! We will go through the fields we are using for our project and I will ask our digital content manager to add our findings to the Google doc.

Cheers,

Grant 

[Grant McNulty (500 Year Archive)]

Thanks a lot, Dan! We will go through the fields we are using for our project and add to the Google doc 

On Wednesday, 6 April 2016 15:20:57 UTC+2, Grant McNulty (500 Year Archive) wrote:

[Grant McNulty (500 Year Archive)]

Hi Dan,

I passed this onto our Digital Content Manager who has been going through the RAD template to see which fields are, and are not, searchable. She has asked if there is a less techy and more user-friendly document to which she might contribute. So, for example, something that shows fields are they are seen in the AtoM interface, e.g. below: 

Title and statement of responsibility area

Title proper

Gourd Snuff Box (detail)

Title notes

• Source of title proper: XXXXX for FHYA using XXX materials

Level of description

Item

Repository

Cambridge Museum 

Thanks,

Grant

[Dan Gillean]

Hi Grant,

Your image showed up *very* strangely, which might be part of why I'm a bit confused as to what you're asking... but because this also seems relevant to another thread you've started, I'm attaching 3 resources for you. I created a description in AtoM with every available RAD field, using the RAD field names and standard numbers where relevant. I then 1) took a screenshot of the entire display, 2) exported the EAD, and 3) used the new clipboard functionality coming in 2.3 to export a CSV of just this description from the user interface. I'm attaching all 3 for you. Hopefully you can use them as a resource. The EAD should import properly in a 2.2 instance - the CSV will not, as it is a 2.3 CSV, and there are significant changes between 2.2 and 2.3 around how events and actors are handled, as I explained in your other thread. This should allow you to see what is kept and what is lost, and how things are mapped. The CSV column headers are also a lot closer to the kinds of internal names we used for the elasticsearch index, so it should allow you to draw inferences as to which ES fields in the document relate to which RAD fields.

I could try to find time to create a separate document with a crosswalk table on it - Field name, ES name, and testing notes, if you think that would help? A lot of them would be conjecture since as I mentioned, I haven't had time to dive into those initial notes and test them myself, but if you think that would be useful, let me know, and I'll try to start something in a public document soon.

Hope this helps!

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

--

You received this message because you are subscribed to the Google Groups "ICA-AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/0bcf2e91-036f-40da-bc5b-a6be41d4b67c%40googlegroups.com.

[Grant McNulty (500 Year Archive)]

Hi Dan,

Thanks for this detailed response! This definitely helps with our thinking around XML versus .csv, as per my other thread. It will also help us document which fields are, and are not searchable. 

Grant

On Wednesday, 20 April 2016 12:17:40 UTC+2, Grant McNulty (500 Year Archive) wrote:

Hi Dan,

I passed this onto our Digital Content Manager who has been going through the RAD template to see which fields are, and are not, searchable. She has asked if there is a less techy and more user-friendly document to which she might contribute. So, for example, something that shows fields are they are seen in the AtoM interface, e.g. below: 

Title and statement of responsibility area

Title prope

Gourd Snuff Box (detail)

[Clara Rosales]

Good morning to all!

Following the document of Dan (https://docs.google.com/document/d/1luhU9GRXnKO5Pr91s_epCMl5zn-IfGfmtKyXosFWGHo/edit) I was trying to recover information by the field "Type of entity" from Autorithy records.

If I am not doing wrong, the formulas would be:

creators.entity_type_id:131 -> Corporate body

creators.entity_type_id:132 -> Persons

creators.entity_type_id:133 -> Family

The result for any of the three queries is always empty, No results.

Please, does anyone know what I'm doing wrong?

I did the search from the general search box and from "Search Authority record" and even tried the "actor" formula instead of "creators" (actor.entity_type_id:131), but always the same result: No results :_(

Thank you very much in advance,

Clara 

[Dan Gillean]

Hi Clara,

This may be a result of my unfinished notes. There's a strange conversion that happens from the database fields to the ES index, where things that in the db use underscores as separators always use camelCase in the index. So I might not have converted what I found yet. Can you try:

creators.entityTypeId:131 -> Corporate body

creators.entitTypeId:132 -> Persons

creators.entityTypeId:133 -> Family

The other thing is to make sure you are trying this search using the dedicated authority record search box on the Authority record browse page - the type of entity will not be indexed in the global search, which is focused much more on archival descriptions.

Let us know how it goes!

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.

Visit this group at https://groups.google.com/group/ica-atom-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/3647a1b6-c058-405c-960a-5977d7a59230%40googlegroups.com.

[Clara Rosales]

Thank you very much, Dan. It works as you indicate!! :)

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/3647a1b6-c058-405c-960a-5977d7a59230%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/CAC1FhZLKMu%3DGma1%2Bk3zwD_XJ%3DPLwOJ1na_UV6Cvj0%3D0E_UA5qw%40mail.gmail.com.

[Grant McNulty (500 Year Archive)]

Hi Dan,

A long time later but here is a link to the (descriptive) searchable fields we found in AtoM:

https://docs.google.com/spreadsheets/d/1IMoHAtCroM5sWcfDavwof9Nig94-DqCCNI1pb8UcdaU/edit?usp=sharing

May be of use to someone.

Grant