Related descriptions

54 views
Skip to first unread message

Nathanael

unread,
Feb 6, 2018, 9:51:23 AM2/6/18
to AtoM Users
Hi

We're finding that the 'related descriptions' field in the 'Allied materials area' is hard to use. Presumably you should be able to enter the identifier of a related description and it will then appear in the dropdown list, allowing you to create the link. However when we type in one of our identifiers (which are usually in the form "SPAB-BAS-045" or similar), often nothing appears in the dropdown. Typing part of the identifer or a word from the title brings up results but there are often a lot of them and only the first 20 are shown, so the one we want might not be visible. Is there any way around this?

Thanks

Nathanael

Jenny Mitcham

unread,
Feb 6, 2018, 10:03:14 AM2/6/18
to ica-ato...@googlegroups.com
Hi Nathanael,

We have a workaround to this issue as described below:

"In some cases you will not be able to locate the entry you want to link to in the drop down menu.  AtoM only displays 20 options at a time and if you are searching for a particularly common term such as ‘letters’ it will not be able to show all the possible entries.  There are two solutions to this,

If you can, try typing a more unusual word from the title of the entry you want to link to. If it is called ‘Letters concerning Germany’, try typing ‘Germany’ or ‘concerning Germany’ instead of ‘Letters’.

If no combination of words from the title is working, open the entry you want to link to and click ‘edit’ in the bottom toolbar.  Then expand the ‘identity area’ and add an unusual word to the title.  Save the changes and then return to your original entry and search for the unusual word in the ‘related descriptions’ field and select the entry you want.  Save the changes (and remember to go back to the linked entry to remove the unusual word!).  The edits will not affect the URL of the linked entry and the link between the two entries will remain in place after you have changed the title back."

I know this isn't ideal but it does at least allow staff here to make the links they need to.

A more robust and scalable solution would be great in the longer term!

Cheers,
Jen


--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/6af779ea-6801-4295-8171-6890d99c39f3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
Jenny Mitcham
Digital Archivist
Borthwick Institute for Archives
University of York
Heslington
York
YO10 5DD

Telephone: 01904 321170

Borthwick Institute website: http://www.york.ac.uk/borthwick/
Digital archiving blog: http://digital-archiving.blogspot.co.uk/
Twitter: @Jenny_Mitcham
Skype: jenny_mitcham

Victoria Peters

unread,
Feb 6, 2018, 10:12:13 AM2/6/18
to ica-ato...@googlegroups.com

This is exactly the workaround I use too. ‘Elephant’ is my chosen word!

 

Victoria

 

From: 'Jenny Mitcham' via AtoM Users [mailto:ica-ato...@googlegroups.com]
Sent: 06 February 2018 15:03
To: ica-ato...@googlegroups.com
Subject: Re: [ica-atom-users] Related descriptions

 

Hi Nathanael,

 

We have a workaround to this issue as described below:

 

"In some cases you will not be able to locate the entry you want to link to in the drop down menu.  AtoM only displays 20 options at a time and if you are searching for a particularly common term such as ‘letters’ it will not be able to show all the possible entries.  There are two solutions to this,

 

If you can, try typing a more unusual word from the title of the entry you want to link to. If it is called ‘Letters concerning Germany’, try typing ‘Germany’ or ‘concerning Germany’ instead of ‘Letters’.

 

If no combination of words from the title is working, open the entry you want to link to and click ‘edit’ in the bottom toolbar.  Then expand the ‘identity area’ and add an unusual word to the title.  Save the changes and then return to your original entry and search for the unusual word in the ‘related descriptions’ field and select the entry you want.  Save the changes (and remember to go back to the linked entry to remove the unusual word!).  The edits will not affect the URL of the linked entry and the link between the two entries will remain in place after you have changed the title back."

 

I know this isn't ideal but it does at least allow staff here to make the links they need to.

 

A more robust and scalable solution would be great in the longer term!

 

Cheers,

Jen

 

On 6 February 2018 at 14:51, Nathanael <nathana...@millsarchive.org> wrote:

Hi

 

We're finding that the 'related descriptions' field in the 'Allied materials area' is hard to use. Presumably you should be able to enter the identifier of a related description and it will then appear in the dropdown list, allowing you to create the link. However when we type in one of our identifiers (which are usually in the form "SPAB-BAS-045" or similar), often nothing appears in the dropdown. Typing part of the identifer or a word from the title brings up results but there are often a lot of them and only the first 20 are shown, so the one we want might not be visible. Is there any way around this?

 

Thanks

 

Nathanael

--

You received this message because you are subscribed to the Google Groups "AtoM Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.



 

--

Jenny Mitcham
Digital Archivist
Borthwick Institute for Archives

University of York
Heslington
York
YO10 5DD

Telephone: 01904 321170

Borthwick Institute website: http://www.york.ac.uk/borthwick/
Digital archiving blog: http://digital-archiving.blogspot.co.uk/

Twitter: @Jenny_Mitcham

Skype: jenny_mitcham

 

--

You received this message because you are subscribed to the Google Groups "AtoM Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.

Dan Gillean

unread,
Feb 6, 2018, 10:26:23 AM2/6/18
to ICA-AtoM Users
Hi all, 

Thank you for sharing your workarounds, Jen and Victoria! 

Nathanael, this is unfortunately a known issue, described in the following ticket: 
Overall, the library we use for the autocomplete element is very old and could stand to be entirely replaced. That's a lot of work however. In the meantime, there is much that could be done to incrementally improve this field with a much smaller investment of work. 

The two biggest issues with the current related materials autocomplete are, as I see it: 

1) The field is still using OR as the default operator, and 
2) every time a space is added between terms, the search resets. 

Because of 2, it's not even fair to say that OR is being used as an operator at all - instead, if you enter "John Smith" then because of the space between terms, John is ignored, and the drop-down results reset and begin returning only matches on "Smith."

It wouldn't solve all issues, but fixing these two bugs would massively improve the results in the related materials autcomplete. Two further improvements that could be made with more time: 
  • Support the use of quotations in the field for exact order matching, as in a global search
  • Add lazy loading to the autocomplete results, so that a "Show more results" option could be clicked to load the next 10 matches if the desired record is not shown in the first 10 results. 
Another tricky issue is that improving this field would not necessarily improve all other autocompletes throughout the application. At this point, different stopgap solutions have been individually applied to different autcompletes to resolve various issues (scalability, permissions, accuracy, etc) - so some autocompletes are using the application's ORM for the query, some are querying the Elasticesearch index, and some may even be using raw SQL to query the database directly. We would likely need to find, analyze, and update various fields individually with this in mind. 

However, the first two improvements could be applied to this particular field with relatively little development work. 

If your institution is interested in prioritizing these enhancements, please feel free to contact me off-list so we can discuss options further. 

Regards, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory

On Tue, Feb 6, 2018 at 10:12 AM, Victoria Peters <victori...@strath.ac.uk> wrote:

This is exactly the workaround I use too. ‘Elephant’ is my chosen word!

 

Victoria

 

From: 'Jenny Mitcham' via AtoM Users [mailto:ica-atom-users@googlegroups.com]
Sent: 06 February 2018 15:03
To: ica-atom-users@googlegroups.com
Subject: Re: [ica-atom-users] Related descriptions

 

Hi Nathanael,

 

We have a workaround to this issue as described below:

 

"In some cases you will not be able to locate the entry you want to link to in the drop down menu.  AtoM only displays 20 options at a time and if you are searching for a particularly common term such as ‘letters’ it will not be able to show all the possible entries.  There are two solutions to this,

 

If you can, try typing a more unusual word from the title of the entry you want to link to. If it is called ‘Letters concerning Germany’, try typing ‘Germany’ or ‘concerning Germany’ instead of ‘Letters’.

 

If no combination of words from the title is working, open the entry you want to link to and click ‘edit’ in the bottom toolbar.  Then expand the ‘identity area’ and add an unusual word to the title.  Save the changes and then return to your original entry and search for the unusual word in the ‘related descriptions’ field and select the entry you want.  Save the changes (and remember to go back to the linked entry to remove the unusual word!).  The edits will not affect the URL of the linked entry and the link between the two entries will remain in place after you have changed the title back."

 

I know this isn't ideal but it does at least allow staff here to make the links they need to.

 

A more robust and scalable solution would be great in the longer term!

 

Cheers,

Jen

 

On 6 February 2018 at 14:51, Nathanael <nathanael.hodge@millsarchive.org> wrote:

Hi

 

We're finding that the 'related descriptions' field in the 'Allied materials area' is hard to use. Presumably you should be able to enter the identifier of a related description and it will then appear in the dropdown list, allowing you to create the link. However when we type in one of our identifiers (which are usually in the form "SPAB-BAS-045" or similar), often nothing appears in the dropdown. Typing part of the identifer or a word from the title brings up results but there are often a lot of them and only the first 20 are shown, so the one we want might not be visible. Is there any way around this?

 

Thanks

 

Nathanael

--

You received this message because you are subscribed to the Google Groups "AtoM Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.



 

--

Jenny Mitcham
Digital Archivist
Borthwick Institute for Archives

University of York
Heslington
York
YO10 5DD

Telephone: 01904 321170

Borthwick Institute website: http://www.york.ac.uk/borthwick/
Digital archiving blog: http://digital-archiving.blogspot.co.uk/

Twitter: @Jenny_Mitcham

Skype: jenny_mitcham

 

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.

Jenny Mitcham

unread,
Feb 6, 2018, 10:55:08 AM2/6/18
to ica-ato...@googlegroups.com
I am reliably informed that the word we favour here is 'Zebra' as it pops up as soon as you start typing the letter ' :-)

On 6 February 2018 at 15:12, Victoria Peters <victori...@strath.ac.uk> wrote:

This is exactly the workaround I use too. ‘Elephant’ is my chosen word!

 

Victoria

 

From: 'Jenny Mitcham' via AtoM Users [mailto:ica-atom-users@googlegroups.com]
Sent: 06 February 2018 15:03
To: ica-atom-users@googlegroups.com
Subject: Re: [ica-atom-users] Related descriptions

 

Hi Nathanael,

 

We have a workaround to this issue as described below:

 

"In some cases you will not be able to locate the entry you want to link to in the drop down menu.  AtoM only displays 20 options at a time and if you are searching for a particularly common term such as ‘letters’ it will not be able to show all the possible entries.  There are two solutions to this,

 

If you can, try typing a more unusual word from the title of the entry you want to link to. If it is called ‘Letters concerning Germany’, try typing ‘Germany’ or ‘concerning Germany’ instead of ‘Letters’.

 

If no combination of words from the title is working, open the entry you want to link to and click ‘edit’ in the bottom toolbar.  Then expand the ‘identity area’ and add an unusual word to the title.  Save the changes and then return to your original entry and search for the unusual word in the ‘related descriptions’ field and select the entry you want.  Save the changes (and remember to go back to the linked entry to remove the unusual word!).  The edits will not affect the URL of the linked entry and the link between the two entries will remain in place after you have changed the title back."

 

I know this isn't ideal but it does at least allow staff here to make the links they need to.

 

A more robust and scalable solution would be great in the longer term!

 

Cheers,

Jen

 

On 6 February 2018 at 14:51, Nathanael <nathanael.hodge@millsarchive.org> wrote:

Hi

 

We're finding that the 'related descriptions' field in the 'Allied materials area' is hard to use. Presumably you should be able to enter the identifier of a related description and it will then appear in the dropdown list, allowing you to create the link. However when we type in one of our identifiers (which are usually in the form "SPAB-BAS-045" or similar), often nothing appears in the dropdown. Typing part of the identifer or a word from the title brings up results but there are often a lot of them and only the first 20 are shown, so the one we want might not be visible. Is there any way around this?

 

Thanks

 

Nathanael

--

You received this message because you are subscribed to the Google Groups "AtoM Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.



 

--

Jenny Mitcham
Digital Archivist
Borthwick Institute for Archives

University of York
Heslington
York
YO10 5DD

Telephone: 01904 321170

Borthwick Institute website: http://www.york.ac.uk/borthwick/
Digital archiving blog: http://digital-archiving.blogspot.co.uk/

Twitter: @Jenny_Mitcham

Skype: jenny_mitcham

 

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Victoria Peters

unread,
Feb 6, 2018, 11:12:49 AM2/6/18
to ica-ato...@googlegroups.com

Even better!

 

From: 'Jenny Mitcham' via AtoM Users [mailto:ica-ato...@googlegroups.com]
Sent: 06 February 2018 15:55
To: ica-ato...@googlegroups.com
Subject: Re: [ica-atom-users] Related descriptions

 

I am reliably informed that the word we favour here is 'Zebra' as it pops up as soon as you start typing the letter ' :-)

 

On 6 February 2018 at 15:12, Victoria Peters <victori...@strath.ac.uk> wrote:

This is exactly the workaround I use too. ‘Elephant’ is my chosen word!

 

Victoria

 

From: 'Jenny Mitcham' via AtoM Users [mailto:ica-ato...@googlegroups.com]
Sent: 06 February 2018 15:03
To: ica-ato...@googlegroups.com
Subject: Re: [ica-atom-users] Related descriptions

 

Hi Nathanael,

 

We have a workaround to this issue as described below:

 

"In some cases you will not be able to locate the entry you want to link to in the drop down menu.  AtoM only displays 20 options at a time and if you are searching for a particularly common term such as ‘letters’ it will not be able to show all the possible entries.  There are two solutions to this,

 

If you can, try typing a more unusual word from the title of the entry you want to link to. If it is called ‘Letters concerning Germany’, try typing ‘Germany’ or ‘concerning Germany’ instead of ‘Letters’.

 

If no combination of words from the title is working, open the entry you want to link to and click ‘edit’ in the bottom toolbar.  Then expand the ‘identity area’ and add an unusual word to the title.  Save the changes and then return to your original entry and search for the unusual word in the ‘related descriptions’ field and select the entry you want.  Save the changes (and remember to go back to the linked entry to remove the unusual word!).  The edits will not affect the URL of the linked entry and the link between the two entries will remain in place after you have changed the title back."

 

I know this isn't ideal but it does at least allow staff here to make the links they need to.

 

A more robust and scalable solution would be great in the longer term!

 

Cheers,

Jen

 

On 6 February 2018 at 14:51, Nathanael <nathana...@millsarchive.org> wrote:

Hi

 

We're finding that the 'related descriptions' field in the 'Allied materials area' is hard to use. Presumably you should be able to enter the identifier of a related description and it will then appear in the dropdown list, allowing you to create the link. However when we type in one of our identifiers (which are usually in the form "SPAB-BAS-045" or similar), often nothing appears in the dropdown. Typing part of the identifer or a word from the title brings up results but there are often a lot of them and only the first 20 are shown, so the one we want might not be visible. Is there any way around this?

 

Thanks

 

Nathanael

--

You received this message because you are subscribed to the Google Groups "AtoM Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.



 

--

Jenny Mitcham
Digital Archivist
Borthwick Institute for Archives

University of York
Heslington
York
YO10 5DD

Telephone: 01904 321170

Borthwick Institute website: http://www.york.ac.uk/borthwick/
Digital archiving blog: http://digital-archiving.blogspot.co.uk/

Twitter: @Jenny_Mitcham

Skype: jenny_mitcham

 

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.

--

You received this message because you are subscribed to the Google Groups "AtoM Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.



 

--

Jenny Mitcham
Digital Archivist
Borthwick Institute for Archives

University of York
Heslington
York
YO10 5DD

Telephone: 01904 321170

Borthwick Institute website: http://www.york.ac.uk/borthwick/
Digital archiving blog: http://digital-archiving.blogspot.co.uk/

Twitter: @Jenny_Mitcham

Skype: jenny_mitcham

 

--

You received this message because you are subscribed to the Google Groups "AtoM Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.

L Snider

unread,
Feb 6, 2018, 12:00:38 PM2/6/18
to ica-ato...@googlegroups.com
Thanks for that tip! I have found that area to be hard to use so it is a good workaround!

Cheers

Lisa

On Tue, Feb 6, 2018 at 10:12 AM, Victoria Peters <victori...@strath.ac.uk> wrote:

Even better!

 

From: 'Jenny Mitcham' via AtoM Users [mailto:ica-atom-users@googlegroups.com]
Sent: 06 February 2018 15:55

--

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.



 

--

Jenny Mitcham
Digital Archivist
Borthwick Institute for Archives

University of York
Heslington
York
YO10 5DD

Telephone: 01904 321170

Borthwick Institute website: http://www.york.ac.uk/borthwick/
Digital archiving blog: http://digital-archiving.blogspot.co.uk/

Twitter: @Jenny_Mitcham

Skype: jenny_mitcham

 

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.

--

You received this message because you are subscribed to the Google Groups "AtoM Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.



 

--

Jenny Mitcham
Digital Archivist
Borthwick Institute for Archives

University of York
Heslington
York
YO10 5DD

Telephone: 01904 321170

Borthwick Institute website: http://www.york.ac.uk/borthwick/
Digital archiving blog: http://digital-archiving.blogspot.co.uk/

Twitter: @Jenny_Mitcham

Skype: jenny_mitcham

 

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.

Nathanael

unread,
Feb 7, 2018, 5:09:22 AM2/7/18
to AtoM Users
Thanks for the suggestions!

Dan Gillean

unread,
Feb 7, 2018, 10:59:39 AM2/7/18
to ICA-AtoM Users
Hi Nathanael, 

I have some good news. The two low-hanging fruit fixes I mentioned (changing the operator to AND; making sure the autocomplete doesn't reset when a space is added between terms) have been fixed! It turns out a developer had a patch for this completed that was never merged into the public code. We have now added the patch to 2.5 and tested it there - we should have time to backport this improvement to stable/2.4.x for inclusion in the 2.4.1 release. Hopefully that will somewhat in the future. 

You can keep an eye on this ticket - when it has been backported to stable/2.4.x, I will change the target release from 2.5 to 2.4.1: 

I discovered something else about the autocomplete fields as I was looking into this. There is an issue right now where any term that is longer than 10 characters will cause the autcomplete to reset and stop providing suggestions. I thought this was a bug, but it seems it is in fact a setting that a system administrator could choose to alter, maintained here: 
I would personally like to see our default value upped to 15 characters rather than 10, but Artefactual's technical team has apparently decided not to change this default value, out of worries that it could impact scalability and performance. I understand the concern - take a term like correspondence for example. With the current settings, if you type past correspond then the related materials autocomplete will close showing no results, which seems frustrating. However, what's happening in the background so that terms can be matched on the fly as you type?

Essentially this term (and every other term in your data that makes it into the index) is being cut up and indexed at every character: 
  • cor
  • corr
  • corre
  • corres
  • corresp
  • correspo
  • etc..
It's also indexing partial terms the other way, as well, e.g.: 
  • nce
  • ence
  • dence
  • ndence
  • ondence
  • etc...
And partials right in the middle of the term, such as: 
  • res
  • rres
  • resp
  • rresp
  • orres
  • orresp
  • respo
  • rrespo
  • orrespo
  • etc...
Which is why you can enter orres into the Related descriptions autocomplete and seem matching results that include terms like correspondence, Forrester, Torres, etc. 

For this one term, at a setting of 10, there are dozens of variations of it in your index (10 partial matches and the full term), all for one word in your data. To change this value could mean adding thousands or even hundreds of thousands of new partial terms to be indexed. So, while upping the default max_gram value seems like a good idea, it might take some time and analysis to figure the impact this could have on a site - especially a large installation. For that reason, we're leaving it as is for now, though I will try to add some notes about this to the documentation somewhere so system administrators can change the default value if desired. 

You can read about this in the ES 1.7 documentation here, and for our 2.5 version (where we have upgraded ES to version 5.2.1) the ES 5.x docs are here. If you want to dig into the technical info on how ngrams work in ES, this is a good place to start

Our developers also added some detauls about this on the ticket where it was first identified, here: 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory

On Wed, Feb 7, 2018 at 5:09 AM, Nathanael <nathana...@millsarchive.org> wrote:
Thanks for the suggestions!

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.
Reply all
Reply to author
Forward
0 new messages