Browse Archival Descriptions with Sort by: Alphabetic not behaving

76 views
Skip to first unread message

Vicky Phillips

unread,
May 4, 2017, 11:58:00 AM5/4/17
to AtoM Users
Hello

Archivists have reported that the Sort Alphabetic doesn’t seem to behave either when logged in or not in version 2.3 and 2.3.1. We’ve set the Sort browser (users) and (anonymous) to sort alphabetic.


https://archives.library.wales/index.php/informationobject/browse

What I've noticed is that it behaves when you select either English language or Welsh language but not when Unique records is selected. Is this a bug in the system?

Thanks,
Vicky


Dan Gillean

unread,
May 4, 2017, 12:50:20 PM5/4/17
to ICA-AtoM Users
Hi Vicky,

I glanced at your site as a public user, and it *seems* at first glance to be behaving much the same for the Unique records facet as for any other language facet. Can you tell me more about the differences you are seeing between the Unique records and the English faceted ones?

It's possible that part of the confusion stems from the nature of the "alphabetic" sort as it is currently implemented in AtoM - that is, the reasonable expectations we have of what is often referred to as "natural sort" versus how it is actually implemented - as ASCII sort.

At the moment AtoM's "alphabetical" sorting is in fact what is sometimes called "ASCIIbetical" - that is, it is a sort based on the ASCII table, not a natural sort as we would expect, which ignores leading punctuation like quotes, capitalization, accents or other special characters, leading spaces or zeroes, etc.  Natural sorting is hard, especially an an international, multilingual application like AtoM, and no one has ever prioritized funding its development so far. Here's an article outlining some of the challenges:

We also have a note on this, with a reference ASCII table, in this section of our documentation:


Essentially, since ASCII sorting depends on the ordering of the ASCII table, this can lead to unexpected results from what we might naturally expect - for example:

  • Za would appear before aardvark (capitalization matters)
  • Numbers would sort 1, 10, 11, 2, etc
  • Leading spaces will affect sort order
  • Accented characters will sort later than non-accented ones - and a capitalized accented character will sort differently than one that is not
  • 'test' will appear before "test"

And so forth.

We could definitely improve this in AtoM - there are some sorting options built into PHP that could be further analyzed and tested. The multilingual nature of AtoM, especially where culture fallback is used in results that might appear in many cultures (such as search/browse) adds complexity, but I think with time we could improve this consistently in the application. This would definitely require community sponsorship for us to be able to implement properly however, as the analysis, testing, and development required is non-trivial. If that's something your institution is interested in discussing further, please feel free to contact me off-list. I've found a number of tickets that further describe the challenges, the attempts we've made over the years to adjust, and their subsequent move to the Wishlist until they can be sponsored - you might find them of interest:

And of course: if I've totally missed what you are pointing to in my quick glance, please feel free to correct me and provide further details, and I can take another look and try to reproduce.

Regards,


Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/1c216538-99ce-45f5-97a3-85761b853695%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Vicky Phillips

unread,
May 5, 2017, 4:19:41 AM5/5/17
to AtoM Users
Hi Dan,
Thanks for your quick response and for explaining how AtoM's sorting works. The problem we have at the moment is that with the Unique records, the pages towards the end of the results list don't seem to follow any Alphabetic sorting, see here
https://archives.library.wales/index.php/informationobject/browse?page=1485&limit=10&sort=alphabetic&topLod=1

Whereas selecting English
https://archives.library.wales/index.php/informationobject/browse?page=1422&languages=en&limit=10&sort=alphabetic&topLod=1

or Welsh
https://archives.library.wales/index.php/informationobject/browse?page=63&languages=cy&limit=10&sort=alphabetic&topLod=1

the alphabetic sorting work fine. I would expect alphabetic sorting of Unique records to behave the same way as the English and Welsh alphabetic sorting. Hope this explains things  a bit better.

Thanks,
Vicky

Dan Gillean

unread,
May 5, 2017, 11:24:55 AM5/5/17
to ICA-AtoM Users
Hi Vicky,

I see what you mean. Since it seems to behave correctly when results are limited to one specific culture, I believe that this is the issue described in the last ticket I linked - see:

One of our developers at the time added a rough estimate on the issue ticket of the time he believed would be involved for a fix. We would have to review this, but it should give you a sense of scale for the issue. As it will require investigation, testing, and a number of changes to our ES index and database, it is a bit beyond what Artefactual can address without community sponsorship.

Regards,


Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.

David Juhasz

unread,
May 5, 2017, 1:00:10 PM5/5/17
to ica-ato...@googlegroups.com
Hi all,

I just want to add a caveat here: this sorting issue is totally unrelated to the treeview sorting problems being discussed in a separate thread.  Fixing this issue will not fix treeview sorting.

The browse and search pages in AtoM use Elasticsearch as the data source and the sorting is done with Elasticsearch.  The treeview is using MySQL as it's data source and for sorting.  Both data sources have their own advantages and disadvantages.

Cheers,
David

--

David Juhasz
Director, AtoM Technical Services Artefactual Systems Inc. www.artefactual.com

On Fri, May 5, 2017 at 8:24 AM, Dan Gillean <d...@artefactual.com> wrote:
Hi Vicky,

I see what you mean. Since it seems to behave correctly when results are limited to one specific culture, I believe that this is the issue described in the last ticket I linked - see:

One of our developers at the time added a rough estimate on the issue ticket of the time he believed would be involved for a fix. We would have to review this, but it should give you a sense of scale for the issue. As it will require investigation, testing, and a number of changes to our ES index and database, it is a bit beyond what Artefactual can address without community sponsorship.

Regards,


Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056
@accesstomemory
On Fri, May 5, 2017 at 4:19 AM, 'Vicky Phillips' via AtoM Users <ica-atom-users@googlegroups.com> wrote:
Hi Dan,
Thanks for your quick response and for explaining how AtoM's sorting works. The problem we have at the moment is that with the Unique records, the pages towards the end of the results list don't seem to follow any Alphabetic sorting, see here
https://archives.library.wales/index.php/informationobject/browse?page=1485&limit=10&sort=alphabetic&topLod=1

Whereas selecting English
https://archives.library.wales/index.php/informationobject/browse?page=1422&languages=en&limit=10&sort=alphabetic&topLod=1

or Welsh
https://archives.library.wales/index.php/informationobject/browse?page=63&languages=cy&limit=10&sort=alphabetic&topLod=1

the alphabetic sorting work fine. I would expect alphabetic sorting of Unique records to behave the same way as the English and Welsh alphabetic sorting. Hope this explains things  a bit better.

Thanks,
Vicky

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-ato...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-users+unsubscribe@googlegroups.com.
To post to this group, send email to ica-atom-users@googlegroups.com.
Visit this group at https://groups.google.com/group/ica-atom-users.
Reply all
Reply to author
Forward
0 new messages