How many files can a series hold?

[dafy...@aol.com]

Dear AtoM users,

The National Library of Wales is creating a catalogue of BBC Scripts in which several series will contain hundreds of files - up to 300 in most cases and in one case 600. AtoM is coping well with what we've done so far, and it's also coping with some very large series in other catalogues that have been migrated from previous systems (which couldn't cope with them), but we're wondering whether there's a limit on how many can be created and processed comfortably. Vicky Phillips has reminded me that AtoM has been designed to deal with large archives, so hopefully this won't be a problem.

If anyone has experience of creating large numbers of records with a single parent, we'd love to hear from you. Thanks!

David Moore (Archivist, NLW)

[Dan Gillean]

Hi David, 

We don't know exactly what the upper limit is exactly - mostly because it depends largely on what you're doing in terms of system resources and deployment strategies to optimize the site, but also because definitions around how usable a site is based on load/response time can sometimes be subjective. 

Making sure you have lots of memory is always a good idea. Using separately resourced VMs for the database, the search index, and the code is a good way to handle scalability. We regularly recommend a 2-site deployment for large sites, because then the read-only front end can be combined with an aggressive cache engine (such as Varnish, etc) and tuned/resourced separately. Finally, keeping your installation version up to date is often helpful, because we try to include performance improvements in each release. 

As far as I'm aware, NLW is already doing most if not all of these things, so.... you tell us! As far as I know there's no built-in limit to the number of records - most elements are configurable anyway, so generally it's a matter of having enough resources, and the page(s) in question not timing out before they can load. 

I will say that if it does start getting too big to be usable as a single fonds, you might consider artificially separating things, i.e. making the series its own top-level record, and then using Related Description linkages, in-description Markdown hyperlinks, custom uploaded finding aids, Static pages, and/or any other method to make it apparent to end users that other top-level series are directly related. Just one idea. 

Cheers, 

Dan Gillean, MAS, MLIS
AtoM Program Manager
Artefactual Systems, Inc.
604-527-2056

@accesstomemory

he / him

--
You received this message because you are subscribed to the Google Groups "AtoM Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ica-atom-user...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ica-atom-users/0c313048-de4c-46c0-98f6-8d6abc889baan%40googlegroups.com.

[Nathanael]

Hi

We have a series of 10647 items, (photographic surveys of mills which the photographer catalogued for us). It seems to work fine, although the treeview takes a moment to load: https://catalogue.millsarchive.org/images-1.

Best wishes

Nathanael