Hello everyone,
I am pleased to announce the Archipelago 1.4.0 Release today, woohoo!
You can find below a fuller overview of the changes baked into this release, broken down by the primary Archipelago Github repos. Sharing the highlights of what we (myself and Diego) think are some of the most notable items in this release reel:
Out-of-the-box integration of IIIF Content Search API functionality: updated and new IIIF Manifest templates, configurations, and under-the-hood code to support this branch of the IIIF APIs tree in Archipelago
New ‘subtitle’ and ‘text’ Strawberry Runners post-processing pipelines: for indexing and feeding IIIF Content Searching
Implementation of Mirador 4
Advanced Search Default Configurations: taking full advantage of the custom Search filters and functionality developed specifically for Archipelago
Incorporation of key Drupal 10+ bug fixes and updates: keeping your Archipelago up-to-date and ready for future incoming waves of Drupal changes
New Metadata API Plugin and default OAI-PMH configurations: for sharing your Archipelago assets via a tailored OAI-PMH endpoint
Experimental implementation of ML assisted semantic search/image and text similarity analysis tooling (for local deployments, for authenticated-users-only; requires enabling process as noted in deployment instructions)
Vector (text and image fragments) integration for Solr
Replayweb and other Javascript integrations updated to the latest versions
New guides for new functionality and updated information throughout our documentation: for all your learning and reference needs
New 2024 & Beyond Archipelago Roadmap: https://github.com/esmero/archipelago-deployment/issues/269
You can find instructions for taking the latest Archipelago 1.4.0 release for a local test drive here:
https://docs.archipelago.nyc/1.4.0/archipelago-deployment-readme/
And you can find instructions for Archipelago Deployment Live here:
https://docs.archipelago.nyc/1.4.0/archipelago-deployment-live-readme/
Earlier today, running a fresh local deployment (again, repeat testing!) took me approximately 5 minutes to complete, from Step 1 for MacOSX, all the way to a completely processed + ingested AMI Demo Set.
To check out the latest IIIF API Content Search integrations, you will need to first wait for the Background Post-processing Queues and your Solr Index to finish updating after you run the AMI Demo Set (feel free to give the Cron a push to start ahead of the default 3 hour schedule). Then you can switch the Display Mode for any of the text-based demo objects (such as ‘The Natural History of Dogs’ Book) to the ‘Digital Object with Mirador Viewer’ mode, and test out a search within the left-hand sidebar of the Mirador viewer.
During a time when there is much fawning over machine mediated algorithmic outputs, I also want to take the time to highlight and share information about the often obscured hidden human labor and professional endeavors that empower Archipelago as an open source software system, and the wonderful content of research and cultural heritage resources that bring Archipelago deployments to life.
Across only the first 15 Archipelagos listed on our ‘In the Wild’ page under the section covering instances that Diego and I are privileged to manage directly or support as the METRO Digital Services Team alongside our institutional partners:
Total Top-Level Digital Objects: 570,000+
Total Digital Objects including all subcomponents (pages, transcripts, mixed media/compound items): 2.65+ million
Total Unique Lines of Twig Code in Metadata Display Templates: ~4200
Total Custom Configuration Yaml File Differentiations (top file level, not reflective of line-level differences): ~3150
Total Custom Solr Fields, Facets, Blocks, and Theme Components: ~2400
Site Status Alerts (for Bot Resource Floods) Responded to between Jan-June 2024: 96
Total Cups of Coffee Per Day Required by our team alone: 4-8 (depending on the day’s to-do’s and those site alerts)
Examples of a few beautifully crafted resources found in Archipelago repositories:
Association Montessori Internationale’s NAMTA Journal Collection (please also see the fuller listings style page)
San Diego State University’s Archivo Histórico de Tijuana: Postales
Hostos Community College’s Museum of Contemporary Hispanic Art Collection
Please browse through the portfolio of Archipelago instances listed on our ‘In the Wild’ page and take a moment to appreciate the many other vibrant collections being shared by institutions around the world.
These release notes would not be complete without a special shout out and thank you to Diego for all of his wonderful work on crafting so much of Archipelago’s codebase, and for his ongoing, dedicated support of the community throughout many years, reflecting a true embodiment of ‘esmero’.
We would also like to thank the contributors of code and resources to Archipelago’s functionality and a few of the largest growing Archipelagos that have crossed our paths these past 6 months: Johannes Baiter and the Bavarian State Library team, Giancarlo Birello and Anna Perin, Mike Bennet, Megan Tyne and Roland Obermair, Ruairí O’Hare, Pat Dunlavey, Joanna DiPasquale and Corinne Chatnick, the entire SDSU repository team, Jennifer Palmentiero and Zack Spalding, Ilya Kreymer and the Webrecorder team, and the wider IIIF and Drupal Communities.
To everyone that we have not the space to name individually: thank you also, the Archipelago community of users and thinkers at libraries, museums, universities and colleges, cultural heritage institutions, and repository services organizations. We appreciate your continued support and your open sharing of growing and well nurtured Archipelago repository collections, your use cases, and your thoughtful discussions and considerations.
If you have any questions or comments related to the latest release, please reach out on this community group or hop into the Archipelago Slack.
Thank you for your time and attention. Wish you all a very happy rest of your summer!
Allison
--
Allison Sherrick
Digital Projects and Services Manager
Metropolitan New York Library Council
Repo-by-repo Detailed Breakdown of 1.4.0 Release Changes
(Some of the numerical figures cited below may bump up a bit as we make a few more minor changes here and there, but all figures are by and large an accurate representation of the release work totals.)
Archipelago Deployment:
https://github.com/esmero/archipelago-deployment/compare/1.3.0...1.4.0
155 changed files with 10,528 additions and 1,741 deletions
Please review the branch comparison above for a fuller breakdown of all the changes.
Archipelago Deployment-Live:
https://github.com/esmero/archipelago-deployment-live/compare/1.3.0...1.4.0
162 changed files with 6,914 additions and 1,811 deletions.
Please review the branch comparison above for a fuller breakdown of all the changes.
Archipelago Multi Importer (AMI):
https://github.com/esmero/ami/releases/tag/v0.8.0-pre
ISSUE-190: Set max-age to 0 for amiSetEntityReportForm.php by @DiegoPino in #191
ISSUE-193: Forced Downloads will force everything + check on return code/status by @DiegoPino in #194
ISSUE-197: Deals with moving target for our VBO Event subscriber conditionals by @DiegoPino in #198
ISSUE-195: CSV queue worker and nested CSVs for EAD + an EAD plugin by @DiegoPino in #196
Update amiSetEntityForm.php for AMI Update Operations changes by @alliomeria in #204
13 changed files with 718 additions and 124 deletions.
Strawberryfield:
https://github.com/esmero/strawberryfield/releases/tag/v1.4.0-pre
ISSUE-294: Fix missing access check on entity query by @DiegoPino in #295
ISSUE-294: Fix missing access check on entity query by @DiegoPino in #296
ISSUE-297: D3JS once v/s Jquery Once fix + small supporting code fixes by @DiegoPino in #298
ISSUE-297: For 1.4.0, fixes once on D3JS by @DiegoPino in #299
ISSUE-300: 1.4.0 version of safer trim for Key Name Providers by @DiegoPino in #302
ISSUE-303: Escape Tags on RAW JSON formatter by @DiegoPino in #304
ISSUE-305: Don't use parent_top_sequence_id if the result Node is the same as the c… by @DiegoPino in #306
ISSUE-308: This adds {1992..1996,...} to Solr indexing + Session ISSUE-310 + ISSUE-313 by @DiegoPino in #309
ISSUE-315: Use a regular expression on Fido's output. That way ... by @DiegoPino in #316
ISSUE-318: Mixed SQL/SearchAPI driven result sets and Query Object by @DiegoPino in #319
ISSUE-320: Adds a new Key value DB class with ranged/listed queries by @DiegoPino in #321
ISSUE-322: Add the famous universal constant of time/space disambiguation by @DiegoPino in #323
ISSUE-325: Vectors for each king, one float to rule them all by @DiegoPino in #326
32 changed files with 869 additions and 211 deletions.
Format Strawberryfield:
https://github.com/esmero/format_strawberryfield/releases/tag/v1.4.0-pre
ISSUE-385: Drupal 10.1 compatibility for our overriden Controllers/facet/exposed forms (Symphony 5 v/s GET) by @DiegoPino in #391
ISSUE-182: Makes Metadata Display Entities revision able by @DiegoPino in #399
ISSUE-400: Advanced Search Classic mode by @DiegoPino in #403
ISSUE-404: Facet summary with separate entries per Input/field for advanced search by @DiegoPino in #405
ISSUE-408: Adds multi value check for required, validate count on non empties by @DiegoPino in #409
ISSUE-412: Tested on production. Finally IP embargoes can cut the Cache crap by @DiegoPino in #413
Issue-414-1.4.0--leaflet-options --> 1.4.0 by @patdunlavey in #416
Adds the ability to change the default base URL for the images of IA BookReader by @roromedia in #420
ISSUE-292: IIIF driven paging/sequence + order drive hints + cover page by @DiegoPino in #419
ISSUE-425 --- Fixes brackets to get settings into JS by @roromedia in #426
ISSUE-427: Adds data property (HTML) driven on click or on load Views loading via AJAX by @DiegoPino in #428
ISSUE-434: Fix 206 response on a HEAD with a Range to a 200 by @DiegoPino in #435
ISSUE-430: Various AJAX/Interactions and Views fixes by @DiegoPino in #431
ISSUE-106b:API builder/controller re-roll for 1.4.0 by @DiegoPino in #394
ISSUE-442:Open API sanity + Controller cleanup by @DiegoPino in #444
Issue-447: missing accessCheck in TwigExtension::entityIdsByLabel by @patdunlavey in #448
ISSUE-449: Fix multiple Contextual filters, some "NULL" and allow Exposed filters. But avoid if already set same "key" via a Cont… by @DiegoPino in #450
ISSUE-436: PDFJS 4.x + VTT Content Search API + Mirador 4.0 alpha 2 + ML by @DiegoPino in #437
@roromedia made their first contribution in #420
73 changed files with 7,307 additions and 1,997 deletions.
Webform Strawberryfield:
https://github.com/esmero/webform_strawberryfield/releases/tag/v1.4.0-pre
ISSUE-171: levenshtein sorting Input v/s result labels (not description) by @DiegoPino in #172
ISSUE-171: Sorry, Forgot to remove the 'Desc' from the label when sorting by @DiegoPino in #173
ISSUE-174: adds a fixed "10" rows limit while previewing any Entity Reference Search API (solr) views by @DiegoPino in #175
ISSUE-177: LoD Autocomplete Fixes and Improvements by @DiegoPino in #178
14 changed files with 241 additions and 86 deletions.
Strawberry Runners:
https://github.com/esmero/strawberry_runners/releases/tag/v0.8.0-pre
ISSUE-87: VTT Processor to MiniOCR with time/space transformation by @DiegoPino in #89
ISSUE-87: a bug by @DiegoPino in #90
ISSUE-91:ML processors by @DiegoPino in #92
27 changed files with 3,999 additions and 90 deletions.
Archipelago Documentation:
https://github.com/esmero/archipelago-documentation/compare/1.3.0...1.4.0
Documentation Updates + New Section for Strawberry Runners by @alliomeria in #185
New Advanced Search docs + other updates by @alliomeria in #187
Search Within Collection by @alliomeria in #188
New Webform LoD via CSV DO Element + Custom Webforms Update by @alliomeria in #189
Updates for Logo and other references by @alliomeria in #190
Create extras.css by @DiegoPino in #191
Archipelago Logo Usage Guidelines by @alliomeria in #192
Docs Updates & New Metadata Display Usage Guide by @alliomeria in #194
New Embargo Docs + Updates by @alliomeria in #196
Update I7 Solr Importer Doc step 4 by @alliomeria in #197
Documentation New Guides & Updates for 1.4.0 - Round 1 by @alliomeria in #200
More 1.4.0 Documentation New & Updates by @alliomeria in #203
38 changed files with 889 additions and 105 deletions.
For all above Archipelago modules:
Total files changes: 514
Total lines of code added: 31465
Total lines lines of code removed: 6165
For only the non-deployment Archipelago modules:
Total files changes: 197
Total lines of code added: 14023
Total lines lines of code removed: 2613