Dearest everyone,
Happy to write you that just 5 hours before Midnight and after countless hours testing, moving pixels left and right and more testing and more YAML file editing and Twig template and JSON and code, we can share with you all (and your closest friends) Archipelago 1.0.0-RC1!!!!!
This has been one exceptionally complex year, good people left us, many other barely holding up and many others tired, world is not a better place and we all lost hope a few times, still we all together, our caring team and with the eternal support (even when quite shy and silent sometimes..mmm) of our community we managed to code and release Archipelago 1.0.0-RC. This worth some celebrating. 2 years ago we started with some code and dreams. Many said we were not right, many others did not believe and to be honest the stakes and ambitions were high. But one day at the time, constantly we move to this current state.
What is new?
I could write you a full Wall of text here but since I really want to do normal things humans do (like say Adiós to 2020) I will make it quick:
1.- Full support for W3C Annotations thanks to wonderful Annotorious from Rainer Simon and our CRUD full Integration directly on Digital Object Image view.
2.- Full support (like magic) for HOCR using direct Solr Highlights thanks to the amazing work of Johannes Baiter at the
Bavarian State Library and the hard work of Giancarlo taming the IAB Book reader beast. Moreover, HOCR gets processed and pushed into Solr per page on the background via...
3.- Strawberry Runners! a hierarchical, do anything you want, post processor for Digital objects. From HOCR to Metadata Enrichment, to File transfiguration to checks, you name it. All possible via the UI. We ship with HOCR for PDFs and WARC to the Amazing WACZ web archiving format from Ilya Kreymer and Emma Dickson. Includes of course the latest WebReplay plugin for faboulous Webarchive Life viewing and searching on Archipelago. We even added range request responses from S3 coded into our Binary deliveries so Gbyte Archives stream without any delay (really!)
4.- So many new Metadata-Linked data Webform Elements. From fancy CSV to JSON importers to all the range (we left nothing behind) of Library Of congress, Getty and Viaf.
5.- Strawberry Hydroponics: A background Service running via reactPHP that processes all your heavy queues (HOCR, etc). Runs for a little, gets done-tired, goes to sleep. Never runs twice. Never gives up.
6.- New Webforms coming from the marvelous hands of Allison Lund (our team's metadata expert), covering all our new elements but also adding some Controlled Rights statements magic and better wording/help. Also Collections (And newspapers, Series) gets a new Webform and new Types.
7.- Total new Stack. Best of the best and faster: Mysql 8, Solr 8.7, Cantaloupe 4.1.7, latest Minio, Natural language Processing Services (who needs dancing robots..), PHP 7.4 and new tools.Composer 2 by default (no more running out of memory)
8.- Of course, full Drupal 9.1 support and we even have a demo deployment for that! (and tested for months, made in Italy)
9.- New curated modules and others had to go. Yes. We also patched Drupal and contributed to the community quite a lot
10. AMI: Mass importer for the Masses. Spreadsheet and GoogleSheets are enabled by default. Generates Ingest sets that can be run via Drush or Hydroponics. Can be deleted, reinvested, etc. Super Huge config forms with every possible option. You can even upload your HUGE files first to S3 if you want and we take them from there or have them remote in one of your other servers (who said migrations??). This also includes (sorry for the lack of super UI, but its a powertool) Find and Replace metadata! And yes, you can handle all your JSON as simple text or go Fancy and do JSON Patching and go precise about all your metadata remediation (who said migrations without changing your source data , twice?). We also made Ingesting 29.5X faster. yes (I could not believe the numbers had to ask my fastest dog - the black one - whippet mix - for advice). Less than 2 seconds for a large book with many pages to identify and metadata to enrich. that!
Ok. There is more but I'm tired as hell. My Github Activity graph has more "lights" than my actual Xmas tree.
Check it out:
https://github.com/esmero/archipelago-deployment (RC1 is the default branch) and give it a try. Is it perfect? No. Its it beautiful? Maybe.. Is it the best repository ever? May be... but its the one I would trust my assets and collections and the one that gives users, archivists, collection owners and humans like you, complete control and also allows you to grow with it.
lastly: Dear Giancarlo my buddy, Allison the best co worker I could ask for and many times the one that kept me going when sleeping was not possible, Nate for believeing, your daily help, encouragement and new ideas, thanks so much!!, whole Metro team that had to hear our Monday updates, Jen + Zack I can never stop thanking you both, I have learned so much of EAD2 and 3, Ilya for your amazing JS/Python coding skills and patience on our calls, Sumitra for all your web archiving knowledge and help, Natalie and OHA bringing Archipelago to other realms, Derek and Pat for the too many calls and Github interactions, ESLN folks, ESIE! we did so much, Tim for believing in this, Carl and your perfect emails and wonderful use cases and old friends Mitchell and Marlo, and my wife too. Thank you all for your support, help, testing, asking, breaking, finding, suggesting, coding, collaborating and for not sending me to hell everyday I was late trying to fix another bug or working extra hours from one day into the other. If I forgot someone it's because it's almost 2021.
Good things are to come and we will keep improving and adding more. In a few weeks Pedro Cambra starts working with us. He is a great Drupal Developer and member of the Drupal Association Board too. We will have more knowledge and more hands to polish this. RC1 is already quite spectacular and I'm proud of you (and myself too)
Feliz año nuevo and sorry for the typos and my bad 2020 dad jokes.
Love.
Diego Pino
Metro.org