Recommendations for best IIIF paid hosting or cloud/AWS-friendly IIIF servers

[Ruth Tillman]

I'm starting the part of our rather patchy IIIF adoption where I sit down with the people in charge of the servers and start to talk about actually putting spinning one up in our long-term game plan. Our current institutional mandate requires us to look at either paid services or things that we can host on AWS or otherwise in the cloud.

Does anyone who's used a paid service feel comfortable sharing their experiences on here or privately to rtil...@nd.edu? 

Would people who have been hosting their IIIF servers on some kind of cloud, preferably on AWS, similarly feel comfortable sharing?

I have information but what I don't have is information about how they function in the real world, whether a particular paid service is a major pain or if it's extremely hard to get a particular service into the cloud.

Thanks,

Ruth Kitchin Tillman

Digital Collections Librarian

Hesburgh Libraries

University of Notre Dame

rtil...@nd.edu

[Stern, Randy]

Hi Ruth,

 

We currently host our IIIF presentation service in AWS. This works fine. This server can obtain its metadata from our repository through web services.

 

We have been looking, and are still looking, into hosting our IIIF image service in the cloud as well. One barrier is that the file system for our image storage cannot be mounted to AWS machines. We will eitherneed to host deliverable copies, or a cache of deliverable copies, in S3, or deliver the copies through a web service to the EC2 image server – but that will likely be too slow. So I think some sort of caching solution in S3 is in our future.

 

Randy

--
-- You received this message because you are subscribed to the IIIF-Discuss Google group. To post to this group, send email to iiif-d...@googlegroups.com. To unsubscribe from this group, send email to iiif-discuss...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/iiif-discuss?hl=en
---
You received this message because you are subscribed to the Google Groups "IIIF Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to iiif-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tom Crane]

Hi Ruth, Randy,

The platform we (Digirati) developed for the Wellcome Library runs on AWS, serving ~30m images where the JP2 in is S3 storage. Images are moved on demand from S3 to a faster disk when required for tile extraction, taking advantage of the extreme long-tail distribution nature of requests for images (the most-used images are on fast disks).

The biggest problem with that approach is when you suddenly get requests for 500 thumbnails for the pages of a book far down in the long tail, but we address that by handling different types of image request in different ways – not all size responses are served by the same image server. 

This platform is called DLCS (Digital Library Cloud Services). We will be packing this "in a box" so that anyone can roll it out under their own AWS estate. We will also offer this as a hosted service early next year.

A number of pilot users have been trying it out as we’ve been developing it for Wellcome – it has an API (which eventually will be IIIF CRUD), you register images with it, and it provides an IIIF Image API endpoint for the images you register. You can make queries against it based on metadata provided at registration time and it will project the results into IIIF resources (e.g., return a manifest). However it is not a read/write Presentation API server, unless your needs are very simple you would keep your Presentation API implementation separate, as Wellcome do.

Tom

[Petr Pridal]

Hey Ruth,

we (KlokanTech) can offer two alternatives:

1) IIIF Image API hosting as a service: http://www.iiifhosting.com/

Just drag&drop a JPEG or TIFF into http://www.iiifhosting.com/ and try it with your images. First 100 MB or 5 images are free forever with the new backend we are just beta-testing now. The service offers HTTPS and caching CDN with 42 servers all around the globe. The core data storage service is located in the European Union - but we are considering to launch a secondary center in the US or Canada - if there are more customers over the ocean. We host quite a lot of maps for our Georeferencer pilots (http://www.georeferencer.com/) with this service - for customers like The British Library, American Geographical Society, etc.

The service has very affordable and direct pricing structure and offers IIIF Image API for integration with your projects. Your can upload us your images from your web browser (even hundreds of gigabytes) or via FTP. Our team works actively on new exciting features which are going to be announced during the next year. Integration with external systems (via webhooks and APIs) is possible. Would be great to get more feedback from the community.

People interested in beta testing and getting more info before the official announcements can contact in...@klokantech.com. ;-)

2) Deployed open-source system on Amazon or Google cloud + optional maintenance of such system on your cloud account

We can deploy for you an instance of the open-source code developed for the Embedr research project and applied (with modifications and improvements) also on the Europeana IIIF service which is now in production. This allows high-performance deploy of IIIF Image API with auto-scaling and JPEG2000 deliverable copy images stored on S3, GCS or similar storage. Such work would be charged per hour for consulting, deployment and customisation for your use-case. The source code of the Embedr research project is at https://github.com/klokantech/embedr, but it is slightly outdated and parts of it revived recently in the Europeana IIIF. This approach would be more expensive then the IIIF hosting service - and make sense if you have special requirements or several terabytes of imagery where you want to control the deployment.

BTW If you want to just launch IIIFServer on your AWS with locally hosted images, you can do that trivially with one command and Docker. See http://www.iiifserver.com/download/.

Best regards,

Petr Pridal, Ph.D.

CEO, Klokan Technologies GmbH, Switzerland

--
-- You received this message because you are subscribed to the IIIF-Discuss Google group. To post to this group, send email to iiif-d...@googlegroups.com. To unsubscribe from this group, send email to iiif-discuss+unsubscribe@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/iiif-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "IIIF Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to iiif-discuss+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Petr Pridal, Ph.D.
CEO

Klokan Technologies GmbH
Hofnerstrasse 98, 6314 Unterageri, Switzerland
Tel: +41 (0)41 511 26 12
Email: in...@klokantech.com
Web: http://www.klokantech.com/

[Colin Greenstreet]

Dear Ruth, Randy, Tom and Petr,

What is currently the state of play (as of April and further product offer improvements in 2018 and early 2019) with the provision of cloud based IIIF services? Specifically we are interested in manuscript image serving integrated with annotations? We are looking for a standard, not a bespoke service, with minimum if any tailoring.

The use case we are exploring involves the Signs of Literacy community we are developing for the exploration of Historical Literacy (1550-1750). We intend in 2019 to work with a number of small (as well as large) archives in the United Kingdom and in the Netherlands, imaging manuscript pages in collaboration with volunteers linked to small archives, and putting the images up on a IIIF cloud server capable of handling crowdsourced annotations, not simply serving images. We could of course use just one central cloud server for multiple archives, but there is an argument for having individual small archives able to experiment with and control their own IIIF cloud server services to help them experiment with IIIF as a concept, over and beyond working with us as content suppliers to the Signs of Literacy community.

We note Tom and Petr's previous responses as of late 2016 to Ruth's original posting.

We note also subsequent developments and use cases such as:

* The ongoing development of the IIIF LibCrowds Viewer [http://blogs.bl.uk/digital-scholarship/2017/11/crowdsourcing-using-iiif-and-web-annotations.html], with the transcriptions collected using the W3C Web Annotations standard [https://www.w3.org/annotation/] and search of the trancriptions made possibel therough the IIIF Content Search API [http://iiif.io/api/search/1.0/]

* The Harvard IIIF Annotations Editor http://iiif.harvard.edu/iiif-annotations-editor/]

* https://github.com/IIIF

* https://github.com/IIIF/awesome-iiif

* https://resources.digirati.com/iiif/an-introduction-to-iiif/

* https://github.com/klokantech/iiifviewer

Best regards

Colin Greenstreet

Co-director, MarineLives; Community organiser, Signs of Literacy; Founder, Chronoscopic Education

E: colin.gr...@gmail.com

M: +44-(0)7769-340229

W: chronoscopic.org

GitHub: https://github.com/Signsofliteracy/Signoff/wiki

We are registering a charitable incorporated organisation under the name of Chronoscopic Education, to be the legal, funding and technology home for MarineLives, our sister project Maphackathon, and for the Signs of Literacy community. The aims of Chronoscopic Education are threefold: (1) To further the teaching of palaeographic, digital research and project management skills, at universities and schools. (2) To apply insights from small teams in management consultancy and R&D project management, together with tools from data science, to the subject matter of history. (3) To build a virtual manuscript-based archive and associated research community, which will foster a culture of collaborative scholarship.

On Tuesday, November 15, 2016 at 2:54:26 PM UTC, Petr Pridal wrote:

Hey Ruth,

we (KlokanTech) can offer two alternatives:

1) IIIF Image API hosting as a service: http://www.iiifhosting.com/

Just drag&drop a JPEG or TIFF into http://www.iiifhosting.com/ and try it with your images. First 100 MB or 5 images are free forever with the new backend we are just beta-testing now. The service offers HTTPS and caching CDN with 42 servers all around the globe. The core data storage service is located in the European Union - but we are considering to launch a secondary center in the US or Canada - if there are more customers over the ocean. We host quite a lot of maps for our Georeferencer pilots (http://www.georeferencer.com/) with this service - for customers like The British Library, American Geographical Society, etc.

The service has very affordable and direct pricing structure and offers IIIF Image API for integration with your projects. Your can upload us your images from your web browser (even hundreds of gigabytes) or via FTP. Our team works actively on new exciting features which are going to be announced during the next year. Integration with external systems (via webhooks and APIs) is possible. Would be great to get more feedback from the community.

People interested in beta testing and getting more info before the official announcements can contact in...@klokantech.com. ;-)

2) Deployed open-source system on Amazon or Google cloud + optional maintenance of such system on your cloud account

We can deploy for you an instance of the open-source code developed for the Embedr research project and applied (with modifications and improvements) also on the Europeana IIIF service which is now in production. This allows high-performance deploy of IIIF Image API with auto-scaling and JPEG2000 deliverable copy images stored on S3, GCS or similar storage. Such work would be charged per hour for consulting, deployment and customisation for your use-case. The source code of the Embedr research project is at https://github.com/klokantech/embedr, but it is slightly outdated and parts of it revived recently in the Europeana IIIF. This approach would be more expensive then the IIIF hosting service - and make sense if you have special requirements or several terabytes of imagery where you want to control the deployment.

BTW If you want to just launch IIIFServer on your AWS with locally hosted images, you can do that trivially with one command and Docker. See http://www.iiifserver.com/download/.

Best regards,

Petr Pridal, Ph.D.

CEO, Klokan Technologies GmbH, Switzerland

On Tue, Nov 15, 2016 at 11:07 AM, Tom Crane <tom....@digirati.co.uk> wrote:

Hi Ruth, Randy,

The platform we (Digirati) developed for the Wellcome Library runs on AWS, serving ~30m images where the JP2 in is S3 storage. Images are moved on demand from S3 to a faster disk when required for tile extraction, taking advantage of the extreme long-tail distribution nature of requests for images (the most-used images are on fast disks).

The biggest problem with that approach is when you suddenly get requests for 500 thumbnails for the pages of a book far down in the long tail, but we address that by handling different types of image request in different ways – not all size responses are served by the same image server. 

This platform is called DLCS (Digital Library Cloud Services). We will be packing this "in a box" so that anyone can roll it out under their own AWS estate. We will also offer this as a hosted service early next year.

A number of pilot users have been trying it out as we’ve been developing it for Wellcome – it has an API (which eventually will be IIIF CRUD), you register images with it, and it provides an IIIF Image API endpoint for the images you register. You can make queries against it based on metadata provided at registration time and it will project the results into IIIF resources (e.g., return a manifest). However it is not a read/write Presentation API server, unless your needs are very simple you would keep your Presentation API implementation separate, as Wellcome do.

Tom

--
-- You received this message because you are subscribed to the IIIF-Discuss Google group. To post to this group, send email to iiif-d...@googlegroups.com. To unsubscribe from this group, send email to iiif-discuss...@googlegroups.com. For more options, visit this group at https://groups.google.com/d/forum/iiif-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "IIIF Discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to iiif-discuss...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Tom Crane]

Dear Colin,

Digirati’s DLCS platform provides IIIF Image API endpoints, management of IIIF Presentation resources, IIIF Search for both human- and machine- contributed plain text and annotations, and IIIF Authentication for hooking into existing access control. All this links to our annotation server for W3C and OA compatible management of annotations. These are variously quite different concerns and handled by different components, which can be used in isolation, but when joined up together as a managed pipeline (which you can hook into via an event/messaging system) they form a flexible text processing application platform; you can see aspects of it in these demos: https://dlcs.info/applications.html

>> ...putting the images up on a IIIF cloud server capable of handling crowdsourced annotations, not simply serving images.

We can provide the components of our DLCS platform as a managed hosted service suitable for use by small archives, in fact it is used like this today, for example https://digirati.com/work/cultural-heritage/royal-college-of-veterinary-surgeons/ - this I think is close to what you describe, allowing individual archives control of their IIIF resources in a shared platform. We can provide the consultancy and integration glue to connect such small archives to our hosted IIIF services, if required.

Other links to add to your list should include some new presentation components for annotations, as well as the crowdsourcing platform we developed for the National Library of Wales, which is proving very popular with their volunteers.

An example project on the NLW crowdsourcing platform:

https://crowd.library.wales/en/s/war-tribunal-records/page/welcome

A project that uses the DLCS for image hosting and annotations:

https://makingscience.royalsociety.org/

Other components:

https://canvas-panel.netlify.com/

http://annotation-studio.netlify.com/examples/

https://digirati.com/updates/insights/a-crowdsourcing-platform-for-wales/

Tom

Tom Crane

Technical Director

Digirati

tom....@digirati.com