Anyone using GPFS storage or IBM spectrum scale for their Dataverse installation?

343 views
Skip to first unread message

Dieuwertje Bloemen

unread,
Oct 27, 2022, 6:11:55 AM10/27/22
to Dataverse Users Community
Hi all,

At KU Leuven, recently new storage was delivered that we will be using for our dataverse installation. However, the storage is GPFS/IBM spectrum scale. Because we don't have any experience with this, we were wondering if there were already other installations using this as their storage and what their experiences are/if they have any tips. Any input would be helpful.

Thanks!
Dieuwertje Bloemen
(product manager KU Leuven RDR)

Vyacheslav Tikhonov

unread,
Oct 27, 2022, 6:17:01 AM10/27/22
to Dataverse Users Community
HI Dieuwertje

GPFS/IBM spectrum should support S3 interface, you can connect it just like any other Cloud Storage

Best,
Slava
DANS-KNAW

Valentina Pasquale

unread,
Apr 12, 2024, 5:12:49 AM4/12/24
to Dataverse Users Community
Hi Dieuwertje, hi all,

Recently, we have also bought the same type of storage and we are now starting to understand how to use it for Dataverse. I kindly ask if you could share some info about how you are using it in Dataverse.
Thanks for any input you may share!

Best regards,

Valentina Pasquale

Dieuwertje Bloemen

unread,
Apr 17, 2024, 8:18:38 AM4/17/24
to Dataverse Users Community
Hi Vaida,

For us the trickiest part was the S3 emulation on the storage (I believe we use openstack), as it doesn't allow for tagging, which is necessary for a lot of Dataverse functionality in S3 (we had issues with previewers, direct uploads and external tools like DVUploader). And S3 is kinda the Dataverse standard, especially for larger data. So, it might be worth it to look at other S3 options with the need for tagging in mind. We also briefly looked at the MinIO S3 gateway, but that is no longer supported, I believe.

Kind regards,
Dieuwertje

James Myers

unread,
Apr 17, 2024, 8:41:22 AM4/17/24
to dataverse...@googlegroups.com

FWIW: KU Leuven’s PR to allow turning tagging off is currently in QA and would be expected to be part of Dataverse v6.3. So as of that point, this restriction goes away.

 

-- Jim

--
You received this message because you are subscribed to the Google Groups "Dataverse Users Community" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dataverse-commu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dataverse-community/1a841dab-6dc6-4f2a-a892-822f3bf5faa0n%40googlegroups.com.

Reply all
Reply to author
Forward
0 new messages