s3 resource plugin help

[joris luijsterburg]

All,

I need a bit of help in attaching a new s3 provider. We are running local s3 objectstores within our facility, and want to attach those to iRODS. We already have one running, and are now attaching a second one. The url is a tad different and I am wondering if that is causing some issues here. Relevant configuration of the working s3 resource is as follows:

location: local.irodsmachine.com
vault: /bucket1
context: S3_DEFAULT_HOSTNAME=s3.local.objectstore.com;

Now I have a new objectstore, where the url is new.local.store.com/s3. Using python and boto3 I was able to send files and access the object store. However, with iRODS I am not succeeding yet. I tried two different setups for the resource, both did not work, with different errors when trying an iput. Do you have an advice on how to continue? Can the s3 resource plugin handle the `s3` prefix before the bucketname, or am I doing something else wrong?


location: local.irodsmachine.com
vault: /bucket2
context: S3_DEFAULT_HOSTNAME=new.local.store.com/s3;...;...;

Error:
log_message: /irods_plugin_source/s3_transport/src/s3_transport.cpp:247 [store_and_log_status] [[138014195967680]]  libs3_types::status: [XmlParseFailure] - 31
log_message: /irods_plugin_source/s3_transport/src/s3_transport.cpp:250 [store_and_log_status] [[138014195967680]]  S3Host: new.local.store.com/s3
log_message: /irods_plugin_source/s3_transport/src/s3_transport.cpp:254 [store_and_log_status] [[138014195967680]]  Function: s3_upload::callback_for_write_to_s3_base::on_response_completion
log_message: [-]        /irods_source/server/api/src/rsFileClose.cpp:120:int _rsFileClose(rsComm_t *, fileCloseInp_t *) :  status [S3_PUT_ERROR]  errno [] -- message [fileClose failed for [3]]
        [-]     /irods_source/server/drivers/src/fileDriver.cpp:167:irods::error fileClose(rsComm_t *, irods::first_class_object_ptr) :  status [S3_PUT_ERROR]  errno [] -- message [failed to call 'close']
                [-]     /irods_plugin_source/s3_transport/include/irods/private/s3_transport/s3_transport.hpp:2306:irods::experimental::io::s3_transport::error_codes irods::experimental::io::s3_transport::s3_transport<char>::s3_upload_file(bool) [CharT = char] :  status [S3_PUT_ERROR]  errno [] -- message [failed in S3_put_object]


log_message: Failed to close file object [error_code=-702000].
log_message: [close_replica_without_catalog_update:288] - error closing replica; ec:[-702000]
log_message: [single_buffer_put:337] - error closing replica; ec:[-702000]


location: local.irodsmachine.com
vault: /s3/bucket2
context: S3_DEFAULT_HOSTNAME=new.local.store.com;...;...;

Error:

log_message: /irods_plugin_source/s3_transport/src/s3_transport.cpp:247 [store_and_log_status] [[138014195967680]]  libs3_types::status: [InvalidBucketNameTooShort] - 8
log_message: /irods_plugin_source/s3_transport/src/s3_transport.cpp:250 [store_and_log_status] [[138014195967680]]  S3Host: new.local.store.com
log_message: /irods_plugin_source/s3_transport/src/s3_transport.cpp:254 [store_and_log_status] [[138014195967680]]  Function: s3_upload::callback_for_write_to_s3_base::on_response_completion
log_message: [-]        /irods_source/server/api/src/rsFileClose.cpp:120:int _rsFileClose(rsComm_t *, fileCloseInp_t *) :  status [S3_PUT_ERROR]  errno [] -- message [fileClose failed for [3]]
        [-]     /irods_source/server/drivers/src/fileDriver.cpp:167:irods::error fileClose(rsComm_t *, irods::first_class_object_ptr) :  status [S3_PUT_ERROR]  errno [] -- message [failed to call 'close']
                [-]     /irods_plugin_source/s3_transport/include/irods/private/s3_transport/s3_transport.hpp:2306:irods::experimental::io::s3_transport::error_codes irods::experimental::io::s3_transport::s3_transport<char>::s3_upload_file(bool) [CharT = char] :  status [S3_PUT_ERROR]  errno [] -- message [failed in S3_put_object]


log_message: Failed to close file object [error_code=-702000].
log_message: [close_replica_without_catalog_update:288] - error closing replica; ec:[-702000]
log_message: [single_buffer_put:337] - error closing replica; ec:[-702000]


Kind regards,

Joris

[James, Justin Kyle]

Some questions/comments:

1. Can you send the entire context string?  Maybe I will see something in it.
2. Is this S3 provider from a different vendor than the original?
3. What was the size of the file you were uploading?
4. Can you increase the logging level for the S3 resource and provide a full log of the put?

---

From: irod...@googlegroups.com <irod...@googlegroups.com> on behalf of joris luijsterburg <jlu...@gmail.com>
Sent: Wednesday, June 24, 2026 3:16 AM
To: iRODS-Chat <irod...@googlegroups.com>
Subject: [iROD-Chat:22562] s3 resource plugin help

 

--
--
The Integrated Rule-Oriented Data System (iRODS) - https://irods.org
 
iROD-Chat: http://groups.google.com/group/iROD-Chat
---
You received this message because you are subscribed to the Google Groups "iRODS-Chat" group.
To unsubscribe from this group and stop receiving emails from it, send an email to irod-chat+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/irod-chat/e34204f3-4177-45ce-908b-9e3e0b0b13aen%40googlegroups.com.