iget from S3 resource of federated zone fails

[j.labrenz]

Hello,

I have a problem with cacheless_attached S3 resources (minio and ECS) and access from a different zone (ZoneB).
I already thought it worked properly since ils and iget was working fine.
Then I recognized iget on files in the S3 resource from ZoneB works only for files up to 32MB. Inside the ZoneA everything is okay. 

Here is the error message from ZoneB (in the shell).

~$ iget -Pf /ZoneA/home/rods/test33M.tst irods-data-playground/
0/1 -  0.00% of files done   0.000/33.000 MB -  0.00% of file sizes done
Processing test33M.tst - 33.000 MB   2020-02-24.11:49:23
remote addresses: 127.0.0.1 ERROR: connectToRhostPortal: connectTo Rhost test-irods port 20080 error, status = -347000
remote addresses: 127.0.0.1 ERROR: getUtil: get error for irods-data-playground//test33M.tst status = -347000 USER_SOCK_CONNECT_TIMEDOUT

~$ iget -Pf /ZoneA/home/rods/test32M.tst irods-data-playground/
0/1 -  0.00% of files done   0.000/32.000 MB -  0.00% of file sizes done
Processing test32M.tst - 32.000 MB   2020-02-24.11:54:03

For 32 MB files its working.

The rodsLog of ZoneA:

remote addresses: 127.0.0.1, 172.21.96.221, 172.21.98.222 ERROR: acceptSrvPortal() -- select timed out
NOTICE: svrPortalPut: acceptSrvPortal error. errno = 107
remote addresses: 127.0.0.1, 172.21.96.221, 172.21.98.222 ERROR: [-]  /irods/server/core/src/rsApiHandler.cpp:540:int readAndProcClientMsg(rsComm_t *, int) :  status [SYS_HEADER_READ_LEN_ERR]  errno [] -- message [failed to call 'read header']
        [-]     /irods/lib/core/src/sockComm.cpp:201:irods::error readMsgHeader(irods::network_object_ptr, msgHeader_t *, struct timeval *) :  status [SYS_HEADER_READ_LEN_ERR]  errno [] -- message [failed to call 'read header']
                [-]     /irods/plugins/network/tcp/libtcp.cpp:194:irods::error tcp_read_msg_header(irods::plugin_context &, void *, struct timeval *) :  status [SYS_HEADER_READ_LEN_ERR]  errno [] -- message [only read [0] of [4]]

I just figured out that increasing the value of "maximum_size_for_single_buffer_in_megabytes": in ZoneA makes the iget work. Seems like parallel transfer fails.

Has anyone else seen a similar behaviour? Or can this be a problem of the settings of the federation or the S3 resource? 

Best regards,
Johannes

[Terrell Russell]

Hi Johannes,

Agreed on your assessment about parallel transfer being the culprit.

Does parallel transfer work correctly when in ZoneB on a data object in ZoneB?

It could be as simple as a firewall issue between the zones - are the high ports open on the network between the two zones?

Terrell

--
--
"iRODS: the Integrated Rule-Oriented Data-management System; A community driven, open source, data grid software solution" https://www.irods.org
 
iROD-Chat: http://groups.google.com/group/iROD-Chat

---
You received this message because you are subscribed to the Google Groups "iRODS-Chat" group.
To unsubscribe from this group and stop receiving emails from it, send an email to irod-chat+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/irod-chat/3046350a-73a7-4e46-9f4b-913bc2ad1cad%40googlegroups.com.

[j.labrenz]

Hi Terrell,

thanks for your help. After allowing the high ports it is working fine. 

$ iget -PfV -R irodsRescRepl test33M.tst irods-data-playground/

0/1 -  0.00% of files done   0.000/33.000 MB -  0.00% of file sizes done

Processing test33M.tst - 33.000 MB   2020-02-24.14:13:25
From server: NumThreads=9, addr:combined, port:20051, cookie=1010579746
test33M.tst - 25.667/33.000 MB - 77.78% done   2020-02-24.14:13:26
test33M.tst - 33.000/33.000 MB - 100.00% done   2020-02-24.14:13:26
   test33M.tst                    33.000 MB | 1.256 sec | 9 thr | 26.268 MB/s

$ iget -PfV /ZoneA/home/rods/test33M.tst irods-data-playground/

0/1 -  0.00% of files done   0.000/33.000 MB -  0.00% of file sizes done

Processing test33M.tst - 33.000 MB   2020-02-24.14:23:26
From server: NumThreads=9, addr:test-irods, port:20048, cookie=891865320
test33M.tst - 33.000/33.000 MB - 100.00% done   2020-02-24.14:23:28
   test33M.tst                    33.000 MB | 2.616 sec | 9 thr | 12.614 MB/s

Best regards, 

Johannes

To unsubscribe from this group and stop receiving emails from it, send an email to irod...@googlegroups.com.