Strange transfer problems

[Yuriy Halytskyy]

Hi,

We have recently installed new globus connect server and seeing intermittent transfer errors. Some transfers get stuck on a particular file and never progress past it. The file has the same size on source and destination so it looks like it has been transferred successfully. The only error message is Input/Output error so I don't know how to debug this. The underlying filesystem looks OK, user quotas are not exceeded, there is nothing in dmesg or any other log. 

Cheers,

Yuriy

[526758] Wed Jun 25 22:01:12 2025 :: Configuration read from /etc/gridftp.conf.
[526758] Wed Jun 25 22:01:12 2025 :: Server started in inetd mode.
[526758] Wed Jun 25 22:01:12 2025 :: New connection from: ec2-54-237-254-194.compute-1.amazonaws.com:14263
[526758] Wed Jun 25 22:01:12 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [CLIENT]: USER :globus-sharing:id=d88c10b8-57a1-4137-be0c-02f8f9ce3676;sharee=tomoosting;sharee_ids=8ca59a7f-ac2b-47f5-992b-9dc3c02a1737,5f56defb-8af1-46fa-af28-04fabc014126,31d222c2-030b-41de-9a29-c650732ccc05;high_assurance=0;
[526758] Wed Jun 25 22:01:12 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [SERVER]: 331 Password required for :globus-sharing:id=d88c10b8-57a1-4137-be0c-02f8f9ce3676;sharee=tomoosting;sharee_ids=8ca59a7f-ac2b-47f5-992b-9dc3c02a1737,5f56defb-8af1-46fa-af28-04fabc014126,31d222c2-030b-41de-9a29-c650732ccc05;high_assurance=0;.
[526758] Wed Jun 25 22:01:12 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [CLIENT]: PASS dummy
[526758] Wed Jun 25 22:01:12 2025 :: TLS connection version: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384
[526758] Wed Jun 25 22:01:15 2025 :: Access allowed for sharing of user 'cwil201' from share id 'd88c10b8-57a1-4137-be0c-02f8f9ce3676'.  Sharee 'tomoosting' is restricted to '/globus/nesi/nobackup/ga02846/AGDR00089'.
[526758] Wed Jun 25 22:01:15 2025 :: DN /C=US/O=Globus Consortium/OU=Globus Online/OU=Transfer User/CN=__transfer__ successfully authorized.
[526758] Wed Jun 25 22:01:15 2025 :: User cwil201 successfully authorized.
[526758] Wed Jun 25 22:01:15 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [CLIENT]: PASS dummy
[526758] Wed Jun 25 22:01:15 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [SERVER]: 230 User cwil201 logged in.
[526758] Wed Jun 25 22:01:15 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [CLIENT]: SITE CLIENTINFO appname="globusonline-fxp";appver="0.1"
[526758] Wed Jun 25 22:01:15 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [SERVER]: 250 OK.
[526758] Wed Jun 25 22:01:15 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [CLIENT]: SITE TASKID 5218af88-51c1-11f0-8f56-0affcfc1d1e5
[526758] Wed Jun 25 22:01:15 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [SERVER]: 250 OK.
[526758] Wed Jun 25 22:01:15 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [CLIENT]: SITE RESTRICT rw/
[526758] Wed Jun 25 22:01:15 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [SERVER]: 250 OK.
[526758] Wed Jun 25 22:01:15 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [CLIENT]: FEAT
[526758] Wed Jun 25 22:01:15 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [SERVER]: 211-Extensions supported
[526758] Wed Jun 25 22:01:15 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [SERVER]: 200 16.10 (gcc64, 1747925922-86) [Globus Toolkit 1747926314 GCS-5.4.85]
[526758] Wed Jun 25 22:01:15 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [CLIENT]: SITE VERSION
[526758] Wed Jun 25 22:01:16 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [CLIENT]: MODE E
[526758] Wed Jun 25 22:01:16 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [SERVER]: 200 Mode set to E.
[526758] Wed Jun 25 22:01:16 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [CLIENT]: TYPE I
[526758] Wed Jun 25 22:01:16 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [SERVER]: 200 Type set to I.
[526758] Wed Jun 25 22:01:16 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [CLIENT]: PBSZ 1048576
[526758] Wed Jun 25 22:01:16 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [SERVER]: 200 PBSZ=1048576
[526758] Wed Jun 25 22:01:16 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [CLIENT]: OPTS RETR markers=10;
[526758] Wed Jun 25 22:01:16 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [SERVER]: 200 OPTS Command Successful.
[526758] Wed Jun 25 22:01:16 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [CLIENT]: OPTS CKSM markers=10;
[526758] Wed Jun 25 22:01:16 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [SERVER]: 200 OPTS Command Successful.
[526758] Wed Jun 25 22:01:16 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [CLIENT]: DCAU A
[526758] Wed Jun 25 22:01:16 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [SERVER]: 200 DCAU A.
[526758] Wed Jun 25 22:01:16 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [CLIENT]: DCSC P LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM4ekNDQWR1Z0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREE5TVFzd0NRWURWUVFHRXdKVlV6RVcKTUJRR0ExVUVDZ3dOUjJ4dlluVnpJRTl1YkdsdVpURVdNQlFHQTFVRUF3d05SbGhRSUVSRFFWVWdRMlZ5ZERBZQpGdzB5TlRBMk1UZ3lNakF4TVRGYUZ3MHlOVEEzTURJeU1qQXhNVEZhTUQweEN6QUpCZ05WQkFZVEFsVlRNUll3CkZBWURWUVFLREExSGJHOWlkWE1nVDI1c2FXNWxNUll3RkFZRFZRUUREQTFHV0ZBZ1JFTkJWU0JEWlhKME1JSUIKSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQXVkOHBIT0dxd3JwLy9JVDQyK2ptS2lSSQo5VnFSNGFZaU8xa2h6TXZUWUJLckpCb1RmaUJrSVhLM1JuM3g4L25Pdk1aRHZhYWE3a3JmUU12ZUVCdVZvZFhLCmF2b2VLTy9tSS9XQnV5ZzZIdjBJL0NzbHZsdGRPTGZwdFJWRk5nd0ZCbytMS1NZMTUzQnlNc2RrdDdFWEs0ZisKMmNTV2VaMk9rUy81aERlUTlueUd0a25EOFRwN0J4RUpnZmc1SFFPamJtZEdINmI4eDh6WTlXYnd4U0dORE1ybwoxalI1cVhuN1JYSGJwMmg5TVE0MHk3NjN4YWlqOWhUNnR1SEZtNzlsSlNvc2JjT2RoS1A0ZWVXOGhXNnRhSzdTCnI4bFRHTGlBcFA4VE9URUQwSDNhMnJmaWtZRFBPN3hYWCtkNS9QVlV6WHU3TlBYM0tuL2kzRUFTd1hDOGVRSUQKQVFBQk1BMEdDU3FHU0liM0RRRUJDd1VBQTRJQkFRQTJsSHJtZmVHY3phMFNGbzZGTG5vcGtyYll2TEJXdXUvWAo3eDB2QWxzK3NrYnlMMGhLeUVPV3VTUXRlYjRIdjFjYXloS1lRL1JqbzBpWVRKQXQ5aWwvNys3YXVFL0xVTnlJCi9mRC9Gb3R2ck5LQ2l6elVjbkVEWHo3UGV5TGlYMjNoMm9yZU9RWGFJSkwrRURPbnZpd2JUdWxKNDBUcjdKeWgKVDVvNklWUjE1VmZFdkR6U2EyLzV5Vis5d1VTYmRnTDA1WldpTjJKNUQ2cm1UdlpadXRHVXpVa0tMK0hVRUdpYgpNWWxXMDlyaU1aS3lTb0k0eTZ2YytqaWVqZ1c4cjJDYW5GeXdWWXdWQ0VLQVUvZXBQV05qQUNUZDUxMEdxN2VvCjFnWktlYzRnVzJPcCtiS1dOTUZ4YTEzN3dMMlI0am9XVXBoaGZ4Nm9MNlBrdjNkK0Ixd2UKLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQotLS0tLUJFR0lOIFBSSVZBVEUgS0VZLS0tLS0KTUlJRXZ3SUJBREFOQmdrcWhraUc5dzBCQVFFRkFBU0NCS2t3Z2dTbEFnRUFBb0lCQVFDNTN5a2M0YXJDdW4vOApoUGpiNk9ZcUpFajFXcEhocGlJN1dTSE15OU5nRXFza0doTitJR1FoY3JkR2ZmSHorYzY4eGtPOXBwcnVTdDlBCnk5NFFHNVdoMWNwcStoNG83K1lqOVlHN0tEb2UvUWo4S3lXK1cxMDR0K20xRlVVMkRBVUdqNHNwSmpYbmNISXkKeDJTM3NSY3JoLzdaeEpaNW5ZNlJML21FTjVEMmZJYTJTY1B4T25zSEVRbUIrRGtkQTZOdVowWWZwdnpIek5qMQpadkRGSVkwTXl1aldOSG1wZWZ0RmNkdW5hSDB4RGpUTHZyZkZxS1AyRlBxMjRjV2J2MlVsS2l4dHc1MkVvL2g1CjVieUZicTFvcnRLdnlWTVl1SUNrL3hNNU1RUFFmZHJhdCtLUmdNODd2RmRmNTNuODlWVE5lN3MwOWZjcWYrTGMKUUJMQmNMeDVBZ01CQUFFQ2dnRUFIQnRYSGZjeENZZkd2VktGU0VuWWJrUjA1T2M0ZXFFUytac0JuYlN2eHM0ago0NDV1NXFweXdjMFZVbHRPdUdoTTN5WnpEVzF4YmJuVU56bnMzMDR3aXh1blk1UmxkYWJMVENJK3B5VE8zeXMvCnpkUWZmY0RXb0xKTTdMVmw5U1dOcC8rU1dWdGVHTnliNURwYkxGemk1bElIRTNxakQ0cG9DZVB4alJFVEFuYXgKdFJ1Qm1jTDMycytLY2tRU1AwbzRmc0x4RS90OXhjWDFha082dkxXNTFPYVJOWjdvSC9iOU91WEVYdjljSWVTdAowd1c2eCt2b2JvQWxuZVF5ejFVdjd2NmFHQisrYVY2MldMYTltTnpjWHlkdGVkcGZQY1ZtUnF5T0tpMG1uZXJJCjB3RDUySlZIU0dWUkNDNW9pQmxnZWo5c1NEK3R5MHI5aEw3bEU4RmxoUUtCZ1FEYzBmSWVndUltMXFJUXc2N2cKbmV3a2hlK1RIWjlURDNydmNMWElocWMzVGdNb3FTb2pSNElyand5dmdWbnVMNStnV092d01WbElGK2xKbnlhNApXb3BvOXJhajRxL2V2MGxHcVRNSnNvOEZoMlZpNkVZUTZaTEk1aC9rc2lQclBRcEhMeEVYY1Rsdzg4RVFUZXQwCml5bCthS3RtNG5sT0V4Q2RITmprcUFyV1ZRS0JnUURYZTl3bTEzV2dLTzZab3Eyak1WdXZOczNrTGtqWDBRajYKUWcrL05QZGNKNXZHalRKSHNBeGJCc3BaVFdZMWl4dU0vQUNBSm9BYmVsZHNDdzczdjdmdXhzZmZQZlF5ZUFLegpCTEJXL1NZSFVMeXdRVHg5WVc2TXBsUlcrdzNGNFVxMFlyTjh1enY4Z25aWHBkeEJhYmFXdkFaUGlmU1R4Kyt2CitacTdhalVKbFFLQmdRREVYSlYzZUYvdXRaSXNZamFTV3h4d1pnSGl5c2F4TCtUY0kxZ2pWMG8vTGNvMVVDaSsKY29wWWZiNkVHdnlEaUNua2h1NVlDeW9mcDlPMXVpYk9iUVVEemRBZjhDY0FadzJjSGozYWM4UW00SDNSOUViSwpKMHAyQjkxNG1zakNHT2VZNFV2TGtzM0htWnFuQ3dORUFiUUdTc0dRbmZmTTY3c3NXMGlzSEpwMmVRS0JnUURTCkl6akxMQnpLWm9CZ2xSTnQ3UWx6TWlmUkdRM3lZU2kzSE10TWRQT3hDL05PeW8wNCtjYmkwaGxzd3gwa3BCYm4KQXkzZHl5M3RIZjY1SDRPbU9YRyt5bDFLS0puR3llQWdBamVSYldlemhKbXQ3TUIzcVJkczFCdTFnemlvTzQ5bApvRWRYSVNSYm1FS2VNT0k5YkVFaWVCUlRacjU3aTJRRUlZOHlxMTB6alFLQmdRRFY1L0VYaVZZYWdRZnR4Ti9XCldiTEovSTViZUw1c0V2U3YrWVFPZlVMbFpiSlZBSm94cmVyRVdWN2JqUURWNnJ5SDhRQzFUa2NaVUpGRTNqck4KYWl6djdCeUVvMTVKcWE5MkhoTllwd2hBWWFtb2tnV3FuRkJJOFVZQ2QwWXJEaWZLRkc0MFJDaUZ5Ymx4bml2NwpLRGVUZXF4OHlUTDBFbitmQzdJSUVrUy9BQT09Ci0tLS0tRU5EIFBSSVZBVEUgS0VZLS0tLS0KLS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM4ekNDQWR1Z0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREE5TVFzd0NRWURWUVFHRXdKVlV6RVcKTUJRR0ExVUVDZ3dOUjJ4dlluVnpJRTl1YkdsdVpURVdNQlFHQTFVRUF3d05SbGhRSUVSRFFWVWdRMlZ5ZERBZQpGdzB5TlRBMk1UZ3lNakF4TVRGYUZ3MHlOVEEzTURJeU1qQXhNVEZhTUQweEN6QUpCZ05WQkFZVEFsVlRNUll3CkZBWURWUVFLREExSGJHOWlkWE1nVDI1c2FXNWxNUll3RkFZRFZRUUREQTFHV0ZBZ1JFTkJWU0JEWlhKME1JSUIKSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQXVkOHBIT0dxd3JwLy9JVDQyK2ptS2lSSQo5VnFSNGFZaU8xa2h6TXZUWUJLckpCb1RmaUJrSVhLM1JuM3g4L25Pdk1aRHZhYWE3a3JmUU12ZUVCdVZvZFhLCmF2b2VLTy9tSS9XQnV5ZzZIdjBJL0NzbHZsdGRPTGZwdFJWRk5nd0ZCbytMS1NZMTUzQnlNc2RrdDdFWEs0ZisKMmNTV2VaMk9rUy81aERlUTlueUd0a25EOFRwN0J4RUpnZmc1SFFPamJtZEdINmI4eDh6WTlXYnd4U0dORE1ybwoxalI1cVhuN1JYSGJwMmg5TVE0MHk3NjN4YWlqOWhUNnR1SEZtNzlsSlNvc2JjT2RoS1A0ZWVXOGhXNnRhSzdTCnI4bFRHTGlBcFA4VE9URUQwSDNhMnJmaWtZRFBPN3hYWCtkNS9QVlV6WHU3TlBYM0tuL2kzRUFTd1hDOGVRSUQKQVFBQk1BMEdDU3FHU0liM0RRRUJDd1VBQTRJQkFRQTJsSHJtZmVHY3phMFNGbzZGTG5vcGtyYll2TEJXdXUvWAo3eDB2QWxzK3NrYnlMMGhLeUVPV3VTUXRlYjRIdjFjYXloS1lRL1JqbzBpWVRKQXQ5aWwvNys3YXVFL0xVTnlJCi9mRC9Gb3R2ck5LQ2l6elVjbkVEWHo3UGV5TGlYMjNoMm9yZU9RWGFJSkwrRURPbnZpd2JUdWxKNDBUcjdKeWgKVDVvNklWUjE1VmZFdkR6U2EyLzV5Vis5d1VTYmRnTDA1WldpTjJKNUQ2cm1UdlpadXRHVXpVa0tMK0hVRUdpYgpNWWxXMDlyaU1aS3lTb0k0eTZ2YytqaWVqZ1c4cjJDYW5GeXdWWXdWQ0VLQVUvZXBQV05qQUNUZDUxMEdxN2VvCjFnWktlYzRnVzJPcCtiS1dOTUZ4YTEzN3dMMlI0am9XVXBoaGZ4Nm9MNlBrdjNkK0Ixd2UKLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
[526758] Wed Jun 25 22:01:16 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [SERVER]: 250 OK.
[526758] Wed Jun 25 22:01:16 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [CLIENT]: PASV
[526758] Wed Jun 25 22:01:16 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [SERVER]: 227 Entering Passive Mode (163,7,145,185,196,99)
[526758] Wed Jun 25 22:01:17 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [CLIENT]: ALLO 1310720
[526758] Wed Jun 25 22:01:17 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [SERVER]: 200 ALLO command successful.
[526758] Wed Jun 25 22:01:17 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [CLIENT]: REST 0-268434145280
[526758] Wed Jun 25 22:01:17 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [SERVER]: 350 Restart Marker OK. Send STORE or RETR to initiate transfer.
[526758] Wed Jun 25 22:01:17 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [CLIENT]: STOR /AGDR00089_hapuka/hapuka_pod5.tar.gz.part01
[526758] Wed Jun 25 22:01:17 2025 :: Starting to transfer "/globus/nesi/nobackup/ga02846/AGDR00089/AGDR00089_hapuka/hapuka_pod5.tar.gz.part01".
[526758] Wed Jun 25 22:01:17 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [SERVER]: 150 Beginning transfer.
[526758] Wed Jun 25 22:01:17 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [SERVER]: 112-Perf Marker
[526758] Wed Jun 25 22:01:17 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [SERVER]: 112-Perf Marker
[526758] Wed Jun 25 22:01:17 2025 :: Failure attempting to transfer "/globus/nesi/nobackup/ga02846/AGDR00089/AGDR00089_hapuka/hapuka_pod5.tar.gz.part01".
[526758] Wed Jun 25 22:01:17 2025 :: Transfer failure:
[526758] Wed Jun 25 22:01:17 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [SERVER]: 111 Range Marker 268434145280-268435456000
[526758] Wed Jun 25 22:01:17 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [SERVER]: 500-Command failed. : globus_xio: System error in close: Input/output error
[526758] Wed Jun 25 22:01:17 2025 :: ec2-54-237-254-194.compute-1.amazonaws.com:14263: [CLIENT]: CKSM SHA1 0 -1 /AGDR00089_hapuka/hapuka_pod5.tar.gz.part01

[Brigitte Raumann]

Hi Yuriy,

If you are still having problems with your specific transfer, you can contact sup...@globus.org with the transfer task ID and they will troubleshoot with you.

Brigitte

To unsubscribe from this group and stop receiving emails from it, send an email to discuss+u...@globus.org.