Error message when uploading files by using the new version of python-irodsclient (1.1.9)

[MBANZA Sara]

Hi,

When I use PRC version 1.1.5 to upload the data, this works good as expected.

I want to use the new version of PRC (1.1.9). But when try to use it and upload the file, I receive the error below, even if I try to upload the small file (example the file of 22 bytes). Do you maybe know how to solve this problem?

CRITICAL:irods.connection:The read operation timed out
ERROR:irods.connection:Could not receive server response
Traceback (most recent call last):
  File "/home/mary/myenv/lib64/python3.6/site-packages/irods/message/__init__.py", line 185, in _recv_message_in_len
    buf = sock.recv(size_left, socket.MSG_WAITALL)
  File "/usr/lib64/python3.6/ssl.py", line 955, in recv
    self.__class__)
ValueError: non-zero flags not allowed in calls to recv() on <class 'ssl.SSLSocket'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/mary/myenv/lib64/python3.6/site-packages/irods/connection.py", line 113, in recv
    msg = iRODSMessage.recv(self.socket)
  File "/home/mary/myenv/lib64/python3.6/site-packages/irods/message/__init__.py", line 275, in recv
    rsp_header_size = _recv_message_in_len(sock, 4)
  File "/home/mary/myenv/lib64/python3.6/site-packages/irods/message/__init__.py", line 187, in _recv_message_in_len
    buf = sock.recv(size_left)
  File "/usr/lib64/python3.6/ssl.py", line 956, in recv
    return self.read(buflen)
  File "/usr/lib64/python3.6/ssl.py", line 833, in read
    return self._sslobj.read(len, buffer)
  File "/usr/lib64/python3.6/ssl.py", line 592, in read
    v = self._sslobj.read(len)
socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "Downloads/Centos_App/gui_main.py", line 610, in slot_upload_file
    self.upload_one_file_to_irods (fname, newobjectname)
  File "Downloads/Centos_App/gui_main.py", line 1634, in upload_one_file_to_irods
    MY_SESSION.data_objects.put (filename, objectname)
  File "/home/mary/myenv/lib64/python3.6/site-packages/irods/manager/data_object_manager.py", line 190, in put
    o.write(chunk)
  File "/home/mary/myenv/lib64/python3.6/site-packages/irods/data_object.py", line 180, in close
    self.conn.close_file(self.desc, **self.options)
  File "/home/mary/myenv/lib64/python3.6/site-packages/irods/connection.py", line 622, in close_file
    self.recv()
  File "/home/mary/myenv/lib64/python3.6/site-packages/irods/connection.py", line 124, in recv
    raise NetworkException("Could not receive server response")
irods.exception.NetworkException: Could not receive server response
Aborted (core dumped)

Here the error from log file:

{"log_category":"legacy","log_facility":"local0","log_level":"error","log_message":"ssl_load_hd_params: can't load DH parameter file. Falling back to built-ins.. SSL error: error:2006D080:BIO routines:BIO_new_file:no such file","XXX","server_pid":3350412,"server_timestamp":"2023-11-21T13:06:37.087Z","server_type":"agent"}

[Kory Draughn]

Hi,

The SSL code was adjusted in PRC 1.1.6.

I'm assuming you're connecting to the iRODS server using SSL based on those error messages.

Have you tried comparing your PRC SSL session construction logic with what's documented in the README?

See https://github.com/irods/python-irodsclient#establishing-a-secure-connection

One more question, has the iRODS server version changed since you saw success with PRC 1.1.5?

Kory Draughn

Chief Technologist

iRODS Consortium

--
--
The Integrated Rule-Oriented Data System (iRODS) - https://irods.org
 
iROD-Chat: http://groups.google.com/group/iROD-Chat
---
You received this message because you are subscribed to the Google Groups "iRODS-Chat" group.
To unsubscribe from this group and stop receiving emails from it, send an email to irod-chat+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/irod-chat/3afc56b0-60aa-490e-b0e8-ce41a6bfb59cn%40googlegroups.com.

[MBANZA Sara]

Hi,

I use SSL Certificate from CA. The codes in PRC are:

       ...

       ssl_context = ssl.create_default_context(purpose=ssl.Purpose.SERVER_AUTH, cafile=None, capath=None, cadata=None)

        ssl_settings =  {
               "client_server_negotiation": "request_server_negotiation",
               "client_server_policy": "CS_NEG_REQUIRE",
               "encryption_key_size": 32,
               "encryption_salt_size": 8,
               "encryption_num_hash_rounds": 16,
               "encryption_algorithm": "AES-256-CBC",
               "ssl_context": ssl_context
        }

        try:

            with iRODSSession(host='XXX', port=1247, authentication_scheme='PAM', user=username, password=passwd, zone='UniZone', **ssl_settings) as MY_SESSION:

           ...

iRODS version is 4.3.0.

I ' m uploading by using different PRC versions. With PRC version 1.1.5, 1.1.6 or 1.1.7 no problem. The error shown in the previous message comes when I upload using PRC version 1.1.8 or 1.1.9. Any suggestion to solve this?

[Alan King]

I'm not the best source here, but it looks like there may be some missing SSL context here...

According to the README, the ssl_settings requires a few extra bits of information if no env_file is being used. Here's the sample provided there:

>>> import ssl
>>> from irods.session import iRODSSession

>>> ssl_settings = {'client_server_negotiation': 'request_server_negotiation',

...                'client_server_policy': 'CS_NEG_REQUIRE',
...                'encryption_algorithm': 'AES-256-CBC',
...                'encryption_key_size': 32,
...                'encryption_num_hash_rounds': 16,
...                'encryption_salt_size': 8,
...                'ssl_context': ssl_context
...                'ssl_verify_server': 'cert',
...                'ssl_ca_certificate_file': '/etc/irods/ssl/irods.crt'
... }

ssl_verify_server and ssl_ca_certificate_file are required in addition to the other ssl_settings you listed. Try adding these members to the ssl_settings being provided to the iRODSSession, obviously adjusting as appropriate for your configuration.

All that being said, @Daniel Moore may have a better idea of what's going on.

Hope that helps.

To view this discussion on the web visit https://groups.google.com/d/msgid/irod-chat/1bec078f-5337-4132-9706-8a6c83a3330en%40googlegroups.com.

--

Alan King

Senior Software Developer | iRODS Consortium

[MBANZA Sara]

I think, these two lines (see below) are needed, if someone uses a self-signed SSL certificate (especially for testing purpose) . For my productive system, I 'm not using a self-signed SSL certificate. Instead, I ' m using a SSL certificate signed by CA.

Has anyone uploaded a file using PRC version 1.1.8 or 1.1.9 without the mentioned error message? (For other versions I have no problem, no error message.)

...                'ssl_verify_server': 'cert',
...                'ssl_ca_certificate_file': '/etc/irods/ssl/irods.crt'

[dmoore.renci]

Just by the way, is there an /etc/irods/ssl/dhparams.pem? I noticed the server was unable to read one.  This may be unimportant, I'm not sure, so I'm continuing to investigate.

Daniel Moore

Applications Engineer

iRODS Consortium

[dmoore.renci]

From v1.1.6 forward, PRC internally generates its own context if not externally generated and provided. 

You may provide your own as in the code example you've given, but if so, cafile and capath may need to be set to "non-None"  values, as shown by this line:

https://github.com/irods/python-irodsclient/blob/96d2cd223cc9f755f3d12f72a99b600553e8d911/irods/connection.py#L170

[MBANZA Sara]

Hello Daniel,

Thank you for your time and the reply.

Following your orientation, I have tried everything that I think could help me to fix the error. Unfortunately, the error is still there, when I use PRC 1.1.8 or 1.1.9.

Python-irodsclient PRC 1.1.5, 1.1.6 or 1.1.7 works without any errors. I' m still wondering why I only see the error for these last two versions 1.1.8 or 1.1.9.

I can continue to use PRC 1.1.7 (although I want to use the new version 1.1.9) and wait for the next version (I think it will be 1.1.10). I hope that PRC 1.1.10 will work for me without the error.

An /etc/irods/ssl/dhparams.pem is there.

Further information: iCommands and CyberDuck clients work very well. This means that the iRODS server has no problem.

Kind Regards,

Sara

[Terrell Russell]

Hi Sara,

This is the set of changes between 1.1.7 and 1.1.8...

https://github.com/irods/python-irodsclient/compare/v1.1.7...v1.1.8

Does anything there look... related to the errors you're seeing?

Terrell

To view this discussion on the web visit https://groups.google.com/d/msgid/irod-chat/dc57fd6f-cf2c-41f9-a0a0-d61b83ea208bn%40googlegroups.com.

[MBANZA Sara]

Hi Terrell,

Thanks.

So far I don't see anything. I will continue to check. Maybe the error is related to something else (instead of PRC), because I'm using/testing the Hamster GUI (https://github.com/Andries-B/Hamster).

Kind Regards,

Sara