problems with Kerberos client

[mp4...@gmail.com]

Hi,

I configured an iRODS ICAT server with:
irods-icat-4.1.7-0.x86_64
irods-database-plugin-postgres-1.7-0.x86_64
irods-auth-plugin-krb-1.1-0.x86_64

and a client with:
irods-icommands-4.1.7-0.x86_64
irods-auth-plugin-krb-1.1-0.x86_64

On the client, when I run ils, I see:

[pasmarco@pollegio ~]$ ils
ERROR: [-]    iRODS/lib/core/src/clientLogin.cpp:293:clientLogin :  status [PLUGIN_ERROR]  errno [] -- message []
    [-]    iRODS/lib/core/src/irods_krb_object.cpp:34:resolve :  status [PLUGIN_ERROR]  errno [] -- message [Failed to load the KRB auth plugin.]
        [-]    iRODS/lib/core/src/irods_auth_manager.cpp:76:init_from_type :  status [PLUGIN_ERROR]  errno [] -- message [Failed to load auth plugin.]
            [-]    iRODS/lib/core/src/irods_auth_manager.cpp:55:load_auth_plugin :  status [PLUGIN_ERROR]  errno [] -- message [Failed to load plugin: "krb".]
                [-]    iRODS/lib/core/include/irods_load_plugin.hpp:194:load_plugin :  status [PLUGIN_ERROR]  errno [] -- message [failed to open shared object file [/var/lib/irods/plugins/auth/libkrb.so] :: dlerror: is [/var/lib/irods/plugins/auth/libkrb.so: undefined symbol: _ZN5irods4auth10delay_loadEPv]]

The missingfile is there though:
[pasmarco@pollegio ~]$ ls -lah /var/lib/irods/plugins/auth/libkrb.so
-rw-r--r-- 1 irods irods 1.5M Aug 24  2014 /var/lib/irods/plugins/auth/libkrb.so

On the server logs I see:
Jan 22 11:24:07 pid:23540 NOTICE: Agent process 23622 started for puser=pasmarco and cuser=pasmarco from 148.187.80.35
Jan 22 11:24:07 pid:23622 NOTICE: readAndProcClientMsg: received disconnect msg from client
Jan 22 11:24:07 pid:23622 NOTICE: Agent exiting with status = 0
Jan 22 11:24:08 pid:23540 NOTICE: Agent process 23622 exited with status 0
Jan 22 11:25:57 pid:23540 NOTICE: Agent process 23636 started for puser=pasmarco and cuser=pasmarco from 148.187.80.35
Jan 22 11:25:57 pid:23636 NOTICE: readAndProcClientMsg: received disconnect msg from client
Jan 22 11:25:57 pid:23636 NOTICE: Agent exiting with status = 0
Jan 22 11:25:58 pid:23540 NOTICE: Agent process 23636 exited with status 0

I followed the guide at: https://docs.irods.org/4.1.7/manual/authentication/#kerberos

Any idea of what I might be doing wrong?

Thanks in advance,
Marco Passerini

[Smeele, A.P.M. (Ton)]

undefined symbol

 

Not sure if this helps in your case:  if the client application .SO component has been compiled/linked against an earlier iRODS client lib release then this may result in undefined symbols for plugins.   We have experienced dynamic linking incompatibilities between 4.1.4 and 4.17.  Either relink the client or install the iRODS client libs for the earlier release as a workaround.

 

Kind regards,

 

Ton

--
--
"iRODS: the Integrated Rule-Oriented Data-management System; A community driven, open source, data grid software solution" https://www.irods.org
 
iROD-Chat: http://groups.google.com/group/iROD-Chat

---
You received this message because you are subscribed to the Google Groups "iRODS-Chat" group.
To unsubscribe from this group and stop receiving emails from it, send an email to irod-chat+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[mp4...@gmail.com]

I used the RPMs provided by the iRODS webpage. I'll try to compile the source code and let you know how it goes.

[Ben Keller]

I'm sorry for the confusion. The Kerberos packages on the irods.org/download site are out-of-date. We will be releasing new versions along with iRODS 4.1.8 in the near term (two weeks from now?).

We will update the website shortly to reflect this.

[mp4...@gmail.com]

Thanks Ben for the reply,

I compiled the latest version from Github and indeed it works better. However, I still have some problem, maybe with my configuration.

On the irods server, the server_config.json file contains:

[...]
    "kerberos_service_principal": "host/irodsserver....@MYDOMAIN.COM",
    "kerberos_keytab": "/var/lib/irods/irods.keytab"
}

Please note that I had to change the string formatting to follow the way YAML works. On the original documentation it uses = instead of columns:
https://docs.irods.org/4.1.0/manual/authentication/#kerberos

I created the keytab on the server and copied it to the specified folder. I gave it ownership to irods:irods and  it's right now world readable, for debugging purposes.

On the client  I have:
    "irods_authentication_scheme": "KRB"

I created an association with iadmin:
myuser myu...@MYDOMAIN.COM


I ran kinit and I have a valid ticket:
[myuser@irodsclient ~]$ klist
Ticket cache: FILE:/tmp/krb5cc_23520
Default principal: myu...@MYDOMAIN.COM

Valid starting     Expires            Service principal
01/27/16 11:01:04  01/27/16 21:01:04  krbtgt/MYDOMA...@MYDOMAIN.COM
    renew until 02/03/16 11:00:57

If I run ils on the client I see:

[myuser@irodsclient ~]$ ils
[-]    libkrb.cpp:1195:krb_auth_client_request :  status [KEY_NOT_FOUND]  errno [] -- message [call to rcAuthRequest failed.]
 failed with error -1800000 KEY_NOT_FOUND

On the server instead I get:

Jan 27 11:27:00 pid:6411 NOTICE: Agent process 6562 started for puser=myuser and cuser=myuser from 148.187.80.35
Jan 27 11:27:00 pid:6562 ERROR: [-]    iRODS/server/api/src/rsAuthPluginRequest.cpp:85:rsAuthPluginRequest :  status [KEY_NOT_FOUND]  errno [] -- message []
    [-]    libkrb.cpp:1234:krb_auth_agent_request :  status [KEY_NOT_FOUND]  errno [] -- message [Failed to fetch Kerberos name from server config.]
        [-]    libkrb.cpp:131:krb_kerberos_name :  status [KEY_NOT_FOUND]  errno [] -- message [Failed reading KerberosServicePrincipal from the server properties.]
            [-]    iRODS/lib/core/include/irods_server_properties.hpp:89:get_property :  status [KEY_NOT_FOUND]  errno [] -- message []
                [-]    iRODS/lib/core/include/irods_lookup_table.hpp:149:get :  status [KEY_NOT_FOUND]  errno [] -- message [failed to find key [kerberos_name] in table.]
Jan 27 11:27:00 pid:6562 DEBUG: On iRODS-Server side:GSS-API error accepting context: Invalid token was supplied
Jan 27 11:27:00 pid:6562 DEBUG: On iRODS-Server side:GSS-API error accepting context: Unknown error
Jan 27 11:27:00 pid:6562 ERROR: [-]    iRODS/server/core/src/rodsAgent.cpp:348:agentMain :  status [KRB_ACCEPT_SEC_CONTEXT_ERROR]  errno [] -- message [Failed during auth plugin agent start for scheme: "krb".]
    [-]    libkrb.cpp:892:krb_auth_agent_start :  status [KRB_ACCEPT_SEC_CONTEXT_ERROR]  errno [] -- message [Failed to establish server side context.]
        [-]    libkrb.cpp:783:krb_establish_context_serverside :  status [KRB_ACCEPT_SEC_CONTEXT_ERROR]  errno [] -- message [Error accepting KRB security context for client: "(null)".]
Jan 27 11:27:00 pid:6562 NOTICE: Agent exiting with status = -966000
Jan 27 11:27:00 pid:6411 NOTICE: Agent process 6562 exited with status 36864



Is there a problem in parsing the configuraiton? These lines suggest me this:
Failed to fetch Kerberos name from server config.
Failed reading KerberosServicePrincipal from the server properties.
Failed to find key [kerberos_name] in table.

Any idea?

Marco Passerini

On Monday, January 25, 2016 at 5:20:16 PM UTC+1, Ben Keller wrote:

I'm sorry for the confusion. The Kerberos packages on the irods.org/download site are out-of-date. We will be releasing new versions along with iRODS 4.1.8 in the near term (two weeks from now?).

We will update the website shortly to reflect this.

On Mon, Jan 25, 2016 at 7:49 AM, <mp4...@gmail.com> wrote:

I used the RPMs provided by the iRODS webpage. I'll try to compile the source code and let you know how it goes.


On Monday, January 25, 2016 at 1:06:11 PM UTC+1, Smeele, A.P.M. (Ton) wrote:

undefined symbol

 

Not sure if this helps in your case:  if the client application .SO component has been compiled/linked against an earlier iRODS client lib release then this may result in undefined symbols for plugins.   We have experienced dynamic linking incompatibilities between 4.1.4 and 4.17.  Either relink the client or install the iRODS client libs for the earlier release as a workaround.

 

Kind regards,

 

Ton

 

From: irod...@googlegroups.com [mailto:irod...@googlegroups.com] On Behalf Of mp4...@gmail.com

Sent: maandag 25 januari 2016 11:40
To: iRODS-Chat
Subject: [iROD-Chat:14713] problems with Kerberos client

Hi,

I configured an iRODS ICAT server with:
irods-icat-4.1.7-0.x86_64
irods-database-plugin-postgres-1.7-0.x86_64
irods-auth-plugin-krb-1.1-0.x86_64

and a client with:
irods-icommands-4.1.7-0.x86_64
irods-auth-plugin-krb-1.1-0.x86_64

On the client, when I run ils, I see:

[myuser@irodsclient ~]$ ils

ERROR: [-]    iRODS/lib/core/src/clientLogin.cpp:293:clientLogin :  status [PLUGIN_ERROR]  errno [] -- message []
    [-]    iRODS/lib/core/src/irods_krb_object.cpp:34:resolve :  status [PLUGIN_ERROR]  errno [] -- message [Failed to load the KRB auth plugin.]
        [-]    iRODS/lib/core/src/irods_auth_manager.cpp:76:init_from_type :  status [PLUGIN_ERROR]  errno [] -- message [Failed to load auth plugin.]
            [-]    iRODS/lib/core/src/irods_auth_manager.cpp:55:load_auth_plugin :  status [PLUGIN_ERROR]  errno [] -- message [Failed to load plugin: "krb".]
                [-]    iRODS/lib/core/include/irods_load_plugin.hpp:194:load_plugin :  status [PLUGIN_ERROR]  errno [] -- message [failed to open shared object file [/var/lib/irods/plugins/auth/libkrb.so] :: dlerror: is [/var/lib/irods/plugins/auth/libkrb.so: undefined symbol: _ZN5irods4auth10delay_loadEPv]]

The missingfile is there though:

[myuser@irodsclient ~]$ ls -lah /var/lib/irods/plugins/auth/libkrb.so

-rw-r--r-- 1 irods irods 1.5M Aug 24  2014 /var/lib/irods/plugins/auth/libkrb.so

On the server logs I see:

Jan 22 11:24:07 pid:23540 NOTICE: Agent process 23622 started for puser=myuser and cuser=myuser from 148.187.80.35

Jan 22 11:24:07 pid:23622 NOTICE: readAndProcClientMsg: received disconnect msg from client
Jan 22 11:24:07 pid:23622 NOTICE: Agent exiting with status = 0
Jan 22 11:24:08 pid:23540 NOTICE: Agent process 23622 exited with status 0

Jan 22 11:25:57 pid:23540 NOTICE: Agent process 23636 started for puser=myuser and cuser=myuser from 148.187.80.35

Jan 22 11:25:57 pid:23636 NOTICE: readAndProcClientMsg: received disconnect msg from client
Jan 22 11:25:57 pid:23636 NOTICE: Agent exiting with status = 0
Jan 22 11:25:58 pid:23540 NOTICE: Agent process 23636 exited with status 0

I followed the guide at: https://docs.irods.org/4.1.7/manual/authentication/#kerberos

Any idea of what I might be doing wrong?

Thanks in advance,
Marco Passerini

--
--
"iRODS: the Integrated Rule-Oriented Data-management System; A community driven, open source, data grid software solution" https://www.irods.org
 
iROD-Chat: http://groups.google.com/group/iROD-Chat

---
You received this message because you are subscribed to the Google Groups "iRODS-Chat" group.

To unsubscribe from this group and stop receiving emails from it, send an email to irod-chat+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Ben Keller]

Thank you for reporting this. 

There is a bug in the documentation:

In your /etc/irods/server_config.json,

• kerberos_service_principal should be KerberosServicePrincipal
• kerberos_keytab should be KerberosKeytab.
  

The documentation for the upcoming 4.1.8 has been updated to use the JSON formatting (as you noted) and to use the corrected server_config.json key names (camel case instead of underscores).

The updates to the documentation can be seen here.

FYI, the code we use to setup our iRODS/Kerberos testing environment (for Ubuntu 14) is here.

[mp4...@gmail.com]

All right, I got it working, as you said, I also had to add the following (based on your example):

    "environment_variables": {
       "KRB5_KTNAME": "/var/lib/irods/irods.keytab"
    },

I must say that after a few months of experience with iRODS... this software is a bit of a mine field! I hope you guys manage to get your release process right in the near future.
Anyway, thanks for your support, this mailing list is very helpful to get problems sorted.

Marco Passerini

On Wednesday, January 27, 2016 at 6:28:34 PM UTC+1, Ben Keller wrote:

Thank you for reporting this. 

There is a bug in the documentation:

In your /etc/irods/server_config.json,

• kerberos_service_principal should be KerberosServicePrincipal
• kerberos_keytab should be KerberosKeytab.
  

The documentation for the upcoming 4.1.8 has been updated to use the JSON formatting (as you noted) and to use the corrected server_config.json key names (camel case instead of underscores).

The updates to the documentation can be seen here.

FYI, the code we use to setup our iRODS/Kerberos testing environment (for Ubuntu 14) is here.

On Wed, Jan 27, 2016 at 5:32 AM, <mp4...@gmail.com> wrote:

Thanks Ben for the reply,

I compiled the latest version from Github and indeed it works better. However, I still have some problem, maybe with my configuration.

On the irods server, the server_config.json file contains:

[...]

    "kerberos_service_principal": "host/irodsserver.mydomain.c...@MYDOMAIN.COM",

[Dan Bedard]

I must say that after a few months of experience with iRODS... this software is a bit of a mine field! I hope you guys manage to get your release process right in the near future.

Dear Marco,

Thank you for the frank feedback. As you have experienced, we are still managing the transition from a time when many iRODS capabilities were implemented as compile-time options. We think we're on the right track: by allowing plugin code to advance at its own rate, the iRODS developer community can grow more rapidly, as plugin developers won't have to be so closely tied in to the core development team.

For iRODS users, we have a lot of improvements in the works: We're bringing Consortium-supported plugins into our own continuous integration and test system. We're teaching developers how to implement their own CI system, so they can certify that their plugins work. And iRODS 4.2 will be installable from a yum/apt package repository, with a more clearly defined and enforceable dependency model.

To help in the meantime, going forward, we are documenting and organizing released packages so that it's clear which plugin packages have been tested to work with which core packages.

We are also discussing ways to improve this forum, to recognize experts for their efforts in supporting other users in the iRODS community. 

I'm being sincere when I say that I appreciate your feedback. Please keep your issue reports and suggestions coming. It's the best way to keep making iRODS better.

Best regards,

Dan Bedard

— 

Dan Bedard
Executive Director

iRODS Consortium

email: danb (at) renci.org

twitter: @DataDanB

iRODS User Group Meeting 2016

June 7-9 -- Chapel Hill, NC -- https://ugm2016.irods.org

[mp4...@gmail.com]

Thanks for your kind reply!

I think you have the right vision for the future of iRODS. For sure it's not easy to keep open source projects consistent and satisfy all requirements but it seems you guys know where you are going and you know how to make it better. Keep up with the good work! I apologize if my comment sounded rude, I didn't mean to criticize anybody in particular.

Best Regards,
Marco Passerini

[Dan Bedard]

Marco,

No need to apologize. This is what I like about being open source: we're all in this together.

Cheers,

Dan

[John Constable]

So, we've been following along at home, because kerberos is key to us as well (we're on 3.3.1 at the moment), and thanks to this thread we've got a lot further than we would have done (so thank you!), however, we hit a different problem, in that when we try and run an icommand as a kerberos enabled user in irods we get the following;

DEBUG: Client side:GSS-API error initializing context: Unspecified GSS failure.  Minor code may provide more information

DEBUG: Client side:GSS-API error initializing context: KDC has no support for encryption type

ERROR: [-] iRODS/lib/core/src/clientLogin.cpp:321:clientLogin :  status [KRB_ERROR_INIT_SECURITY_CONTEXT]  errno [] -- message []

[-] libkrb.cpp:628:krb_auth_establish_context :  status [KRB_ERROR_INIT_SECURITY_CONTEXT]  errno [] -- message [Failed initializing KRB context. Major status: 0 Minor status: 0]

In the RodsLog we have;

Jan 29 14:59:25 pid:2778 NOTICE: Agent process 7272 started for puser=irods_srv_test and cuser=irods_srv_test from 127.0.0.1

Jan 29 14:59:26 pid:7272 DEBUG: On iRODS-Server side:GSS-API error accepting context: Invalid token was supplied

Jan 29 14:59:26 pid:7272 DEBUG: On iRODS-Server side:GSS-API error accepting context: Unknown error

Jan 29 14:59:26 pid:7272 ERROR: [-]     iRODS/server/core/src/rodsAgent.cpp:348:agentMain :  status [KRB_ACCEPT_SEC_CONTEXT_ERROR]  errno [] -- message [Failed during auth plugin agent start for scheme: "krb".]

        [-]     libkrb.cpp:892:krb_auth_agent_start :  status [KRB_ACCEPT_SEC_CONTEXT_ERROR]  errno [] -- message [Failed to establish server side context.]

                [-]     libkrb.cpp:783:krb_establish_context_serverside :  status [KRB_ACCEPT_SEC_CONTEXT_ERROR]  errno [] -- message [Error accepting KRB security context for client: "(null)".]

Jan 29 14:59:26 pid:7272 NOTICE: Agent exiting with status = -966000

Jan 29 14:59:26 pid:2778 NOTICE: Agent process 7272 exited with status 36864

the kerberos ticket we have for the user uses the following encryption;

renew until 30/01/2016 14:39, Etype (skey, tkt): aes256-cts-hmac-sha1-96, aes256-cts-hmac-sha1-96 

The error message implies that we are using incompatible encryption types (we use AD as our Kerberos server, and have had this problem in the past when Microsoft patched AD to change the encryption order), any idea if this is a valid hypothesis, or any better ones?

Cheers,

John

[Terrell Russell]

This has been solved offline with John using this workaround...

  https://docs.irods.org/master/manual/authentication/#weak-encryption-workaround

To override this mismatch and allow a weaker algorithm to be sufficient, set allow_weak_crypto = yes in the libdefaults stanza of /etc/krb5.conf:

$ head /etc/krb5.conf
[libdefaults]
        default_realm = EXAMPLE.ORG
        allow_weak_crypto = yes
...

This will allow the Kerberos handshake to succeed, which allows the iRODS connection to continue.

Terrell