CAT_INVALID_AUTHENTICATION for service and native users after restart

37 views
Skip to first unread message

J.P. Mc Farland

unread,
Aug 10, 2026, 7:19:13 AMAug 10
to iRODS-Chat
Hi Everybody!

After updating rules (security and other) on one of our test zone servers (4.3.4) and restarting, the service user and all "native" users cannot authenticate (CAT_INVALID_AUTHENTICATION), but "pam_password" users can.  The configuration did not change and the rules were effectively "read-only" in nature (only checks with *_pre PEPs).  Reverting the rules and restarting has not helped.  Running iinit on native users (including the service user) has not helped.  Restarting the server process and entire server has not helped.  Using an appropriately reconfigured server recently cloned from the first has not helped (ICAT issue?).  There is nothing of note in the logs except the CAT_INVALID_AUTHENTICATION errors.  Neither iinit nor iexit was run on any of the native users prior to this occurrence.  Relevant documentation (https://docs.irods.org/4.3.4/system_overview/troubleshooting/#service-account-ran-iexit-or-otherwise-is-not-authenticated) indicates simply rerunning iinit should do the trick, but it does not.  We even tried running iexit prior to iinit and still no change.

Has anybody else seen such behavior?  Are there any other areas where we should check or perform tests?

Cheers,

--John

J.P. Mc Farland

unread,
Aug 20, 2026, 4:18:53 AM (12 days ago) Aug 20
to iRODS-Chat
Hi Everybody,

It turns out that the problem had to do with the addition of the unset IRODS_DATABASE_USER_PASSWORD_SALT environment variable.  The original:

    "environment_variables": {},

was changed to:

    "environment_variables": {
        "IRODS_DATABASE_USER_PASSWORD_SALT": ""
    },

and the problem apparently was caused by the difference between a NULL value and a null string.  More documentation on this would be appreciated.

Also, a previous issue we commented on here was peripherally involved.  Using a pam_password user of type rodsadmin, it appears possible to update a native user's password.  Updating an existing native user's password from a client machine (not any server in the zone) simply fails silently, but creating a new native user and setting the initial password actually succeeds.  If that same pam_password rodsadmin runs the commands from the server machine, the operation proceeds as before, but the password for existing native user is properly set now.

While trying to diagnose this issue, the service user's password was updated with different password salt settings, but all failed giving a misleading result due to what I just described.  When I reset the salt as it originally was and updated the password while on the zone server, everything returned to normal.

It would be nice to have at least a brief explanation of how the password salt is applied in the various circumstances (for native and pam_password users and any possible differences) somewhere in the documentation.  Something like this could have likely reduced the amount of time we spent on diagnosis and resolution.  That is the main reason for this followup post.

Cheers,

--John

Kory Draughn

unread,
Aug 25, 2026, 9:22:42 AM (7 days ago) Aug 25
to irod...@googlegroups.com
Hi John,

I've opened https://github.com/irods/irods_docs/issues/427 to address documentation for the database salt.

Kory Draughn
Chief Technologist
iRODS Consortium


--
--
The Integrated Rule-Oriented Data System (iRODS) - https://irods.org
 
iROD-Chat: http://groups.google.com/group/iROD-Chat
---
You received this message because you are subscribed to the Google Groups "iRODS-Chat" group.
To unsubscribe from this group and stop receiving emails from it, send an email to irod-chat+...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/irod-chat/438c898e-74fe-4de2-bfbf-74b4321b8d76n%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages