Local (Rundeck) node breaks in specific projects

[Dan Landerman]

Hi,

I am running Rundeck 3.3.8-20210111 on RHEL 7 and seeing issues when trying to run jobs against the local Rundeck node for specific projects. The behavior seems to be that the local node as a resource breaks within a project at some point.

I found this specifically trying to implement a file upload job that just takes a single file and  uploads it to a folder on the Rundeck server. The job would just hang and not start. I also see the following error when trying to show all nodes in the project:

If I remove the local node from the project this works as expected. Also, if I export the entire project to a new one the local node works again. I did this once previously thinking it was just an issue with my project but now it's happened again.

From the logs, all I really see is a nullpointerexception and a lot of info that doesn't mean much to me. Happy to provide if it would help with resolution. 

[Dan L]

In the time it took to clear moderation I found this to be permissions related my account being in multiple groups with Admin not taking precedence. I am however still having issues adding the local Rundeck server to my ACL. This is what I currently have, I have also tried rundeck_server: 'true' instead of localhost.  Jobs using the local node either result in 500 errors or they hang. Do I have it in the wrong place or misformatted?

context:

  application: 'rundeck'

description: "System Access"

for:

  project:

    - match:

        name: 'myproj'

      allow: [read]

by:

  group: my_group

---

context:

  application: 'rundeck'

description: "Key Access"

for:

  storage:

    - match:

        path: 'keys'

      allow: [read]

by:

  group: my_group

---

context:

  project: 'myproj'

description: "Job Access"

for:

  resource:

    - equals:

        kind: 'node'

      allow: [read]

    - equals:

        kind: 'job'

      allow: [read]

    - equals:

        kind: 'event'

      allow: [read]

  adhoc:

    - deny: '*'

  job:

    - match:

        group: 'NonProd/.*'

      allow: [read,run,view,view_history]

  node:

    - contains:

        tags: 'NonProd'

      allow: [read,run]

    - equals:

        nodename: 'localhost'

      allow: [read,run]

by:

  group: my_group

[rac...@rundeck.com]

Hi Dan,

I reproduced your issue and it seems related to your node tags (your localhost is tagged as “NonProd”?), anyway, I leave a simplified ACL to test (check the application context block), feel free to modify it:

description: project context.
context:
  project: ProjectEXAMPLE

for:
  resource:
    - equals:
        kind: 'node'
      allow: [read]
    - equals:
        kind: 'job'
      allow: [read]
    - equals:
        kind: 'event'
      allow: [read]
  adhoc:
    - deny: '*'
  job:
    - equals:
        group: NonProd
      allow: [run,read]
  node:
    - equals:
        nodename: 'localhost'
      allow: [read,run]
by:
  group: mygroup

---

description: app context.
context:
  application: 'rundeck'

for:
  project:
    - match:
        name: ProjectEXAMPLE
      allow: [read]
  storage:
    - match:
        path: 'keys/.*'
      allow: [read]
by:
  group: mygroup

Hope it helps!

​

[Dan L]

Thanks a lot for looking at this!

I guess what I am trying to accomplish is having this group be able to 'read, run' against nodes with the NonProd tag AND the Rundeck server. We have Prod hosts in this project and I am trying to set up some additional restrictions around that. I have been doing a lot of testing and I've been able to accomplish NonProd OR Rundeck but not both.

The server error when viewing nodes I shared before is still a concern. It seems anytime you restrict nodes within the Project and try to 'select all' this is the behavior and it's not very user-friendly. I'm considering either making Prod a new Project or allowing 'read, run' on all nodes since the jobs are still restricted to the NonProd Group and 'adhoc' is disabled. 

[SysadminX]

I'm also experiencing this same issue after upgrading to Rundeck 3.4.0 on Ubuntu 18.04. I tried the above suggestion as well as several variations, but it seems that the problem is if there are two statements in the node clause, then the one for the rundeck server is ignored.

This does not work (the rundeck server is inaccessible):

  node:

  - allow: [read,run]

    equals:

      rundeck_server: 'true'

  - allow: [read,run]

    contains:

      tags: mytag

This does work:

  node:

  - allow: [read,run]

    equals:

      rundeck_server: 'true'

How can I restrict which nodes a user or group can run jobs on (e.g. by specifying a tag) and also allow the user and group to run jobs on the rundeck server itself? This worked back in Rundeck 3.2.x. Thanks!

[rac...@rundeck.com]

Hi SysadminX,

With the following ACL, it’s possible (tested on 3.4.0). The ACL is focused on the “user” group (you can use username: myuser instead of the group statment).

description: project context.
context:
  project: ProjectEXAMPLE

for:
  resource:
    - equals:
        kind: 'node'
      allow: [read]
    - equals:
        kind: 'job'
      allow: [read]
    - equals:
        kind: 'event'
      allow: [read]
  adhoc:
    - allow: '*'
  job:
    - equals:
        name: JobONE
      allow: [run,read]
  node:
    - equals:
        tags: 'db'
      allow: [read,run]
    - equals:
        nodename: 'localhost'
      allow: [read,run]
by:
  group: user

---

description: app context.
context:
  application: 'rundeck'

for:
  project:
    - match:
        name: ProjectEXAMPLE
      allow: [read]
  storage:
    - match:
        path: 'keys/.*'
      allow: [read]
by:
  group: user

At the moment of executing the job, applying the .* filter, the user only can execute on the db tagged nodes and localhost (Rundeck server).

Hope it helps!

​

[SysadminX]

Thank you, that worked! Specifically, I was able to use the following:

for:

  node:

    - equals:

        tags: 'db'

      allow: [read,run]

    - equals:

        rundeck_server: 'true'

      allow: [read,run]

It seems that the problem is using contains: with tags:. If instead I use equals: with tags: (as you suggested), the ACLs work as expected. I see this issue on Github which appears to report this changed behavior as a bug. Should contains: no longer be used with tags:, or is this indeed a bug?

Also, can you clarify how multiple statements inside the node: block are evaluated together? For example, in the ACL snippet I posted above, would that allow the user to read and run on nodes either with the db tag or the rundeck server itself? What if it is configured as two separate blocks in two separate files:

file1.aclpolicy:

for:

  node:

    - equals:

        tags: 'db'

      allow: [read,run]

file2.aclpolicy:

for:

    - equals:

        rundeck_server: 'true'

      allow: [read,run]

Does that change the behavior of how these two rules are joined together?

Thanks!

[rac...@rundeck.com]

Hi,

Yes, the bug is related with “contains” statment. If you put those rules in two separate files Rundeck evaluates one by one, and keeps the same behavior of:

  node:
    - equals:
        tags: 'db'
      allow: [read,run]
    - equals:

        rundeck_server: 'true'
      allow: [read,run]

So, I think that the best approach is put those rules on same file.

Regards!

​

[SysadminX]

While I can now see the rundeck server node, I now cannot seem to figure out how to show nodes with a specific tag. Here's a complete example:

description: Limited user access to myproject

context:

  application: 'rundeck'

for:

  resource:

    - equals:

        kind: 'system'

      allow: [read] # allow read of system info

  project:

    - match:

        name: 'myproject'

      allow: [read] # allow view of all projects

by:

  group: mygroup

---

description: mygroup run Test job

context:

  project: 'myproject'

for:

  resource:

  - equals:

      kind: 'node'

    allow: [read,refresh]

  - equals:

      kind: 'job'

    allow: [read]

  - equals:

      kind: 'event'

    allow: [read]

  adhoc:

  - allow: [read,run,kill]

  job:

  - allow: [read,run,kill]

    equals:

       name: 'Test'

  node:

  - equals:

      tags: 'mytag'

    allow: [read,run]

  - equals:

      rundeck_server: 'true'

    allow: [read,run]

by:

  group: mygroup

Yet when using the filter .* in the list of nodes, the only node I can see is the rundeck server (I would expect to see all of the nodes with the mytag tag listed too). In rundeck.audit.log I see the following:

[2021-06-23T07:57:49,605] INFO  authorization.LoggingAuthorization - Evaluating Decision for: res<nodename:myhost, hostname:myhost, type:node, username:rundeck, tags:tag1, mytag, tag3, rundeck_server:false> subject<Username:myuser Group:group1 Group:mygroup Group:group2 Group:group3> action<read> env<rundeck:auth:env:project:production>: authorized: false: REJECTED, reason: REJECTED, evaluations:  ACLRule</etc/rundeck/mygroup.aclpolicy[3][type:job][rule: 1]>{'mygroup run Test job' context={project='myproject'} type='job' match , resource={name=Test} for: { group='mygroup'} allow=[read, run, kill]} REJECTED for action read => REJECTED (1ms)

This doesn't make sense to me - the action read is rejected but yet it says right there that allow=[read, run, kill] is allowed. What am I missing? I tried feeding this log excerpt to rd acl create -i but could not figure out how to get the command to ingest it like rd-acl used to do; do you have a working example of that?

Thanks!

[rac...@rundeck.com]

Hi SysadminX,

I tested again with your ACL and works as expected (just changing the project name, group name, job name, and the node tag name).

description: project context.
context:
  project: ProductionPROJECT

for:
  resource:
    - equals:
        kind: 'node'
      allow: [read]
    - equals:
        kind: 'job'
      allow: [read]
    - equals:
        kind: 'event'
      allow: [read]
  adhoc:
    - allow: '*'
  job:
    - equals:
        name: HelloWorld
      allow: [run,read]
  node:
    - equals:
        tags: 'db'
      allow: [read,run]
    - equals:
        rundeck_server: 'true'
      allow: [read,run]
by:
  group: user

---

description: app context.
context:
  application: 'rundeck'

for:
  project:
    - match:
        name: ProductionPROJECT
      allow: [read]
  storage:
    - match:
        path: 'keys/.*'
      allow: [read]
by:
  group: user

Ensure that another ACL does not interfere with the current one. Also, take a look at this.

Regards!

​

[SysadminX]

Thanks for the additional information. I cannot seem to get the example you sent to work, so I am wondering if two ACL policies are conflicting. What happens if a user is a member of both group1 and group2 and the following two ACL policies exist?

group1.aclpolicy:

  job:

    - equals:

        name: Job1

      allow: [run,read]

  node:

    - equals:

        tags: 'db'

      allow: [read,run]

    - equals:

        rundeck_server: 'true'

      allow: [read,run]

group2.aclpolicy:

  job:

    - equals:

        name: Job2

      allow: [run,read]

  node:

    - equals:

        tags: 'web'

      allow: [read,run]

    - equals:

        rundeck_server: 'true'

      allow: [read,run]

I would expect that the user would be able to run Job1 on any node with tag db and Job2 on any node with tag web, but is that not the case? How are multiple ACL policy files combined (e.g. are the job + node sections in a single policy ANDed together, or are all of the job sections from all policies ANDed together first)?

Thanks!

[rac...@rundeck.com]

Hi SysadminX,

In that case the best approach is to define separate user profiles for what you want to run. If the user belongs to both groups, will see all the rules defined for both. There is currently no ACL granularity level that defines which specific nodes a specific job can run (the user can see all nodes defined on both acl rules, and all jobs too), anothher approach you can use is to simply use node filters passing via options, or call the jobs from RD-CLI inside on inline script depending on ${job.username} variable.

Greetings!

[SysadminX]

Hello,

Thanks for the additional suggestions like using node filters on the jobs. Unfortunately this would create a lot of duplicate jobs (all with the same contents but each with a different node filter corresponding to a different group of users) which I'd like to avoid.

I did some additional testing and believe the problem to be that the nodes I am working with have several different tags (for example, a node has db, linux, and servers all as its tags). Therefore, this ACL does not allow me to run Job1 on a node because equals requires that ONLY the db tag, exclusively, be present on the node (and since other tags are present on the node, it fails):

  job:

    - equals:

        name: Job1

      allow: [run,read]

  node:

    - equals:

        tags: 'db'

      allow: [read,run]

    - equals:

        rundeck_server: 'true'

      allow: [read,run]

I reviewed the documentation for the "Matching" entries and it sounds like contains is the only one that will fit my use case (allow setting ACL rules based on one of a node's tags while allowing the node to have other tags also). Is there a timeline or status update for this GitHub issue? I don't think I can upgrade to 3.4.0 until this is resolved.

Thanks!