loading large rundeck jobxml results in many authorization calls

[Charles Scott]

Testing rundeck v1.5.2 (rpm dist).

I am uploading a very large set of jobs which are concatenated in a single job xml (as opposed to loading many jobxml's with 1 per file) and using api v5 to upload via curl

This is to expedite the job loading process.  Unfortunately it appears to take a very long time and sometimes my curl api connection will terminate (probably apache timing it out as the async process will continue)

The rundeck.audit.log after iterating a few times has grown  very large in just a very short time.   

As you can see, the uploaded jobxml is very long

-shell-3.2$ wc -l /tmp/tmp.BULGZc2052/jobsFO.xml

84718 /tmp/tmp.BULGZc2052/jobsFO.xml

-shell-3.2$ grep -c '<uuid>' /tmp/tmp.BULGZc2052/jobsFO.xml

2313

And yes a lot of jobs (2313).

It appears that there is a lot of  access control activity going on for each individual job entry:

[chuck@wsgc-dev-rhel59-2 rundeck]$ ls -l rundeck.audit.log 

-rw-r--r-- 1 rundeck rundeck 44558218 Jun 12 09:16 rundeck.audit.log

[chuck@wsgc-dev-rhel59-2 rundeck]$ tail -f rundeck.audit.log 

…

…

2013-06-12 09:14:08,257 - Evaluating Decision for: res<type:job, name:stopPKAppDynamics, group:deploy/uat4/frontend/pk> subject<Username:cscott2 Group:GS-EComm-WebAdmins Group:SNC-Ecommerce Performance Group:Ecom EO DevOps Group:GS-Misg-MIS-CORP Group:UnixUsers Group:GS-Members-MISMO-CORP Group:VPN-Users Associates Group:IT eCommerce Organization Group:SNC-Ecommerce Search Group:GS-APMQALDAP Group:All North Point - Associates Group:Clarity Users Group:GS-WebSiteAccess-MIS-CORP Group:SNC-Ecommerce Middleware Group:SNC-Ecommerce Non-Production Systems Group:GS-PrivNAS-AK-CORP Group:WebAdmins Group:GS-Everyone-MISMO-CORP Group:$MISMO Group:GS-APMPRDLDAP Group:GS-MAC Users-Merch> action<update> env<http://dtolabs.com/rundeck/env/project:wsgc>: authorized: true: GRANTED, reason: GRANTED, evaluations:  /etc/rundeck/WebAdmins.aclpolicy[1][rule: 1: {allow=[create, read, update, delete, run, kill]}] for actions: [update, delete, kill, read, run, create] => GRANTED_ACTIONS_AND_COMMANDS_MATCHED (0ms)

2013-06-12 09:14:08,355 - Evaluating Decision for: res<type:job, name:stopPK, group:deploy/uat4/frontend/pk> subject<Username:cscott2 Group:GS-EComm-WebAdmins Group:SNC-Ecommerce Performance Group:Ecom EO DevOps Group:GS-Misg-MIS-CORP Group:UnixUsers Group:GS-Members-MISMO-CORP Group:VPN-Users Associates Group:IT eCommerce Organization Group:SNC-Ecommerce Search Group:GS-APMQALDAP Group:All North Point - Associates Group:Clarity Users Group:GS-WebSiteAccess-MIS-CORP Group:SNC-Ecommerce Middleware Group:SNC-Ecommerce Non-Production Systems Group:GS-PrivNAS-AK-CORP Group:WebAdmins Group:GS-Everyone-MISMO-CORP Group:$MISMO Group:GS-APMPRDLDAP Group:GS-MAC Users-Merch> action<update> env<http://dtolabs.com/rundeck/env/project:wsgc>: authorized: true: GRANTED, reason: GRANTED, evaluations:     /etc/rundeck/WebAdmins.aclpolicy[1][rule: 1: {allow=[create, read, update, delete, run, kill]}] for actions: [update, delete, kill, read, run, create] => GRANTED_ACTIONS_AND_COMMANDS_MATCHED (0ms)

2013-06-12 09:14:08,451 - Evaluating Decision for: res<type:job, name:deployPKAppDynamics, group:deploy/uat4/frontend/pk> subject<Username:cscott2 Group:GS-EComm-WebAdmins Group:SNC-Ecommerce Performance Group:Ecom EO DevOps Group:GS-Misg-MIS-CORP Group:UnixUsers Group:GS-Members-MISMO-CORP Group:VPN-Users Associates Group:IT eCommerce Organization Group:SNC-Ecommerce Search Group:GS-APMQALDAP Group:All North Point - Associates Group:Clarity Users Group:GS-WebSiteAccess-MIS-CORP Group:SNC-Ecommerce Middleware Group:SNC-Ecommerce Non-Production Systems Group:GS-PrivNAS-AK-CORP Group:WebAdmins Group:GS-Everyone-MISMO-CORP Group:$MISMO Group:GS-APMPRDLDAP Group:GS-MAC Users-Merch> action<update> env<http://dtolabs.com/rundeck/env/project:wsgc>: authorized: true: GRANTED, reason: GRANTED, evaluations:        /etc/rundeck/WebAdmins.aclpolicy[1][rule: 1: {allow=[create, read, update, delete, run, kill]}] for actions: [update, delete, kill, read, run, create] => GRANTED_ACTIONS_AND_COMMANDS_MATCHED (0ms)

2013-06-12 09:14:08,551 - Evaluating Decision for: res<type:job, name:statusPK, group:deploy/uat4/frontend/pk> subject<Username:cscott2 Group:GS-EComm-WebAdmins Group:SNC-Ecommerce Performance Group:Ecom EO DevOps Group:GS-Misg-MIS-CORP Group:UnixUsers Group:GS-Members-MISMO-CORP Group:VPN-Users Associates Group:IT eCommerce Organization Group:SNC-Ecommerce Search Group:GS-APMQALDAP Group:All North Point - Associates Group:Clarity Users Group:GS-WebSiteAccess-MIS-CORP Group:SNC-Ecommerce Middleware Group:SNC-Ecommerce Non-Production Systems Group:GS-PrivNAS-AK-CORP Group:WebAdmins Group:GS-Everyone-MISMO-CORP Group:$MISMO Group:GS-APMPRDLDAP Group:GS-MAC Users-Merch> action<update> env<http://dtolabs.com/rundeck/env/project:wsgc>: authorized: true: GRANTED, reason: GRANTED, evaluations:   /etc/rundeck/WebAdmins.aclpolicy[1][rule: 1: {allow=[create, read, update, delete, run, kill]}] for actions: [update, delete, kill, read, run, create] => GRANTED_ACTIONS_AND_COMMANDS_MATCHED (0ms)

It appears that each job entry has to be granted access.   

In an older rundeck (1.3) I believe this may not be the case, however, I will need to test that as we are using a similar method over api v1.

Does this appear to be an unexpected amount of overhead?

Thanks,

Chuck

[Alex Honor]

Hi Chuck,

Yes it does seem like it is extra overhead. One question is should the ACL check be performed just once (eg, can the user add/edit jobs in general for this project) vs check each job.

Thanks

--
You received this message because you are subscribed to the Google Groups "rundeck-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rundeck-discu...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--

Alex Honor

[SimplifyOps, Inc | a...@simplifyops.com ]

Be sure to comment and vote on Rundeck Feature Development!

[ptitlus]

Hi,

I install the rundeck launcher (v2.6.0) and get the same problem.

My rundeck.audit.log is very large.

Every action is evaluate and log grow larger !!

Have you found a solution?

Thanks 

[jjouanin]

Hi,

I experienced the same problem :

1 project / 1000 jobs approx => Many messages "INFO  com.dtolabs.rundeck.core.authorization.RuleEvaluator - Evaluating Decision for: [...]"

I tried to change log level from INFO to WARN :

log4j.rootLogger=WARN, stdout

=> OK no more messages "INFO  com.dtolabs.rundeck.core.authorization.RuleEvaluator - Evaluating Decision for: [...]"

J

[ptitlus]

Hi,

Thanks !

I did the same !

No more problems.

Just for info, the file is:

$RDECK_BASE/server/exp/webapp/WEB-INF/classes/log4j.properties

bye