Upgrade Process w/ Launcher?

[AC]

I had a Rundeck 1.3.2 installation that was working well (no issues at
all), and I'm trying to upgrade to version 1.4.0.1. The upgrade
documentation is rich with details around the aclpolicy format
changes, but it stops short of actually describing the upgrade process
itself. For example, do I just drop the 1.4.0.1 launcher JAR into my
existing installation root, then change scripts & config files to
reference the new JAR file? Or should I be creating a fresh 1.4.0.1
installation, and then re-implementing all of my customizations, jobs,
projects, etc. into the new installation?

Using my best guess, I tried the process below, but the result is a
"No such property: subject for class:
com.dtolabs.rundeck.core.authentication.NoAuthentication" stack trace
(complete trace shown at bottom) when I try to do anything within
Rundeck (even with "admin" user). I'm not sure if this is due to the
process I used, or something else gone awry.

1) Copied my 1.3.2 installation into a 1.4.0 directory (cp -r -p /
apps/rundeck-1.3.2 /apps/rundeck-1.4.0).
2) Changed all text files (scripts, configs, etc.) to reference the
new location.
3) Deleted the 1.3.2 launcher JAR file, and copied the 1.4.0.1
launcher JAR file into new location.
4) Ran the launcher JAR file with command "java -jar rundeck-
launcher-1.4.0.1.jar" from the new location.
5) Replaced my "admin.aclpolicy" file with the example file shown at
http://rundeck.org/docs/upgrading/index.html#example-admin.aclpolicy
6) Started rundeck and logged in as "admin" user.

org.codehaus.groovy.grails.web.servlet.mvc.exceptions.ControllerExecutionException:
Executing action [nodesFragment] of controller [FrameworkController]
caused exception: No such property: subject for class:
com.dtolabs.rundeck.core.authentication.NoAuthentication
at
org.codehaus.groovy.grails.web.servlet.mvc.SimpleGrailsControllerHelper.executeAction(SimpleGrailsControllerHelper.java:
250)
at
org.codehaus.groovy.grails.web.servlet.mvc.SimpleGrailsControllerHelper.handleURI(SimpleGrailsControllerHelper.java:
202)
at
org.codehaus.groovy.grails.web.servlet.mvc.SimpleGrailsControllerHelper.handleURI(SimpleGrailsControllerHelper.java:
137)
at
org.codehaus.groovy.grails.web.servlet.mvc.SimpleGrailsController.handleRequest(SimpleGrailsController.java:
88)
at
org.springframework.web.servlet.mvc.SimpleControllerHandlerAdapter.handle(SimpleControllerHandlerAdapter.java:
48)
at
org.codehaus.groovy.grails.web.servlet.GrailsDispatcherServlet.doDispatch(GrailsDispatcherServlet.java:
255)
at
org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:
716)
at
org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:
647)
at
org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:
563)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:
511)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1166)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:
70)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1157)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:
70)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1157)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:
70)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1157)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:
388)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:
216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:
182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:
765)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:
418)
at org.mortbay.jetty.servlet.Dispatcher.forward(Dispatcher.java:327)
at org.mortbay.jetty.servlet.Dispatcher.forward(Dispatcher.java:126)
at
org.codehaus.groovy.grails.web.util.WebUtils.forwardRequestForUrlMappingInfo(WebUtils.java:
301)
at
org.codehaus.groovy.grails.web.util.WebUtils.forwardRequestForUrlMappingInfo(WebUtils.java:
277)
at
org.codehaus.groovy.grails.web.util.WebUtils.forwardRequestForUrlMappingInfo(WebUtils.java:
269)
at
org.codehaus.groovy.grails.web.mapping.filter.UrlMappingsFilter.doFilterInternal(UrlMappingsFilter.java:
187)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:
76)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1157)
at
org.codehaus.groovy.grails.web.sitemesh.GrailsPageFilter.obtainContent(GrailsPageFilter.java:
249)
at
org.codehaus.groovy.grails.web.sitemesh.GrailsPageFilter.doFilter(GrailsPageFilter.java:
140)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1157)
at
org.codehaus.groovy.grails.web.servlet.mvc.GrailsWebRequestFilter.doFilterInternal(GrailsWebRequestFilter.java:
65)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:
76)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1157)
at
org.codehaus.groovy.grails.web.filters.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:
63)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:
76)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1157)
at
org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:
88)
at
org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:
76)
at
org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:
237)
at
org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:
167)
at org.mortbay.jetty.servlet.ServletHandler
$CachedChain.doFilter(ServletHandler.java:1157)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:
388)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:
216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:
182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:
765)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:
418)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:
152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:
536)
at org.mortbay.jetty.HttpConnection
$RequestHandler.content(HttpConnection.java:930)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:747)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:405)
at org.mortbay.jetty.bio.SocketConnector
$Connection.run(SocketConnector.java:228)
at org.mortbay.thread.QueuedThreadPool
$PoolThread.run(QueuedThreadPool.java:582)
Caused by: groovy.lang.MissingPropertyException: No such property:
subject for class:
com.dtolabs.rundeck.core.authentication.NoAuthentication
at
FrameworkService.authorizeProjectResourceAll(FrameworkService.groovy:
114)
at FrameworkService$authorizeProjectResourceAll.call(Unknown Source)
at FrameworkController$_closure4.doCall(FrameworkController.groovy:
315)
at FrameworkController$_closure4.call(FrameworkController.groovy)

[Greg Schueler]

Hi AC,

Good question, the simple answer is: remove the following directories and try it again: "server/exp", "server/lib", "tools"

I neglected to add that basic upgrade info, sorry. I will add some instructions and update the rundeck.org site

[AC]

Yep, that resolved the issue - thanks!

[AC]

Rundeck appears to be working great after the suggestion below, but I
just noticed something very concerning with the authorization layer.
My 1.3.2 installation was Active-Directory enabled, and this still
works fine with version 1.4.0.1. My admin.aclpolicy file is still the
same as the example shown at bottom of the upgrade documentation page,
and I'm using the 'out of the box' apitoken.aclpolicy file as well.
When I log in with my domain account, I seem to have the ability to
view job details, create new jobs, and execute commands/jobs, all
while not having any specific entries in the aclpolicy file allowing
this. I do not belong to any "admin" groups in my AD, so I'm not sure
why Rundeck is allowing me to do so much in the application. The only
thing that's failing is the actual execution of commands or jobs; at
which time I get a "No matched nodes" error. When this happens, the
Rundeck audit log gets some entries like the one shown below.
However, all actions leading up to this point do not generate anything
at all in the Rundeck audit log.

I believe right now anyone in our corporate AD could probably log into
the GUI and rummage around in my Rundeck installation, although I
haven't verified this yet. Is this normal behavior, given the
configuration described here? If this is because of the default
apitoken.aclpolicy, then how would I restrict it, such that access can
only be obtained by people who belong to a given AD group?

Group assignments as seen by Rundeck:
Application.SMS.Global.ComplianceReporting,
Application.GBIT.Rundeck.Admins, Application.Clearcase.Users,
Application.Citrix.Netscaler.POC.Users, Application.SRP.Contributors,
Application.Citrix.Concord.Users, Application.Stream.Users,
Application.GMS.Site.Singapore.Owner, Application.GMS.Site.Singapore,
Application.Clearcase.Admins, Application.GMS.CoreTeam.AppArch,
Application.GBIT.Monitoring, Application.BNX.Developers,
Application.BNX.Administrators, Application.BNX.Testers,
Application.YouSendIt.Users, Application.CNET.BAR.ITAdmins,
Application.Citrix.Shared.DPA.Users, Application.Stream.Admins,
Application.Internet.USA.Group1.Users, Application.PcAnywhere.Users,
Application.GMS.Clearcase.admin, Application.SRPUpgrade.WHQ.Users,
Application.GMS.CoreTeam.WebTeam, Application.CiscoWorks.Users,
Application.ConsumerNet.WHQ.Users, Application.BNX.Operations

Audit log entry generated when "No matched nodes" is shown in GUI:
2011-11-01 17:49:43,077 - Evaluating Decision for: res<osFamily:unix,
tags:acs, apache, atlassian, controltier, linux, prod, rundeck,
tomcat, tools, username:brandops, osArch:x86_64, osVersion:5.5,
description:Brand Tools Server PROD, nodename:nke-lnx-gbt-p002,
hostname:nke-lnx-gbt-p002, type:node, osName:Linux,
rundeck_server:true> subject<Username:acraw1
Group:Application.Citrix.Concord.Users
Group:Application.GMS.Clearcase.admin
Group:Application.BNX.Administrators
Group:Application.Citrix.Shared.DPA.Users
Group:Application.Citrix.Netscaler.POC.Users
Group:Application.ConsumerNet.WHQ.Users
Group:Application.GMS.CoreTeam.AppArch
Group:Application.CNET.BAR.ITAdmins
Group:Application.GMS.Site.Singapore
Group:Application.Internet.USA.Group1.Users
Group:Application.GMS.CoreTeam.WebTeam
Group:Application.Clearcase.Admins Group:Application.Stream.Users
Group:Application.YouSendIt.Users Group:Application.CiscoWorks.Users
Group:Application.SMS.Global.ComplianceReporting
Group:Application.Clearcase.Users Group:Application.GBIT.Monitoring
Group:Application.GBIT.Rundeck.Admins
Group:Application.GMS.Site.Singapore.Owner
Group:Application.BNX.Testers Group:Application.SRPUpgrade.WHQ.Users
Group:Application.PcAnywhere.Users Group:Application.BNX.Developers
Group:Application.Stream.Admins Group:Application.BNX.Operations
Group:Application.SRP.Contributors> action<run> env<http://dtolabs.com/
rundeck/env/project:Brand_Ops>: authorized: false: No context
matches subject or environment => REJECTED_NO_SUBJECT_OR_ENV_FOUND
(1ms): No context matches subject or environment =>
REJECTED_NO_SUBJECT_OR_ENV_FOUND

On Nov 1, 12:43 pm, Greg Schueler <g...@dtosolutions.com> wrote:

[Greg Schueler]

Hi AC,

are there any other *.aclpolicy files in the "etc" directory?

[AC]

No, just the initial two (admin.aclpolicy + apitoken.aclpolicy).

> ...
>
> read more »

[AC]

It looks like the logging level is set for the audit log within file
"server/exp/webapp/WEB-INF/classes/log4j.properties", but can you
confirm what setting would result in logging even successful actions
within Rundeck? Perhaps a higher logging level would show us
Rundeck's reasoning for letting my domain account have so much
access? The settings I found are shown below:

# Enable audit logging
log4j.logger.com.dtolabs.rundeck.core.authorization=info, audit
log4j.additivity.com.dtolabs.rundeck.core.authorization=false

> ...
>
> read more »

[Greg Schueler]

Hi AC,

I don't think changing the log4j will result in more output, it should be logging at INFO level all evaluations.

Can you verify that removing the aclpolicy files results in basically not being granted any access to things in the GUI?

[AC]

I renamed the policy files as shown below, and now the "admin" user
behaves just like my domain account. The admin user can view job
details, edit jobs, run commands, etc. - but the actual execution
fails with a "No matched nodes" error.

drwxr-xr-x 2 brandops users 4096 Nov 2 11:37 .
drwxr-xr-x 9 brandops users 4096 Nov 1 16:48 ..
-rw-rw-r-- 1 brandops users 1261 Nov 1 11:28
admin.aclpolicy_DEFAULT
-r--r--r-- 1 brandops users 937 Nov 1 17:07
apitoken.aclpolicy.DEFAULT
-rw-r--r-- 1 brandops users 414 Oct 24 18:47 console-
log4j.properties
-rw-r--r-- 1 brandops users 4705 Nov 1 10:55 framework.properties
-rw-r--r-- 1 brandops users 889 Oct 24 18:47 log4j.properties
-rw-r--r-- 1 brandops users 5053 Nov 1 11:07 preferences.properties
-rw-r--r-- 1 brandops users 781 Oct 26 12:45 profile
-rw-r--r-- 1 brandops users 567 Oct 24 18:47 project.properties
-rw-rw-r-- 1 brandops users 156735 Nov 1 11:43 stacktrace.log

> ...
>
> read more »

[Greg Schueler]

I think your upgrade/installation might not be correct then. are you sure you are using the 1.4.0.1 without any 1.3 code?

If you have *no* aclpolicy files in the etc/ dir, then rundeck 1.4 will not allow any actions (even though you may log in)

So I think you are somehow running 1.3 code

[AC]

Okay, I'll do some digging. Right off the bat, I found some 1.3 JAR
files in the "libext" directory (shown below). You may want to add
this directory to the list of things to clear out during an upgrade.
I'll let you know if I find other remnants as well.

brandops@gbt-p002:~$ ll /apps/rundeck/libext/
total 52
-rw-r--r-- 1 brandops users 10140 Oct 28 16:47 rundeck-script-
plugin-1.3.2.jar
-rw-rw-r-- 1 brandops users 13199 Nov 1 18:08 rundeck-script-
plugin-1.4.0.1.jar
-rw-r--r-- 1 brandops users 4592 Oct 28 16:47 rundeck-stub-
plugin-1.3.2.jar
-rw-rw-r-- 1 brandops users 6242 Nov 1 18:08 rundeck-stub-
plugin-1.4.0.1.jar

> ...
>
> read more »

[AC]

Those two JAR's in the "libext" folder were the only obvious remnants
I could find. I'm not familiar enough with Rundeck code to know if
there might be some non-obvious files elsewhere. I deleted those two
1.3 JAR files, then restarted Rundeck, but it's still behaving the
same way. Could it be something stored within Rundeck's database? I
removed the old 1.3.2 installation from disk (after backing it up), so
there's no way the 1.4.0 installation could be accessing 1.3 files
from the old path. Not sure what else to look at...?

> ...
>
> read more »

[Greg Schueler]

Can you try doing a clean install of 1.4.0.1, and applying your authentication and aclpolicies to that, just to verify that it works in a clean state?

[AC]

Confirmed - this same behavior occurs with a fresh installation of
v1.4.0.1. I created a brand new installation, created an initial
'test' project, and then verified that the "admin" user could execute
commands against the rundeck host (just to verify basic out of box
setup). I then changed the "server/config/jaas-loginmodule.conf" file
to look like below (although I'm obfuscating real values). Nothing
else was changed in the entire installation. When I restart Rundeck,
I can log in with my domain account, and can do almost anything within
the GUI. I actually created a new project, browsed the initial "test"
project created by "admin" user previously, etc. When I actually
execute a command, it fails with the "No matched nodes" error.

brandops@gbt-p002:/apps/rundeck/server/config$ cat jaas-
loginmodule.conf
RDpropertyfilelogin {
org.mortbay.jetty.plus.jaas.spi.PropertyFileLoginModule sufficient
debug="true"
file="/apps/rundeck/server/config/realm.properties";

com.dtolabs.rundeck.jetty.jaas.JettyCachingLdapLoginModule
sufficient
debug="true"
contextFactory="com.sun.jndi.ldap.LdapCtxFactory"
providerUrl="ldap://ldap.company.com:389"
bindDn="cn=adminuser,ou=All Users,dc=ad,dc=company,dc=com"
bindPassword="somepass"
authenticationMethod="simple"
forceBindingLogin="true"
userBaseDn="ou=All Users,dc=ad,dc=company,dc=com"
userRdnAttribute="cn"
userIdAttribute="sAMAccountName"
userPasswordAttribute="unicodePwd"
userObjectClass="user"
roleBaseDn="OU=Application,OU=Groups,DC=ad,DC=company,DC=com"
roleNameAttribute="cn"
roleMemberAttribute="member"
roleObjectClass="group"
cacheDurationMillis="300000"
reportStatistics="true";
};

> ...
>
> read more »

[AC]

New finding - I set the "server/config/jaas-loginmodule.conf" file
back to the default (out of box), and logged into Rundeck as the
predefined "user" ID. The same behavior occurs for this "user" ID, so
it's not related to the domain/ldap setup. The out of box policy
files only grant access to the "admin" group and the "api_token_group"
group, and the "user" ID does not appear to belong to either of these
groups. Unless it is getting assigned to the "api_token_group"
automatically upon GUI login?

> ...
>
> read more »

[Greg Schueler]

Hi AC, You're right, I'm seeing some strange behavior for out of the box "user" account. investigating...

[Greg Schueler]

Ok, after digging into this I think I've tracked down the cause. There are a few issues that contributed:

1. grails "filters" are applied in nondeterministic order, however we use them to set up the credentials to check later and depended on a certain ordering of them
2. the fallback authorization implementation is "NoAuthorization" class which just silently allows all authorization checks
3. if the credentials were not set up correctly (due to #1) then the fallback authorization implementation was used

So, long story short, I will have to produce another build, probably 1.4.0.2, which will add these changes:

* set up the correct filter ordering at startup
* change fallback authorization to deny all checks
* add audit logging to the fallback auth checks

the nondeterministic ordering of filters was really a "treat" to discover :-(