Rundeck in Kubernetes: Automatic configuration (Rundeck as Code)

973 views
Skip to first unread message

Werner Dijkerman

unread,
May 12, 2021, 10:37:34 AM5/12/21
to rundeck-discuss
Hi,

I am building Docker images on top of the "rundeck/rundeck:x.x.xx" image and the goal is to completely automate the installation/configuration of the Rundeck instance once it is deployed on a Kubernetes Cluster. So this Docker image contains the Rundeck jobs (and scripts) and the idea is that once Rundeck is running it will do some basic configuration automatically for Rundeck (Rundeck as Code). Short summary:
  1. Start Rundeck, with environment settings provided by Rundeck;
  2. In the background a script will be started that does the following:
    1. Waiting to rundeck be completely running;
    2. Check if project exist and if not, import project with "rd";
  3. Probably some other configuration things that I have not thought about yet;
  4. Users can make use of the instance by starting/running their own jobs;
So I probably want to have the following in the Docker image:
  1.  A basic user/acl policy (or hardcoded api token) with only access to certain health/metrics endpoints to be used by Rundeck for monitoring;
  2. A basic user/acl policy (or hardcoded api token) with only access to validate the existance of projects and if they don't exist, create one via the "rd" tool;
No further configuration inside the Rundeck instance or making changes in the UI to get something done should be done (I really hate doing thing manually). But I have some questions on how to proceed:

  1. Is it possible to both use "local" users and users from LDAP? As I have a user used by monitoring purposes and the actual users that make use of Rundeck for starting jobs etc coming from LDAP.
  2. And if so, it is possible to have a file present in the Docker image that contains a predefined username/password/authtoken with policy, that also can be configured as healthchecks in the Kubernetes Deployment file?
Can you advice where to look and how this can be done? I can not find to see answers or very old ones (Like suggesting to "use Zabbix for monitoring", which I do not want and thats not being asked ;-).)

Sidenote:
I am fully into Automation (as code), Kubernetes and Docker, but I lack Rundeck knowledge/experience.

Thanks in advance.
May the force be with you

Kind regards,
Werner

Bruce Becker

unread,
May 12, 2021, 11:32:55 AM5/12/21
to rundeck-discuss
Hi Werner,

I have a similar problem to you and have gone down a similar path, but perhaps with a different toolset.
We deploy our rundeck into kubernetes clusters, and configure most of the settings using the built-in remco templates. This goes for the application configuration, and some AuthZ/N, but not keys, storage, projects, nodes or jobs.

I have found that using the Terraform provider for rundeck works very well - have you had a look at that?

The workflow is more or less as follows:

  1. build docker image (from packer definition) this includes things are immutable:
    1. base libraries
    2. custom remco templates
    3. realm properties
  2. Build helm chart with environment variables for remco to consume when the pod is restarted
  3. -> continuously deploy these changes as the packer definition or deployment definition changes
In a separate loop:

  1.  Define Rundeck internal resources (projects, storage, keys, jobs, nodes/resources) in terraform
  2. Continuously apply these changes against the deployed rundeck instance as the need arises.
A typical case is when the need for a new job arises... we have a chat with the team which needs the job to define which resources and workflow it should use, then write a terraform definition for that. This is tested by applying it against a test instance, then if that passes, it is applied against the prod instance.

Doing things like this alllows us to factorise out infrastructure-related tasks and user requests.

Hope that helps,
Bruce

kulmam92

unread,
Aug 10, 2021, 11:06:47 AM8/10/21
to rundeck-discuss
Hi Bruce,

Do you mind sharing the image build script and helm chart? I'm trying to do the same as you. The sample script will be a big help in shortening the project timeline.

Best,
Yong

Sean

unread,
Aug 10, 2021, 11:49:12 PM8/10/21
to rundeck-discuss
Hello Bruce, I’m too looking for helm chart and image for rundeck deployment on my kubernetes cluster. I tried helm chart from incubator artifactory repo but no luck as my pods were falling in liveliness probe. Could you please help here ?


Thanks
Sean

Bruce Becker

unread,
Aug 11, 2021, 1:29:27 AM8/11/21
to rundeck-discuss
Hi Yong,

I can share parts of the packer template we use to provision the image:

# Packer 1.7 template for the Rundeck controller image.
variable "upstream_version" {
type = string
description = "Tag of the upstream rundeck image used to build this image."
default = "3.4.0"
}

variable "cli_version" {
type = string
description = "Version of the rundeck CLI to install"
default = "1.3.4"
}

variable "version" {
type = string
default = "${env("VERSION")}"
}

source "docker" "rundeck" {
exec_user = "rundeck"
changes = ["ENTRYPOINT /tini -- docker-lib/entry.sh"]
commit = true
image = "rundeck/rundeck:${var.upstream_version}"
}

build {
sources = ["source.docker.rundeck"]

# Add basic tools needed to set up repos
provisioner "shell" {
inline = [
"sudo apt-get update -qqq",
"sudo apt-get install -qqq apt-utils software-properties-common"
]
}

# Set Timezone
provisioner "shell" {
environment_vars = [ "TZ=Europe/Rome", "DEBIAN_FRONTEND=noninteractive" ]
inline = [
"sudo ln -s /usr/share/zoneinfo/$TZ /etc/localtime",
"sudo bash -c 'echo $TZ > /etc/timezone'",
"sudo apt-get install tzdata"
]
}
# Configure tool repositories for HashiCorp, Microsoft and Rundeck.
provisioner "shell" {
inline = [
# Add Hashicorp GPG key
"curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -",
# Add Hashicorp repository
"sudo apt-add-repository \"deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main\"",
# Add microsoft repo
"curl -fsSL https://packages.microsoft.com/config/ubuntu/$(lsb_release -rs)/packages-microsoft-prod.deb > packages-microsoft-prod.deb",
"sudo dpkg -i packages-microsoft-prod.deb",
"sudo add-apt-repository universe",
"curl -sSL https://github.com/rundeck/rundeck-cli/releases/download/v${var.cli_version}/rundeck-cli_${var.cli_version}-1_all.deb > /home/rundeck/rundeck-cli.deb",
"sudo dpkg -i /home/rundeck/rundeck-cli.deb"
]
}

# Add tools
provisioner "shell" {
# install packages
inline = [
"sudo apt-get update -qqq",
"sudo apt-get -qqq install bash tree unzip vim python3 git python3-pip python3-setuptools python3-virtualenv virtualenv coreutils binutils bind9-host apt-transport-https inetutils-ping inetutils-traceroute jq telnet bind9utils dnsutils ldap-utils lsb-core lsb-security lsb-base lsb powershell vault terraform",
"virtualenv --python=python3 $HOME/tools/virtualenv",
"source $HOME/tools/virtualenv/bin/activate ; pip install --upgrade ansible==2.9.8 kubernetes boto3 awscli boto"
]
}

# Add rundeck plugins
provisioner "shell" {
inline = [
"curl -sSL https://upload.wikimedia.org/wikipedia/commons/c/cc/Uefa_2013.png > /home/rundeck/user-assets/uefa.png",
"curl -sSL https://github.com/rundeck-plugins/slack-incoming-webhook-plugin/releases/download/v1.2.5/slack-incoming-webhook-plugin-1.2.5.jar > $HOME/libext/slack-incoming-webhook-plugin-1.2.5.jar",
]
}

# Add our custom realm properties adding users to groups
provisioner "file" {
source = "realm.properties"
destination = "/home/rundeck/server/config/realm.properties"
}

# Custom remco templates
provisioner "file" {
source = "remco/templates/"
destination = "/etc/remco/templates"
}

# Add our custom startup file
provisioner "file" {
source = "entry.sh"
destination = "/home/rundeck/docker-lib/entry.sh"
}

You can see the last few steps are adding the custom realm properties, remco templates and Docker entrypoint script. There are a few internal modifications there such as adding APM to rundeck at startup.

Hope this helps.
Bruce

Bruce Becker

unread,
Aug 11, 2021, 1:32:56 AM8/11/21
to rundeck-discuss
Hi Sean

I am using the helm chart from the incubator repo too, it has served me ok. All I can say about the liveness probe is that it can take a very long time to satisfy, even around 5-10 minutes. I had to tune the grace period and probe intervals accordingly.

The liveness probe was looking on /login.

I wonder if this is your case too?

Thanks,
Bruce

Sean

unread,
Aug 11, 2021, 3:16:08 AM8/11/21
to rundeck-discuss

Thanks Bruce for your reply, yeah I added grace period for 240 sec - 4 mins. I will change it to 5-10 mins as you suggested. can we use latest rundeck image in deployment.yaml file ?

Thanks 
Sean

Sean

unread,
Aug 26, 2021, 8:04:04 AM8/26/21
to rundeck-discuss
Hi Bruce,

Incubator chart not working for me, modified packer(below) file. Can build new image locally, but failing in running the image. is it possible for you to share the helm chart or values.yaml file you modified ? 

Thanks
Sean

variable "upstream_version" {
type = string
description = "Rundeck Image"
default = "3.4.3"
}

source "docker" "rundeck" {
image = "rundeck/rundeck:${var.upstream_version}"
commit = true
}

build {
name = "rundeck"
sources = ["source.docker.rundeck"]

provisioner "shell" {
execute_command = "{{.Vars}} sudo -E -S bash '{{.Path}}'"
inline = [
"echo Hello Rundeck",
"sudo -H apt-get update -qqq",
"sudo -H apt-get -qqq install bash vim python3 git python3-pip python3-setuptools coreutils apt-transport-https",
"sudo -H pip3 install virtualenv",
"virtualenv --python=python3 $HOME/tools/virtualenv",
"source $HOME/tools/virtualenv/bin/activate ; pip3 install --upgrade ansible==2.9.8 kubernetes boto3 awscli boto",
"echo check Python version",
"python --version",
"rm -rf /tmp"
]
}

post-processor "docker-tag" {
repository = "rundeck/rundeck"
tags = ["new"]
only = ["docker.rundeck"]
}

}

Reply all
Reply to author
Forward
0 new messages