Organizing Playbooks

[rambius]

Hello,

We are adopting ansible and as a proof of concept I converted some tasks to ansible playbooks. For example, we are using Fisheye tool for code reviews and I created playbooks for upgrading it. I would like to have some feedback about them.

We have two instance of that tool - one for production and one for qa purposes such as testing upgrades. The upgrade procedure is

1) Export the data from the production instance
2) Import it in the qa instance
3) Upgrade the qa instance to a new version of the tool
4) Test the new version in qa with the production data
5) If tests are syccessful upgrade the production instance to the new version

While 4) involves mainly manual testing, the other points can be automated and I created two playbooks for them. The first one covers 1) and 2) - syncing qa instance with production data. I export production data in a compressed file with a command provided by the tool, then I move the file from the production box to the qa box and finally I import it to the qa instance using another command that comes from the tool:

fisheye-sync/site.yml
---

- name: Fetch Fisheye Contents
  hosts: fisheye
  tasks:
  - name: Export Fisheye
    command: "{{ fisheye_dir }}/bin/fisheyectl.sh backup --cache -f {{ remote_backup_file }}"
  - name: Fetch Fisheye Export
    fetch: src="{{ remote_backup_file }}" dest="{{ local_backup_file }}" flat=yes

- name: Import Fisheye Contents
  hosts: qafisheye
  vars_prompt:
    - name: dbpass
      prompt: "QA Fisheye's database password"
      private: yes
  tasks:
  - name: Copy Fisheye Export
    copy: src="{{ local_backup_file }}" dest="{{ remote_backup_file }}"
  - name: Check if Fisheye is Running
    command: pgrep -f fisheyeboot.jar
    register: is_fisheye_running
    ignore_errors: true
  - name: Fail if Fisheye is running
    fail: msg="Fisheye is still running"
    when: is_fisheye_running.rc == 0
  - name: Import Fisheye Contents
    command: "{{ fisheye_dir }}/bin/fisheyectl.sh restore --force --username fisheye --password {{ dbpass }} --jdbcurl {{ jdbc_url_qa }} --dbtype oracle --file {{ remote_backup_file }}"
  - name: Change Fisheye's URL in config files
    replace: backup=yes dest="{{ ansible_env.FISHEYE_INST }}/config.xml" regexp="fisheye\.company\.com" replace="qafisheye.company.com"

fisheye-sync/group_vars/all
---
remote_home: "{{ ansible_env.HOME }}"
fisheye_basedir: "/data/fisheye"
fisheye_dir: "{{ fisheye_basedir }}/fecru"
backup_file_name: "backup-fecru.zip"
local_backup_file: "fisheye-backups/{{ backup_file_name }}"
remote_backup_file: "{{ remote_home }}/fisheye-backups/{{ backup_file_name }}"
jdbc_url_qa: "jdbc:oracle:thin:@dbs01:1521:dev"

I call this playbook with

$ ansible-playbook -i /data/ansible/ansible_hosts site.yml

and after it completes the qa instance is synced with production data. Then I upgrade the qa instance. The upgrade consists of installing a suitable JDK, backing up the existing installation, fetching the new version from the internet and extracting it:

fisheye-updgrade/site.yml
---
- name: Install JDK
  hosts: fisheyeservers
  tasks:
  - name: Extract JDK
    unarchive: src="{{ jdk_archive }}" dest="{{ remote_home }}" creates="{{ jdk_home }}"

- name: Backup Fisheye Installation
  hosts: fisheyeservers
  tasks:
  - name: Get Current Fisheye Version
    command: "sed -n -e 's/ *<title>FishEye \\(.*\\)<\\/title>/\\1/p' {{ fisheye_dir }}/README.html"
    register: fisheye_version
  - name: Backup Fisheye
    command: "tar -cvzf {{ fisheye_inst_backup }} {{ fisheye_dir }}"
    args:
      creates: "{{ fisheye_inst_backup }}"

- name: Install New Fisheye
  hosts: fisheyeservers
  tasks:
  - name: Verify New Version is Set
    fail: msg="Please provide fisheye_version_new var in --extra-vars option"
    when: fisheye_version_new is not defined
  - name: Fetch Fisheye Distribution
    local_action: get_url url="https://www.atlassian.com/software/fisheye/downloads/binary/fisheye-{{ fisheye_version_new }}".zip dest="{{ local_home }}/fisheye-{{ fisheye_version_new }}.zip"
  - name: Extract Fisheye Distribution
    unarchive: src="{{ remote_home }}/fisheye-{{ fisheye_version_new }}.zip" dest="{{ fisheye_basedir }}" creates="{{ fisheye_basedir }}/fecru-{{ fisheye_version_new }}"

- name: Repoint Fisheye Symbolic Link
  hosts: fisheyeservers
  tasks:
  - name: Check if Fisheye is Running
    command: pgrep -f fisheyeboot.jar
    register: is_fisheye_running
    ignore_errors: true
  - name: Fail if Fisheye is running
    fail: msg="Fisheye is still running"
    when: is_fisheye_running.rc == 0
  - name: Create Symlink
    file: src="{{ fisheye_basedir }}/fecru-{{ fisheye_version_new }}" dest="{{ fisheye_dir }}" state="link" force="yes"

fisheye-updgrade/group_vars/all
---
local_home: "{{ lookup('env','HOME') }}"
remote_home: "{{ ansible_env.HOME }}"
jdk_version: "8"
jdk_update: "45"
jdk_archive: "jdk-{{ jdk_version }}u{{ jdk_update }}-linux-x64.tar.gz"
jdk_home: "{{ remote_home }}/jdk1.{{ jdk_version }}.0_{{ jdk_update }}"
fisheye_basedir: "/data/fisheye"
fisheye_dir: "{{ fisheye_basedir }}/fecru"
fisheye_backup_dir: "{{ remote_home }}/fisheye-backups"
fisheye_inst_backup: "{{ fisheye_backup_dir }}/fecru-{{ fisheye_version.stdout }}.tar.gz"

To upgrade the qa instance I call this playbook with

$ ansible-playbook -i /data/ansible/ansible_hosts -l qafisheye site.yml --extra-vars "fisheye_version_new=3.7.1"


While these playbooks work, I think that they may be organized better, more specifically:

-) Some variables are duplicated in the playbooks' group_vars/all files. How to remove that duplication?
-) Some tasks are duplicated in playbooks's site.yml files - the ones that check if fisheye is running. How to remove that duplication?
-) In fisheye-sync playbook I export production data and import it to the qa instance. I think this is not flexible enough. I may want to export production data not only during an upgrade. I may also want to not import it to the qa instance but for back up purposes. How can I make it more flexible?

I know this is a long post and I will be very grateful for any feedback you give me.

Thank you in advance.

Regards
Rambius