New Module Idea - Version

[Patrick Heeney]

I have been working with ansible for a few weeks now and I thought of a useful module idea that I really am finding myself needing a lot. I am posting here to open up a discussion in hopes of getting it implemented. I don't know python at the moment so I am not in a position to implement this. 

The goal of the module would be to parse version strings of different services (maybe files?) and break them into usable chunks for conditionals. I am finding myself needing this a lot since I need to put different configuration files in place depending on what version is running. For example if the playbook is running on ubuntu 10.04 I need mysql 5.1 configuration entries whereas ubuntu 12.04 I need mysql 5.5. 

Here is some ideas of how it would look:

- version: name=mysql

  register: mysql_version

- debug: var=mysql_version

mysql_version: {

  version: "5.5.34"

  major: 5

  minor: 5

  patch: 34

}

Which would be parsed from: mysql -v: "Server version: 5.5.34-0ubuntu0.12.04.1 (Ubuntu)"

Another example:

- version: name=apache2

  register: apache_version

- debug: var=apache_version

apache_version: {

  version: "2.2.22"

  major: 2

  minor: 2

  patch: 22

}

Which would be parsed from apache2 -v: "Server version: Apache/2.2.22 (Ubuntu)"

If the service is not running or not found it could just return blank.

Right now I only need it for services. In order to implement it, it can maybe regex search search http://stackoverflow.com/questions/82064/a-regex-for-version-number-parsing ?. 

It could potentially be expanded to search files such as: version: path=/etc/lsb-release register: distro_version which would return version: 12.04, major 12, minor 4, patch ''. Optionally it could have the regex as an argument and perhaps a grep to narrow it down if the file has a lot of numbers or different format:

- version: path=/etc/lsb-release line=RELEASE regexp='^(\d+\\.)?(\d+\\.)?(\\*|\d+)$'

Which could parse cat /etc/lsb-release | grep RELEASE

What do you think?

[Michael DeHaan]

So we do have a bit of a strategic plan regarding modules to store inventory type information, in particular with fact caching, so I have preferences on where this goes.

Namely, I want things to record the package versions of all installed services and not have to rely on a register.

Once this happens, something like this should be easily possible:

package_information["httpd"]["version"]

etc

What you are asking to do regarding splitting things up, however, might be difficult -- if not everything follows that exact format.

Something like a lookup plugin that uses Python's "LooseVersion" support to compare two different versions might be better than trying to split them out and do comparisons peacemeal.

http://epydoc.sourceforge.net/stdlib/distutils.version.LooseVersion-class.html

Hypothethically assume the following might exist:

{{ version_compare(alpha, beta) }}

which should return -1, 0, 1 depending on order.

Perhaps this should be a Jinja2 filter, more like this:

{{ alpha | version_compare(beta) }}

In either case, the idea of sampling the installed versions seems to require fact caching (this release) to really be thought out correctly first, and I'd like to keep the numerical comparison filter/function seperate -- and I think that avoids the need to name each part of the version string.

(This type of discussion is probably a better fit for ansible-devel, FWIW)

--
You received this message because you are subscribed to the Google Groups "Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

Michael DeHaan <mic...@ansibleworks.com>
CTO, AnsibleWorks, Inc.
http://www.ansibleworks.com/

[Patrick Heeney]

This sounds great. The only issue I have however is the fact that sometimes my roles need to access versions that other roles may not setup. For example if I install ubuntu and it comes with apache, will it automatically lookup the facts of the apache version when running the playbook even though I didn't install it? If I uninstall apache in a role and install nginx for example then I assume it would add the nginx facts and remove the apache ones? 

I just want to make sure that in any use case I can access the exact version of different services that are installed. Mainly right now due to the need of different configurations for different major versions. 

I think the plan for version compare sounds good. Hypothetically if version_compare compares two depending on the order, we would need another one to return 1 or 0 based on a condition or can it be handled by the same thing?

- template: src=apache/5.1/httpd.conf dest=/etc/apache2/httpd.conf

- when: version_compare(package_information["httpd"]["version"], '>=2.0,<=2.4') or version_compare(version, '2.*')

or

- when: package_information["httpd"]["version"]|float >= 2.0 or version|match('2.*')

I get what your saying about versions being completely different, but perhaps this function could be a regex match or strip alpha chars so you can test integers when you know what the version "should" be. 

I didn't know about ansible-devel. I will have to make a note to post there next time. It's great you have a plan for this. 1.5 is the one that will include fact caching?

--
You received this message because you are subscribed to a topic in the Google Groups "Ansible Project" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ansible-project/P_Rp4fVJYHk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ansible-proje...@googlegroups.com.

[Michael DeHaan]

I'm suggesting we'll create a module to interrogate all package versions, not just ones that you've setup in the role, so I don't think this precludes the other.

I'd probably also suggest (if you have cycles) attempting version_compare as a filter plugin.   It seems to fit in most logically and lookup plugins are a little awkard with multiple parameters, and it's not really querying another datasource anyway.

I'm not sure if I understand the question about the 0's and 1's, but I'm imagining this:

- debug: msg="packages match"

  when: current|version_compare(future) == 0

- debug: msg="there's no upgrade coming"

  when: current|version_compare(future) != 1

I would highly suggest against trying to compare package versions with regexes, both are tricksey.

And yes, fact caching is going into 1.5.   This doesn't really require that, we could totally make a package_inventory type module now, but I was trying to limit myself with the numbers of open design discussions open at one time and I'm a bit unready to specify what form these should take just yet for similar reasons.

That being said, nothing preventing you from executing yum/apt/other as needed (or having your own module) and nothing preventing a version_compare being implemented now either.

[Patrick Heeney]

The module interrogation sounds awesome. The question about the 0s and 1s stems from the fact that only certain configuration should be applied when specific versions are met. Without regex or breaking the version into integers I don't know how you would compare specific versions. Heres some examples:

- debug: msg="this should run for apache 2.4.1 to 2.4.100"

  when: apache_version|version_compare('2.4.*') == 1

- debug: msg="this should NOT run for apache 2.4.1 to 2.4.100"

  when: apache_version|version_compare('2.4.*') == -1

Without the above and I wanted to run for specific versions it seems like it would end up like this:

- debug: msg="this should run for apache 2.2.1 to 2.4.100"

  when: apache_version|version_compare('2.2.1') == 1 and

    apache_version|version_compare('2.4.100') == 1 and .... hundreds of times

If we can only compare one number to another and have it return yes, same, no without a way to do regex or integer comparisons then the conditionals are going to get crazy to account for all the variations. If instead I can have another function that returns yes this number matches this group, or not it doesnt is what I meant with 0 and 1. 

In my specific use case I need to add a configuration file for 5.1.X only. I need to apply another specific configuration file for 5.5.X only. Without accounting for the X I either have to add a conditional for every version, or rely on regex when I know the follow sem ver so it should always be major.minor.patch.

Whatever you decide, that is what I am am hoping to solve with my original post. Right now I am running mysql -v | grep version register: mysql_version and running string comparison like mysql_version.stdout in ['5.1'] and mysql_verison.stdout in ['5.5']. It works but its pretty hacky.

[Michael DeHaan]

Ah, so in your case, you want to do things like get the minor version out and so forth.

This seems to imply it would be best to add filters for things like major and minor version in your case:

when: apache_version|major ...

etc

OTOH, my philosophy is usually to maintain a yum mirror of exactly what I want on all of my systems, and then have the packages do state=latest all the time, rather than have mixed minor versions but consistent major versions throughout my environment.

Thus, if a security update for Apache comes out, promote that package to your yum/apt mirror, etc...

IIRC (as I don't use this), the yum module also supports >= math if you want too though, like: yum: name=foo>1.2 

The apt module does not.

[CS]

FWIW, on a related note to managing packages and versions...

> OTOH, my philosophy is usually to maintain a yum mirror of exactly what I want on all of my systems, and then have the packages do state=latest all the time, rather than have mixed minor versions but consistent major versions throughout my environment.

In our case our yum repos get updated from CI, so state=latest wouldn't work for us because we need to be able to install specific builds and pin environments to them. Additionally we need to be able to roll back builds, which requires some hackery...

The problem with rollbacks is that state=installed is technically an upgrade command, so specifying a previous version of an RPM actually causes it to do nothing and return success. So we do a string compare on the RPM name to see if it differs and then remove the old RPM before installing the desired one, which works around the issue.

Not sure if it would make more sense to make the yum module smarter, so you could just say state=install-abs-version (or something) and it would determine if it's an upgrade or a downgrade.  Or just add a state=downgrade option to the yum module and use the version_compare() filter suggested to determine whether to call state=downgrade or state=installed. Any thoughts?

[Michael DeHaan]

What I was saying above:

yum: name=foo-1.2.3

installs version 1.2.3 from the yum repo.

It supports yum version specifiers in the name.

We do not need extra states to do this, in other words.

If we did, regardless, we'd just do state=version and would not want to specify upgrade/downgrade, as that presumes knowledge of the previous state of the system.

[C. S.]

The name specifier only works on upgrades though, not downgrades…

e.g. This would succeed, but you’d still have 1.2.3 installed instead of 1.2.2.

yum: name=foo-1.2.3

yum: name=foo-1.2.2

[Michael DeHaan]

Tested?

[C. S.]

Yes. It’s the documented behavior of yum actually, “install” means “upgrade to”, and the ansible module extends that behavior to the playbooks. 

The yum module also mentions it being an upgrade command as well.

e.g. line 103 [1]:

- name: install the latest version of Apche from the testing repo
  yum: name=httpd enablerepo=testing state=installed

It would be nice if it worked as you’d expect, but I guess yum historically an "update manager", so the commands are interpreted with that understanding, and changing the module at this point it might cause people’s playbooks to downgrade something unexpectedly… maybe just starting a new module all together is another option… 

[1] https://github.com/ansible/ansible/blob/devel/library/packaging/yum#L103

[Michael DeHaan]

Starting a new module is not acceptable, but adding state=version semantics (which did not exist before), I'd be fine with.

[Michael DeHaan]

Perhaps a better way to do this would be "version=" as a new (optional) parameter with a note that it can be expressed in the package name as well, and then the apt module could also (maybe) follow suit.

[C. S.]

Makes sense. As I think about it, the likelihood of someone having a package accidentally downgraded due to the behavior change would be a bit of an odd case anyway since they would have to had specified a version in the name= and then upgraded the package outside of Ansible.