https://github.com/pki-io/build/pull/12 and some testing thoughts

[Athanasios Kostopoulos]

Hi all,
https://github.com/pki-io/build/pull/12 is ready for review and merging.

How that we are able to make .deb/.rpm packages how we can test them in a short amount of time? My idea is to fire up a few DO VMs, install the packages there and do some testing to see if they work.

I am all open to ideas

[Jon Bonazza]

Are you wanting to test the package itself (installation,  etc...) or the code inside the package?

--
You received this message because you are subscribed to the Google Groups "pki.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pki_io+un...@googlegroups.com.
To post to this group, send email to pki...@googlegroups.com.
Visit this group at http://groups.google.com/group/pki_io.
To view this discussion on the web, visit https://groups.google.com/d/msgid/pki_io/CA%2BaGpV9ge61q5ruJADfbLW2ikNrZB4rgah80bEPhHARUhjGi%2Bw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Fraser Scott]

Is rather use vagrant to avoid depending on cloud service. That way any one can check out the build repo, install vagrant and build every package with ease.

To view this discussion on the web, visit https://groups.google.com/d/msgid/pki_io/CAK0%3D6mN10C2ed6hbbTN4j1tCkoPeZxmjzA8md1HMwCZnsxPr7g%40mail.gmail.com.

[Athanasios Kostopoulos]

Hi all,

I verified that the generated packages install pki.io at /opt/pki.io/bin (both .rpm and .deb)

I want to make sure that the packages compiled work (e.g. when I type pki.io whatever, we get a correct result). Is there an automated way of doing this?

Regarding Vagrant vs DO, for me it is easier to spin up a few VMs for less than 1 hr and spend a few cents than go to the trouble of creating Vagrant VMs (lazy I know). I think both approaches are valid however and we should support them both.

Thoughts?

 

[Fraser Scott]

On 2 May 2015 at 20:17, Athanasios Kostopoulos <athanasios...@gmail.com> wrote:

Hi all,

I verified that the generated packages install pki.io at /opt/pki.io/bin (both .rpm and .deb)

Cool, is that a test?

 

I want to make sure that the packages compiled work (e.g. when I type pki.io whatever, we get a correct result). Is there an automated way of doing this?

The best way of testing correctness would be to run the bats tests from the admin project. Not sure the best way of implementing that though. It belongs in admin because it is testing the correctness of the code, regardless of packaging etc. But we also want to keep things DRY, so I guess we would have to check out admin and make bats tests use a specified binary. At the moment it uses

 CMD="$SOURCE_PATH/pki.io"

as specified it fixtures/basics.bash

We could change it to look for an env variable and use that if found.

 

Regarding Vagrant vs DO, for me it is easier to spin up a few VMs for less than 1 hr and spend a few cents than go to the trouble of creating Vagrant VMs (lazy I know). I think both approaches are valid however and we should support them both.

What do you mean with "creating Vagrant VMs"? The vagrant file in the build project uses the Bento boxes provided by Chef. I'd rust them as much as a DO OS install or any other cloud OS. The only pain is downloading the box for the first time.

Having said that, supporting both is a good approach, as long as we make it easy for people to use. There are a handful of things that piss me off as a sysadmin, and one of them is convoluted and undocumented build processes. One can build an entire Linux kernel with "make deb-pkg", and yet some projects make you jump through hoops just to build a 10k line tool.

[Jon Bonazza]

We already test the applications themselves,  though. What value is there in testing the applications when installed via a package? I'm not certain i understand what this is trying to accomplish.

[Fraser Scott]

It comes down to how the packages were built and packaged, but in theory it shouldn't make much difference. Conceptually though there is

1) I have written this code and tested that it works as expected

2) I have installed this package and it functions as expected

For 2 it is probably sufficient that there are some basics tests to check files are put where expected and with the correct permissions etc. Or for (1) you might turn off logging, but for (2) you might want to check that stuff is written to the log file correctly (mainly for a daemon). Maybe also check that it actually runs by running --version and checking the output. 

I also think there is some value in being able to say "Tested on X, Y, Z".

[Athanasios Kostopoulos]

Hi all,

thank you Fraser for elaborating my point. While I agree that it we should not expect much deviation, before release I want to be as sure as possible that our packages work as expected. The last thing I want is a bug report (if we are lucky!) along the lines "Hi, I installed pki.io and it fails with foo failed library dependency" from a user of a mainstream distribution.

I am open to discussing a testing approach for our packages - I agree with the points Fraser made.

There is a lot of value in being able to say "Tested on Centos6, Centos7, Debian Jessie, Ubuntu 14.04" etc. As a sidenote, after a quick demo to my colleagues at $DAYJOB, they also expressed interest but they asked if we have tested the output of our binary with the products whose logos they are using, so maybe we should make some quick tests in this direction.

I am on IRC, we can talk there as well ...

[Fraser Scott]

On Sunday, 3 May 2015 08:25:16 UTC+1, athanasioskostopoulos wrote:

There is a lot of value in being able to say "Tested on Centos6, Centos7, Debian Jessie, Ubuntu 14.04" etc. As a sidenote, after a quick demo to my colleagues at $DAYJOB, they also expressed interest but they asked if we have tested the output of our binary with the products whose logos they are using, so maybe we should make some quick tests in this direction.

As per the brief conversation on IRC, definitely a good idea. Could do it using Chef. 

[Fraser Scott]

Long term that is. In the short term a manual verification would be fine.