Those of you using Jenkins and Autopkg

[Gaute]

Did you try to set up mail notifications when Autopkg adds a new package to your Munki-repo? Trying to figure out if it is possible to use the Jenkins email-ext plugin for this.. 

Thanks,

Gaute

[Fridolin Koch]

Hello!

I use the email-ext-Plugin to do exactly that, I get an email with the console output of the Autopkg run. Works fine so far.

Just install the Plugin and then add the "email"-Task to your job.

Bye, Frido.

[Gaute]

Hello Fridolin - I suppose then you get the output of the run also if it says "Nothing downloaded, packaged or imported."?

Gaute

[Fridolin Koch]

Yes, I am still working on that, but there are ways through the email-ext-plugin to grep for certain terms or just email the last couple of lines.

Bye, Fridolin.

[tim]

Do you mean by using the pre-send script?

I just posted a Python check script I used to use with AutoPkg and Jenkins, using the ScriptTrigger plugin to decide whether to build (in this case a "normal" run with a recipe list) based on the output of a script.

https://gist.github.com/timsutton/7237350

It doesn't make any use of AutoPkg's preferences for known repos or override directories, but that's just an implementation choice on my part, as is the use of AutoPkg directly from Git. The actual code to wrap the command the plist output from --report-plist is quite short and (probably overly) simple.

A script run by ScriptTrigger runs before the rest of the build "environment" is created, so it's not without its complications depending on how your Jenkins job(s) are configured.

But, hopefully this might be a useful enough proof-of-concept for a few people, should you choose to set up Jenkins in this manner.

-Tim

[Fridolin Koch]

I just thought about using the truncate log functions of the email-ext plugin (which I can´t find at the moment, but that`s no wonder at 8 am on a Sunday...:-)

Will report back as soon as I have a solution.

Thanks for the check script, will look into that also.

Bye, Frido.

[Anthony Hunt]

Hi All,

I'm in the process of doing the exact same thing.

Tim:  Nice work on autopkg, and all the other bits you've been working on!

Just one thought.  Couldn't autopkg exit with a different error code if an update was needed? ie. autopkg run -c AdobeAcrobatProXUpdate.munki exits with 0 if no update or 1 if update available.  This could be caught by scripttrigger directly to then email update available; build; email completed.

Just a thought as essentially we would have just two similar lines to deal with: autopkg run -c AdobeAcrobatProXUpdate.munki & autopkg run AdobeAcrobatProXUpdate.munki

Obviously this is excluding the possibility of multiple input packages, but surely this could be handled the same way?  

Any thoughts on whether this would be easier? better?

Thanks for all your hard work.

Anthony.

[Gregory Neagle]

On Nov 20, 2013, at 11:26 AM, Anthony Hunt <ant....@gmail.com> wrote:

> Hi All,
>
> I'm in the process of doing the exact same thing.
>
> Tim: Nice work on autopkg, and all the other bits you've been working on!
>
> Just one thought. Couldn't autopkg exit with a different error code if an update was needed? ie. autopkg run -c AdobeAcrobatProXUpdate.munki exits with 0 if no update or 1 if update available. This could be caught by scripttrigger directly to then email update available; build; email completed.

Finding an update is _not_ an error. UNIX convention is that a successful tool exit is 0; anything else is an error. I don't think that convention should be violated in order to work within Jenkins' capabilities.

[Gregory Neagle]

autopkg does not exit with 1 if there is a new download/update, but Tim's wrapper script here does: https://gist.github.com/timsutton/7237350

Perhaps that's what you are referring to?

-Greg

On Nov 20, 2013, at 11:33 AM, Anthony Hunt <ant....@gmail.com> wrote:

ok... I may have already been pre-empted by this.  Just testing this does seem to give me an exit code of 1 if there is an update and 0 if there isn't.  

--
You received this message because you are subscribed to the Google Groups "autopkg-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to autopkg-discu...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Anthony Hunt]

Yeah... something weird happened and when I rechecked it didn't return 1.  So I deleted the post.

Yes, I've been looking into the script.  I'm having difficulties seeing how is check on a per package basis?  I'm guessing it doesn't and thus forces a complete update?  The beauty of autopkg doeng the check would be that you could get information on success and failure on a per package/build basis.  If this isn't already happening and I'm missing it.

BTW.  Nice talk greg.  

Thanks for all your hard work!

Anthony.

[Gregory Neagle]

Putting this back on list:

Certainly one can argue for multiple return codes for autopkg.

But if some are actually errors, and some aren't, it rapidly gets confusing.

-Greg

On Nov 20, 2013, at 11:44 AM, Anthony Hunt <ant....@gmail.com> wrote:

> I agree and disagree.
>
> If you take grep for instance, you could say that using the exit code for whether it has found a string is not an error? it's returning the equivalent of yes or no. I think (as a programmer) I should be flexible with this approach and returning an exit code for the purpose of not just passing an error is reasonable.
>
> Taking it further you could look at it like this: autopkg runs in check mode to check if there are no updates. It doesn't find an update so returns 0 exiting cleanly. When it does find an update it throws an error code essentially error'ing because the current version is not the most up to date.
>
> Anthony.

[Anthony Hunt]

I agree that it could be confusing, so lets treat it as an error and deal with it accordingly?

exit 0: everything works and is current

exit 1: recipe does not exist

exit 2: unable to connect/404

exit 3: blah, blah disk space

exit 4: timeout

exit 5: your version is not up to date

then additionally on the adding to munki:

exit 6: repo not found

exit 7: unable to write to repo

etc

etc

This would enable better error handling in a variety of situations (some in Jenkins, some not).  I know and understand that it actually needs someone with the skills to actually make it happen!  Which isn't me at this time!!! maybe soon though!? Time permitting to re-learn most of the languages my brain has unknowing chosen to forget.

I can't believe it's taken me so long to find munki and autopkg etc.  My life is already so much better!

Thank you all!

Anthony.

[Gregory Neagle]

So where in that list is: 

recipe executed successfully, but found nothing

recipe executed successfully and downloaded something new

recipe executed successfully, found something new, imported it into munki

And how do you make a meaningful code when a given autopkg execution can run multiple recipes?

-Greg

[Anthony Hunt]

I understand what your saying, but at that point your already second guessing what you've asked it to do and this is where logic paths have to take over.  We need to be able to tell autopkg exactly what we want it to do, which is what we already do.  We already have a check and a run flag.  

If we have passed it a standard or munki recipe running an "I am are the latest version of" check we are expecting it to return 0 if it is the latest version or an error code as above.

If we have passed it a standard recipe we are already expecting a run without -c to do a check and then a download if needed.  Therefore, we already know that a successful run returning 0 will have performed a check and will either have exited as it's up to date or will have exited because it will have downloaded successfully.  Any other exit code means as above.

If we have passed it a munki recipe we are already expecting a run without a -c to do a check and then a download if needed, and then an import of the (if needed) download into munki. Because we have already told it to this by giving it a munki recipe. Therefore, we already know that a successful run returning 0 will have performed a check and will either have exited as it's up to date or will have exited because it will have downloaded successfully and imported the package into munki successfully.  Any other code as above.

When run on multiple recipes the same logic applies.  If doing a check run against 5 recipes, and only one requires an update the same code should be thrown. When a standard run against 5 recipes is perfomed and one update needs to be done, the same logic applies.  We know what we are expecting the recipe to do as we passed it.  So at this point we are only expecting it to complete what we have asked and return 0 or to exit with code as above.

Sorry, maybe I'm missing something?  

Anthony.

[Gregory Neagle]

On Nov 21, 2013, at 2:22 AM, Anthony Hunt <ant....@gmail.com> wrote:

I understand what your saying, but at that point your already second guessing what you've asked it to do and this is where logic paths have to take over.  We need to be able to tell autopkg exactly what we want it to do, which is what we already do.  We already have a check and a run flag.  

If we have passed it a standard or munki recipe running an "I am are the latest version of" check we are expecting it to return 0 if it is the latest version or an error code as above.

If we have passed it a standard recipe we are already expecting a run without -c to do a check and then a download if needed.  Therefore, we already know that a successful run returning 0 will have performed a check and will either have exited as it's up to date or will have exited because it will have downloaded successfully.  Any other exit code means as above.

If we have passed it a munki recipe we are already expecting a run without a -c to do a check and then a download if needed, and then an import of the (if needed) download into munki. Because we have already told it to this by giving it a munki recipe. Therefore, we already know that a successful run returning 0 will have performed a check and will either have exited as it's up to date or will have exited because it will have downloaded successfully and imported the package into munki successfully.  Any other code as above.

When run on multiple recipes the same logic applies.  If doing a check run against 5 recipes, and only one requires an update the same code should be thrown. When a standard run against 5 recipes is perfomed and one update needs to be done, the same logic applies.  We know what we are expecting the recipe to do as we passed it.  So at this point we are only expecting it to complete what we have asked and return 0 or to exit with code as above.

Sorry, maybe I'm missing something?  

I'm definitely missing what the expected return is for:

recipe executed successfully, but found nothing
recipe executed successfully and downloaded something new
recipe executed successfully, found something new, imported it into munki

And more importantly, what the expected return is if:

One recipe downloads a new item, and another recipe has some sort of error.

If autopkg could run only a single recipe in a given execution we wouldn't have to worry about this.

-Greg

[Anthony Hunt]

On Thursday, 21 November 2013 14:04:11 UTC, Greg Neagle wrote:

I'm definitely missing what the expected return is for:

recipe executed successfully, but found nothing

It returns 0; It passed the first check in either check mode or run mode that "Is x package at the latest version?". Because at that point it has only done a check.

 

recipe executed successfully and downloaded something new

It returns 0; It passed the first check in run mode and either way, exited because it was either at the latest version or because it downloaded something (as we've told it to do by not running in check only mode).

recipe executed successfully, found something new, imported it into munki

It returns 0; It passed the first check in run mode and either way, exited because it was either at the latest version or because it downloaded something and imported into munki (as we've told it do by not running in check mode and by using a recipe with .munki as the extension)

Any other exit code means as the above.  Which means it has either failed to be the latest version or has failed with a communication error, repo error etc etc.

And more importantly, what the expected return is if:

One recipe downloads a new item, and another recipe has some sort of error.

Why does it need to return anything other than what it already would do for a single recipe?  You chain a number of recipies together.  If there is an update to do you on any of them fire the whole thing off again with the run flag (is that not what your Jenkins does).  If it fails, you check why it failed?  Don't forget the verbose output would still be there, so either if running it directly from the command prompt you still see the exact same output to why it failed.  And, furthermore, Jenkins captures this in it's log.  Any email report would contain this and you'd easily be able to see what and why it failed. You'd then be able to have Jenkins respond to the exit code in a number of ways depending on the code?

If it failed on recipe 3 of 5, what difference does the completion of recipe 1 or 2 have on the entire job?  Your emailed the console log (or see the red dot in jenkins).  Reading that you find that jobs 1 & 2 are ok, 3 failed on the url.  You fix the url and on the next scheduled run everything goes through.  It exits with 0 that all went well?

If autopkg could run only a single recipe in a given execution we wouldn't have to worry about this.

I don't see that it makes any difference. Although given that most people using jenkins are running only one recipe at a time (taken from your demo)? Maybe it should only parse one job at a time?

Sorry if I've caused an undue amount of upset regarding this.  I've not intended to and only meant to help in the development of what is already an extremely useful product which is also free!!! which is very more useful to tired admins :(.

Seriously, THANKS FOR ALL YOUR TIME, EFFORT AND HARDWORK!!!!

Anthony.

[Gregory Neagle]

On Nov 22, 2013, at 4:20 AM, Anthony Hunt <ant....@gmail.com> wrote:

On Thursday, 21 November 2013 14:04:11 UTC, Greg Neagle wrote:

I'm definitely missing what the expected return is for:

recipe executed successfully, but found nothing

It returns 0; It passed the first check in either check mode or run mode that "Is x package at the latest version?". Because at that point it has only done a check.

 

recipe executed successfully and downloaded something new

It returns 0; It passed the first check in run mode and either way, exited because it was either at the latest version or because it downloaded something (as we've told it to do by not running in check only mode).

recipe executed successfully, found something new, imported it into munki

It returns 0; It passed the first check in run mode and either way, exited because it was either at the latest version or because it downloaded something and imported into munki (as we've told it do by not running in check mode and by using a recipe with .munki as the extension)

Any other exit code means as the above.  Which means it has either failed to be the latest version or has failed with a communication error, repo error etc etc.

I am terribly confused about what you are asking for, then. Originally I thought you were advocating that autopkg exit with a non-zero code to indicate that it _had_ downloaded/packaged/imported something new. Now you are saying it should exit 0 in these cases, and exit with an error code if there is a problem. That is the current behavior, so I guess we can declare the problem non-existent and move on to other things.

-Greg

And more importantly, what the expected return is if:

One recipe downloads a new item, and another recipe has some sort of error.

Why does it need to return anything other than what it already would do for a single recipe?  You chain a number of recipies together.  If there is an update to do you on any of them fire the whole thing off again with the run flag (is that not what your Jenkins does).  If it fails, you check why it failed?  Don't forget the verbose output would still be there, so either if running it directly from the command prompt you still see the exact same output to why it failed.  And, furthermore, Jenkins captures this in it's log.  Any email report would contain this and you'd easily be able to see what and why it failed. You'd then be able to have Jenkins respond to the exit code in a number of ways depending on the code?

If it failed on recipe 3 of 5, what difference does the completion of recipe 1 or 2 have on the entire job?  Your emailed the console log (or see the red dot in jenkins).  Reading that you find that jobs 1 & 2 are ok, 3 failed on the url.  You fix the url and on the next scheduled run everything goes through.  It exits with 0 that all went well?

If autopkg could run only a single recipe in a given execution we wouldn't have to worry about this.

I don't see that it makes any difference. Although given that most people using jenkins are running only one recipe at a time (taken from your demo)? Maybe it should only parse one job at a time?

Sorry if I've caused an undue amount of upset regarding this.  I've not intended to and only meant to help in the development of what is already an extremely useful product which is also free!!! which is very more useful to tired admins :(.

Seriously, THANKS FOR ALL YOUR TIME, EFFORT AND HARDWORK!!!!

Anthony.

[Anthony Hunt]

I am terribly confused about what you are asking for, then. Originally I thought you were advocating that autopkg exit with a non-zero code to indicate that it _had_ downloaded/packaged/imported something new. Now you are saying it should exit 0 in these cases, and exit with an error code if there is a problem. That is the current behavior, so I guess we can declare the problem non-existent and move on to other things.

Yes, that's exactly what I was advocating, which I later went on (how I read it) that we could also add other exit codes to help determine the type of failure.  I'll try and rephrase.  Does below help?

autopkg run -c skype.download

exit 0: local version is the most up to date version.

exit 1: recipe does not exist

exit 2: unable to connect/404

exit 3: newer version available 

autopkg run skype.download

exit 0: local version is the most up to date version or successful download. 

exit 1: recipe does not exist

exit 2: unable to connect/404

exit 3: newer version available (not applicable as we have already asked it to download the latest version by not doing -c)

exit 4: blah, blah disk space

exit 5: timeout

autopkg run skype.munki

exit 0: local version is the most up to date version or successful download and import to munki. 

exit 1: recipe does not exist

exit 2: unable to connect/404

exit 3: newer version available (not applicable as we have already asked it to download the latest version by not doing -c)

exit 4: blah, blah disk space

exit 5: timeout

exit 6: repo not found

exit 7: unable to write to repo 

 

autopkg run -c Skype.munki CyberDuck.munki VLC.download

exit 0: local version is the most up to date version. 

exit 1: recipe does not exist

exit 2: unable to connect/404

exit 3: newer version available

autopkg run Skype.munki CyberDuck.munki VLC.download

exit 0: local version is the most up to date version or successful download and import to munki. 

exit 1: recipe does not exist

exit 2: unable to connect/404

exit 3: newer version available (not applicable as we have already asked it to download the latest version by not doing -c)

exit 4: blah, blah disk space

exit 5: timeout

exit 6: repo not found

exit 7: unable to write to repo 

Kind regards,

Anthony. 

[Gregory Neagle]

OK, that's clearer, but I think the result would still be terribly confusing (at least to me). See below.

On Nov 22, 2013, at 6:31 AM, Anthony Hunt <ant....@gmail.com> wrote:

I am terribly confused about what you are asking for, then. Originally I thought you were advocating that autopkg exit with a non-zero code to indicate that it _had_ downloaded/packaged/imported something new. Now you are saying it should exit 0 in these cases, and exit with an error code if there is a problem. That is the current behavior, so I guess we can declare the problem non-existent and move on to other things.

Yes, that's exactly what I was advocating, which I later went on (how I read it) that we could also add other exit codes to help determine the type of failure.  I'll try and rephrase.  Does below help?

autopkg run -c skype.download

exit 0: local version is the most up to date version.

exit 1: recipe does not exist

exit 2: unable to connect/404

exit 3: newer version available 

autopkg run skype.download

exit 0: local version is the most up to date version or successful download. 

exit 1: recipe does not exist

exit 2: unable to connect/404

exit 3: newer version available (not applicable as we have already asked it to download the latest version by not doing -c)

What does "not applicable" mean? It should not emit this error? It should, but the caller should ignore it?

exit 4: blah, blah disk space

exit 5: timeout

autopkg run skype.munki

exit 0: local version is the most up to date version or successful download and import to munki. 

exit 1: recipe does not exist

exit 2: unable to connect/404

exit 3: newer version available (not applicable as we have already asked it to download the latest version by not doing -c)

exit 4: blah, blah disk space

exit 5: timeout

exit 6: repo not found

exit 7: unable to write to repo 

 

autopkg run -c Skype.munki CyberDuck.munki VLC.download

exit 0: local version is the most up to date version. 

exit 1: recipe does not exist

exit 2: unable to connect/404

exit 3: newer version available

I don't understand this. What if Skype's local version is the most up-to-date version, CyberDuck's recipe does not exist, and VLC has a newer version available? These exit codes _might_ make sense for a single recipe; they seem to be useless for multiple recipes.

autopkg run Skype.munki CyberDuck.munki VLC.download

exit 0: local version is the most up to date version or successful download and import to munki. 

exit 1: recipe does not exist

exit 2: unable to connect/404

exit 3: newer version available (not applicable as we have already asked it to download the latest version by not doing -c)

exit 4: blah, blah disk space

exit 5: timeout

exit 6: repo not found

exit 7: unable to write to repo 

Same as above.

I think you are just going to have to ask for plist output and examine it for the things you want information on.

Kind regards,

Anthony. 

[Anthony Hunt]

 I'm just going to rename those cases just to ease talking about it.

Case 1

autopkg run -c skype.download

exit 0: local version is the most up to date version.

exit 1: recipe does not exist

exit 2: unable to connect/404

exit 3: newer version available 

Case 2 

autopkg run skype.download

exit 0: local version is the most up to date version or successful download. 

exit 1: recipe does not exist

exit 2: unable to connect/404

exit 3: newer version available (not applicable as we have already asked it to download the latest version by not doing -c)

What does "not applicable" mean? It should not emit this error? It should, but the caller should ignore it?

Correct.  You wouldn't get an exit 3 as you've already asked it to download the latest version.  So yes, this exit code wouldn't exist.  I just left it there for reference. 

exit 4: blah, blah disk space

exit 5: timeout

Case 3 

autopkg run skype.munki

exit 0: local version is the most up to date version or successful download and import to munki. 

exit 1: recipe does not exist

exit 2: unable to connect/404

exit 3: newer version available (not applicable as we have already asked it to download the latest version by not doing -c)

exit 4: blah, blah disk space

exit 5: timeout

exit 6: repo not found

exit 7: unable to write to repo 

 

Case 4 

autopkg run -c Skype.munki CyberDuck.munki VLC.download

exit 0: local version is the most up to date version. 

exit 1: recipe does not exist

exit 2: unable to connect/404

exit 3: newer version available 

I don't understand this. What if Skype's local version is the most up-to-date version, CyberDuck's recipe does not exist, and VLC has a newer version available? These exit codes _might_ make sense for a single recipe; they seem to be useless for multiple recipes.

True, at this point I can see that 3 possible exit codes are viable. In my opinion it should only return the most fatal exit code. We have a verbose log we can see, so on inspection we can clearly see why it went wrong. 

Skype's version being up to date is irrelevant as the check passed.

CyberDuck not existing, is that a problem? guess not as it doesn't affect the run or should we error and abort the rest of the run?.

VLC having an update, flag for update?

 

Case 5 

autopkg run Skype.munki CyberDuck.munki VLC.download

exit 0: local version is the most up to date version or successful download and import to munki. 

exit 1: recipe does not exist

exit 2: unable to connect/404

exit 3: newer version available (not applicable as we have already asked it to download the latest version by not doing -c)

exit 4: blah, blah disk space

exit 5: timeout

exit 6: repo not found

exit 7: unable to write to repo 

 

Same as above.

I think you are just going to have to ask for plist output and examine it for the things you want information on.

I see where your coming from.  I just think that the plist method makes it overly complicated.  Maybe autopkg should only accept single recipes at a time for exit code to be given?  I suppose I look at it in an old fashioned light where when you put something into a process you should get something meaningful back from the process. ie. at the moment you get nothing really useful other than an exit code of 1 or 0.  This is useless from any scripting process unless your reading the verbose output.  At least with defined exit codes for a single recipe you can deal with result of any number of failures via simple scripting.  ie. run the update or send a failure email to systems or send an email failure to build team.

Tell me if I'm wrong about Tim's script... But from how I read it it doesn't give responses based on individual packages?  It checks *all* packages from the recipes you give it.  Then downloads any updates, then triggers a build of *all* of them if it finds anything? So any output from here is also pretty useless as you'd need to read the returning log by hand to check each went in?

As in Case 4 above, what happens with his script?  Does this script (the entire thing) run for just one recipe at a time? How can it pass anything else useful about a list of recipes other than it ran and triggered a build?  Are we not back to the same point?

Maybe we need a different script to anaylise the plist that Tim's script generates via autopkg? So a check script runs after Tim's script (via jenkins) which calls more jenkins tasks based on the packages that need updating via autopkg?

Et voila?  A much longer set of scripts to essentially do what autopkg could do with a single recipe in 1 line?

Ok, so tell me if I'm reading this all wrong.  Please do.  I'm new to jenkins, autopkg and only a 5 month year old with munki.  But I have a single task that updates my autopkg recipes.  When this completes it fires off an autopkg task for each recipe that I'm interested in. One task for Skype, one task for VLC etc etc.  When all of these tasks has completed I have another task which is fired that rebuilds the catalogue.

Is this about right so far?  Currently I have to look at jenkins every day to see if something has gone wrong.  Which it did when Firefox got updated 2 days ago.  What I'd like is for me to be able to have jenkins email me when a package gets updated or email me when it fails.  For both I would like to know what package we're talking about.  So running these tasks individually makes a lot of sense.  Checking the exit code of the check script tells me a few things.  If there is an update, or if something erroneous went wrong and where it went wrong.  If *Every* package send me a failure email, I'm going to be looking pretty hard at our internet connection or the hard drive space! If just one fails I can look straight at it.

So, maybe I am missing something?

I'm so glad to be finished for the weekend, I need the rest! 

A.

[Timothy Sutton]

So far I'm of the opinion that, at least by default, AutoPkg should reserve nonzero exit codes for when AutoPkg itself has uncaught exceptions or errors. For example, an error exit code is raised when you do a 'repo-add' and it can't locate a Git binary, or you mis-use a command option. This is similar to how other utilities on UNIX and OS X behave.

By default, Jenkins actually will not proceed with build steps if they exit non-zero, which you're probably aware of by now. I'm sure there are ways around that, but they most likely involve either installing a new plugin (which might necessitate another plugin) or some other kind of hack, as this deviates from the idea of what Jenkins is typically used for. There are some things in Jenkins that can be surprisingly challenging to work around if doing non-standard "build software and test" tasks without getting your hands pretty dirty.

When Per wrote AutoPkg, he built in pretty robust error handling from the beginning, which is what allows AutoPkg to run and collect error info from failing recipes into its final output report (either what's printed to screen or with --report-plist, or in the output receipt plist written to disk).

I also think there _could_ be the argument to have an _option_ to exit with error codes instead, but what event "category" (new download, new import, recipe error, etc.) trumps another when multiple recipes are being run? Everyone who would use such an option would make different assumptions, need to consult yet more documentation that tries to explain the hierarchy of error code precedence, etc. I think the argument for optional nonzero exit codes makes much more sense when you can only run a single recipe.

You mentioned earlier: "You'd then be able to have Jenkins respond to the exit code in a number of ways depending on the code?"

Do you know how you would do this (I don't)? You can write more shell scripts to try to make Jenkins work exactly how you'd like it to, but if you want to do this in the context of using the ScriptTrigger plugin you'll still likely need to defer that logic further down the build step with some other kind of wrapper that parses something.. and your best bet then is the machine-parsable output of --report-plist or the receipt file. And, you're back full circle, with more scripts, parsing the output and deciding what to do based on it. The idea of "more wrapper scripts to do what AutoPkg could do in one line" might apply for this specific scenario of yours and how you'd imagine exit codes to work, but it will definitely cause confusion and isn't even easy to explain, as this discussion thread indicates.

This is all part of why there isn't one single "this is how you use AutoPkg with Jenkins" guide, even though some people have been great to document how they've set it up in their environment. There's also definitely more information that could be added to the report plist - for example, it might make sense (though would be a breaking change) to structure the output so that it can be keyed by, or at least include, the recipe identifier, making it easy to pick out which recipes have new downloads and run only these.

The script I've posted, as the documentation states, is a rough demonstration of how you could parse the report plist to do the dumb thing of continuing on with the build if there's a new download.

How you would then decide what recipes to run, and how to run them, is up to you. I would probably start by simply running the same recipe list without --check and not worry about the fact that it's re-running other recipes. This is why the Munki import processor is idempotent - you can re-run the recipes and not keep re-importing the items to the repo.

Jenkins is one tool for doing centralized automated tasks, but it's not the only one, and it is a CI environment after all, so it's heavily biased towards those types of workflows. BuildBot is another popular server app which, I gather, is much more flexible because it works more like a Python library and isn't configured via a webapp. So you might come closer to your ideal using that, but it certainly will require you to write more code. You can also simply write a LaunchDaemon and pipe the output to sendmail given when certain keywords are present in the console output.

In the end, I don't think you're going to get around needing to craft something that glues some pieces together to fit your needs. I think that's what we all do, because people's needs and expectations vary.

-Tim

[Anthony Hunt]

 Hi Tim,

Thank you for such an in depth and considered reply.

I've only started working with Jenkins since Greg's presentation (as is probably apparent from all the above!), so please don't feel offended by the comments I have made.  They are in no way intended to be critical of the work you have put into these developments.  I see you mention "some people have been great to document how they've set it up in their environment".  This is actually the part I have struggled most with and probably where my difficulties begin.  I feel I have been setting this up in line with what others have been doing.  I've search this group and the munki-dev group and not really found much in the way of other people set ups.

We have a really fast internet connection here so I shouldn't be too worried about the bandwidth used, but I suppose this is essentially always on my mind.  For me, running a full check followed by another full download seemed a bit wasteful.  That said, it probably doesn't use that much bandwidth.  I'm going to spend some more time over Christmas considering our best of scenario.  Maybe we don't need to know if something gets updated?

Any how,  if your States bound, happy Thanksgiving.  If not, like me, have a good weekend.

Anthony.

[Timothy Sutton]

On 2013-11-29, at 11:49 AM, Anthony Hunt wrote:

> Hi Tim,
>
> Thank you for such an in depth and considered reply.
>
> I've only started working with Jenkins since Greg's presentation (as is probably apparent from all the above!), so please don't feel offended by the comments I have made. They are in no way intended to be critical of the work you have put into these developments. I see you mention "some people have been great to document how they've set it up in their environment". This is actually the part I have struggled most with and probably where my difficulties begin. I feel I have been setting this up in line with what others have been doing. I've search this group and the munki-dev group and not really found much in the way of other people set ups.

No, I've also only seen a couple people document their particular Jenkins + AutoPkg setup, and besides my example of using the ScriptTrigger plugin, I haven't seen any other documentation for a setup that does periodic checks and runs a build (and sends an e-mail) only if there's a newly-imported item.

It's nontrivial partly because Jenkins was made to trigger builds on things like changes to a Git branch or watching for filesystem changes. But this is what setting up integration automation is - there is always glue required to make the thing work how you'd want, because the tools need to be general enough and make few assumptions in order to be useful to a large number of people.

> We have a really fast internet connection here so I shouldn't be too worried about the bandwidth used, but I suppose this is essentially always on my mind. For me, running a full check followed by another full download seemed a bit wasteful. That said, it probably doesn't use that much bandwidth. I'm going to spend some more time over Christmas considering our best of scenario. Maybe we don't need to know if something gets updated?

The beauty of the URLDownloader processor is that it won't re-download the file unless AutoPkg believes it's changed on the server, or unless you've manually cleared the cache.

The "work" the computer is doing in repeated full runs of Munki recipes is the additional steps following the EndOfCheckPhase processor, which is mostly mounting DMGs, calling makepkginfo, checking the Munki repo, possibly some pre-processing unpackaging/repacking steps, depending on the recipe.

The resource being consumed is your Mac's disk heads or SSD cells as it does repeated runs. Since it won't be re-importing the items into the repo, you won't be filling up space on the Munki repo either.

> Any how, if your States bound, happy Thanksgiving. If not, like me, have a good weekend.

Thanks - our Thanksgiving was last month!


-Tim

[Anthony Hunt]

Oh!!! sorry!!!  I'll never believe anything my Outlook diary says again... Although... that said... Microsoft... I'll stick with my Mac!

[Derak]

I accomplished this behavior by greping the output of a run for the "rebuilt" message. I call a build successful if something is added to munki and fail it otherwise. I then use the email-ext plugin to send an email only on successful builds.

Here is what my build command looks like:

https://gist.github.com/derak/2b5d68f745984df19329

I use the JOB_NAME variable that is identical to my recipe name.

[Childress, Matt]

I did something similar using email-ext:  email-ext is setup to send an e-mail on both success and failure status (determined by Jenkins) along with the log file and then if it's a successful build BUT it does nothing (no download or import) it cancels the send by setting cancel = true in the pre-send script (not the template).  Simple groovy script grep of the log file (had to be simple - I don't know java, groovy or python - I'm a hack)

 For the life of me I couldn't get setting cancel = true  to work in the email template, but the same exact code works beautifully in the pre-send. 

So you get a copy of the log file on a failure, and the log file on a successful import or download, and no email if the job build ran successfully but did nothing. 

I'll see about posting this tonight on the group and then getting something setup on GitHub.  It's pretty simple code but it wasn't easy to figure out how to stop spamming myself with successful "nothing done" emails due to being unable to find documentation on the cancel = true not working in the template.   

I'm a n00b with Jenkins/AutoPkg but the trip has been wonderful so far since others have blazed a pretty easy to follow trail : we now have an auto-updating Mac software repository share that also happens to single-source with our munki_repo/pkgs (that notifies when new stuff is download and imported or failed, that with Jenkins AD-auth and https is secure and has a clean and easy to use interface making it easy to see when something's gone wrong and diagnose :-)

M@

Sent from my iThneed

--

You received this message because you are subscribed to the Google Groups "autopkg-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to autopkg-discu...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Childress, Matt]

Ok, the specifics:

Our Setup:
We have a master Jenkins scheduled job that calls all of the individual autopkg software jobs (unscheduled) in Jenkins, every 3 hours M-F, 7am - 10pm.  Having individual small jobs makes it easy to tell which software download/pkg/munki import is broken/not working (FYI, this technique doesn't work with aamporter:  all your adobe plists need to be called at once to get --munkiimport to have the proper 'update_for' for updates that impact multiple adobe apps). 

The trick is that your last job of the bunch is the 'autopkg run recipe.munki.MakeCatalogs' (we also have that setup as a Jenkins job).  When you run an individual job by hand you either have to remember to do that manually or add it to each of the individual jobs post-build section.  Iif you don't have logic built to separate manual jobs from the scheduled/timer jobs (we don't yet), your timer job ends up running MakeCatalogs A LOT more than necessary.  If you fail to run the autopkg MakeCatalogs recipe manually when run a jenkins job outside of your timed master job you get a lot of __1, __2 duplication in your software repository (pkgs & pkgsinfo).  So far I've opted for the "make the computer do waaaay more work than it should need to" by putting it at the end of all the little jobs AND at the end of the big job.  Eventually I might figure out the logic of having a job run based on whether it was NOT called by the timer function.  But for now it works.

I like Jenkins as it gives both a simple and attractive high level overview of success/brokenness (good for demos to co-workers/boss:  sunny/blue = good, grey/cloudy = iffy, stormy/red = bad), scheduling and does this for both autopkg, aaimporter AND other scheduled jobs (we're looking at using it for a remote backup using rsync) -- you can see what broke, what the log file / command line output for the jobs are split out for each software recipe, and you can test/build it quickly.  And now e-mail notification ;-)

Setting up Email-ext:

Each individual small job (individual software recipe) under its Configure/Configuration page at the bottom is the Editable Email Notification and most are set to the defaults, with the exception of the Grey Advanced Settings... button:  this is set to Pre-send Script field = $DEFAULT_PRESEND_SCRIPT (which I'm pretty sure is the default) and the Triggers are set to Failure - Any, Send To Recipient List and an additional trigger Success Send To Recipient List.

Under the main Manage Jenkins -> Configure System page:

I set the System Admin e-mail address field to MacUpdates @ Servername (Jenkins) <my-email@address> as that is what appears in the FROM: field for notifications of new software imported/downloaded and failed jobs.  Under Extended E-mail Notification we like plain text for the Default Content Type, Default Recipients is set to a comma separated list of our mac admin team members, Default Subject is $PROJECT_NAME - Build # $BUILD_NUMBER - $BUILD_STATUS! (this may also be a default setting) and the Default Content field is set to ${SCRIPT, template="autopkg-munki-new-items.groovy"}  This script (autopkg-munki-new-items.groovy) lives in Jenkins/Home/email-templates and is attached below after the Default Pre-send Script.

The crux was figuring out that the e-mail template must be teamed up with a Default Pre-send Script as the cancel = true that is supposed to kill the sending of the e-mail that we asked to be triggered for a successful Jenkins job BUT that did no real work so does not need to report in.  For some reason it wasn't working from within the template.  Even so I left it in the template as that is the logical place for it, and later I may try to move it back there not realizing I've been down that road.

As mentioned before, this works but I don't claim to be a groovy java coder ;-)  And it would be nice to have something similar for aamporter... hmm...

M@

================ Default Pre-send Script ======================

// Load up the first 1000 lines of the log file into a variable
def log = build.getLog(1000)

// Let's setup a boolean of the result of searching for the string that appears
// in the log file when a job successfully completes but no work occurred.

def NothingDone = log =~ /Nothing downloaded, packaged or imported/

// And now let's test against that boolean and kill the e-mail (cancel = true) if it found the
// "Nothing downloaded, packaged or imported" string

if (NothingDone) {
        // Build is successful, but nothing imported... so no e-mail
        cancel=true;
}

// Else run the script in the template in the Default Content.


======== Jenkins/Home/email-templates/autopkg-munki-new-items.groovy (the Default Content script) ===========

GENERAL INFO
BUILD ${build.result}
Build URL: ${rooturl}${build.url}
Project: ${project.name}
Date of build: ${it.timestampString}
Build duration: ${build.durationString}
<%
import java.util.regex.*
def log = build.getLog(1000)
def NewItemsImportedSearch = log =~ /new items were imported/
def NewItemsDownloaded = log =~ /new items were downloaded/
def NothingDone = log =~ /Nothing downloaded, packaged or imported/

if (NothingDone) {
        // Build is successful, but nothing imported... so no e-mail
        cancel=true;
        // M@ 17Oct2014: THIS IS BEING HANDLED BY A PRE-SEND SCRIPT
        // as it seemed to do no good here :-(
       
%>

Nothing downloaded, packaged or imported... SO WHY ARE WE SEEING THIS E-MAIL?!?@#??%??!

<%
} else if (NewItemsImportedSearch) {
%>

NEW ITEM AVAILABLE IN MUNKI
A new version of ${project.name} was downloaded (autopkg: /Users/Shared/AutoPkg/Cache/${project.name} and imported into the munki repository (/Users/Shared/munki_repo/pkgs/).

<%
} else if (NewItemsDownloaded) {
%>

NEW ITEM DOWNLOADED TO AutoPkg Cache
A new version of ${project.name} was downloaded to the AutoPkg Cache -- (autopkg: /Users/Shared/AutoPkg/Cache/${project.name} BUT NOT imported into the munki repository.

<%
} else {
%>

FAILURE?
<%
}
%>

CONSOLE OUTPUT
<% log.each()
{ line -> %>
${line}
<% }
%>

---

From: Childress, Matt
Sent: Monday, October 20, 2014 9:18 PM
To: <autopkg...@googlegroups.com>
Subject: Re: [autopkg-discuss] Re: Those of you using Jenkins and Autopkg

[Matt Childress]

I did some work improving our Jenkins/AutoPkg e-mail notification scripts last night as the AutoPkg FAQ item #1, https://github.com/autopkg/autopkg/wiki/FAQ -- as the "Every time I run a recipe it downloads something even if it didn't change. Why?" was triggering an e-mail and driving my team and I nutz.  So I made it such that if you add a 'IS_TROUBLEMAKER' environmental variable to the Project Build Configuration page it turns off the 'on download but no import to munki repo' e-mail notification.

Hope it helps and comments/crtique on improving clarity in both code and README very much appreciated!

M@

https://github.com/childrss/jenkins-autopkg-email

[Tim Sutton]

Thanks for sharing!

The next version of AutoPkg, to be released soon, will have a setting that can relax the mechanism for detecting "new" items based on the server's HTTP headers and fall back to size only. The master branch of AutoPkg has this feature already, but I think the input variable name will change from what it is currently so that it easy for a user to override the behavior.

Tim

--