Recipe auditing, security and maintenance - Seal of approval

[Erik]

Lately I've spent a considerable amount of time auditing autopkg recipes and I have found a somewhat alarming issue: 1 in every 4 recipes are missing code signature verification or SSL downloads. While my scope of recipes have been limited to about 45 different software packages (and roughly 800 recipes), I think we have an issue large enough that we should have a discussion. Outlined below are a few issues from highest priority to lowest that I think we somehow need to tackle.

Issue 1 - Code Signature validation

Most modern applications now have code signing, however many recipes are not validating the code signature. While some packages cannot be properly validated (*glares at Google Chrome*), authors should at minimum check for pkg/app code signatures. 

Issue 2 - Original author no longer maintaining recipes

As I've audited recipes, I have sent many PR's to recipe authors. One in particular informed me that he is no longer actively maintaining the recipes, but would gladly accept PR's. https://github.com/autopkg/justinrummel-recipes/pull/9 While this is great, we cannot ensure that recipe authors will stay within our community nor accept PRs. Perhaps our current workflow needs to re-designed.

Issue 3 - Package downloaded from insecure location.

This particular issue is less of a problem so long as Code Signature validation is present, however many of the recipes I audited had neither. Some companies also had improper CDN/proxy configurations and I could not move to HTTPS. Regardless, an author should attempt to use encryption if possible.

Issue 4 - Cleanup

Many recipes do not include any cleanup of the files that were originally downloaded. While my audit did not send PR's for fixing this, I do think recipe authors should attempt to cleanup as much as possible. runs can download a significant amount of data and within a few weeks require pruning. Cleaning up after a run would greatly help with this.

Issue 5 - Old recipes that should be abandoned.

https://github.com/autopkg/recipes/blob/master/TextMate/TextMate.download.recipe is a perfect example of a recipe that should be abandoned and possibly removed. It has no Code Signature validation nor SSL and after reaching out to the vendor, I was told there are no plans to fix these issues. The software also has many errors on modern OS X/macOS versions. 

Issue 6 - Duplicate recipes

I will admit that I am guilty of this as well (https://github.com/autopkg/autopkg/issues/288). While this issue is difficult to resolve, we do need to prune the duplicates and pick the one that adheres to the best standards.

So with all of these issues, what do we do? Well first, the community of businesses/macadmins should come together and audit the recipes at large. I think the initial focus should be on securing the packages (SSL, Code Signature or both) and then tackle the other outstanding issues. If you would like an example on a PR I issued, please see https://github.com/autopkg/cgerke-recipes/pull/9 

With regards to maintenance and the larger issues, we need to find a way to effectively list out the recipes that adhere to the best practices and ensure these are most visible to new autopkg users. While I do not have all of the issues, I hope that we can start a dialogue and come to some agreement with how to move forward.

[Elliot Jordan]

Hi Erik,

I'm really glad you started up this discussion.

After giving my "How not to do bad things with AutoPkg" speech a couple times now, I've started to feel guilty that all my recommendations are suggestions to individual administrators instead of to the community as a whole. Individual administrators are (and will continue to be) responsible for auditing AutoPkg's security as it relates to their organizations' needs, BUT some administrators will always just click the Easy Button. So, we as a community should make the defaults as safe as possible.

Central standardization is not always the answer, but I think it can definitely help with some of the issues you bring up. I'm going to follow your lead and audit a few of the recipes I run to make sure they follow established conventions like code signature verification and using HTTPS when possible. Kudos to you for taking that on.

There are currently three thought bubbles floating over my head, as follows:

AutoPkgr

Often, when we talk about "new AutoPkg users" we're also talking about AutoPkgr users. The Venn diagram isn't a perfect circle, but there's plenty of overlap. Therefore, I want to do what I can to make "the easy way" and "the secure way" one and the same for that crowd. For this, I may need help from those with Objective-C talent, because AutoPkgr is not the Linde Group's main gig, as many of you know.

One goal: Make AutoPkgr a bit more verbose about the "right" way to do things, for the benefit of administrators who may not yet know about the security implications of AutoPkg recipes. For example, nagging when the "update repos before every run" box is checked. (Or even better, a feature that allows easily reviewing and accepting repo changes.)

Regarding what you said here...

"effectively list out the recipes that adhere to the best practices and ensure these are most visible to new autopkg users"

...here's an idea we've been throwing around for a while: What if AutoPkgr collected anonymous statistics about public* recipe popularity? If we were able to see how many AutoPkgr instances are using each recipe in the AutoPkg org, we would be able to do two things:

1. Focus our auditing and testing energy on the recipes that make the most impact, and
2. Provide recipe popularity information in the AutoPkgr interface as another potential data point in answering the question, "Which of these recipes is the best?"

Of course, "popular" is rarely the same as "best," but I think it would correlate better than the number of GitHub stars a repository has.

(*It goes without saying that we have no need or desire to collect usage information about recipes that aren't in public repos. That information should remain private.)

Recipe Robot

Whatever conventions and guidelines we as a community designate as "good ideas," I want to build those ideas into Recipe Robot. This will make it easier for new recipe authors to do the "right" thing from the beginning, and will make it easy for experienced recipe authors to audit existing recipes for potential improvements, as Erik is doing now.

One example: Your issue 4 (Cleanup) is an interesting one, and I like the idea — proposed on Slack — of an ephemeral "workspace" area that is deleted after the recipe run is complete. Recipe Robot could easily be modified to enact this idea using existing processors, even with no modifications to AutoPkg itself.

Trusted Recipes / Seal of Approval

Myself and others are thinking about how to embed some sort of trust into AutoPkg recipes. More on this soon.

I'm glad I'm not the only one thinking about these things.

Elliot

--
You received this message because you are subscribed to the Google Groups "autopkg-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to autopkg-discu...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Hannes Juutilainen]

Oh, AutoPkg security, my favorite subject! A lot of good points already in this discussion and yes, thank you Erik for auditing a lot of recipes and starting this discussion.

HTTPS downloads and verifying the signature are the most important steps a recipe author can do to enhance recipe security. While I've not (yet) seen a compromised download processed by AutoPkg, I think it's only a matter of time before that happens. A perfect example of this is the Transmission ransomware incident earlier this year: http://researchcenter.paloaltonetworks.com/2016/03/new-os-x-ransomware-keranger-infected-transmission-bittorrent-client-installer/

While I think not many of us are deploying Transmission, the above is the most viable threat when blindly trusting what recipes download. CodeSignatureVerifier would have blocked that update immediately since the app was not signed with the expected certificate. Too bad the Transmission download recipe did not use CodeSignatureVerifier at all when this happened. We should be prepared for this happening again.

And this brings us to the attack vectors and threat models for AutoPkg end users. As an AutoPkg end user just running recipes, there's two really different things you're trusting:
1. You're trusting the downloaded item is in fact the item you were expecting to come out of the depths of internet.
    - This is mostly out of AutoPkg's control since we're connecting to some random servers in some random network location through some random network route. Keeping your AutoPkg machine's operating system as up-to-date as possible, using CodeSignatureVerifier and using HTTPS can help mitigate most of the risks involved here.
2. You're trusting the recipe author to not to do bad things.
    - Once the item is downloaded (and some sub item from it is verified) you're essentially handing all control to the recipe author.
    - The only way to know what's going to happen next is to manually read through the recipes involved and actually understand what they're doing. And keep going through the recipes every time there are updates for them.

I've become so paranoid about the item 2 that we're keeping private git repos for every publicly available recipe repo and manually merging upstream changes from the GitHub repos. Our AutoPkg machine will only see these private repos. And as a helper tool, I wrote the VirusTotalAnalyzer post processor to give a "second opinion" of the contents of downloaded files: https://github.com/hjuutilainen/autopkg-virustotalanalyzer


For the issue list by Erik:

Issue 1 - Code Signature validation
Most modern applications now have code signing, however many recipes are not validating the code signature. While some packages cannot be properly validated (*glares at Google Chrome*), authors should at minimum check for pkg/app code signatures.

I agree.

Issue 2 - Original author no longer maintaining recipes
As I've audited recipes, I have sent many PR's to recipe authors. One in particular informed me that he is no longer actively maintaining the recipes, but would gladly accept PR's. https://github.com/autopkg/justinrummel-recipes/pull/9 While this is great, we cannot ensure that recipe authors will stay within our community nor accept PRs. Perhaps our current workflow needs to re-designed.

There should be a process/workflow for retiring recipe repos.

Issue 3 - Package downloaded from insecure location.
This particular issue is less of a problem so long as Code Signature validation is present, however many of the recipes I audited had neither. Some companies also had improper CDN/proxy configurations and I could not move to HTTPS. Regardless, an author should attempt to use encryption if possible.

I agree.

Issue 4 - Cleanup
Many recipes do not include any cleanup of the files that were originally downloaded. While my audit did not send PR's for fixing this, I do think recipe authors should attempt to cleanup as much as possible. runs can download a significant amount of data and within a few weeks require pruning. Cleaning up after a run would greatly help with this.

I agree with this although I'm not sure what the best approach would be.

Issue 5 - Old recipes that should be abandoned.
https://github.com/autopkg/recipes/blob/master/TextMate/TextMate.download.recipe is a perfect example of a recipe that should be abandoned and possibly removed. It has no Code Signature validation nor SSL and after reaching out to the vendor, I was told there are no plans to fix these issues. The software also has many errors on modern OS X/macOS versions.

I have a few of these in my recipe repo too. I'm a bit wary of completely removing them as that will cause unneeded error messages if users have overrides for them. Maybe there's a need for a simple core processor that would just print log messages. With that, a recipe could send a deprecation message in AutoPkg output.

Issue 6 - Duplicate recipes
I will admit that I am guilty of this as well (https://github.com/autopkg/autopkg/issues/288). While this issue is difficult to resolve, we do need to prune the duplicates and pick the one that adheres to the best standards.

I'm not yet guilty of this but it has been close a few times. Some sort of popularity/quality indicator would be perfect for this and AutoPkgr might be the best place to implement it.

--

Hannes Juutilainen

[Gregory Neagle]

Get ready to do some reading...

https://github.com/autopkg/autopkg/commit/ca7ce2fb59ea435c867859e1f85e4003ea5caccc

Introduces a new 'audit' verb. Some sample output:

autopkg audit --recipe-list ~/Library/AutoPkg/disneyanimation

AdobeAIR.munki

    File path:        /Users/gneagle/Library/AutoPkg/RecipeOverrides/AdobeAIR.munki.recipe

    Parent recipe(s): /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/AdobeAIR/AdobeAir.munki.recipe

                      /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/AdobeAIR/AdobeAIR.pkg.recipe

                      /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/AdobeAIR/AdobeAIR.download.recipe

    The following http URLs were found in the recipe:

        Input:

            URL: http://airdownload.adobe.com/air/mac/download/latest/AdobeAIR.dmg

    The following processors make modifications and their use in this recipe should be more closely inspected:

        PkgCreator

        PlistEditor

        PkgInfoCreator

        Copier

        PathDeleter

AdobeFlashPlayer.munki

    File path:        /Users/gneagle/Library/AutoPkg/RecipeOverrides/AdobeFlashPlayer.munki.recipe

    Parent recipe(s): /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/AdobeFlashPlayer/AdobeFlashPlayer.munki.recipe

                      /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/AdobeFlashPlayer/AdobeFlashPlayer.download.recipe

    The following processors are non-core and can execute arbitrary code, performing any action.

    Be sure you understand what the processor does and/or you trust its source:

        AdobeFlashURLProvider

AdobeReader.munki

    File path:        /Users/gneagle/Library/AutoPkg/RecipeOverrides/AdobeReader.munki.recipe

    Parent recipe(s): /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/AdobeReader/AdobeReader.munki.recipe

                      /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/AdobeReader/AdobeReader.download.recipe

    The following processors are non-core and can execute arbitrary code, performing any action.

    Be sure you understand what the processor does and/or you trust its source:

        AdobeReaderURLProvider

<snip>

munkitools2.munki

    File path:        /Users/gneagle/Library/AutoPkg/RecipeOverrides/munkitools2.munki.recipe

    Parent recipe(s): /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/munkitools/munkitools2.munki.recipe

    Missing CodeSignatureVerifier

    The following processors make modifications and their use in this recipe should be more closely inspected:

        FlatPkgPacker

MakeCatalogs.munki

    File path:        /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/Munki/MakeCatalogs.munki.recipe

    The following processors are non-core and can execute arbitrary code, performing any action.

    Be sure you understand what the processor does and/or you trust its source:

        MakeCatalogsProcessor

Summary:

    39 recipes audited

    37 recipes with audit issues

    2 recipes with no issues to report

`autopkg audit` currently flags the following things if found in a recipe (or its parents):

1) If the recipe (or its parents) contains a download processor ('CURLDownloader', 'URLDownloader') and does not contain a CodeSignatureVerifier step, this is flagged.

2) Any http: URLs are flagged.

3) Any "creator processor" ('DmgCreator', 'FlatPkgPacker', 'PkgCreator') causes a flag -- use of these implies that the recipe modifies the item provided by the vendor.

4) Any non-core processor is flagged: these can do any arbitrary thing, including repackaging, and therefore may modify the item provided by the vendor.

37 out of 39 recipes with audit issues doesn't reduce the noise much (yet). But 14 these have in their issues list either an absent CodeSignatureVerifier step or an http URL. If these issues were addressed, we'd have 23 recipes to look at more closely. For example:

MSWord2016.munki

    File path:        /Users/gneagle/Library/AutoPkg/RecipeOverrides/MSWord2016.munki.recipe

    Parent recipe(s): /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/MSOfficeUpdates/MSWord2016.munki.recipe

                      /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/MSOfficeUpdates/MSWord2016.download.recipe

    The following processors are non-core and can execute arbitrary code, performing any action.

    Be sure you understand what the processor does and/or you trust its source:

        MSOffice2016URLandUpdateInfoProvider

MSExcel2016.munki

    File path:        /Users/gneagle/Library/AutoPkg/RecipeOverrides/MSExcel2016.munki.recipe

    Parent recipe(s): /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/MSOfficeUpdates/MSExcel2016.munki.recipe

                      /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/MSOfficeUpdates/MSExcel2016.download.recipe

    The following processors are non-core and can execute arbitrary code, performing any action.

    Be sure you understand what the processor does and/or you trust its source:

        MSOffice2016URLandUpdateInfoProvider

MSPowerPoint2016.munki

    File path:        /Users/gneagle/Library/AutoPkg/RecipeOverrides/MSPowerPoint2016.munki.recipe

    Parent recipe(s): /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/MSOfficeUpdates/MSPowerPoint2016.munki.recipe

                      /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/MSOfficeUpdates/MSPowerPoint2016.download.recipe

    The following processors are non-core and can execute arbitrary code, performing any action.

    Be sure you understand what the processor does and/or you trust its source:

        MSOffice2016URLandUpdateInfoProvider

MSOutlook2016.munki

    File path:        /Users/gneagle/Library/AutoPkg/RecipeOverrides/MSOutlook2016.munki.recipe

    Parent recipe(s): /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/MSOfficeUpdates/MSOutlook2016.munki.recipe

                      /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/MSOfficeUpdates/MSOutlook2016.download.recipe

    The following processors are non-core and can execute arbitrary code, performing any action.

    Be sure you understand what the processor does and/or you trust its source:

        MSOffice2016URLandUpdateInfoProvider

MSOneNote2016.munki

    File path:        /Users/gneagle/Library/AutoPkg/RecipeOverrides/MSOneNote2016.munki.recipe

    Parent recipe(s): /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/MSOfficeUpdates/MSOneNote2016.munki.recipe

                      /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/MSOfficeUpdates/MSOneNote2016.download.recipe

    The following processors are non-core and can execute arbitrary code, performing any action.

    Be sure you understand what the processor does and/or you trust its source:

        MSOffice2016URLandUpdateInfoProvider

These recipes all use the same non-core processor: MSOffice2016URLandUpdateInfoProvider.

If we could tell `autopkg audit` that this processor was "trusted", that would eliminate five more recipes from the list.

I have some ideas on how we could do this in a reasonably secure manner such that future changes to that processor would cause the warnings to resurface.

Here's my thinking:

The admin could add additional data to a recipe override (some of this might be done automatically by `autopkg make-override` eventually):

1) Each parent recipe could be explicitly listed, along with a checksum. As long as the checksum for each parent recipe doesn't change, the parent recipes would be "trusted".

2) Each non-core processor used by the recipe (or its parents) could be explicitly listed, also with a checksum. Again, as long as the checksum does not change, the non-core processor would be "trusted".

Trusting the parents and the non-core processors would be a manual option -- the admin would use a command to add this information to a recipe override. When `autopkg audit` runs, it would use this additional data to decide whether or not to warn about a potential issue for a given recipe.

Eventually, the audit mechanism might be tied into the `repo-add` and `repo-update` verbs to either automatically do an audit when repos are added or updated, or at least to remind the admin that it's a good idea to do an audit.

Let the discussion begin.

-Greg

On Jul 6, 2016, at 10:27 AM, Erik <eriknico...@gmail.com> wrote:

[Nick McSpadden]

This is fantastic work, but I'd like to take it a step further and add a preference to automatically reject any recipe which doesn't meet these trust guidelines.  

If there's an untrusted processor that isn't checksummed correctly, or isn't checksummed at all, this entire recipe should be skipped and an error should be logged.  If this is part of a run of multiple recipes, move on to the next one (rather than abort the run).

--

--
Nick McSpadden
nmcsp...@gmail.com

[Gregory Neagle]

On Jul 7, 2016, at 4:22 PM, Nick McSpadden <nmcsp...@gmail.com> wrote:

This is fantastic work, but I'd like to take it a step further and add a preference to automatically reject any recipe which doesn't meet these trust guidelines.  

Maybe we'll get there eventually, but not today. Baby steps.

[Erik]

Greg, this is great work.

I've attached a gist after running the audit on a significant amount of download recipes (I would caution others to _not_ do this unless they have a Mac Pro).

https://gist.github.com/erikng/3c03684d93d7bba65ca8c7fbfd5684ff

Summary:

    878 recipes audited

    665 recipes with audit issues

    213 recipes with no issues to report

We have a lot of work ahead of us. :)

[Erik Gomez]

Apologies to some recipe authors but I did not run a git pull prior to this audit. Tomorrow I will start from scratch and post a new one. 

[Tim Sutton]

I had some initial thoughts to Erik's post but since then Greg's come up with the audit tool which looks like a fantastic start. Have a couple points left outside of the `autopkg audit` discussion:

Issue 2 - Original author no longer maintaining recipes

As I've audited recipes, I have sent many PR's to recipe authors. One in particular informed me that he is no longer actively maintaining the recipes, but would gladly accept PR's. https://github.com/autopkg/justinrummel-recipes/pull/9 While this is great, we cannot ensure that recipe authors will stay within our community nor accept PRs. Perhaps our current workflow needs to re-designed.

It would be nice if an AutoPkg recipe could at least specify a 'redirect' to at least a new recipe location where a currently-mainted version of the same recipe can be found. Then AutoPkg would, instead of running it, print out the info about the new repo. I'd imagine that one may also want to "deprecate" an entire repo rather than each recipe and AutoPkg could just offer to remove it from one's cloned repos.

 

Issue 3 - Package downloaded from insecure location.

This particular issue is less of a problem so long as Code Signature validation is present, however many of the recipes I audited had neither. Some companies also had improper CDN/proxy configurations and I could not move to HTTPS. Regardless, an author should attempt to use encryption if possible.

I think what's most important that for cases where there's a Sparkle feed or a website being scraped, is HTTPS. When I installed something from the App Store last week, my Caching Server's logs indicated that it downloaded the actual pkg via HTTP. But the App Store goes through so much crypto just to get to the point of actually downloading a package.

 

Issue 4 - Cleanup

Many recipes do not include any cleanup of the files that were originally downloaded. While my audit did not send PR's for fixing this, I do think recipe authors should attempt to cleanup as much as possible. runs can download a significant amount of data and within a few weeks require pruning. Cleaning up after a run would greatly help with this.

Cleaning up makes sense to me, although you need to _at least_ keep the original downloaded file, if you want AutoPkg to not download it again on a subsequent run. It stores the server check metadata as an extended attribute on the file.

The idea of having a "workspace" in a temp dir seems logical to me to avoid recipe maintainers needing to care about cleaning up particular files. Maybe CACHE_DIR is something that could be special-cased to persist files only when they should be, and all intermediate files use a workspace, but since AutoPkg doesn't have any hard and fast rules about what's the "final" output of a recipe it seems like there are some complex heuristics required to try and guess what's important to keep and what isn't.

 

Issue 5 - Old recipes that should be abandoned.

https://github.com/autopkg/recipes/blob/master/TextMate/TextMate.download.recipe is a perfect example of a recipe that should be abandoned and possibly removed. It has no Code Signature validation nor SSL and after reaching out to the vendor, I was told there are no plans to fix these issues. The software also has many errors on modern OS X/macOS versions.

I agree this recipe should be removed. What we don't have right now in AutoPkg is a very simple way of communicating any such removals to the user. Someone can do a repo-update, the recipe is gone, and AutoPkg reports next time that it cannot find it. Perhaps the above idea about redirecting could also potentially be extended to "deprecating" a recipe where it just exists as a reference but it is clear to the user that it's no longer supported and should be removed.

[Mr. Alan Siu]

I wonder if, after the audit process is completely agreed upon, it makes sense to restructure the GitHub repo to be similar to how Ubuntu does their repos—theirs is based on open source licensing, but the AutoPkg repos could be organized based on the audit report score.

Ubuntu starts off with Main and Universe, I believe, and then you have the option to add Multiverse and various PPAs.

AutoPkg could have a core set of recipes that use https and code signature verification, with that repository added in by default, and then you have the option to add additional repositories that may be less secure.

Right now the organization is based on recipe maintainer instead of by recipe quality. Would a reorganization of the recipes be too big an undertaking with too little benefit? It may also break (URL) links that people have to specific recipes.

Alan Siu

Client Systems Analyst

St. Ignatius College Preparatory

[Tim Sutton]

As far as how the repos are organized, we've been able to get by so far without having to write and maintain our own registry-like service that would be able to track such metadata. This wouldn't prevent someone, however, from scripting an audit across many repos (like I believe Erik is doing now) and then posting the results for one or more community members to review or post issues/PRs for fixes.

I think this goes back to the discussion about which elements of an audit are "obvious" fixes and which are just things the consumer of a given recipe should be aware of.

The organization of repos isn't strictly by maintainer although it's evolved that way - repos are free to add more maintainers or to group themselves by something other than who maintains them.

Not that you were suggesting this, but as a bit of an aside I'm not personally in favour of moving everything into a completely mono repo like Homebrew, because the problem of how to "package manage" software on one platform is very different than the problem of packing one app (which may have a few variants) into many different distribution platforms, and I'm not sure it's sustainable to coerce every potential recipe maintainer into a strict set of requirements for inclusion into a "core" repo. Of course with Homebrew there is still first-class support for supporting alternate repos where maintainers can do whatever they like.

Tim

[Mr. Alan Siu]

I think this goes back to the discussion about which elements of an audit are "obvious" fixes and which are just things the consumer of a given recipe should be aware of.

Yes, I'd definitely want to know, of the recipes I could potentially use, which ones are essentially unfixable (there is no version 2 to use code signature verifier on and/or no available https download location) and which ones are just yet-to-be-fixed.

For the latter, I'd have the option to even make a pull request. The former would just waste my time to investigate if the recipe maintainer had already looked into https and code signature verification.

Alan Siu

Client Systems Analyst

St. Ignatius College Preparatory

[Gregory Neagle]

On Jul 8, 2016, at 10:04 AM, Mr. Alan Siu <as...@siprep.org> wrote:

I think this goes back to the discussion about which elements of an audit are "obvious" fixes and which are just things the consumer of a given recipe should be aware of.

Yes, I'd definitely want to know, of the recipes I could potentially use, which ones are essentially unfixable (there is no version 2 to use code signature verifier on and/or no available https download location) and which ones are just yet-to-be-fixed.

For the latter, I'd have the option to even make a pull request. The former would just waste my time to investigate if the recipe maintainer had already looked into https and code signature verification.

The maintainer could add this information into the recipe description.

[Mr. Alan Siu]

Thanks, Greg. I wasn't sure if it made sense for there to be some kind of flag or other way to tell apart from a note in the description. Since, presumably, those will be the exception rather than the rule, a description note totally makes sense.

Alan Siu

Client Systems Analyst

St. Ignatius College Preparatory

[Erik]

I have rerun the audit and as of a few minutes ago here is the status of _all_ .download.recipes across the entire autopkg recipe base.

txt form

https://gist.github.com/erikng/3c03684d93d7bba65ca8c7fbfd5684ff#file-audit-txt

plist form

https://gist.github.com/erikng/3c03684d93d7bba65ca8c7fbfd5684ff#file-audit-plist 

Summary:

    1182 recipes audited

    914 recipes with audit issues

    268 recipes triggered no audit flags

[Gregory Neagle]

On Jul 7, 2016, at 4:06 PM, Gregory Neagle <gregn...@mac.com> wrote:

I have some ideas on how we could do this in a reasonably secure manner such that future changes to that processor would cause the warnings to resurface.

Here's my thinking:

The admin could add additional data to a recipe override (some of this might be done automatically by `autopkg make-override` eventually):

1) Each parent recipe could be explicitly listed, along with a checksum. As long as the checksum for each parent recipe doesn't change, the parent recipes would be "trusted".

2) Each non-core processor used by the recipe (or its parents) could be explicitly listed, also with a checksum. Again, as long as the checksum does not change, the non-core processor would be "trusted".

Trusting the parents and the non-core processors would be a manual option -- the admin would use a command to add this information to a recipe override. When `autopkg audit` runs, it would use this additional data to decide whether or not to warn about a potential issue for a given recipe.

Eventually, the audit mechanism might be tied into the `repo-add` and `repo-update` verbs to either automatically do an audit when repos are added or updated, or at least to remind the admin that it's a good idea to do an audit.

Here's a start of that:

% ./autopkg run TestAdobeFlashPlayerOverride.munki
Processing TestAdobeFlashPlayerOverride.munki...
Failed.

The following recipes failed:
    TestAdobeFlashPlayerOverride.munki
        Could not verify trust of parent recipes and custom processors:
Expected hash 2c3456a7cd28037ab7de1b8fd5883743954b665fb7d5aaaec37bf4829a4824e4 for non-core processor AdobeFlashURLProvider, got fda060422a2fd9d1df7b28cd304b60660810cfa9ecf8619e59732b90767964f8
(Processor AdobeFlashURLProvider has been changed.)
Expected hash 5d444afa08a86b73cb2eea9c94de0ee201c03a42471f512ea345baa0851581ef for parent recipe com.github.autopkg.download.FlashPlayer, got d3f2ae0f10f07489c083b8ad7f4f00e988c049850e9be20f41ea6a9dc568e878
(Parent recipe com.github.autopkg.download.FlashPlayer has been changed.)

This will be enabled by new data that is stored in a recipe override. Here's the data for the TestAdobeFlashPlayerOverride.munki recipe:

<key>ParentRecipeTrustInfo</key>

<dict>

<key>non_core_processors</key>

<dict>

<key>AdobeFlashURLProvider</key>

<string>2c3456a7cd28037ab7de1b8fd5883743954b665fb7d5aaaec37bf4829a4824e4</string>

</dict>

<key>parent_recipes</key>

<dict>

<key>com.github.autopkg.download.FlashPlayer</key>

<string>5d444afa08a86b73cb2eea9c94de0ee201c03a42471f512ea345baa0851581ef</string>

<key>com.github.autopkg.munki.FlashPlayerNoRepackage</key>

<string>2d9c62120dd57503be1d3a3b8e49f6baa7ecf79ae6298c843769b7c35058f598</string>

</dict>

</dict>

The string stored with the names of each non-core processor and each parent recipe identifier is a sha256 hash of the file containing the processor or parent recipe.

The plan is that `autopkg make-override` will automatically add this information to new recipe overrides. By making a recipe override you are saying "I trust its parents and the non-core processors they use _as of the time I make this override_". If the parents or the non-core processors change, the trust is broken, which should cause the autopkg admin to re-examine/re-audit the changes.

There will need to be a new verb to update the trust info for an existing override.

I suppose there will need to be a new flag for `autopkg run` to tell it to ignore parent trust failures as well.

-Greg

[Nick McSpadden]

Thank you for doing this, I'll give this a spin very soon.

--
You received this message because you are subscribed to the Google Groups "autopkg-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to autopkg-discu...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Gregory Neagle]

I haven't committed anything yet -- I'm still in a experimental phase. I'll announce when I have something to test -- it will likely be in a new branch at first. 

Sent from my iPhone

[Gregory Neagle]

More experimenting:

% ./autopkg run TestAdobeFlash.munki

Processing TestAdobeFlash.munki...

Failed.

The following recipes failed:

    TestAdobeFlash.munki

        Could not verify trust of parent recipes and custom processors:

Processor AdobeFlashURLProvider has been changed.

Processor path: /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/AdobeFlashPlayer/AdobeFlashURLProvider.py

diff --git a/AdobeFlashPlayer/AdobeFlashURLProvider.py b/AdobeFlashPlayer/AdobeFlashURLProvider.py

index bc6379d..3805bb0 100644

--- a/AdobeFlashPlayer/AdobeFlashURLProvider.py

+++ b/AdobeFlashPlayer/AdobeFlashURLProvider.py

@@ -24,7 +24,7 @@ from autopkglib import Processor, ProcessorError

 

 __all__ = ["AdobeFlashURLProvider"]

 

-UPDATE_XML_URL = ("http://fpdownload2.macromedia.com/"

+UPDATE_XML_URL = ("http://fpdownload2.malicious.com/"

                   "get/flashplayer/update/current/xml/version_en_mac_pl.xml")

 

 DOWNLOAD_TEMPLATE_URL = ("https://fpdownload.macromedia.com/"

Nothing downloaded, packaged or imported.

Not only is a difference flagged, but you are given a git diff highlighting the changes.

Another example:

% ./autopkg run TestAdobeFlash.munki

Processing TestAdobeFlash.munki...

Failed.

The following recipes failed:

    TestAdobeFlash.munki

        Could not verify trust of parent recipes and custom processors:

Parent recipe com.github.autopkg.download.FlashPlayer has been changed.

Recipe path: /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/AdobeFlashPlayer/AdobeFlashPlayer.download.recipe

diff --git a/AdobeFlashPlayer/AdobeFlashPlayer.download.recipe b/AdobeFlashPlayer/AdobeFlashPlayer.download.recipe

index 2824fc4..7aab179 100644

--- a/AdobeFlashPlayer/AdobeFlashPlayer.download.recipe

+++ b/AdobeFlashPlayer/AdobeFlashPlayer.download.recipe

@@ -24,6 +24,8 @@

             <string>URLDownloader</string>

             <key>Arguments</key>

             <dict>

+                <key>url</key>

+                <string>http://my.malware.site/flash.dmg</string>

                 <key>filename</key>

                 <string>%NAME%.dmg</string>

             </dict>

Nothing downloaded, packaged or imported.

[Erik Gomez]

That's pretty awesome. 

[Gregory Neagle]

My current experiments around parent recipe trust verification are available here:

https://github.com/autopkg/autopkg/tree/recipe-checksumming

`autopkg make-override Foo.recipe`  "automagically" adds parent recipe trust verification info to the override.

`autopkg verify-trust-info some.recipe` verifies the trust info

`autopkg update-trust-info some.recipe` updates existing trust info or adds it if there is none. It's meant to be used on recipe overrides, but might be useful on any child recipe.

`autopkg run some.recipe` will fail a given recipe if trust verification fails. this can be overridden by adding --ignore-parent-trust-verification-errors

Have fun!

-Greg

[Gregory Neagle]

On Jul 21, 2016, at 4:40 PM, Gregory Neagle <gregn...@mac.com> wrote:

My current experiments around parent recipe trust verification are available here:

https://github.com/autopkg/autopkg/tree/recipe-checksumming

`autopkg make-override Foo.recipe`  "automagically" adds parent recipe trust verification info to the override.

`autopkg verify-trust-info some.recipe` verifies the trust info

`autopkg update-trust-info some.recipe` updates existing trust info or adds it if there is none. It's meant to be used on recipe overrides, but might be useful on any child recipe.

`autopkg run some.recipe` will fail a given recipe if trust verification fails. this can be overridden by adding --ignore-parent-trust-verification-errors

An example workflow showing how this might be useful:

I added trust info to ~50 recipe overrides:

autopkg update-trust-info ~/Library/AutoPkg/RecipeOverrides/*.recipe

This also found three "abandoned" overrides -- overrides I created a long time ago for recipes that no longer exist.

I then updated all repos:

autopkg repo-update all

This resulted in over 800 lines of output, telling me about updates for over 400 recipes, most of which I do not use. It would be tedious to parse this list to find the changes for the recipes I do use.

Next, I attempted to re-verify all my recipe overrides:

autopkg verify-trust-info ~/Library/AutoPkg/RecipeOverrides/*.recipe

Only _two_ of my overrides did not pass verification. Here's one:

Parent trust info verification failed for /Users/gneagle/Library/AutoPkg/RecipeOverrides/LibreOffice.munki.recipe

Could not verify trust of parent recipes and custom processors:

Expected non-core processor list: [u'LibreOfficeURLProvider']

Actual non-core processor list: []

Our stored checksum for processor LibreOfficeURLProvider does not match the current content of that processor.

Perhaps processor LibreOfficeURLProvider has been changed.

Processor path: None

Our stored checksum for parent recipe io.github.hjuutilainen.download.LibreOffice does not match the current content of that recipe.

Perhaps parent recipe io.github.hjuutilainen.download.LibreOffice has been changed.

Recipe path: /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.hjuutilainen-recipes/LibreOffice/LibreOffice.download.recipe

diff --git a/LibreOffice/LibreOffice.download.recipe b/LibreOffice/LibreOffice.download.recipe

index d0ee986..16f0765 100644

--- a/LibreOffice/LibreOffice.download.recipe

+++ b/LibreOffice/LibreOffice.download.recipe

@@ -3,17 +3,18 @@

 <plist version="1.0">

 <dict>

  <key>Description</key>

- <string>Downloads the latest LibreOffice.</string>

+ <string>Downloads the latest LibreOffice. Set RELEASE to either "fresh" or "still".

+

+LibreOffice Still is the stable version that has undergone more testing (over a longer time). It is usually recommended for more conservative use.

+LibreOffice Fresh is the stable version with the most recent features. Users interested in taking advantage of our most innovative features should download and use our fresh version.</string>

  <key>Identifier</key>

  <string>io.github.hjuutilainen.download.LibreOffice</string>

  <key>Input</key>

  <dict>

  <key>NAME</key>

  <string>LibreOffice</string>

- <key>TYPE</key>

- <string>mac-x86_64</string>

- <key>USER_AGENT</key>

- <string>Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_3) AppleWebKit/536.28.10 (KHTML, like Gecko) Version/6.0.3 Safari/536.28.10</string>

+ <key>RELEASE</key>

+ <string>fresh</string>

  </dict>

  <key>MinimumVersion</key>

  <string>0.2.0</string>

@@ -21,11 +22,13 @@

  <array>

  <dict>

  <key>Processor</key>

- <string>LibreOfficeURLProvider</string>

+ <string>URLTextSearcher</string>

  <key>Arguments</key>

  <dict>

- <key>type</key>

- <string>%TYPE%</string>

+ <key>re_pattern</key>

+ <string>(?P&lt;DOWNLOAD_URL&gt;download.documentfoundation.org/libreoffice/stable/[\d\.]+/mac/x86_64/LibreOffice_(?P&lt;version&gt;[\d\.]+)_MacOS_x86-64.dmg)</string>

+ <key>url</key>

+ <string>https://www.libreoffice.org/download/libreoffice-%RELEASE%/?type=mac-x86_64</string>

  </dict>

  </dict>

  <dict>

@@ -33,11 +36,8 @@

  <string>URLDownloader</string>

  <key>Arguments</key>

  <dict>

- <key>request_headers</key>

- <dict>

- <key>user-agent</key>

- <string>%USER_AGENT%</string>

- </dict>

+ <key>url</key>

+ <string>https://%DOWNLOAD_URL%</string>

  </dict>

  </dict>

  <dict>

Our stored checksum for parent recipe io.github.hjuutilainen.munki.LibreOffice does not match the current content of that recipe.

Perhaps parent recipe io.github.hjuutilainen.munki.LibreOffice has been changed.

Recipe path: /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.hjuutilainen-recipes/LibreOffice/LibreOffice.munki.recipe

diff --git a/LibreOffice/LibreOffice.munki.recipe b/LibreOffice/LibreOffice.munki.recipe

index b60ec1c..6205d83 100644

--- a/LibreOffice/LibreOffice.munki.recipe

+++ b/LibreOffice/LibreOffice.munki.recipe

@@ -3,13 +3,18 @@

 <plist version="1.0">

 <dict>

  <key>Description</key>

- <string>Downloads the current release version of LibreOffice and imports into Munki.</string>

+ <string>Downloads the current release version of LibreOffice and imports into Munki. Set RELEASE to either "fresh" or "still".

+

+LibreOffice Still is the stable version that has undergone more testing (over a longer time). It is usually recommended for more conservative use.

+LibreOffice Fresh is the stable version with the most recent features. Users interested in taking advantage of our most innovative features should download and use our fresh version.</string>

  <key>Identifier</key>

  <string>io.github.hjuutilainen.munki.LibreOffice</string>

  <key>Input</key>

  <dict>

  <key>NAME</key>

  <string>LibreOffice</string>

+ <key>RELEASE</key>

+ <string>fresh</string>

  <key>MUNKI_REPO_SUBDIR</key>

  <string>apps/LibreOffice</string>

  <key>MUNKI_CATEGORY</key>

This is probably too much data and would be overwhelming for many people; I think I should probably display less detail by default and use the -v/--verbose flag to provide more detail.

But it does let us know these things:

1) A parent used a non-core processor named "LibreOfficeURLProvider"; the updated recipe no longer does.

2) The parent LibreOffice.download.recipe and LibreOffice.munki.recipe recipes have changed:

a) the LibreOffice.download.recipe replaces the use of the non-core LibreOfficeURLProvider processor with the core URLTextSearcher processor.

b) other changes consistent with the above change

Note that I would not have had to explicitly verify the trust info; I would get the same info if I had tried to run any of the affected recipes:

autopkg run LibreOffice.munki

Processing LibreOffice.munki...

Failed.

The following recipes failed:

    LibreOffice.munki

        Could not verify trust of parent recipes and custom processors:

Expected non-core processor list: [u'LibreOfficeURLProvider']

Actual non-core processor list: []

<etc>

This mechanism makes it possible for autopkg admins to be notified when there are parent recipe changes the admin should probably pay attention to, without overwhelming him/her with irrelevant changes.

[Tim Sutton]

While I haven't yet had a chance to try out this new branch, what I like about this approach is that leaves the responsibility of trust with the AutoPkg admin.

To me it's analogous to the various programming language package managers' support for managing a package dependency that resides in a Git repo, but pinned to a particular point in time.

While perhaps there could be ways to show more/less info so that it's not overwhelming, at least the capability is there to ensure that you're running the recipes you think you are, and to make an explicit commitment that you trust the parent recipe chain at any given point in time.

Tim

[Gregory Neagle]

Added support for -v when verifying trust info. The default level provide a very basic overview of the changes:

autopkg verify-trust-info LibreOffice.munki 
Parent trust info verification failed for LibreOffice.munki

Could not verify trust of parent recipes and custom processors:
    Expected non-core processor list: [u'LibreOfficeURLProvider']
    Actual non-core processor list: []

    Processor LibreOfficeURLProvider differs from expected.
    Processor LibreOfficeURLProvider can't be found.
    Parent recipe io.github.hjuutilainen.download.LibreOffice differs from expected.
    Recipe path: /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.hjuutilainen-recipes/LibreOffice/LibreOffice.download.recipe
    Parent recipe io.github.hjuutilainen.munki.LibreOffice differs from expected.
    Recipe path: /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.hjuutilainen-recipes/LibreOffice/LibreOffice.munki.recipe

One -v adds some checksum info (I don't really know if this is actually useful):

autopkg verify-trust-info LibreOffice.munki -v
Parent trust info verification failed for LibreOffice.munki

Could not verify trust of parent recipes and custom processors:
    Expected non-core processor list: [u'LibreOfficeURLProvider']
    Actual non-core processor list: []

    Processor LibreOfficeURLProvider differs from expected.
    Expected checksum: 77d0c983825fed168a856e63355deefd90f101483325ce669b32fa54c7feec91
    Actual checksum: None
    Processor LibreOfficeURLProvider can't be found.
    Parent recipe io.github.hjuutilainen.download.LibreOffice differs from expected.
    Expected checksum: 28d04d55de407b29f561b2000e53e5b47e3e149aa865c3659b5af610099ff263
    Actual checksum: e6adc0690acbb6061ed86c7ca5093bd2ff94d6518aaa7d4798176297527a4380
    Recipe path: /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.hjuutilainen-recipes/LibreOffice/LibreOffice.download.recipe
    Parent recipe io.github.hjuutilainen.munki.LibreOffice differs from expected.
    Expected checksum: f431d09a84ec72bf9528ed039b73e068b1f5c0c231ee8f72f158f5ebd40d6a4d
    Actual checksum: 6629394ea06504bd72c3ef9630f041f834b5092481a20d1d9c62aa1515f46241
    Recipe path: /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.hjuutilainen-recipes/LibreOffice/LibreOffice.munki.recipe

Two -vv options adds Git diffs if available:

autopkg verify-trust-info LibreOffice.munki -vv
Parent trust info verification failed for LibreOffice.munki

Could not verify trust of parent recipes and custom processors:
    Expected non-core processor list: [u'LibreOfficeURLProvider']
    Actual non-core processor list: []

    Processor LibreOfficeURLProvider differs from expected.
    Expected checksum: 77d0c983825fed168a856e63355deefd90f101483325ce669b32fa54c7feec91
    Actual checksum: None
    Processor LibreOfficeURLProvider can't be found.
    Parent recipe io.github.hjuutilainen.download.LibreOffice differs from expected.
    Expected checksum: 28d04d55de407b29f561b2000e53e5b47e3e149aa865c3659b5af610099ff263
    Actual checksum: e6adc0690acbb6061ed86c7ca5093bd2ff94d6518aaa7d4798176297527a4380

    Parent recipe io.github.hjuutilainen.munki.LibreOffice differs from expected.
    Expected checksum: f431d09a84ec72bf9528ed039b73e068b1f5c0c231ee8f72f158f5ebd40d6a4d
    Actual checksum: 6629394ea06504bd72c3ef9630f041f834b5092481a20d1d9c62aa1515f46241

[Gregory Neagle]

Changed my mind. No -v's product this output:

autopkg verify-trust-info LibreOffice.munki
Parent trust info verification failed for LibreOffice.munki

a single -v:

autopkg verify-trust-info LibreOffice.munki -v
Parent trust info verification failed for LibreOffice.munki

    Expected non-core processor list: [u'LibreOfficeURLProvider']
    Actual non-core processor list: []
    Processor LibreOfficeURLProvider differs from expected.
    Processor LibreOfficeURLProvider can't be found.
    Parent recipe io.github.hjuutilainen.download.LibreOffice differs from expected.
    Recipe path: /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.hjuutilainen-recipes/LibreOffice/LibreOffice.download.recipe
    Parent recipe io.github.hjuutilainen.munki.LibreOffice differs from expected.
    Recipe path: /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.hjuutilainen-recipes/LibreOffice/LibreOffice.munki.recipe

Two -vs would have the above output plus the Git diff if available.

-Greg

[Erik Gomez]

Again, this all looks great. I sadly haven't had time to test this but I hope to soon. 

Thanks for all of your work on this. 

[Erik]

So I've had a little bit of time to test and here are a few thoughts:

1. Old overrides won't have trust information. 

./autopkg verify-trust-info AdobeFlashPlayer.munki

AdobeFlashPlayer.munki: NO TRUST INFO

Currently an autopkg run will not warn the admin that the override doesn't contain trust information.

2. Perhaps a new preference: RequireOverrideTrustInfo. If someone attempts to remove the trust information from an override, autopkg will refuse to run that particular override. Perhaps this key would also require each recipe that is ran to have an override, containing the trust info. You would not be able to run the default recipes.

3. The git diffs will only work if you are using autopkg repo-add. If you are managing repos and ignoring each recipe repo's own git information, this information will not be generated. I think for the few people doing something like this, that will be OK.

Other than that, it looks really good.

-Greg

To unsubscribe from this group and stop receiving emails from it, send an email to autopkg-discuss+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "autopkg-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to autopkg-discuss+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "autopkg-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to autopkg-discuss+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "autopkg-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to autopkg-discuss+unsubscribe@googlegroups.com.

[Gregory Neagle]

Once again I would like to draw attention to https://github.com/autopkg/autopkg/tree/recipe-checksumming

In the 'recipe-checksumming' branch, I've added several things to assist autopkg admins in maintaining a level of trust in the recipes they use.

Here's the expected workflow.

1) Admin decides he/she wants to use a community-shared recipe. He/she looks through the recipe and processors and tests the recipe's output and sees that it does what is expected.

2) Admin creates a recipe override: `autopkg make-override Foo.recipe`. As part of that override, parent recipe trust information is stored. This information contains the identifiers and checksums for any parent recipes and names and checksums for any "non-core" processors used by the recipe and its parents.

3) Admin can now use the recipe normally -- just as with any previous release of autopkg.

4) Later, the admin does `autopkg repo-update all` or any other repo update operation that updates one or more parent recipes and/or custom processors used by the Foo.recipe override.

5) On the next attempted run of the Foo.recipe override, the run is stopped with a trust verification failure since parts of the recipe have changed.

6) Running `autopkg verify-trust-info -vv Foo.recipe` provides a lot of detail on what changed. The admin can then review the changes to be sure nothing malicious or suspicious was introduced.

7) If the admin wants to "re-trust" the recipe, he/she runs `autopkg update-trust-info Foo.recipe` and the stored trust info is updated to reflect the current set of parents and custom processors.

The new verbs:

autopkg update-trust-info --help
Usage: autopkg update-trust-info [options] recipe_override_name [...]
Update or add parent recipe trust information for a recipe override.

Options:
  -h, --help            show this help message and exit
  -d DIRECTORY, --search-dir=DIRECTORY
                        Directory to search for recipes. Can be specified
                        multiple times.
  --override-dir=DIRECTORY
                        Directory to search for recipe overrides. Can be
                        specified multiple times.

autopkg verify-trust-info --help
Usage: autopkg verify-trust-info [options] recipe_override_name [...]
Verify parent recipe trust information for a recipe override.

Options:
  -h, --help            show this help message and exit
  -l TEXT_FILE, --recipe-list=TEXT_FILE
                        Path to a text file with a list of recipes to verify.
  -v, --verbose         Verbose output.
  -d DIRECTORY, --search-dir=DIRECTORY
                        Directory to search for recipes. Can be specified
                        multiple times.
  --override-dir=DIRECTORY
                        Directory to search for recipe overrides. Can be
                        specified multiple times.

You can disable parent trust verification when running recipes:

  --ignore-parent-trust-verification-errors
                        Run recipes even if they fail parent trust verification.

Currently, by default, a recipe that does not have any parent trust information is treated the same as one that has invalid trust info -- it causes the recipe run to fail. You can override this by defining a ALLOW_RECIPES_WITHOUT_TRUST_INFO preference -- either in ~/Library/Preferences/com.github.autopkg.plist:

defaults write com.github.autopkg ALLOW_RECIPES_WITHOUT_TRUST_INFO -bool true

 or via the command line for a specific run: -k ALLOW_RECIPES_WITHOUT_TRUST_INFO=yes

I plan to merge this to master very soon, but I encourage people to check this code out sooner rather than later as it is a major change to how autopkg operates. If there are tweaks that should be made to the functionality, earlier is always better.

-Greg

[Erik]

After testing the latest commit, I believe the support issues are going to be significant at first:

One Recipe:

./autopkg run AutoDMG.munki
Processing AutoDMG.munki...

Failed.
The following recipes failed:

    AutoDMG.munki
        No trust information present

Nothing downloaded, packaged or imported.

So the admin verifies the trust info, and finds out it fails.

./autopkg verify-trust-info AutoDMG.munki
AutoDMG.munki: FAILED

They look over it and everything looks good so they validate the info.

./autopkg verify-trust-info AutoDMG.munki
AutoDMG.munki: OK 

Now they are good.

./autopkg run AutoDMG.munki
Processing AutoDMG.munki...

Nothing downloaded, packaged or imported. 

Now they decided to do another application:

./autopkg run AutoDMG.munki Firefox.munki
Processing AutoDMG.munki...
Processing Firefox.munki...

Failed.
The following recipes failed:

    Firefox.munki
        No trust information present

Nothing downloaded, packaged or imported. 

While this makes sense to me, I think some admins (especially newer) ones will be caught off-guard by this. 

Here's what I think we should do:

1. Update the Getting Started wiki to explicitly show people how to make an override and how it creates the trust chain.
2. Write another wiki article specifically about the trust component.
3. Release Autopkg 0.6.2 and instead of hard breaking a run, warn with a direct wiki link to the upcoming changes.
4. In 2-3 months, release Autopkg 1.0 or 0.6.3/0.7.0 that specifically requires the trust component. This gives people ample time to change their processes and also allows people that wrap around Autopkg to create the proper tooling.

As a side note, this will also require that _all_ recipes require an override, so this may break the autopkg install feature. Further documentation may be needed that I have not thought about.

[Gregory Neagle]

On Aug 1, 2016, at 3:49 PM, Erik <eriknico...@gmail.com> wrote:

As a side note, this will also require that _all_ recipes require an override, so this may break the autopkg install feature.

Even though I wrote that feature, I'd be perfectly OK to see it go away. `autopkg install Foo` is something you'd do on a given machine usually exactly once. So auditing the recipe, creating an override to trust it and then running the override just to install Firefox is just too much work.

"install" recipes would benefit from a special "tested, blessed and trusted" recipe repo, I suppose. I'm still not comfortable with the proposals so far about creating and maintaining such a repo.

-Greg

[Erik]

I would _highly_ encourage people that are active in the autopkg community to test the latest commit from the recipe-checksumming branch (https://github.com/autopkg/autopkg/tree/recipe-checksumming) and give proper feedback.

There are significant changes that may impact your current autopkg/autopkgr process. To date, only I have given any feedback to Greg. While I am pretty happy with the changes, other may not be. Speak now before it's committed to master.

[Gregory Neagle]

In the latest commits, support for ALLOW_RECIPES_WITHOUT_TRUST_INFO has been removed.

Recipes without trust info will cause a warning to be printed, but will run:

autopkg run OracleJava8.munki OracleJava8JDK.munki MakeCatalogs.munki -k FAIL_RECIPES_WITHOUT_TRUST_INFO=''

Processing OracleJava8.munki...

WARNING: OracleJava8.munki is missing trust info and FAIL_RECIPES_WITHOUT_TRUST_INFO is not set. Proceeding...

Processing OracleJava8JDK.munki...

WARNING: OracleJava8JDK.munki is missing trust info and FAIL_RECIPES_WITHOUT_TRUST_INFO is not set. Proceeding...

Processing MakeCatalogs.munki...

WARNING: MakeCatalogs.munki is missing trust info and FAIL_RECIPES_WITHOUT_TRUST_INFO is not set. Proceeding...

You can set FAIL_RECIPES_WITHOUT_TRUST_INFO in autopkg's preferences, or via CLI:

autopkg run OracleJava8.munki OracleJava8JDK.munki MakeCatalogs.munki -k FAIL_RECIPES_WITHOUT_TRUST_INFO=yes

Processing OracleJava8.munki...

Failed.

Processing OracleJava8JDK.munki...

Failed.

Processing MakeCatalogs.munki...

Failed.

The following recipes failed:

    OracleJava8.munki

        No trust information present. Audit the recipe, then create an override to trust it.

    OracleJava8JDK.munki

        No trust information present. Audit the recipe, then create an override to trust it.

    MakeCatalogs.munki

        No trust information present. Audit the recipe, then create an override to trust it.

--
You received this message because you are subscribed to the Google Groups "autopkg-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to autopkg-discu...@googlegroups.com.

[Eric Holtam]

Interesting conversations and movement on this feature.  I've cloned and tested the latest build on the branch on an autopkg clean system.

What's the purpose of the --override-dir option?  If it's to point to an override directory to parse thru to apply or verify trust info I'm not able to get it to work.  I've tried a few variations but it doesn't seem to do anything when used with verify-trust-info and update-trust-info.

`autopkg verify-trust-info --verbose --override-dir=/Users/eholtam/Library/AutoPkg/RecipeOverrides/*.recipe`

`autopkg verify-trust-info --verbose --override-dir=/Users/eholtam/Library/AutoPkg/RecipeOverrides/`

`autopkg verify-trust-info --verbose --override-dir=~/Library/AutoPkg/RecipeOverrides`

`autopkg verify-trust-info --verbose --override-dir=/Users/eholtam/Library/AutoPkg/RecipeOverrides/ /Users/eholtam/Library/AutoPkg/RecipeOverrides/*.recipe`

All result in zero feedback.  It just jumps back to the prompt.  

The only way I was able to get trust info to write to override by Greg's previous example of `autopkg update-trust-info ~/Library/AutoPkg/RecipeOverrides/*.recipe`

Another thing I noticed is when running a recipe with any level of verbosity there's no output about the recipe being successfully trusted.  A clean run acts as if there is no trust information present.  I'd think at least acknowledging the trust info is there and valid would be beneficial to some level of verbose output.

-Eric

[Gregory Neagle]

On Aug 2, 2016, at 6:36 PM, Eric Holtam <eho...@gmail.com> wrote:

Interesting conversations and movement on this feature.  I've cloned and tested the latest build on the branch on an autopkg clean system.

What's the purpose of the --override-dir option?

The override-dir option works the same as in every other subcommand that accepts it: it provides a directory to search for overrides.  `autopkg run` supports an --override-dir option:

autopkg run --help

Usage: autopkg run [options] [recipe ...]

Run one or more recipes.

Options:

  -h, --help            show this help message and exit

  --pre=PREPROCESSOR, --preprocessor=PREPROCESSOR

                        Name of a processor to run before each recipe. Can be

                        repeated to run multiple preprocessors.

  --post=POSTPROCESSOR, --postprocessor=POSTPROCESSOR

                        Name of a processor to run after each recipe. Can be

                        repeated to run multiple postprocessors.

  -c, --check           Only check for new/changed downloads.

  -k KEY=VALUE, --key=KEY=VALUE

                        Provide key/value pairs for recipe input. Caution:

                        values specified here will be applied to all recipes.

  -l TEXT_FILE, --recipe-list=TEXT_FILE

                        Path to a text file with a list of recipes to run.

  -p PKG_OR_DMG, --pkg=PKG_OR_DMG

                        Path to a pkg or dmg to provide to a recipe.

                        Downloading will be skipped.

  --report-plist=OUTPUT_PATH

                        File path to save run report plist.

  -v, --verbose         Verbose output.

  -d DIRECTORY, --search-dir=DIRECTORY

                        Directory to search for recipes. Can be specified

                        multiple times.

  --override-dir=DIRECTORY

                        Directory to search for recipe overrides. Can be

                        specified multiple times.

Here's the help for `autopkg verify-trust-info`:

autopkg verify-trust-info --help

Usage: autopkg verify-trust-info [options] recipe_override_name [...]

Verify parent recipe trust information for a recipe override.

Options:

  -h, --help            show this help message and exit

  -l TEXT_FILE, --recipe-list=TEXT_FILE

                        Path to a text file with a list of recipes to verify.

  -v, --verbose         Verbose output.

  -d DIRECTORY, --search-dir=DIRECTORY

                        Directory to search for recipes. Can be specified

                        multiple times.

  --override-dir=DIRECTORY

                        Directory to search for recipe overrides. Can be

                        specified multiple times.

recipe_override_name can be a simple name or a path -- if it's just a name, the override directories are searched. The override directories default to:

[
        "~/Library/AutoPkg/RecipeOverrides"
]

But can also be set in autopkg's preferences (https://github.com/autopkg/autopkg/blob/master/Code/autopkg#L595-L605), or specified at the command-line using the --override-dir option.

Again, this is not new/unique to verify-trust-info or update-trust-info -- these options have been available for other autopkg sub-commands for a long time.

 If it's to point to an override directory to parse thru to apply or verify trust info I'm not able to get it to work.

Nope,  that's not it. You need to give autopkg verify-trust-info the _names_ or _paths_ of one or more recipe.

 I've tried a few variations but it doesn't seem to do anything when used with verify-trust-info and update-trust-info.

`autopkg verify-trust-info --verbose --override-dir=/Users/eholtam/Library/AutoPkg/RecipeOverrides/*.recipe`

`autopkg verify-trust-info --verbose --override-dir=/Users/eholtam/Library/AutoPkg/RecipeOverrides/`

`autopkg verify-trust-info --verbose --override-dir=~/Library/AutoPkg/RecipeOverrides`

`autopkg verify-trust-info --verbose --override-dir=/Users/eholtam/Library/AutoPkg/RecipeOverrides/ /Users/eholtam/Library/AutoPkg/RecipeOverrides/*.recipe`

All result in zero feedback.  It just jumps back to the prompt.  

Then we need better error messages! It should complain that you haven't given it any recipes to verify (because you haven't)

The only way I was able to get trust info to write to override by Greg's previous example of `autopkg update-trust-info ~/Library/AutoPkg/RecipeOverrides/*.recipe`

This is shell globbing at work. ~/Library/AutoPkg/RecipeOverrides/*.recipe is expanded by the _shell_ (and not by autopkg) to the paths of all the items ending with ".recipe" in the ~/Library/AutoPkg/RecipeOverrides/ directory.

Another thing I noticed is when running a recipe with any level of verbosity there's no output about the recipe being successfully trusted.  A clean run acts as if there is no trust information present.  I'd think at least acknowledging the trust info is there and valid would be beneficial to some level of verbose output.

Open for discussion on that point. I don't think /usr/sbin/installer tells you anything at any level of verbosity if a signed pkg is OK, but I understand your point.

[Gregory Neagle]

On Aug 2, 2016, at 6:56 PM, Gregory Neagle <gregn...@mac.com> wrote:

 I've tried a few variations but it doesn't seem to do anything when used with verify-trust-info and update-trust-info.

`autopkg verify-trust-info --verbose --override-dir=/Users/eholtam/Library/AutoPkg/RecipeOverrides/*.recipe`

`autopkg verify-trust-info --verbose --override-dir=/Users/eholtam/Library/AutoPkg/RecipeOverrides/`

`autopkg verify-trust-info --verbose --override-dir=~/Library/AutoPkg/RecipeOverrides`

`autopkg verify-trust-info --verbose --override-dir=/Users/eholtam/Library/AutoPkg/RecipeOverrides/ /Users/eholtam/Library/AutoPkg/RecipeOverrides/*.recipe`

All result in zero feedback.  It just jumps back to the prompt.  

Then we need better error messages! It should complain that you haven't given it any recipes to verify (because you haven't)

Addressed here:

https://github.com/autopkg/autopkg/commit/95a16fe6439476bf546d6cc17d9974545bfdc254

-Greg

[Eric Holtam]

Thanks for the explanation above about the override-dir option.  Makes sense.

With the commit above I'm getting proper output when fubaring the command.

$ autopkg verify-trust-info --override-dir=/Users/eholtam/Library/AutoPkg/RecipeOverrides

Need at least one recipe name or path!

Usage: autopkg verify-trust-info [options] recipe_override_name [...]

Verify parent recipe trust information for a recipe override.

Other utilities use the method where ending a path with a trailing / means apply to the contents of the directory but that doesn't appear to be the case here. The first example below is just pathing to the directory.  The second successful run is globbing that same directory path. Is that expected due to the shell globbing as well?

$ autopkg update-trust-info ~/Library/AutoPkg/RecipeOverrides/

Didn't find a recipe for /Users/eholtam/Library/AutoPkg/RecipeOverrides/.

Cannot find a recipe for /Users/eholtam/Library/AutoPkg/RecipeOverrides/.

$ autopkg update-trust-info ~/Library/AutoPkg/RecipeOverrides/*

Wrote updated /Users/eholtam/Library/AutoPkg/RecipeOverrides/PS4RemotePlay.download.recipe

Wrote updated /Users/eholtam/Library/AutoPkg/RecipeOverrides/SizeUp.download.recipe

-Eric

[Eric Holtam]

When a verification fails evaluating an override the error message is confusing.  The message is "No trust information present. Audit the recipe, then create an override to trust it." but the override already exists and is the recipe that is being evaluated.  My understanding is that these verbs would only be run on overrides in general, correct?  Would instructing the admin to use update-trust-info instead be more beneficial.  Something like:  "No trust information present. Audit the parent recipe, then use update-trust-info to trust it."

$ ./autopkg verify-trust-info -v ~/Library/AutoPkg/RecipeOverrides/* 

/Users/eholtam/Library/AutoPkg/RecipeOverrides/PS4RemotePlay.download.recipe: OK

/Users/eholtam/Library/AutoPkg/RecipeOverrides/SizeUp.download.recipe: OK

/Users/eholtam/Library/AutoPkg/RecipeOverrides/TextWrangler.download.recipe: FAILED

    No trust information present. Audit the recipe, then create an override to trust it.

[Gregory Neagle]

On Aug 2, 2016, at 7:50 PM, Eric Holtam <eho...@gmail.com> wrote:

Other utilities use the method where ending a path with a trailing / means apply to the contents of the directory but that doesn't appear to be the case here. The first example below is just pathing to the directory.  The second successful run is globbing that same directory path. Is that expected due to the shell globbing as well?

$ autopkg update-trust-info ~/Library/AutoPkg/RecipeOverrides/

Didn't find a recipe for /Users/eholtam/Library/AutoPkg/RecipeOverrides/.

Cannot find a recipe for /Users/eholtam/Library/AutoPkg/RecipeOverrides/.

$ autopkg update-trust-info ~/Library/AutoPkg/RecipeOverrides/*

Wrote updated /Users/eholtam/Library/AutoPkg/RecipeOverrides/PS4RemotePlay.download.recipe

Wrote updated /Users/eholtam/Library/AutoPkg/RecipeOverrides/SizeUp.download.recipe

A simple example of using update-trust-info:

autopkg update-trust-info Firefox.munki

A more complex example:

autopkg update-trust-info Firefox.munki GoogleChrome.munki

`autopkg update-trust-info` expects the names and/or paths to one or more recipes. At no time is the path to a directory an expected input. This is precisely the same as `autopkg run`:

autopkg run ~/Library/AutoPkg/RecipeOverrides/

Didn't find a recipe for /Users/gneagle/Library/AutoPkg/RecipeOverrides/.

Search GitHub AutoPkg repos for a /Users/gneagle/Library/AutoPkg/RecipeOverrides/ recipe? [y/n]: 

One could take advantage of shell globbing with autopkg run as well:

autopkg run ~/Library/AutoPkg/RecipeOverrides/*

(which would fail on any files inside ~/Library/AutoPkg/RecipeOverrides that were not recipes, so the safer variant is:

autopkg run ~/Library/AutoPkg/RecipeOverrides/*.recipe

Supporting autopkg is hard enough -- shell globbing really has nothing at all to do with autopkg. Compare for example:

cat ~/Library/AutoPkg/RecipeOverrides/

cat ~/Library/AutoPkg/RecipeOverrides/*

[Gregory Neagle]

On Aug 2, 2016, at 8:07 PM, Eric Holtam <eho...@gmail.com> wrote:

When a verification fails evaluating an override the error message is confusing.  The message is "No trust information present. Audit the recipe, then create an override to trust it." but the override already exists and is the recipe that is being evaluated.  My understanding is that these verbs would only be run on overrides in general, correct?  Would instructing the admin to use update-trust-info instead be more beneficial.  Something like:  "No trust information present. Audit the parent recipe, then use update-trust-info to trust it."

$ ./autopkg verify-trust-info -v ~/Library/AutoPkg/RecipeOverrides/* 

/Users/eholtam/Library/AutoPkg/RecipeOverrides/PS4RemotePlay.download.recipe: OK

/Users/eholtam/Library/AutoPkg/RecipeOverrides/SizeUp.download.recipe: OK

/Users/eholtam/Library/AutoPkg/RecipeOverrides/TextWrangler.download.recipe: FAILED

    No trust information present. Audit the recipe, then create an override to trust it.

This is good feedback. Earlier, it just said something like:

/Users/eholtam/Library/AutoPkg/RecipeOverrides/TextWrangler.download.recipe: FAILED
    No trust information present.

with no info on what to do. I may go back to that, since there are multiple ways to have a recipe fail:

autopkg verify-trust-info Firefox.munki

Just like `autopkg run Firefox.munki` , "Firefox.munki" is searched for. What is found _might_ be an override, and might be a "regular" recipe. If it's an override, the correct way to fix a failed trust verification is to audit the parent recipe(s), and then run `autopkg update-trust-info Firefox.munki`.  But if "Firefox.munki" resolves to a "regular" recipe, especially one that is part of a repo you added, the correct way to fix a failed trust verification is to audit this recipe itself and any parents, followed by creating an override with `autopkg make-override Firefox.munki`.

The current text refers to the second scenario. I had forgotten about the first scenario -- I've already added trust info to all my recipe overrides.

Since the full explanation is rather long, I don't think it should be printed after every failed verification, so I might just go back to "No trust information present.'

-Greg

On Tuesday, August 2, 2016 at 10:02:21 PM UTC-4, Greg Neagle wrote:

On Aug 2, 2016, at 6:56 PM, Gregory Neagle <gregn...@mac.com> wrote:

 I've tried a few variations but it doesn't seem to do anything when used with verify-trust-info and update-trust-info.

`autopkg verify-trust-info --verbose --override-dir=/Users/eholtam/Library/AutoPkg/RecipeOverrides/*.recipe`

`autopkg verify-trust-info --verbose --override-dir=/Users/eholtam/Library/AutoPkg/RecipeOverrides/`

`autopkg verify-trust-info --verbose --override-dir=~/Library/AutoPkg/RecipeOverrides`

`autopkg verify-trust-info --verbose --override-dir=/Users/eholtam/Library/AutoPkg/RecipeOverrides/ /Users/eholtam/Library/AutoPkg/RecipeOverrides/*.recipe`

All result in zero feedback.  It just jumps back to the prompt.  

Then we need better error messages! It should complain that you haven't given it any recipes to verify (because you haven't)

Addressed here:

https://github.com/autopkg/autopkg/commit/95a16fe6439476bf546d6cc17d9974545bfdc254

-Greg

[Gregory Neagle]

How about for scenario #1 (with different levels of verbosity):

autopkg verify-trust-info MSLync.munki 

MSLync.munki: FAILED

autopkg verify-trust-info MSLync.munki -v

MSLync.munki: FAILED

    No trust information present.

autopkg verify-trust-info MSLync.munki -vv

MSLync.munki: FAILED

    No trust information present.

    Audit the parent recipe, then run:

     autopkg update-trust-info MSLync.munki

and for scenario #2:

autopkg verify-trust-info Zulip.munki 

Zulip.munki: FAILED

autopkg verify-trust-info Zulip.munki -v

Zulip.munki: FAILED

    No trust information present.

autopkg verify-trust-info Zulip.munki -vv

Zulip.munki: FAILED

    No trust information present.

    Audit the recipe, then store trust info by running:

     autopkg make-override Zulip.munki

-Greg

[Eric Holtam]

I think more verbiage with higher verbosity is a good compromise and what most would expect.

-Eric

[Mr. Alan Siu]

Greg, first of all, amazing work on this!

I'm sorry I haven't had a chance to do some testing on this until now. I have, though, and the flow is great.

I agree with Erik, though, that the change may throw some people off if it just says something like

autopkg verify-trust-info AdobeFlashPlayer.munki

AdobeFlashPlayer.munki: FAILED

I think you should keep the code as is but then we should update the wiki to recommend people use the verbose flags when running overrides that fail, because this output is extremely helpful:

AdobeFlashPlayer.munki: FAILED

    Parent recipe com.github.autopkg.munki.FlashPlayerNoRepackage differs from expected.

    Recipe path: /Users/USERNAME/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/AdobeFlashPlayer/AdobeFlashPlayer.munki.recipe

    diff --git a/AdobeFlashPlayer/AdobeFlashPlayer.munki.recipe b/AdobeFlashPlayer/AdobeFlashPlayer.munki.recipe

    index 5bab13c..b62ed63 100644

    --- a/AdobeFlashPlayer/AdobeFlashPlayer.munki.recipe

    +++ b/AdobeFlashPlayer/AdobeFlashPlayer.munki.recipe

    @@ -3,7 +3,7 @@

     <plist version="1.0">

     <dict>

         <key>Description</key>

    -    <string>Downloads Adobe Flash Player installer, imports into Munki.</string>

    +    <string>Downloads Adobe Flash installer, imports into Munki.</string>

         <key>Identifier</key>

         <string>com.github.autopkg.munki.FlashPlayerNoRepackage</string>

         <key>Input</key>

Alan Siu

Client Systems Analyst

St. Ignatius College Preparatory

--
You received this message because you are subscribed to the Google Groups "autopkg-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to autopkg-discu...@googlegroups.com.

[Tim Sutton]

I finally had some time to try out this branch and was happy to find that pretty much everything behaved as I'd expect, despite my not having read this entire thread very closely.

Getting the signature verification failure with -vv printing the diff output is great. My only piece of feedback here would be that it could be helpful to also show the commit message that's relevant to those changes. For example, I got a FAILED for Jesse Peterson's VMwareTools.munki recipe, but the diff was enormous because his change was having converted tabs to spaces. However, that commit also had another change within it, but it wasn't obvious from just looking at the diff because it was large.

I realize you'd need to then be able to triangulate which commits were relevant to those particular files since the signature was captured, which might not be trivial. Maybe something we could look at in the future, this is obviously a nitpick. I think for the vast majority of people who really care about verifying all recipe changes, they'll have to be comfortable parsing diffs because that's actually the point. It would just be also helpful to be able to see someone's reasoning behind the changes.

I'm not necessarily recommending that we try and do that before merging this, just as something to think about.

I also had one exceptional case where I've made my own download recipe for something not public, that was stored along with my overrides. I saw the "FAILED" status for verify-trust-info because it has no parent recipe info to trust anyway. At first I found this potentially misleading but when I ran verify-trust-info with -vv it instructed me to make an override for it, which made sense. And I doubt anyone is storing their own recipes alongside overrides.

Tim

To unsubscribe from this group and stop receiving emails from it, send an email to autopkg-discuss+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "autopkg-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to autopkg-discuss+unsubscribe@googlegroups.com.

[Anthony Reimer]

Further to this specific point…

> On Aug 5, 2016, at 8:40 AM, Tim Sutton <t...@synthist.net> wrote:
>
> I think for the vast majority of people who really care about verifying all recipe changes, they'll have to be comfortable parsing diffs because that's actually the point. It would just be also helpful to be able to see someone's reasoning behind the changes.

Can AutoPkg access the description and/or comment included with the latest commit? In the case Tim cites, just adding that one line could help separate the significant differences from the insignificant ones. Even a link to the commit on GitHub might be enough.

In the case Tim cited, the ImageConverter.py processor had a multi-line comment attached to the commit (thanks, Greg) that makes it very clear what the changes do. Whether that is too much info is a point of discussion. And, of course, that info is only as good as the person who supplied it. But it might give additional guidance, especially to the less-experienced admin who is trying to adopt best practices.


Anthony Reimer

[Gregory Neagle]

What happens if the user runs repo-update foo, and pulls in several different commits because he/she hasn't run repo-update in a long time? The last commit may not be the _relevant_ commit. We'd want to see if we can extract _all_ of the commit messages since our git commit hash...

-Greg

> --
> You received this message because you are subscribed to the Google Groups "autopkg-discuss" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to autopkg-discu...@googlegroups.com.

[Anthony Reimer]

I definitely saw that as a possible issue. But since I didn't (and still don't) know what is possible in this regard, I thought the most recent (flagged as such) would at least be a starting point.

Anthony Reimer

[Gregory Neagle]

Certainly we can try to give the admin information. But ultimately they are going to have to understand at least some things about what is going on. If we can give more _relevant_ info, great. If we can only present additional info that may or may not be relevant, I'm less inclined to do it.

-Greg

[Gregory Neagle]

And now for an alternative view:

If I was a malicious actor, I'd create a recipe repo with a recipe that was perfectly valid and functional, and wait for many people to start using it.

Then later, I'd slip in my malicious changes, but in the same commit, introduce other changes, perhaps with lots of noise (meaningless extra changes), and provide a commit message that directed attention away from the malicious change.

So I'm not really sure providing commit messages is helpful from a security angle, if the author _intends_ mischief.

-Greg

[Tim Sutton]

I agree that being able to have a commit message isn't necessarily to help security, it's just to provide additional context to someone who's auditing any changes.

And to your earlier point, I agree it's non-trivial depending on the state of one's checked-out repo (both locally and relative to upstream changes). So I'm more in favour of getting in what's been done so far and releasing it than addressing the additional complexities there.

Tim

On Fri, Aug 5, 2016 at 12:16 PM, Gregory Neagle <gregn...@mac.com> wrote:

And now for an alternative view:

If I was a malicious actor, I'd create a recipe repo with a recipe that was perfectly valid and functional, and wait for many people to start using it.

Then later, I'd slip in my malicious changes, but in the same commit, introduce other changes, perhaps with lots of noise (meaningless extra changes), and provide a commit message that directed attention away from the malicious change.

So I'm not really sure providing commit messages is helpful from a security angle, if the author _intends_ mischief.

-Greg

> On Aug 5, 2016, at 8:44 AM, Gregory Neagle <gregn...@mac.com> wrote:
>
> Certainly we can try to give the admin information. But ultimately they are going to have to understand at least some things about what is going on. If we can give more _relevant_ info, great. If we can only present additional info that may or may not be relevant, I'm less inclined to do it.
>
> -Greg
>
>> On Aug 5, 2016, at 8:40 AM, Anthony Reimer <are...@ucalgary.ca> wrote:
>>
>> I definitely saw that as a possible issue. But since I didn't (and still don't) know what is possible in this regard, I thought the most recent (flagged as such) would at least be a starting point.
>>
>> Anthony Reimer
>>
>>
>>
>>> On Aug 5, 2016, at 9:35 AM, Gregory Neagle <gregn...@mac.com> wrote:
>>>
>>> What happens if the user runs repo-update foo, and pulls in several different commits because he/she hasn't run repo-update in a long time? The last commit may not be the _relevant_ commit. We'd want to see if we can extract _all_ of the commit messages since our git commit hash...
>>>
>>> -Greg
>>>
>>>
>>>> On Aug 5, 2016, at 8:30 AM, Anthony Reimer <are...@ucalgary.ca> wrote:
>>>>
>>>> Further to this specific point…
>>>>
>>>>> On Aug 5, 2016, at 8:40 AM, Tim Sutton <t...@synthist.net> wrote:
>>>>>
>>>>> I think for the vast majority of people who really care about verifying all recipe changes, they'll have to be comfortable parsing diffs because that's actually the point. It would just be also helpful to be able to see someone's reasoning behind the changes.
>>>>
>>>>
>>>> Can AutoPkg access the description and/or comment included with the latest commit? In the case Tim cites, just adding that one line could help separate the significant differences from the insignificant ones. Even a link to the commit on GitHub might be enough.
>>>>
>>>> In the case Tim cited, the ImageConverter.py processor had a multi-line comment attached to the commit (thanks, Greg) that makes it very clear what the changes do. Whether that is too much info is a point of discussion. And, of course, that info is only as good as the person who supplied it. But it might give additional guidance, especially to the less-experienced admin who is trying to adopt best practices.
>>>>
>>>>
>>>> Anthony Reimer
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google Groups "autopkg-discuss" group.

>>>> To unsubscribe from this group and stop receiving emails from it, send an email to autopkg-discuss+unsubscribe@googlegroups.com.

>>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups "autopkg-discuss" group.

>>> To unsubscribe from this group and stop receiving emails from it, send an email to autopkg-discuss+unsubscribe@googlegroups.com.

>>> For more options, visit https://groups.google.com/d/optout.
>>
>> --
>> You received this message because you are subscribed to the Google Groups "autopkg-discuss" group.

>> To unsubscribe from this group and stop receiving emails from it, send an email to autopkg-discuss+unsubscribe@googlegroups.com.

>> For more options, visit https://groups.google.com/d/optout.
>
> --
> You received this message because you are subscribed to the Google Groups "autopkg-discuss" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to autopkg-discuss+unsubscribe@googlegroups.com.

> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "autopkg-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to autopkg-discuss+unsubscribe@googlegroups.com.

[Gregory Neagle]

Here's relevant commit log output added:

autopkg verify-trust-info -vvv LibreOffice.munki.recipe
LibreOffice.munki.recipe: FAILED

    Expected non-core processor list: [u'LibreOfficeURLProvider']
    Actual non-core processor list: []
    Processor LibreOfficeURLProvider differs from expected.
    Processor LibreOfficeURLProvider can't be found.

    Parent recipe io.github.hjuutilainen.download.LibreOffice differs from expected.
    Recipe path: /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.hjuutilainen-recipes/LibreOffice/LibreOffice.download.recipe
    commit 8b6a277dd5915929ac29ae5e76a1de449f7ed10a
    Author: Hannes Juutilainen <hjuuti...@mac.com>
    Date:   Fri Jul 8 14:02:47 2016 +0300
    
        Added documentation and defaults for "fresh" and "still" release types in LibreOffice recipes
    
    commit 42ddcf54aef79d22b8db39e6e63c97ca81fce779
    Author: Hannes Juutilainen <hjuuti...@mac.com>
    Date:   Fri Jul 8 14:00:42 2016 +0300
    
        Removed custom URL provider from LibreOffice download recipe and updated it to use HTTPS

    diff --git a/LibreOffice/LibreOffice.download.recipe b/LibreOffice/LibreOffice.download.recipe
    index d0ee986..16f0765 100644
    --- a/LibreOffice/LibreOffice.download.recipe
    +++ b/LibreOffice/LibreOffice.download.recipe

    @@ -3,17 +3,18 @@

     <plist version="1.0">
     <dict>
      <key>Description</key>

    - <string>Downloads the latest LibreOffice.</string>
    + <string>Downloads the latest LibreOffice. Set RELEASE to either "fresh" or "still".
    +
    +LibreOffice Still is the stable version that has undergone more testing (over a longer time). It is usually recommended for more conservative use.

    +LibreOffice Fresh is the stable version with the most recent features. Users interested in taking advantage of our most innovative features should download and use our fresh version.</string>
      <key>Identifier</key>

    Parent recipe io.github.hjuutilainen.munki.LibreOffice differs from expected.
    Recipe path: /Users/gneagle/Library/AutoPkg/RecipeRepos/com.github.autopkg.hjuutilainen-recipes/LibreOffice/LibreOffice.munki.recipe
    commit 8b6a277dd5915929ac29ae5e76a1de449f7ed10a
    Author: Hannes Juutilainen <hjuuti...@mac.com>
    Date:   Fri Jul 8 14:02:47 2016 +0300
    
        Added documentation and defaults for "fresh" and "still" release types in LibreOffice recipes

    diff --git a/LibreOffice/LibreOffice.munki.recipe b/LibreOffice/LibreOffice.munki.recipe
    index b60ec1c..6205d83 100644
    --- a/LibreOffice/LibreOffice.munki.recipe
    +++ b/LibreOffice/LibreOffice.munki.recipe

    @@ -3,13 +3,18 @@

     <plist version="1.0">
     <dict>
      <key>Description</key>

    - <string>Downloads the current release version of LibreOffice and imports into Munki.</string>
    + <string>Downloads the current release version of LibreOffice and imports into Munki. Set RELEASE to either "fresh" or "still".
    +
    +LibreOffice Still is the stable version that has undergone more testing (over a longer time). It is usually recommended for more conservative use.

    +LibreOffice Fresh is the stable version with the most recent features. Users interested in taking advantage of our most innovative features should download and use our fresh version.</string>
      <key>Identifier</key>

      <string>io.github.hjuutilainen.munki.LibreOffice</string>
      <key>Input</key>
      <dict>
      <key>NAME</key>
      <string>LibreOffice</string>
    + <key>RELEASE</key>
    + <string>fresh</string>
      <key>MUNKI_REPO_SUBDIR</key>
      <string>apps/LibreOffice</string>
      <key>MUNKI_CATEGORY</key>

What's missing from an "ideal" version of this output (to me) would be the commit/log message where the LibreOfficeURLProvider was removed. I'm not sure how we'd get that since the path no longer exists.

-Greg

To unsubscribe from this group and stop receiving emails from it, send an email to autopkg-discu...@googlegroups.com.

[Gregory Neagle]

And now the relevant log info when a processor or recipe is removed:

autopkg verify-trust-info LibreOffice.munki.recipe -vv

LibreOffice.munki.recipe: FAILED

    Expected non-core processor list: [u'LibreOfficeURLProvider']

    Actual non-core processor list: []

    Processor LibreOfficeURLProvider differs from expected.

    Processor LibreOfficeURLProvider can't be found.

    commit f4d4a33efe9267f8617cb56aa63c517d954a9752

    Author: Hannes Juutilainen <hjuuti...@mac.com>

    Date:   Fri Jul 8 14:01:14 2016 +0300

    

        Deleted LibreOfficeURLProvider.py

[Gregory Neagle]

I've committed new code; you'll want to update the trust info on any overrides you have since I've changed what data is tracked.

verification output has been simplified a bit from the sample below; it's a tiny bit clearer and easier to read.

-Greg

[Erik Gomez]

Is it possible to have all commits in the recipe directories require at least two people? This would require any two people in the autopkg group for any subsequent commits. While this might be annoying (if possible) I think it would significantly increase the overall security to recipes. 

[Andrew Valentine]

As an AutoPkg user without the coding skills to really contribute, I thought I'd test some common tasks on this branch.

Using this thread, I've been able to follow the changes made, and produce the expected results, outlined below.

First, testing that verify-trust-info and update-trust-info work as expected on an existing override:

> git clone -b recipe-checksumming https://github.com/autopkg/autopkg.git ~/mac-tools/autopkg

> cd ~/mac-tools/autopkg

> ./Code/autopkg verify-trust-info ~/Library/AutoPkg/RecipeOverrides/GoogleChrome.munki.recipe

/Users/andrew/Library/AutoPkg/RecipeOverrides/GoogleChrome.munki.recipe: FAILED

> ./Code/autopkg update-trust-info ~/Library/AutoPkg/RecipeOverrides/GoogleChrome.munki.recipe

Wrote updated /Users/andrew/Library/AutoPkg/RecipeOverrides/GoogleChrome.munki.recipe

> ./Code/autopkg verify-trust-info ~/Library/AutoPkg/RecipeOverrides/GoogleChrome.munki.recipe

/Users/andrew/Library/AutoPkg/RecipeOverrides/GoogleChrome.munki.recipe: OK

> cat ~/Library/AutoPkg/RecipeOverrides/GoogleChrome.munki.recipe

---SNIP---

    <key>ParentRecipeTrustInfo</key>

    <dict>

        <key>non_core_processors</key>

        <dict/>

        <key>parent_recipes</key>

        <dict>

            <key>com.github.autopkg.download.googlechrome</key>

            <dict>

                <key>git_hash</key>

                <string>f29f399c54adee7122e9fdc3c1ea7ed4c67388a4</string>

                <key>path</key>

                <string>/Users/andrew/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/GoogleChrome/GoogleChrome.download.recipe</string>

                <key>sha256_hash</key>

                <string>880dbbc39342ba18bb4609a4746be75fa4b41667840201876c0f936aef7599e6</string>

            </dict>

            <key>com.github.autopkg.munki.google-chrome</key>

            <dict>

                <key>git_hash</key>

                <string>5f5db38265797f1754def56335de7bfc75811305</string>

                <key>path</key>

                <string>/Users/andrew/Library/AutoPkg/RecipeRepos/com.github.autopkg.recipes/GoogleChrome/GoogleChrome.munki.recipe</string>

                <key>sha256_hash</key>

                <string>8fd2feaa1c196109ee3bc4149b2447936782336f913c39875a4ad6c3e6dc4baa</string>

            </dict>

        </dict>

    </dict>

---SNIP---

Second, testing that trust information is created automatically on a new override:

> ./Code/autopkg make-override OmniOutliner.munki

Override file saved to /Users/andrew/Library/AutoPkg/RecipeOverrides/OmniOutliner.munki.recipe

> cat ~/Library/AutoPkg/RecipeOverrides/OmniOutliner.munki.recipe

---SNIP---

    <key>ParentRecipe</key>

    <string>com.github.autopkg.munki.omnioutliner</string>

    <key>ParentRecipeTrustInfo</key>

    <dict>

        <key>non_core_processors</key>

        <dict/>

        <key>parent_recipes</key>

        <dict>

            <key>com.github.autopkg.download.omnigroupproduct</key>

            <dict>

                <key>git_hash</key>

                <string>ed89f39e833d51ec92d0f977e444c9c717114413</string>

                <key>sha256_hash</key>

                <string>714d5006583cc813a5d5bbda5e3fb6e55f82c1fa9aac861453c76c1c01d98e30</string>

            </dict>

            <key>com.github.autopkg.download.omnioutliner</key>

            <dict>

                <key>git_hash</key>

                <string>eb989410d827c8ebda4fe8007c8701ba4bbdaf1d</string>

                <key>sha256_hash</key>

                <string>af1e775d960ee969cc6b41c49acde2bd3f4f3e27cb7c45a9a1b5be3ceda93649</string>

            </dict>

            <key>com.github.autopkg.munki.omnioutliner</key>

            <dict>

                <key>git_hash</key>

                <string>a5258794ddc9e34b8304149d4511697e69f70121</string>

                <key>sha256_hash</key>

                <string>9d2f7ee6cb34500f42c1dfb5e51ff582fc37f0e97aa48838d7a92ad70328eac7</string>

            </dict>

        </dict>

    </dict>

---SNIP---

Third, performing an `autopkg run` against a recipe_list.txt. I haven't set FAIL_RECIPES_WITHOUT_TRUST_INFO, so this completed successfully, the only errors being 'normal' ones, such as changed vendor URLs. I know that the changes made shouldn't affect the core operation, but I did this just for the sake of completeness, and so the maintainers know that normal usage is being tested successfully in 'the wild'.

I have one question about setting FAIL_RECIPES_WITHOUT_TRUST_INFO:

> defaults write ~/Library/Preferences/com.github.autopkg FAIL_RECIPES_WITHOUT_TRUST_INFO 0

> ./Code/autopkg run Go.munki

Processing Go.munki...

Failed.

The following recipes failed:

    Go.munki

        No trust information present.

Nothing downloaded, packaged or imported.

> defaults write ~/Library/Preferences/com.github.autopkg FAIL_RECIPES_WITHOUT_TRUST_INFO 1

> ./Code/autopkg run Go.munki

Processing Go.munki...

Failed.

The following recipes failed:

    Go.munki

        No trust information present.

Nothing downloaded, packaged or imported.

> defaults delete ~/Library/Preferences/com.github.autopkg FAIL_RECIPES_WITHOUT_TRUST_INFO

> defaults write ~/Library/Preferences/com.github.autopkg FAIL_RECIPES_WITHOUT_TRUST_INFO

2016-08-07 15:49:41.492 defaults[6907:37955]

Rep argument is not a dictionary

Defaults have not been changed.

In my testing, it looks like FAIL_RECIPES_WITHOUT_TRUST_INFO needs an arbitrary value assigned to it to function as expected. Is that correct?

Thanks for all your work on this. This is a very useful enhancement, and thanks to the efforts that have gone into it, I've found it straightforward to test and have no concerns about implementing it on a production setup.

[Gregory Neagle]

On Aug 7, 2016, at 8:07 AM, Andrew Valentine <av1...@bristol.ac.uk> wrote:

In my testing, it looks like FAIL_RECIPES_WITHOUT_TRUST_INFO needs an arbitrary value assigned to it to function as expected. Is that correct?

It's a bit more subtle than that, and is related to how the various components like Python, defaults, and the shell behave.

To have autopkg pay attention to FAIL_RECIPES_WITHOUT_TRUST_INFO, you can set it to any value that Python will not treat as equivalent to False.

Some of the "false-like" values in Python you could set in a plist:

<false/>

<integer>0</integer>

<string></string> (an empty string)

<array/> (an empty array)

<dict/> (an empty dictionary)

And possibly <real>0.0</real>

In your tests you set two non-empty strings, "0" and "1". (To set integers, you would have had to use `defaults write com.github.autopkg FAIL_RECIPES_WITHOUT_TRUST_INFO  -int 0` -- but more likely you'd want set set a boolean value `defaults write com.github.autopkg FAIL_RECIPES_WITHOUT_TRUST_INFO  -bool NO`)

When using the CLI, for example:

autopkg run OracleJava8.munki -k FAIL_RECIPES_WITHOUT_TRUST_INFO=yes

Processing OracleJava8.munki...

Failed.

The following recipes failed:

    OracleJava8.munki

        No trust information present.

Nothing downloaded, packaged or imported.

The _only_ thing the shell allows you to pass is strings -- you can't set booleans or integers directly. So if you needed to set it to a "falsey" value, you'd do:

autopkg run OracleJava8.munki -k FAIL_RECIPES_WITHOUT_TRUST_INFO=""

Processing OracleJava8.munki...

WARNING: OracleJava8.munki is missing trust info and FAIL_RECIPES_WITHOUT_TRUST_INFO is not set. Proceeding...

Nothing downloaded, packaged or imported.

Note we're setting to an empty string, which Python will evaluate in boolean context as equivalent to False.

-Greg

[Andrew Valentine]

Thanks for the information. That makes sense, and your recommendations worked perfectly:

> defaults write com.github.autopkg FAIL_RECIPES_WITHOUT_TRUST_INFO -bool YES

> ./mac-tools/autopkg/Code/autopkg run Go.munki

Processing Go.munki...

Failed.

The following recipes failed:

    Go.munki

        No trust information present.

Nothing downloaded, packaged or imported.

> defaults write com.github.autopkg FAIL_RECIPES_WITHOUT_TRUST_INFO -bool NO

> ./mac-tools/autopkg/Code/autopkg run Go.munki

Processing Go.munki...

WARNING: Go.munki is missing trust info and FAIL_RECIPES_WITHOUT_TRUST_INFO is not set. Proceeding...

Nothing downloaded, packaged or imported.

>

Cheers!

Andrew

[Gregory Neagle]

This code branch has been merged into autopkg/master and the branch has been removed.

-Greg

[Gregory Neagle]

Initial documentation:

https://github.com/autopkg/autopkg/wiki/Autopkg-and-recipe-parent-trust-info

-Greg

[Tim Sutton]

A couple very minor recent updates to verify-trust-info:

- Properly handle Unicode characters in git log messages or diff output: https://github.com/autopkg/autopkg/commit/cf4de177f09e2f78ce0d840bdf6b90848b7b6abb

For reference, this commit to CharlesProxy.download from Allister is the log message that triggered a UnicodeDecodeError that prompted this fix: https://github.com/autopkg/arubdesu-recipes/commit/0cb08e4d42996c6c3a7d49f80be61710242bfbe1

- Output git diff in color: https://github.com/autopkg/autopkg/commit/611f9e187e1fec8caa3eb9f90fad0131be9bea1e

The latter I find helpful in viewing verify-trust-info output in the terminal. I think that with how color output is formatted in the terminal, any process capturing output that's not aware of ANSI color codes (like AutoPkgr?) should just do "the right thing" and display it without color. But, it would be helpful if anyone running AutoPkg in such contexts could test this further.

Tim