Public Munki Repository

[michaeljsmalley]

I had a spark of imagination this evening. I've been using Munki at
my company, for about 6 months now. It's working great, and it moves
my developer team ever closer toward consistent development MacBooks.
One year ago, one developer couldn't be sure another had the same
system setup. Now they can have increased confidence that they're on
the same page.

My idea is this:

Debian has apt with public repos
Red Hat has rpm with public repos
Suse has yum with public repos
Gentoo has portage with public repos
OS X has MacPorts/Fink/Homebrew with public repos... but they don't
support DMG and PKG files or native OS X installers, they just
download binaries and dump them into an arbitrary directory like /opt/
local/bin...

Why not create a public Munki repository? There are tons of freely
available packages available for OS X, and not everyone wants to use
the App Store. That, and the App Store isn't scriptable, and not all
developers/vendors make their software available via the App Store. A
public Munki repo would give the end user a voice. In the App Store
infrastructure, if a developer doesn't submit their app or make it
through the approval process, their app will never end up in the App
Store. In a munki infrastructure, the developer doesn't even need to
get involved. If their software is something users want in the repo,
they can request it get added (or maybe even add it themselves down
the line). Because munki operates over port 80, no ports would need
to be forwarded on their internal networks to get access to such a
repo.

I'm willing to put in the work needed to get such a repo up and
running, but I'd like to hear your input. I would also need a web host
that would allow such traffic in/out of the server. Maybe EC2 is an
option, but I don't know much about it so there'd be a bit of a
learning curve. It would also cost money, so some sort of fund-
raising would most definitely be in order.

I brought this idea up to Greg, and he suggested I propose it to the
munki-dev list, so here I am. What are your thoughts on this idea?

[Nathan Felton]

I'm personally majorly in favor of this. Something I have been looking at doing myself is creating a way to automate the download and (re)packaging of a lot of the freeware apps that are out there, e.g. Adobe Products, Web Browsers, FTP clients, etc. If we had a repo that already housed all the packages for these types of applications, i'm sure it would make a lot of peoples lives a lot simpler.

As far as web hosting goes, I would be interested in setting up a mirroring option for the repository, much like Firefox (http://www.mozilla.org/community/mirroring.html) and other Open Source projects do. This way people who have available resources (e.g. disk space and bandwidth) could add to the mirror pool. Here at RIT for example we host a lot of linux distros at http://mirrors.rit.edu/.

I personally have no experience yet with setting up a mirror environment, however, at first glimpse it would be nice to have a git repo somewhere as the master that gets replicated out to the mirror sites. This way only a select few could commit to the master repo to prevent malicious or erroneous changes to the repo and have them disseminate out to the mirrors.

Just my quick thoughts and 2 cents. I love the idea though. It would help me concentrate on keeping my local repo up to date without having to worry too much about the packages that change on a more frequent basis.

--

Nathan Felton

n8fe...@gmail.com

315-945-5656

[Justin McWilliams]

<plug>

Have you looked at Simian, the Google App Engine-based Munki solution
that myself and John Randolph developed and maintain?
http://code.google.com/p/simian/

Maybe fitting for such a use case.

- Justin

</plug>

On Thu, Dec 15, 2011 at 11:08 PM, michaeljsmalley
<michael...@gmail.com> wrote:

[Justin McWilliams]

However, maybe the idea of a public Munki server is flawed given that
it requires to run as root, has the idea of managed_installs with
unattended_install (background), force reboot, uninstall, etc. For
public you really only want 1) updating existing things 2) allow me to
optionally install other things. and nothing more.

[Michael Smalley]

Though these sorts of features may be considered overly intrusive for a public repository  ("Hey, all I did was install XCode and now my machine is rebooting itself!?!?")

That said, I don't think it's a deal-breaker.  I just think that initially these types of packages may be something worth excluding from the repo, and secondly, down the line, it may be worth investigating a way to make this a more user-friendly process in the first place (perhaps some sort of disclaimer before the install happens, or maybe grouping software that engages in some sort of activity that could be considered intrusive?)

--

Michael J. Smalley

 E:  michaeljsmalley <at> gmail.com

 W:  blog.smalleycreative.com

[Rob Middleton]

And - who do you trust? I've toyed with the idea. It is easier to trust a DMG type drag&drop installer. You are then delegating permission for a particular folder to someone else not the ability to change arbitrary files on the system. Even then - who signs that package? Preferably the original developer, not a repository maintainer -- the user has already given an implicit level of trust of the application developer.

I could see that applications signed/secured with the Sparkle framework update method could be handled with less problem.

Rob.

[Justin McWilliams]

there are still pre and post install scripts in pkginfos - for drag n drop dmgs even - that can run arbitrary code.

[Rob Middleton]

Indeed. Not saying it is possible to setup the trust boundaries within the present code. Just that drag & drop can be isolated, while package installers can't really (well maybe, but only with lots of work looking at what level of sandbox permissions the installer needs to run & then for each application allowing specific additional writeable paths to sandbox-exec for the installer).

[Andrew Rae]

On 16/12/2011 04:08, "michaeljsmalley" <michael...@gmail.com> wrote:
<snip>

>Why not create a public Munki repository? There are tons of freely
>available packages available for OS X, and not everyone wants to use
>the App Store. That, and the App Store isn't scriptable, and not all
>developers/vendors make their software available via the App Store. A
>public Munki repo would give the end user a voice. In the App Store
>infrastructure, if a developer doesn't submit their app or make it
>through the approval process, their app will never end up in the App
>Store. In a munki infrastructure, the developer doesn't even need to
>get involved. If their software is something users want in the repo,
>they can request it get added (or maybe even add it themselves down
>the line). Because munki operates over port 80, no ports would need
>to be forwarded on their internal networks to get access to such a
>repo.
>
>I'm willing to put in the work needed to get such a repo up and
>running, but I'd like to hear your input. I would also need a web host
>that would allow such traffic in/out of the server. Maybe EC2 is an
>option, but I don't know much about it so there'd be a bit of a
>learning curve. It would also cost money, so some sort of fund-
>raising would most definitely be in order.

I still think (I got flamed down once about this) that a publicly
modifiable wiki or 'repo' for just the pkginfo information for each
version of the software - a bit like app-deploy.com
<http://app-deploy.com> or wpkg.org <http://wpkg.org> on the pc side of
things, and maybe the ability to include the steps necessary to
(re)package software for deploying by munki. I don't see the benefit to
re-hosting already freely available software.
In my short time working with Munki - I've yet to see anything that needs
drastic work doing to be made deployable..

Just my 2p on the matter. Feel free to ignore!
Andrew

[Nate]

I think it would be best if people submitted pkginfos to be added to the existing munki repo.  I'm not sure if Greg agrees, but I think it would be ideal if pkginfos were submitted and looked over before being posted.  I don't like the idea of random pkginfos being posted by whoever without review.  I think newer munki users might be more apt to just grab pkginfo bits and expect them to work.  If they are reviewed before posting, there is a higher chance that the pkginfos will work for them as a template.

Nate

[Andrew Rae]

On 16/12/2011 12:16, "Nate" <nate....@gmail.com> wrote:

>I think it would be best if people submitted pkginfos to be added to the
>existing munki repo. I'm not sure if Greg agrees, but I think it would
>be ideal if pkginfos were submitted and looked over before being posted.
>I don't like the idea of random pkginfos being posted by whoever without
>review. I think newer munki users might be more apt to just grab pkginfo
>bits and expect them to work. If they are reviewed before posting, there
>is a higher chance that the pkginfos will work for them as a template.
>
>Nate

I agree - possibly a group of people that could look over them or test
them out. Or even a 'untested' and 'tested' code section. Once someone
trustworthy checks it out and verifies it, move to the 'tested' section.

I know there are a few people who've posted up their own code on various
sites, but its always good to be able to come back to the origin of the
software :)

Andrew

><http://app-deploy.com> or wpkg.org <http://wpkg.org> <http://wpkg.org>

[Michael Smalley]

This could work. :)

Obviously, it's a lot less ambitious, but it removes the burden and potentially risky legal nature of dealing with binaries, while still greatly assisting the Munki community.

[Ricky Chilcott]

I think Github's pull requests could be really handy here.  For instance, the munki-pkginfo repository could be created on GitHub and then as people would like to share, they can fork the repo, add their changes/pkginfos and then do a pull request to the owners of the repo (which could be a number of volunteers).  

Some people already have their Munki repos in GitHub  (with the exception of manifests and the actual packages).

Just a thought.

Ricky

[Jeff Strunk]

While it is a potential security issue, it is an issue Linux distributions
have dealt with for years.

* There needs to be a group of maintainers who trust each other.
* They need to build a reputation with the community that is going to use
software delivered from their repository. This happens just by existing and
putting software in their repository that is not malicious.
* The software and pkginfos need to be signed by the repository maintainers.
They are the ones gauranteeing that they did not put anything malicious into
the package.
* People who use a repository should be careful to only use reputable ones on
sensitive systems.

However, I also think there should be some abstraction. The Munki client
probably shouldn't get manifests from 3rd party repositorys. I think that a
Munki server in an organization could point to catalogs on 3rd party servers.
Administrators would then make manifests referencing packages in the 3rd party
repos for their clients to access.

Jeff

[Greg Neagle]

On Dec 16, 2011, at 4:16 AM, Nate wrote:

I think it would be best if people submitted pkginfos to be added to the existing munki repo.  I'm not sure if Greg agrees, but I think it would be ideal if pkginfos were submitted and looked over before being posted.  I don't like the idea of random pkginfos being posted by whoever without review.  I think newer munki users might be more apt to just grab pkginfo bits and expect them to work.  If they are reviewed before posting, there is a higher chance that the pkginfos will work for them as a template.

I think all of this is great, but a few comments:

1) I don't want to have to do any of this. This would be great for other Munki admins to pick up and run with. The pkginfo examples are already in a separate repo.

2) If an item can just be imported using munkiimport with no additional pkginfo editing, there's no reason for it to to be in the examples.

3) People should be aware of keys that are a matter of admin choice: catalogs, unattended_install, unattended_uninstall, force_install_after_date, etc.

4) Keys like installer_item_location can vary depending on admin choices

5) installer_item_hash can potentially vary as well.

-Greg

[Zack Williams]

On Dec 15, 2011, at 22:19 , Rob Middleton wrote:
>
> I could see that applications signed/secured with the Sparkle framework update method could be handled with less problem.

What you want is a trust chain - I implemented this (most of the way) in the MunkiAppcast (Appcast == Sparkle) code I wrote a while back but haven't had time to work on recently. That code takes a Sparkle feed and and developer's signing key, downloads, verifies, then puts an installable packages in a Munki repo.

I'm wondering if the greater problem could be solved by having a repo of "fetch scripts" that work like a BSD style ports tree - an organized collection of scripts that download and prep specific packages. This would be much simpler/smaller than duplicating binaries in a common repository, gets around issues of re-hosting other people's software (which some vendors really don't like), and could handle commercial software as well via a "mount the DVD/image, run the script" method.

On a similar note, has anyone added the ability to cryptographically sign and verify manifests in munki?

- Zack

--
Zack Williams - Artisan Computer Services - 520.867.8701
z...@artisancomputer.com http://www.artisancomputer.com
Apple Certified System Administrator, Apple Consultants Network
Microsoft Certified Professional, Small Business Specialist
Sun Certified System Administrator
Linux Professional Institute LPIC-1

[Greg Neagle]

On Dec 16, 2011, at 9:08 AM, Zack Williams wrote:

> On Dec 15, 2011, at 22:19 , Rob Middleton wrote:
>>
>> I could see that applications signed/secured with the Sparkle framework update method could be handled with less problem.
>
>
> What you want is a trust chain - I implemented this (most of the way) in the MunkiAppcast (Appcast == Sparkle) code I wrote a while back but haven't had time to work on recently. That code takes a Sparkle feed and and developer's signing key, downloads, verifies, then puts an installable packages in a Munki repo.
>
> I'm wondering if the greater problem could be solved by having a repo of "fetch scripts" that work like a BSD style ports tree - an organized collection of scripts that download and prep specific packages. This would be much simpler/smaller than duplicating binaries in a common repository, gets around issues of re-hosting other people's software (which some vendors really don't like), and could handle commercial software as well via a "mount the DVD/image, run the script" method.

MunkiServer from Simon Fraser University has methods for automatically retrieving updates using feeds as well.

>
> On a similar note, has anyone added the ability to cryptographically sign and verify manifests in munki?

Is that even needed if you use https on your Munki server? You are verifying the server at that point -- why would the manifests need to be verified further?

[Justin McWilliams]

On Fri, Dec 16, 2011 at 12:16 PM, Greg Neagle <gregn...@mac.com> wrote:
> On Dec 16, 2011, at 9:08 AM, Zack Williams wrote:
>
>> On Dec 15, 2011, at 22:19 , Rob Middleton wrote:
>>>
>>> I could see that applications signed/secured with the Sparkle framework update method could be handled with less problem.
>>
>>
>> What you want is a trust chain - I implemented this (most of the way) in the MunkiAppcast (Appcast == Sparkle) code I wrote a while back but haven't had time to work on recently.  That code takes a Sparkle feed and and developer's signing key, downloads, verifies, then puts an installable packages in a Munki repo.
>>
>> I'm wondering if the greater problem could be solved by having a repo of "fetch scripts" that work like a BSD style ports tree - an organized collection of scripts that download and prep specific packages.  This would be much simpler/smaller than duplicating binaries in a common repository, gets around issues of re-hosting other people's software (which some vendors really don't like), and could handle commercial software as well via a "mount the DVD/image, run the script" method.
>
> MunkiServer from Simon Fraser University has methods for automatically retrieving updates using feeds as well.
>
>>
>> On a similar note, has anyone added the ability to cryptographically sign and verify manifests in munki?
>
> Is that even needed if you use https on your Munki server? You are verifying the server at that point -- why would the manifests need to be verified further?

If the server is hacked, the hacker would be able to replace the
manifests and catalogs. If they were required to be signed by the
admin before the client would accept them, the admin would also have
to get hacked to own any of the machines managed by the server.

[Zack Williams]

On Dec 16, 2011, at 10:16 , Greg Neagle wrote:
> MunkiServer from Simon Fraser University has methods for automatically retrieving updates using feeds as well.

I'll have to take a look at that.

>> On a similar note, has anyone added the ability to cryptographically sign and verify manifests in munki?
>
> Is that even needed if you use https on your Munki server? You are verifying the server at that point -- why would the manifests need to be verified further?

I guess I was thinking of extending the trust chain to not rely only on SSL.

Does the Munki client verify the server's SSL cert against a certificate store that contains only the CA key that issued the server's cert, to avoid say a rouge CA problem? (I do admit this is somewhat farfetched)

[Nathan Felton]

On Dec 16, 2011, at 9:08 AM, Zack Williams wrote:
> On Dec 15, 2011, at 22:19 , Rob Middleton wrote:
>>
>> I could see that applications signed/secured with the Sparkle framework update method could be handled with less problem.
>
>
> What you want is a trust chain - I implemented this (most of the way) in the MunkiAppcast (Appcast == Sparkle) code I wrote a while back but haven't had time to work on recently.  That code takes a Sparkle feed and and developer's signing key, downloads, verifies, then puts an installable packages in a Munki repo.
>
> I'm wondering if the greater problem could be solved by having a repo of "fetch scripts" that work like a BSD style ports tree - an organized collection of scripts that download and prep specific packages.  This would be much simpler/smaller than duplicating binaries in a common repository, gets around issues of re-hosting other people's software (which some vendors really don't like), and could handle commercial software as well via a "mount the DVD/image, run the script" method.

 

MunkiServer from Simon Fraser University has methods for automatically retrieving updates using feeds as well.

Though I am not aware of how MunkiServer works yet, this was originally the idea was going with for downloading the commonly available and used applications. Much like the InstaDMGs "Up2Date" addon, create a catalog file that would list the locations to download the pkg from, along with the sha1 of the package. You would then have the repo automatically update itself with the latest versions. Again like "Up2Date" the catalog could be placed on GitHub and downloaded by sysadmins as desired.

This would skip the need for trusting anyone other than the developers of the application, which we already are trusting when we manually download their dmgs/pkgs from their sites now. We would then only be entrusting the sha1 in the catalog files on the repo, which we could easily verify for ourselves if the integrity of the catalog repo were in question.

Just more of my own 2 cents. This is the direction i'm looking at going. I eventually would like to have my environment run itself essentially and simply email me when a new package is available on the repo for testing before rolling out to production.

[John Randolph]

On Fri, Dec 16, 2011 at 12:46 PM, Zack Williams <z...@artisancomputer.com> wrote:

On Dec 16, 2011, at 10:16 , Greg Neagle wrote:
> MunkiServer from Simon Fraser University has methods for automatically retrieving updates using feeds as well.

I'll have to take a look at that.

>> On a similar note, has anyone added the ability to cryptographically sign and verify manifests in munki?
>
> Is that even needed if you use https on your Munki server? You are verifying the server at that point -- why would the manifests need to be verified further?

I guess I was thinking of extending the trust chain to not rely only on SSL.

Does the Munki client verify the server's SSL cert against a certificate store that contains only the CA key that issued the server's cert, to avoid say a rouge CA problem?   (I do admit this is somewhat farfetched)

this is not as far fetched as you might think. see Comodo or diginotar.nl hacks.

munki lets you set SoftwareRepoCAPath and SoftwareRepoCACertificate to tighten down your ssl usage.

- Zack

--
Zack Williams - Artisan Computer Services - 520.867.8701
z...@artisancomputer.com   http://www.artisancomputer.com
Apple Certified System Administrator, Apple Consultants Network
Microsoft Certified Professional, Small Business Specialist
Sun Certified System Administrator
Linux Professional Institute LPIC-1

--
John Randolph -- Google New York -- Corp Platforms Eng