Pkg on a Dmg and version numbers

[John Lockwood]

It is common for App updates to be distributed on a disk image and Munki would do a Drag and Drop type install for these.

I have no problems creating AutoPkg recipes for this type of installation.

It is common for Apple Installer packages to be distributed as just pkg files and again I have no problems with this.

It is less common but not unknown to have a pkg in a Disk Image but with the pkg name always being the same no matter the version and usually the .dmg file name changing to reflect the version.

What I am struggling with is a Disk Image with a consistent unchanging file name containing an installer pkg file which has a changing file name incorporating the version number in the file name.

The Pkg appears to be a 'Distribution' pkg.

The URL for the Dmg is https://downloads.systoolsgroup.com/mac-pst-converter.dmg

The current version of the above contains 

SysTools Mac PST Converter v8.2.pkg

My problem is that whilst I can get AutoPkg to import the pkg in to Munki and set a correct version number, when the recipe is run again on the same Dmg this results in two, three, etc. copies all being added to Munki.

This is not therefore the more common issue of Munki repeatedly installing the same version which can be solved by the usual methods, e.g. the Installs array.

[Nick McSpadden]

You need to rebuild the Munki catalogs in between imports. Typically, with AutoPkg, you would add MakeCatalogs.munki to the end of your runlist, or run it like this:
```
autopkg run -v MyRecipeName.munki MakeCatalogs.munki
```

With the catalogs up to date, MunkiImporter should correctly be able to figure out that a copy already exists in the repo and gracefully do nothing.

--
You received this message because you are subscribed to the Google Groups "autopkg-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to autopkg-discu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/autopkg-discuss/cc7ad3f5-bf9e-443b-9a7f-a2ada1c1084bn%40googlegroups.com.

--

--
Nick McSpadden
nmcsp...@gmail.com

[John Lockwood]

It is not (unfortunately) the MakeCatalogs process, that is working and as a result multiple copies are being successfully added to the Munki Repo and showing up, that is complete with pkginfo etc.

The main issue is the stupid way the dev has made the disk image and likely my own stupidity regarding recipes.

To summarise the disk image is called SysTools Mac PST Converter.dmg it contains in its current version the installer package called SysTools Mac PST Converter v8.2.pkg 

My initial attempt of recipes using RecipeRobot resulted in using the URL https://downloads.systoolsgroup.com/mac-pst-converter.dmg and RecipeRobot  generated recipes which were hardcoded to look for a pkg called SysTools Mac PST Converter v8.2.pkg 

It was easy to modify the recipe to look for a wildcard pkg filename on the downloaded disk image so then it would look for either "SysTools Mac PST Converter*.pkg" or simply "*.pkg" this change worked. However then in order to 'find' the version of the pkg I appear to need to use FltPkgUnpacker. I can first use a PkgCopier to copy e.g. "SysTools Mac PST Converter v8.2pkg" to "SysTools Mac PST Converter.pkg" and then point FltPkgUnpacker to that but FltPkgUnpacker is extracting and producing along with some other files "SysTools_Mac_PST_Converter_v8.2.pkg" which as you can see has the version number hardcoded in it. PkgPayloadUnpacker needs to be told this name and UNLESS I hardcode it, this seems impossible to derive and pass to it as part of a variable. If I hardcode it, then it works but obviously this is not going to be able to handle other versions, if I try using wildcards or other approaches it fails with an error.

Even if I hardcode it and it works it then as originally reported it has been repeatedly copying and adding to the Munki Repo the same version.

[John Lockwood]

Ignoring for the moment the repeated copies the PkgPayloadUnpacker below works which is hardcoded

<dict>
<key>Arguments</key>
<dict>
<key>destination_path</key>
<string>%RECIPE_CACHE_DIR%/payload</string>
<key>pkg_payload_path</key>
<string>%RECIPE_CACHE_DIR%/unpack/SysTools_Mac_PST_Converter_v8.2.pkg/Payload</string>
<key>purge_destination</key>
<true/>
</dict>
<key>Processor</key>
<string>PkgPayloadUnpacker</string>

the following does not work

<dict>
<key>Arguments</key>
<dict>
<key>destination_path</key>
<string>%RECIPE_CACHE_DIR%/payload</string>
<key>pkg_payload_path</key>
<string>%RECIPE_CACHE_DIR%/unpack/*.pkg/Payload</string>
<key>purge_destination</key>
<true/>
</dict>
<key>Processor</key>
<string>PkgPayloadUnpacker</string>

As mentioned FltPkgExtractor itself is including the version number in filename of the pkg it produces. If it did not then I would not have this problem.

[John Lockwood]

Slightly good news, it is now no longer repeatedly doing copies but I still need help regarding the hardcoded version as explained above in PkgPayloadUnpacker.

[Nick McSpadden]

Use the FileFinder processor to locate a wildcard with a glob: 

https://github.com/autopkg/autopkg/wiki/Processor-FileFinder

To view this discussion on the web visit https://groups.google.com/d/msgid/autopkg-discuss/4b3e821d-61b2-4eb9-890a-b08323109fa6n%40googlegroups.com.

[Gregory Neagle]

It really sounds to me like you are working too hard, if the need here is to have autopkg download the latest dmg-containing-a-pkg and then to import that into Munki.

MunkiImporter in the .munki recipe will figure out the version of the pkg inside the dmg.

So you need a .download recipe that downloads the dmg and a .munki recipe that has the download recipe as a parent. The Process array of the munki recipe would be something like:

    <key>Process</key>

    <array>

        <dict>

            <key>Arguments</key>

            <dict>

                <key>pkg_path</key>

                <string>%pathname%</string>

                <key>repo_subdirectory</key>

                <string>%MUNKI_REPO_SUBDIR%</string>

            </dict>

            <key>Processor</key>

            <string>MunkiImporter</string>

        </dict>

    </array>

You do not need to jump through hoops to determine the version of the pkg; Munki’s code will do that.

The autopkg recipes that do do a lot of work determining the version usually do so for the benefit of Jamf admins, since the tools to import pkgs into Jamf don’t extract the version.

-Greg

To view this discussion on the web visit https://groups.google.com/d/msgid/autopkg-discuss/CAJySx-%3DcaR75Z6DtbMpQ5oQ-AKvjbbzTa-C_13f2y%2BxomZ7ERw%40mail.gmail.com.

[John Lockwood]

Unfortunately Greg, that is what the original recipe was doing and with that approach it DOES keep adding duplicate copies to the Munki Repo.

If I force it to look at the version it does not add duplicates to the Munki Repo.

:(

[Anthony Reimer]

We’re working with incomplete information. What would be helpful is a verbose run (-vv) of the recipe chain (or even supplying the recipes themselves), suitably edited for privacy. If for some reason MunkiImporter can’t determine the version number, then Nick’s suggestion of using FileFinder in conjunction with expanding the pkg should do the trick. (Rich Trouton’s repo is full of examples of doing just this for his pkg recipes, like Firefox.pkg.)

If you want a real kludge, you can extract the version number from the filename if you deem that to be a reliable source, but that should be a last resort.

Anthony Reimer


> On Jul 27, 2023, at 10:44 AM, John Lockwood <jeloc...@gmail.com> wrote:
>
> [△EXTERNAL]

> To view this discussion on the web visit https://groups.google.com/d/msgid/autopkg-discuss/313627a3-0a02-44bd-ae05-b0ff7c7d4860n%40googlegroups.com.

[Nick McSpadden]

I agree with Anthony - it would be most helpful to show a complete `autopkg run -vv` of this recipe, because I suspect we are not operating with the same details.

To view this discussion on the web visit https://groups.google.com/d/msgid/autopkg-discuss/914D97C8-FAFA-42FF-8537-BED2AAAB4BBB%40ucalgary.ca.

[John Lockwood]

Thank you everyone for your help.

It turned out that using FileFinder solved the problem. My recipe flow is now as follows.

• Absolutely bog standard download recipe which downloads the DMG.
  
• A pkg recipe which copies the pkg out of the DMG, then FltPkgUnpacker, then FileFinder to find the pkg produced which has a version number in the name, then does PayLoadUnpacker, then is able to use Versioner to get the version from the App Info.plist
  
• Then a bog standard Munki recipe doing the MunkiImport of the original (not unpacked) Pkg
  

This is now running with no errors and is not resulting in multiple copies being added to the Munki repo.

If anyone is interested the recipes are available at https://github.com/jelockwood/recipes/tree/master/SysTools%20Mac%20PST%20Converter

[Anthony Reimer]

You may wish to add a PathDeleter step at the end to clean up the unpack directory you create during the process. While it might not be absolutely necessary, doing so makes certain that there is nothing there the next time the recipe expands to that location and also keeps disk usage lower. Again, there is lots of prior art in the repos (rtrouton, even mine).

Anthony Reimer

> To view this discussion on the web visit https://groups.google.com/d/msgid/autopkg-discuss/22a69c16-f0c3-400e-bf16-30b228cde367n%40googlegroups.com.

[John Lockwood]

@Anthony Reimer

Good point. I have just now successfully added a PathDeleter processor at the end of the Munki recipe and defined an array of items to delete and it worked. Equally importantly it did not break anything. :)

[Anthony Reimer]

It should be at the end of pkg recipe. Thus, if someone wants to leverage your pkg recipe to put into a different management system (e.g., Jamf Pro), they still get the cleanup.

Anthony Reimer

> To view this discussion on the web visit https://groups.google.com/d/msgid/autopkg-discuss/0a1e3ed1-6dd1-4f6f-a11c-1d60d66262c8n%40googlegroups.com.

[Gregory Neagle]

On Jul 28, 2023, at 1:32 AM, John Lockwood <jeloc...@gmail.com> wrote:

• A pkg recipe which copies the pkg out of the DMG, then FltPkgUnpacker, then FileFinder to find the pkg produced which has a version number in the name, then does PayLoadUnpacker, then is able to use Versioner to get the version from the App Info.plist

This implies to me that that package itself does not have usable version info, or that the version info does not match the version of the software itself (which you are presumably defining as the value of CFBundleShortVersionString in an installed app’s Info.plist).

-Greg

[John Lockwood]

@Anthony Reimer I have moved the PathCleaner, however I found I had to leave one file and I can live with that.

@Greg quite likely. I always make sure to put the version in my own custom pkgs but sadly see many commercial ones which fail to do so. Bizarrely even when I started off with very basic download and Munki recipes it found a valid version and put this info in the pkginfo but kept adding duplicate copies. Now with reading the version from the app Info.plist it still shows the same version but does not result in duplicates.

For the curious the link to my now updates recipes on my Github is earlier in this discussion.