Supporting new "HyperDrive" Adobe CC installers

[Tim Sutton]

This week there were new Adobe CC "major updates" - which apparently weren't major enough to be branded 2016, but were major enough to be given unique app IDs and which install as new products. So in reality, these aren't technically updates, they're new base products.

These use a completely new installer engine called "HyperDrive" or "HD", and while CCP can build packages for them, the current Munki code does not yet know how to extract useful metadata like installs items.

I've begun some work to add this in the adobe-hyperdrive branch:

https://github.com/munki/munki/tree/adobe-hyperdrive

So far this has only been additions to the code used by makepkginfo (and thus munkiimport). My idea so far is to at least get parity with the RIBS (i.e. now "legacy") installer metadata as extracted by makepkginfo, and make sure the install/uninstall behaviour is sound.

These new installer metadatas seem like it might not be too difficult to also attempt extracting actual application install metadata in the future.

Big thanks to Patrick Fergus and Clayton Burlison for helping poke through some of this. With people getting ready for PSUMAC next week and uncertainty with how the update workflow will work optimally for future updates, it may be some time before the dust settles on these. Anyone interested in testing this is welcome to do so!


Tim

[Erik Gomez]

Slightly off topic but is the new installer better? I haven't been in the loop. 

It's still disappointing that they can't make native/normal packages. 

--
Find related discussion groups here:
https://github.com/munki/munki/wiki/Discussion-Group
---
You received this message because you are subscribed to the Google Groups "munki-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to munki-dev+...@googlegroups.com.
To post to this group, send email to munk...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Rod Christiansen]

The installers are much faster than before. That is the only good news, all the rest is a mess ... 

Thanks Tim for this!

[Tim Sutton]

Like Rod mentioned the new ones are much faster. What's also noteworthy is that Adobe now offers every version as a single self-contained installer, so CCP packages built won't need to be a single base installer + some additional updates.

The old "aam20" update feed used from CS5 up to now doesn't look like it will be used for future patches to applications that have already switched over to the HyperDrive installers.

The Creative Cloud Desktop App, from what I've heard at Adobe, still understands the notion of "patches", in that it won't require users to download large (up to 2GB) installers for every patch update. However it's not yet clear to me whether these will be easy to figure out how to download into our own systems and apply in the same manner that Munki has supported for Adobe's Creative app patch updaters using the "AdobePatchInstaller" tools.

Or, for example, being able to easily apply a patch to one common support package like Camera Raw, without needing to manually import large installers for apps including these updates. Plenty of unknowns here so far.

Tim

[Tim Sutton]

Another thing to point out about the current adobe-hyperdrive branch: it does not yet calculate an 'installed_size' pkginfo key for these new payloads, nor extract their metadata to store in the pkginfo under a 'payloads' key.

Tim

[Tim Sutton]

Having had some time to build out CCP installers for 8 new app versions, I've found that makepkginfo/munkiimport crashes on importing an Illustrator CC 2015.3 installer. Further inspection needed as to what's different about that particular package.

Some other items on the TODO list (any thoughts on these welcome):

- calculate installed_size

- generate better version numbers and display names if a single product is present in a package

- consider generating installs apps for the actual app bundles when possible, instead of figuring out Adobe's "receipt" xml files in /L/Application Support/PCF, which function similarly to the RIBS installers' Uninstall.db files

It would also be nice to generate a payloads array as previous Adobe CS/CC installers did, although this seems less critical as 1) they aren't used for the uninstallation, and 2) the install times are an order of magnitude faster (2 minutes to install four large apps in a VM after Munki had cached the installers), so having progress info throughout the install seems less relevant (as long as Munki properly reports success or failure, which it seems to so far in my limited testing).

Tim

[Gregory Neagle]

On Jun 28, 2016, at 2:53 PM, Tim Sutton <t...@synthist.net> wrote:

Having had some time to build out CCP installers for 8 new app versions, I've found that makepkginfo/munkiimport crashes on importing an Illustrator CC 2015.3 installer. Further inspection needed as to what's different about that particular package.

Some other items on the TODO list (any thoughts on these welcome):

- calculate installed_size

- generate better version numbers and display names if a single product is present in a package

- consider generating installs apps for the actual app bundles when possible, instead of figuring out Adobe's "receipt" xml files in /L/Application Support/PCF, which function similarly to the RIBS installers' Uninstall.db files

It would also be nice to generate a payloads array as previous Adobe CS/CC installers did, although this seems less critical as 1) they aren't used for the uninstallation, and 2) the install times are an order of magnitude faster (2 minutes to install four large apps in a VM after Munki had cached the installers), so having progress info throughout the install seems less relevant (as long as Munki properly reports success or failure, which it seems to so far in my limited testing).

If we decide we don't care about displaying progress info, then perhaps we should just avoid all the special Munki handling of these items and just hand the pkg over to /usr/sbin/installer. (and also have munkiimport treat this exactly as it does any other pkg) This would also leave a pkg receipt in the receipt database, which might avoid the need to generate an installs array...

-Greg

[Tim Sutton]

That's interesting. At least one thing I think would still be needed, thought, would be for Munki to know that it's to do the uninstallation via the uninstaller package that's generated by CCP.

The new HD installers support doing a command-line uninstall (which could be added to adobeutils.py), but since it's possible to build a CCP package containing both legacy (RIBS) and new installers, it seems like continuing to just use the uninstaller package would be much simpler.

One difference between uninstalling via the pkg vs the CLI, is that if your package includes a device activation or serial, then the package uninstallation will also remove that, whereas via the CLI the license will remain activated but the product uninstalled.

Tim

[Gregory Neagle]

On Jun 28, 2016, at 4:30 PM, Tim Sutton <t...@synthist.net> wrote:
>
> One difference between uninstalling via the pkg vs the CLI, is that if your package includes a device activation or serial, then the package uninstallation will also remove that, whereas via the CLI the license will remain activated but the product uninstalled.

<headdesk>

<facepalm>

[Clayton Burlison]

Hey that's supposedly going to be fixed in a future update (July). So we'll be able to close that bug from April of 2015! :D

Source: https://macadmins.slack.com/archives/adobeinstalls/p1466624698002038 w/ answer https://macadmins.slack.com/archives/adobeinstalls/p1466625753002116

As for the HD installer stuff, I have a few ideas on how we can obtain the required bits for the TODO list above. I _think_ we have enough information we can generate the data needed for munki however it might be Friday before I have enough back to back time for me to test my theory. Pretty sure all the data needed is inside of the json files we just need to combine it and manipulate it. That also means we would have to trust Adobe to do their job and be consistent in their packages. At least with 8 new applications now I'll be able to see if they are consistent up to this point.

.../HD_based_Install.pkg/Contents/Resources/HD/SAP##/Application.json

[Tim Sutton]

Yes, with the length of time it's taken for Adobe to address that issue about removing licenses when other applicable products are also installed on the system, that for a long time I questioned what the desired behaviour was (and another year into deploying CC I'm that much more cynical, and now realize that many behaviours are undefined anyway).

I think all of the above in my TODOs are doable given the metadata that's available (certainly sizes, display names, versions, etc. are all there).

As for relying on just receipts, Adobe's included pkg receipt isn't very descript. A sample:

<key>CFBundleDisplayName</key>

<string>AdobePhotoshopCC2015.5</string>

<key>CFBundleGetInfoString</key>

<string>Copyright 2009-2012 Adobe Systems Incorporated. All rights reserved.</string>

<key>CFBundleIdentifier</key>

<string>com.adobe.Enterprise.install.EC279CA7-4E16-440E-B30B-62CB367402F7</string>

<key>CFBundleShortVersionString</key>

<string>1</string>

<key>IFMajorVersion</key>

<integer>1</integer>

<key>IFMinorVersion</key>

<integer>0</integer>

We _could_ use this receipt to track installed status, though I feel it's more robust to use something more significant to the installation (at least similar to the Uninstall.db files) in a situation like, for example, a single payload fails but for whatever reason the `installer` session succeeds. This is just speculative of course, I can't say whether I've seen this happen.

The version info in this pkg Info plist is of course bogus, but it will have a differing UUID in the identifier. I can only hope that future packages of the same configuration (but later versions of an app) would have a new UUID.

I think that given the new updaters will be full installers, it's possible that there will be many more CCP-style packages being used for applying future updates (at least until another alternative workflow similar to aamporter emerges), 

The required system version (10.6.8, not shown here) is also wrong. I haven't found minimum system versions in places other than the actual feed used by CCDA, which I've so far only inspected on a local mirror replicated with the new AUSST 4. However if I got ambitious enough to attempt automatic generation of installs keys for apps, many of these apps seem to at least properly define LSMinimumSystemVersion, so perhaps makepkginfo would handle inserting minimum_os_version for us later as it does with standard copy_from_dmg apps.

Tim

Tim

[Tim Sutton]

The adobe-hyperdrive branch has been merged to master.

Still, more testing by anyone who plans to distribute any kind of Adobe CCP packages (containing apps from any generation) would be most welcome, since the composition of CCP packages in Munki probably varies a lot across organizations.

Tim

To unsubscribe from this group and stop receiving emails from it, send an email to munki-dev+unsubscribe@googlegroups.com.

[Sam Tarr]

Worked great for us. I used the new build and installed the latest versions of Acrobat, Auditions, Photoshop, Illustrator, InDesign, Bridge, Premier, Light Room and After Effects on about 60 machines all running 10.11.6. 

To unsubscribe from this group and stop receiving emails from it, send an email to munki-dev+...@googlegroups.com.

[Tim Sutton]

Patrick Fergus also pointed out to me that there are a few outstanding issues with generating installs metadata for CCP packages, which isn't any change from prior behaviour (i.e. these never did the right thing, and needed user-crafted metadata to be added) but figured I'd point it out anyway. The following types of CCP package contents don't generate any installs metadata:

- Acrobat DC

- Creative Cloud Desktop Application-only packages 

- RIBS-based update-only packages

Tim

To unsubscribe from this group and stop receiving emails from it, send an email to munki-dev+unsubscribe@googlegroups.com.

[Martinus Verburg]

Just did a basic test with the august 12 build where the hyperdrive branch was merged.  It installs and runs the imported CCP package fine on a test client running 10.10.5 with an older munki from august last year. Not sure if I should update the munkitools on the client Macs as well?  

I tested a CCP package with the Design&Web apps.  I also switched on the Elevated Privileges for the Creative Cloud desktop app, which appears to work to install extra apps for non-admin users.  When I tested it the first time around with a CCP package with only the CC desktop app in it, the serialization didn't work though.  I'm wondering whether this might eliminate the need for aamporter - you think?

I'm not sure what the consequence of the missing installs metadata for Acrobat DC and others is, but this did install OK in my basic test.

Martinus

[Gregory Neagle]

On Sep 9, 2016, at 9:14 AM, Martinus Verburg <mart.v...@me.com> wrote:

Just did a basic test with the august 12 build where the hyperdrive branch was merged.  It installs and runs the imported CCP package fine on a test client running 10.10.5 with an older munki from august last year. Not sure if I should update the munkitools on the client Macs as well?  

The only changes affect munkiimport and makepkginfo to help them generate more accurate/more useful pkginfo for the items. There were no changes (that I recall) to the Adobe installation code, so updating your clients is not necessary.

I tested a CCP package with the Design&Web apps.  I also switched on the Elevated Privileges for the Creative Cloud desktop app, which appears to work to install extra apps for non-admin users.  When I tested it the first time around with a CCP package with only the CC desktop app in it, the serialization didn't work though.  I'm wondering whether this might eliminate the need for aamporter - you think?

Only if you are happy with running yet another closed-source binary with root privileges from the company that brought you Adobe Flash.

And the bandwidth considerations of having all your clients download updates from the Internet, though that, presumably could be mitigated with AUSST.  So you get to maintain a Munki repo, possibly an Apple Software Update repo, now an Adobe repo. Later maybe a Microsoft repo. Will that all scale well? Won't it be fun to have to learn vendor-specific update tools for each vendor?

I'm not sure what the consequence of the missing installs metadata for Acrobat DC and others is, but this did install OK in my basic test.

It all comes down to https://github.com/munki/munki/wiki/How-Munki-Decides-What-Needs-To-Be-Installed

If munkiimport can't generate info Munki can use to determine installation status, you'll have to do it manually.

-Greg

[Miq Viq]

Hi,

tested installing and removing the latest CC 2015.x Photoshop and Illustrator with https://munkibuilds.org/2.8.1.2844/

Munki Status Window displays the finished payloads as expected.

Thanks!

[Gregory Neagle]

Thank you for the testing and the feedback!

-Greg

[foigus]

Tested against Photoshop CC 2015.3 (17.0), Photoshop CC 2015.3.1 (17.0.1), Illustrator CC 2015.3.1 (20.1), and Premiere Pro CC 2015.3 (10.4).  It works--Adobe appears to be skipping unneeded payloads, combined with Munki only seeing completed payloads leads to interesting progress feedback.  But it's definitely more information than we had before.

I also tested against a mixture of RIBS and HyperDrive single-application packages and installations were successful there as well.

- Patrick

[Tim Sutton]

Yesterday I downloaded an "All Apps" CC package from the AE Dashboard, which contains a mix of RIBS and HD installers, as well as the Acrobat Pro DC package.

I imported it into Munki with no modifications to the pkginfo, which had nearly 20 installs items generated, and had it install on a test client with no previous Adobe installations. It installed all the items and reported progress along the way. The only issue was that it enumerated 389 payloads to install and completed once it had counted 73, but payload "counting" has been somewhat off for a while and probably not trivial to fix..

The install was complete and on a subsequent Munki run it (correctly) didn't consider anything not installed.

Tim

To unsubscribe from this group and stop receiving emails from it, send an email to munki-dev+unsubscribe@googlegroups.com.

[Gregory Neagle]

On Sep 27, 2016, at 7:15 AM, Tim Sutton <t...@synthist.net> wrote:

Yesterday I downloaded an "All Apps" CC package from the AE Dashboard, which contains a mix of RIBS and HD installers, as well as the Acrobat Pro DC package.

I imported it into Munki with no modifications to the pkginfo, which had nearly 20 installs items generated, and had it install on a test client with no previous Adobe installations. It installed all the items and reported progress along the way. The only issue was that it enumerated 389 payloads to install and completed once it had counted 73, but payload "counting" has been somewhat off for a while and probably not trivial to fix..

Nor is it really _important_ to fix.

It's decidedly non-trivial, since we'd have to know/determine how many of the payloads will actually be _required_. I'm quite sure Adobe's installer can figure that out, but I'm not terribly interested in replicating its logic.

Consider the progress feedback to be exactly what it is: eye-candy to convince you the thing hasn't locked up doing nothing useful.

-Greg

[Tim Sutton]

Agreed!