Weird startup delay on OSX Catalina

[Nicolas de Loof]

Hi,

I'm working by Docker and investigating a weird issue reported on docker-compose, which is packaged as a standalone binary using PyInstaller. Thanks for maintaining this awesome project on your spare time by the way :)

Trying to reproduce this issue, I upgraded to Catalina on my own macbook, and noticed when I run an unknown command, which just display the "usage" help message, I always get an HTTP request sent to https://api.apple-cloudkit.com (captured by Charles Proxy). AFAIK CloudKit is the client library to access iCloud, so I can't see how this relates to setting up a standalone Python executable, but this is systematic on every run, and may explain few seconds delay running the command.

If I switch of my Wifi the same command just run within a second.

I'd be happy to run any further analysis that could help diagnose this issue, just let me know.

Original issue here :https://github.com/docker/compose/issues/6956

--

Nicolas

[Hartmut Goebel]

Am 15.10.19 um 11:48 schrieb 'Nicolas de Loof' via PyInstaller:

Thanks for maintaining this awesome project on your spare time by the way :)

If there is no serious funding, I will retire from maintaining. See <https://github.com/pyinstaller/pyinstaller/issues/4404>.

And to be frank: I expect companies like docker to pay a noteworthy part of this funding. Please step forward to you management.

I always get an HTTP request sent to https://api.apple-cloudkit.com (captured by Charles Proxy). AFAIK CloudKit is the client library to access iCloud, so I can't see how this relates to setting up a standalone Python executable, but this is systematic on every run, and may explain few seconds delay running the command.

This is not related to PyInstaller, which does not include such code. Please check your sources.

--
Schönen Gruß
Hartmut Goebel
Dipl.-Informatiker (univ), CISSP, CSSLP, ISO 27001 Lead Implementer
Information Security Management, Security Governance, Secure Software Development

Goebel Consult, Landshut
http://www.goebel-consult.de

Blog: https://www.goe-con.de/blog/frauenhofer-promoted-gnunet-mit-heisser-luft
Kolumne: https://www.goe-con.de/hartmut-goebel/cissp-gefluester/2011-09-kommerz-uber-recht-fdp-die-gefaellt-mir-partei

[Nicolas de Loof]

On Tue, Oct 15, 2019 at 12:23 PM Hartmut Goebel <h.go...@goebel-consult.de> wrote:

Am 15.10.19 um 11:48 schrieb 'Nicolas de Loof' via PyInstaller:

Thanks for maintaining this awesome project on your spare time by the way :)

If there is no serious funding, I will retire from maintaining. See <https://github.com/pyinstaller/pyinstaller/issues/4404>.

And to be frank: I expect companies like docker to pay a noteworthy part of this funding. Please step forward to you management.

Forwarded.

I always get an HTTP request sent to https://api.apple-cloudkit.com (captured by Charles Proxy). AFAIK CloudKit is the client library to access iCloud, so I can't see how this relates to setting up a standalone Python executable, but this is systematic on every run, and may explain few seconds delay running the command.

This is not related to PyInstaller, which does not include such code. Please check your sources.

As you can guess I checked many times, and actually went into some extra experiments :

I disabled system integrity check on OSX (csrutil disable from recovery console)

Then the exact same executable just run as expected within a second, and charles proxy don't capture such a http request anymore.

If I re-enable system integrity the issue is back, as well as the http request

this makes me thing integrity do analyze the executable when ran and (?) send some report/analytics to Apple

As PyIntsaller do unpack python runtime to execute, maybe the issue is due to this fresh new (temporary) executable being checked before it can actually run. I'm not confident with the notarization process required on Catalina, but maybe this unpacking step used by PyInstaller has some impact in this very specific context.

 

--
Schönen Gruß
Hartmut Goebel
Dipl.-Informatiker (univ), CISSP, CSSLP, ISO 27001 Lead Implementer
Information Security Management, Security Governance, Secure Software Development

Goebel Consult, Landshut
http://www.goebel-consult.de

Blog: https://www.goe-con.de/blog/frauenhofer-promoted-gnunet-mit-heisser-luft
Kolumne: https://www.goe-con.de/hartmut-goebel/cissp-gefluester/2011-09-kommerz-uber-recht-fdp-die-gefaellt-mir-partei

--
You received this message because you are subscribed to the Google Groups "PyInstaller" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyinstaller...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pyinstaller/af1dd1fe-e9dd-cd70-de1d-6cf0327e0b33%40goebel-consult.de.

[Nicolas de Loof]

To give a bit more detailed, I experimented a bit from this minimalist "hello" app and pyinstaller : when packaged as a directory, first run of the app is a bit slow, then subsequent runs are faster, and no request to apple-cloudkit is captured by charles proxy . This seem to demonstrate system scan being able to flag a file as "checked" for further usage. As the "single file" packaging do upack on every run into a temporary folder, such a scan will take place on every run, explaining the delay reported for docker-compose (which is a pretty large application)

On Tue, Oct 15, 2019 at 5:03 PM Nicolas de Loof <nicolas...@docker.com> wrote:

I was able to reproduce this issue with a minimalist main.py "application" : print "hello"

packaged as a single file distributable, and ran on OSX Catalina, a request get sent to api.apple-cloudkit.com.

I assume, as a fresh new Python installation is created under /tmp/_MExxx for every run, system software protection on Catalina do scan this "new" executable and some live diagnostic is sent to apple...

[Nicolas de Loof]

I was able to reproduce this issue with a minimalist main.py "application" : print "hello"

packaged as a single file distributable, and ran on OSX Catalina, a request get sent to api.apple-cloudkit.com.

I assume, as a fresh new Python installation is created under /tmp/_MExxx for every run, system software protection on Catalina do scan this "new" executable and some live diagnostic is sent to apple...

On Tue, Oct 15, 2019 at 2:25 PM Nicolas de Loof <nicolas...@docker.com> wrote: