Apple SUS not updating
72 views
Skip to first unread message
Thomas Maerz
May 10, 2016, 5:01:50 PM5/10/16
to Simian Discuss
| OS Version | Synced From Apple | Unstable | Testing | Stable |
|---|---|---|---|---|
| 10.10 | Feb. 17, 2016, 6 a.m. (2 months, 3 weeks ago) | May 9, 2016, 4 p.m. (1 day, 4 hours ago) | May 9, 2016, 4 p.m. (1 day, 4 hours ago) | May 9, 2016, 4 p.m. (1 day, 4 hours ago) |
| 10.11 | Feb. 17, 2016, 6 a.m. (2 months, 3 weeks ago) | May 9, 2016, 3:59 p.m. (1 day, 4 hours ago) | May 9, 2016, 4 p.m. (1 day, 4 hours ago) | May 9, 2016, 4 p.m. (1 day, 4 hours ago) |
| 10.8 | Feb. 5, 2016, 6 a.m. (3 months ago) | May 9, 2016, 4 p.m. (1 day, 4 hours ago) | May 9, 2016, 3:59 p.m. (1 day, 4 hours ago) | May 9, 2016, 4 p.m. (1 day, 4 hours ago) |
| 10.9 | Feb. 10, 2016, 6 a.m. (3 months ago) | May 9, 2016, 3:59 p.m. (1 day, 4 hours ago) | May 9, 2016, 4 p.m. (1 day, 4 hours ago) | May 9, 2016, 4 p.m. (1 day, 4 hours ago) |
| 10.7 | Feb. 17, 2016, 6 a.m. (2 months, 3 weeks ago) | May 9, 2016, 4 p.m. (1 day, 4 hours ago) | May 9, 2016, 4 p.m. (1 day, 4 hours ago) | May 9, 2016, 4 p.m. (1 day, 4 hours ago) |
I'm not getting Apple SUS catalogs syncing for some reason. The only thing I see in the logs is
2016-05-10 20:38:50 WARN Ignoring msu_log time; datetime in the future: 2016-05-10 21:46:06
Which is curious but I am not sure if it's related or not. I checked with the remote python shell and my system time is set to UTC and is correct so I don't think that's the problem. I dug around in the filesystem and cron.log is showing a bunch of these two events:
</body></html><html> <head> <title>500 Internal Server Error</title> </head> <body> <h1>500 Internal Server Error</h1> The server has either erred or is incapable of performing the requested operation.<br /><br />
</body></html>and
<html><head><title>504 Gateway Time-out</title></head><body bgcolor="white"><center><h1>504 Gateway Time-out</h1></center><hr><center>nginx/1.1.19</center></body></html><html><body><h1>503 Service Unavailable</h1>No server is available to handle this request.</body></html><html><body><h1>503 Service Unavailable</h1>No server is available to handle this request.</body></html><html><body><h1>503 Service Unavailable</h1>No server is available to handle this request.</body></html><html><body><h1>503 Service Unavailable</h1>No server is available to handle this request.</body></html><html><body><h1>503 Service Unavailable</h1>No server is available to handle this request.</body></html><html> <head> <title>500 Internal Server Error</title> </head> <body> <h1>500 Internal Server Error</h1> The server has either erred or is incapable of performing the requested operation.<br /><br />The main app log shows these events for the Apple SUS cronjob:
Invalid and/or missing SSL certificate for URL: https://swscan.apple.com/content/catalogs/others/index-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog
Traceback (most recent call last):
File "/root/appscale/AppServer/lib/webapp2-2.5.2/webapp2.py", line 1535, in __call__
rv = self.handle_exception(request, response, e)
File "/root/appscale/AppServer/lib/webapp2-2.5.2/webapp2.py", line 1529, in __call__
rv = self.router.dispatch(request, response)
File "/root/appscale/AppServer/lib/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "/root/appscale/AppServer/lib/webapp2-2.5.2/webapp2.py", line 1102, in __call__
return handler.dispatch()
File "/root/appscale/AppServer/lib/webapp2-2.5.2/webapp2.py", line 572, in dispatch
return self.handle_exception(e, self.app.debug)
File "/root/appscale/AppServer/lib/webapp2-2.5.2/webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "/var/apps/simian/app/simian/mac/cron/applesus.py", line 291, in get
if self._UpdateCatalogIfChanged(untouched_catalog, url):
File "/var/apps/simian/app/simian/mac/cron/applesus.py", line 141, in _UpdateCatalogIfChanged
url, headers=headers, deadline=30, validate_certificate=True)
File "/root/appscale/AppServer/google/appengine/api/urlfetch.py", line 270, in fetch
return rpc.get_result()
File "/root/appscale/AppServer/google/appengine/api/apiproxy_stub_map.py", line 615, in get_result
return self.__get_result_hook(self)
File "/root/appscale/AppServer/google/appengine/api/urlfetch.py", line 414, in _get_fetch_result
'Invalid and/or missing SSL certificate for URL: ' + url)
SSLCertificateError: Invalid and/or missing SSL certificate for URL: https://swscan.apple.com/content/catalogs/others/index-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog
Any idea what might be causing this?
Thomas
Justin McWilliams
May 10, 2016, 6:02:24 PM5/10/16
to Simian Discuss
Thomas,
The main log is the relevant failure here. I'm not familiar what SSL validation AppScale is doing for you, but this works as expected from App Engine. It's possible the machine you're running AppScale on has an incompatible root CA chain.
Although not advised, to get around this you could disable cert validation here: https://github.com/google/simian/blob/master/src/simian/mac/cron/applesus.py#L131
Alternatively, do you have a support contract with AppScale which you could leverage for help? This should be reproducible with the single linked call above.
- Justin
--
You received this message because you are subscribed to the Google Groups "Simian Discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to simian-discus...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Thomas Maerz
May 12, 2016, 3:07:40 PM5/12/16
to Simian Discuss
AppScale guys said to disable cert validation. I think they are just wanting to isolate it. I just built the latest master and noticed that during make test I am seeing this:
Could this be related or is this a normal error during make test?
WARNING:root:/Users/bsitadmin/simian/src/tests/gae_server.zip/lib/cacerts/urlfetch_cacerts.txt missing; without this urlfetch will not be able to validate SSL certificates.WARNING:root:No ssl package found. urlfetch will not be able to validate SSL certificates.Testing uploadfile_test.WARNING:root:file_type=log , file_name=WARNING:root:file_type= , file_name=fooname.----------------------------------------------------------------------Could this be related or is this a normal error during make test?
Justin McWilliams
May 12, 2016, 3:11:58 PM5/12/16
to Simian Discuss
On Thu, May 12, 2016 at 3:07 PM, Thomas Maerz <deathb...@gmail.com> wrote:
AppScale guys said to disable cert validation. I think they are just wanting to isolate it. I just built the latest master and noticed that during make test I am seeing this:
Could this be related or is this a normal error during make test?
Unrelated. gae_server.zip is only used for tests, and never deployed to App Engine.
Thomas Maerz
May 12, 2016, 4:06:29 PM5/12/16
to Simian Discuss
Their SDK's default validation mechanism has bundled ssl cacerts and I think they're out of date. They're gonna fix it: http://redmine.appscale.com/issues/210
Justin McWilliams
May 12, 2016, 4:23:10 PM5/12/16
to Simian Discuss
Glad the problem in AppScale was identified. That said, I'd be reluctant to blindly fall back to the system's default (the issue states "Python's default validation mechanism", but I think they really mean the system root chain).
Simian bundle it's own cert chain for the client to use[1] (due concerns around blindly trusting system defaults), which are then used by the client[2]. On App Engine, thankfully we rely on the platform to host a trustworthy and updated root chain.
Thomas Maerz
May 13, 2016, 11:48:17 AM5/13/16
to Simian Discuss
Ideally, AppScale should provide trustworthy certs as well, but It's obviously not as refined or widely deployed as GAE. My system's root chain was working fine, but they have some mechanism within their URLfetch that has it's own root chain. What actually broke this for us was previous versions of AppScale ignored the validate_certificate:True. The default is still False for this (which is less than ideal) but in the current version it no longer ignores the flag.
As always, thanks for the guidance!
Reply all
Reply to author
Forward
0 new messages