IOS builds failing 1/3/2018

[Dave Dyer]

My current IOS builds (essentially same as the old IOS builds) are failing.  The assumption in
previous discussions has been that some timeout is kicking in, but (in this case) the fails are
happening in less than 7 minutes of clock time start to finish.

1) It would be helpful if these "timeout kills" were identified as such in the log, if that's what's going on.

2) The current brouhaha about intel processors taking a bit performance hit (5-30%) to fix a security problem
are likely to affect AWS performance, and therefore your timeouts.

[Shai Almog]

We do print out the timeout error. Not sure why it's not showing as I said we currently provide 6 minutes for that stage which is a huge amount of time for the first stage.

We don't use AWS for this (barely use it at all). We use physical Macs for this bit. As far as I read the issue Mac OS wouldn't be as severely impacted because of its usage of PCID. If we need to increase timeouts to compensate that we'll do it.

[Dave Dyer]

Could I have another minute please?

[Shai Almog]

OK, but again you should reign this in... This barely leaves time for the actual build and when you do a release build for two platforms you will just run out of time.

[Dave Dyer]

I have a couple push requests still pending that will affect this phase of the build favorably.

I'd love to be able to do my builds in a more modular way.

[Shai Almog]

They are still assigned for the 3.8 milestone. I'm not sure about the status of those pull requests.

The main issue with building pieces etc. is state/security. We don't store anything in the servers and wipe them clean. This is essential. It also helps with consistent repeatable results.
I've noticed that some things that work well for people offline fail really badly on the servers. E.g.  a user submitted a patch to speed up builds by merging all the include files to one HUGE file. He said it sped up his builds considerably. In our case it caused them to always timeout.