triple IOS build time

[Dave Dyer]

My IOS builds were taking 8-10 minutes.  Now they are taking 25-30 minutes.
My build is getting incrementally larger, but something else must have changed.

[Shai Almog]

Sometimes the UI doesn't refresh so try reloading to verify. When build is slow there is a distinction between "building" and "queued" if you spend a lot of time queued this could mean there are a lot of people building to our servers at this point in time.

[Dave Dyer]

I'm commenting based on the "took" time on the build page.

[Shai Almog]

That includes the queued time in the total. We should probably split that into more verbose stats in the future.

[Dave Dyer]

The builds visibly move from queued to in progress very quickly. Android builds are still quick.
Is there a way to get a build log for a successful build? Or otherwise tease out more info about
The timing? It's really behaving as if there's a N^2 factor slowdown, scaled by the amount of
Code in the build.

[Shai Almog]

I'll look into it and see if I can see a difference with our test cases.

[Dave Dyer]

Android builds are 2-3 minutes, iOS 25-35 minutes for the full load. Small projects don't have such a large ratio.

[Shai Almog]

That means you have a lot of files so the cost is in the process itself. It might be related to the contributed performance optimizations we recently added but we didn't notice this for most typical Codename One projects. I'm assuming you have a REALLY large number of Java classes?

[Dave Dyer]

About 850 source files.

[Dave Dyer]

Build times continue to be very slow on IOS.  I've topped out the amount of code, so the final score
is that Android builds take 2-3 minutes, IOS builds take 25-40 minutes.  That seems like a pretty
severe differential.

One thing that occurs to me is to ask about the configuration of your build servers.  For example,
I know from personal experience that Macs suck at networked file systems.  Building using a remote
disk, even one on the same LAN, can take an order of magnitude longer than building with a local
file system.

[Shai Almog]

All of our servers with the exception of the windows servers are SSD based.

[Dave Dyer]

Just to keep the fire under this problem;  My android build times are pretty consistent at 2:30 (when they succeed)

My IOS build time for the same version currently ranges from a low of 22 minutes to a looong time of 72 minutes.

[Shai Almog]

Looking at your other thread its pretty obvious why. Your build sizes are HUGE.

[Dave Dyer]

As previously noted, the Android build for the same code+resources take 2.5 minutes,
so the best case is IOS takes 10x as long as Android, worst case (so far) is 30x as
long.  Sure seems like something is out of whack to me.

[Shai Almog]

Android != iOS. The implementations have nothing in common.

[Dave Dyer]

So I see; on this thread https://groups.google.com/forum/#!topic/codenameone-discussions/4mNmzIe3AFA
I dug into the details of the build, and discovered that the actual source code being compiled for IOS is gargantuan - 350 mb
derived from 4mb of java jars, and that compiling that blob takes 40 minutes on my mac.  So the actual build is pretty much
absolved, but the process that prepared the code to be built is pushing the envelope in an atypical direction.

[Shai Almog]

FYI we got a great test case from Stefan: http://stackoverflow.com/questions/41930957/what-could-cause-long-build-times/41951064

And just posted a fix for his issue, this might impact your build times as well. I'd be curious to know.

[Dave Dyer]

Sadly, the IOS build seems to be broken just now.  Maybe it's a coincidence.  3 build attempts in a row
fail with this error:

/ArchiveIntermediates/Develop/IntermediateBuildFilesPath/Develop.build/Release-iphoneos/Develop.build/Objects-normal/armv7/com_codename1_ui_geom_GeneralPath.o
/var/folders/zh/kb_4hqhn4kg1h0r5dp_6htcm0000gn/T/build489511495227273694xxx/dist/Develop-src/com_codename1_ui_geom_GeneralPath.m:1011:1: error: expected statement
}
^
1 error generated.

[Dave Dyer]

I finally for a good build, and assuming it's not because you reverted your change, my build time is substantially unchanged.
-- currently weighing in at about 35 minutes for a debug build.   My last analysis of slow builds remains that the long times
in my builds are caused by excessive re-inclusion of irrelevant header files.