New AppVeyor Windows CI Builds
72 views
Skip to first unread message
Bryan Petty
Nov 4, 2015, 8:57:27 PM11/4/15
to wxWidgets Development
Hi,
I built out some initial configurations that make use of AppVeyor to
provide some new Windows continuous integration builds. According to
our BuildBot master, we're down to just two wxMSW configurations that
only cover VC8 using nmake on 3.0 branch and master using Windows XP.
That's good, and we still need those, but I now have the following
configurations configured and running as well:
1. MSVC 2013 (12.0) - MSBuild, x64, DLL Release, STL=1
2. MSVC 2008 (9.0) - nmake, x86, release, STL=0
3. MSVC 2015 (14.0) - nmake, amd64, debug, STL=1
4. MinGW (GCC 4.8.1) - x86, debug, static, STL=0
5. MSYS2/MinGW32 (GCC 5.2.0) - x86, debug, shared, STL=0
6. Cygwin (GCC 4.9.3) - x86, debug, static, STL=0
You can see these builds running here right now:
https://ci.appveyor.com/project/tierra/wxwidgets
All builds are running on Windows Server 2012 R2 (x64). More info
about installed software is available here:
http://www.appveyor.com/docs/installed-software
AppVeyor has conflicting reports about max run time for builds. I've
seen both 40 minutes and 30 minutes in official docs, however, in
practice, it's left builds running for a full hour before killing
them. This is still a problem for us, specifically for GCC-based
builds, and especially because their VMs are restricted to just two
cores, so they aren't very powerful. I've been required to scale back
those GCC builds because of this.
So for right now, all builds are only compiling the library. They
don't compile or run tests yet, and they don't compile samples.
Additionally, the GCC-based builds trade off disabled features for
faster builds. Here's what I disabled for each GCC build:
MinGW: STC, Ribbon, Propgrid, and AUI
MSYS2: XRC (just this alone is the biggest build time factor)
Cygwin: HTML, RichText, and STC
I also just stuck with x86 builds for GCC, but maybe at least one of
those can be bumped to x64, probably the MSYS2 build since it's easy
to switch to MinGW64.
Also in the interest of speed, the GCC builds (except maybe MinGW
right now) are configured to pre-install system libraries like libpng,
libjpeg, libtiff, and libexpat, and don't compile the builtin versions
of these. Just the MSVC builds use the builtins.
Here's the main changes I made that make this all possible:
https://github.com/wxWidgets/wxWidgets/commit/209098848
You might notice that right now, I do have one configuration that is
failing (MinGW). I'd love it if someone investigated why GCC 4.8.1
doesn't like our "#if wxUSE_SOMETHING" checks even though it's fine
with them in the other builds.
It would also be nice if someone patched up the MSVC builds to also
compile and run tests since they're plenty fast enough to do so.
Also note that unlike Travis CI, AppVeyor only gives us one build at a
time, so the full time it takes to run one full build is longer than
Travis CI already, and we'll need to still keep this total down to
something reasonable (preferably under 3 hours I'd say).
All of these new configurations could, and should run against the 3.0
branch. I've left it enabled in the config, but I still haven't
actually backported the necessary script to run them.
Team Access:
Note that all configuration happens in the appveyor.yml config file in
the repo. There is nothing you need to touch on AppVeyor's site,
however, it might be necessary to cancel a build. Everyone with GitHub
commit access to wxWidgets should have the ability to cancel a build
if you login with your GitHub account.
If you would prefer though, it's ridiculously easy to pull these new
changes into your own forked repo, and enable AppVeyor on that repo.
Your builds will run completely independently from the main wxWidgets
builds.
--
Regards,
Bryan Petty
I built out some initial configurations that make use of AppVeyor to
provide some new Windows continuous integration builds. According to
our BuildBot master, we're down to just two wxMSW configurations that
only cover VC8 using nmake on 3.0 branch and master using Windows XP.
That's good, and we still need those, but I now have the following
configurations configured and running as well:
1. MSVC 2013 (12.0) - MSBuild, x64, DLL Release, STL=1
2. MSVC 2008 (9.0) - nmake, x86, release, STL=0
3. MSVC 2015 (14.0) - nmake, amd64, debug, STL=1
4. MinGW (GCC 4.8.1) - x86, debug, static, STL=0
5. MSYS2/MinGW32 (GCC 5.2.0) - x86, debug, shared, STL=0
6. Cygwin (GCC 4.9.3) - x86, debug, static, STL=0
You can see these builds running here right now:
https://ci.appveyor.com/project/tierra/wxwidgets
All builds are running on Windows Server 2012 R2 (x64). More info
about installed software is available here:
http://www.appveyor.com/docs/installed-software
AppVeyor has conflicting reports about max run time for builds. I've
seen both 40 minutes and 30 minutes in official docs, however, in
practice, it's left builds running for a full hour before killing
them. This is still a problem for us, specifically for GCC-based
builds, and especially because their VMs are restricted to just two
cores, so they aren't very powerful. I've been required to scale back
those GCC builds because of this.
So for right now, all builds are only compiling the library. They
don't compile or run tests yet, and they don't compile samples.
Additionally, the GCC-based builds trade off disabled features for
faster builds. Here's what I disabled for each GCC build:
MinGW: STC, Ribbon, Propgrid, and AUI
MSYS2: XRC (just this alone is the biggest build time factor)
Cygwin: HTML, RichText, and STC
I also just stuck with x86 builds for GCC, but maybe at least one of
those can be bumped to x64, probably the MSYS2 build since it's easy
to switch to MinGW64.
Also in the interest of speed, the GCC builds (except maybe MinGW
right now) are configured to pre-install system libraries like libpng,
libjpeg, libtiff, and libexpat, and don't compile the builtin versions
of these. Just the MSVC builds use the builtins.
Here's the main changes I made that make this all possible:
https://github.com/wxWidgets/wxWidgets/commit/209098848
You might notice that right now, I do have one configuration that is
failing (MinGW). I'd love it if someone investigated why GCC 4.8.1
doesn't like our "#if wxUSE_SOMETHING" checks even though it's fine
with them in the other builds.
It would also be nice if someone patched up the MSVC builds to also
compile and run tests since they're plenty fast enough to do so.
Also note that unlike Travis CI, AppVeyor only gives us one build at a
time, so the full time it takes to run one full build is longer than
Travis CI already, and we'll need to still keep this total down to
something reasonable (preferably under 3 hours I'd say).
All of these new configurations could, and should run against the 3.0
branch. I've left it enabled in the config, but I still haven't
actually backported the necessary script to run them.
Team Access:
Note that all configuration happens in the appveyor.yml config file in
the repo. There is nothing you need to touch on AppVeyor's site,
however, it might be necessary to cancel a build. Everyone with GitHub
commit access to wxWidgets should have the ability to cancel a build
if you login with your GitHub account.
If you would prefer though, it's ridiculously easy to pull these new
changes into your own forked repo, and enable AppVeyor on that repo.
Your builds will run completely independently from the main wxWidgets
builds.
--
Regards,
Bryan Petty
Bryan Petty
Nov 4, 2015, 9:09:35 PM11/4/15
to wxWidgets Development
On Wed, Nov 4, 2015 at 6:57 PM, Bryan Petty <br...@ibaku.net> wrote:
> You might notice that right now, I do have one configuration that is
> failing (MinGW). I'd love it if someone investigated why GCC 4.8.1
> doesn't like our "#if wxUSE_SOMETHING" checks even though it's fine
> with them in the other builds.
Oops, I just figured this out. I forgot to include the wxUSE_STL
> You might notice that right now, I do have one configuration that is
> failing (MinGW). I'd love it if someone investigated why GCC 4.8.1
> doesn't like our "#if wxUSE_SOMETHING" checks even though it's fine
> with them in the other builds.
setting in the config for this build. Sorry about that.
--
Regards,
Bryan Petty
Vadim Zeitlin
Nov 6, 2015, 8:49:29 AM11/6/15
to wxWidgets Development
On Wed, 4 Nov 2015 18:57:24 -0700 Bryan Petty wrote:
BP> I built out some initial configurations that make use of AppVeyor to
BP> provide some new Windows continuous integration builds.
Hi Bryan,
Thanks a lot for working on this!
BP> According to
BP> our BuildBot master, we're down to just two wxMSW configurations that
BP> only cover VC8 using nmake on 3.0 branch and master using Windows XP.
BP> That's good, and we still need those,
Yes, we do, especially because we can run GUI tests on them, unlike with
AppVeyor (nor Travis for that matter), but it's definitely nice to have
more builds.
BP> but I now have the following configurations configured and running as
BP> well:
BP>
BP> 1. MSVC 2013 (12.0) - MSBuild, x64, DLL Release, STL=1
BP> 2. MSVC 2008 (9.0) - nmake, x86, release, STL=0
BP> 3. MSVC 2015 (14.0) - nmake, amd64, debug, STL=1
BP> 4. MinGW (GCC 4.8.1) - x86, debug, static, STL=0
BP> 5. MSYS2/MinGW32 (GCC 5.2.0) - x86, debug, shared, STL=0
BP> 6. Cygwin (GCC 4.9.3) - x86, debug, static, STL=0
Looks like a very reasonable selection.
BP> You can see these builds running here right now:
BP> https://ci.appveyor.com/project/tierra/wxwidgets
I wonder if it would be possible to replace your user name with
"wxwidgets" above to make this URL simpler to remember? Also, on a
tangentially related note, perhaps setting this up would be worth a news
post on the web site main page (on one hand, this is something that is
mostly interested to us ourselves and not our users, but OTOH the last news
is from more than 6 months ago...)?
BP> MinGW: STC, Ribbon, Propgrid, and AUI
BP> MSYS2: XRC (just this alone is the biggest build time factor)
BP> Cygwin: HTML, RichText, and STC
I'm really surprised about XRC, but as long as everything is still built
somewhere, why not.
BP> Here's the main changes I made that make this all possible:
BP> https://github.com/wxWidgets/wxWidgets/commit/209098848
Thanks for writing this PowerShell snippet, I certainly wouldn't have been
capable of this.
BP> If you would prefer though, it's ridiculously easy to pull these new
BP> changes into your own forked repo, and enable AppVeyor on that repo.
BP> Your builds will run completely independently from the main wxWidgets
BP> builds.
Yes, but considering that AppVeyor is a free (and very useful) service, I
think we should try to not waste their resources needlessly, so I'd prefer
to have just one set of wxWidgets builds if possible.
Thanks again for setting this up!
VZ
BP> I built out some initial configurations that make use of AppVeyor to
BP> provide some new Windows continuous integration builds.
Hi Bryan,
Thanks a lot for working on this!
BP> According to
BP> our BuildBot master, we're down to just two wxMSW configurations that
BP> only cover VC8 using nmake on 3.0 branch and master using Windows XP.
BP> That's good, and we still need those,
Yes, we do, especially because we can run GUI tests on them, unlike with
AppVeyor (nor Travis for that matter), but it's definitely nice to have
more builds.
BP> but I now have the following configurations configured and running as
BP> well:
BP>
BP> 1. MSVC 2013 (12.0) - MSBuild, x64, DLL Release, STL=1
BP> 2. MSVC 2008 (9.0) - nmake, x86, release, STL=0
BP> 3. MSVC 2015 (14.0) - nmake, amd64, debug, STL=1
BP> 4. MinGW (GCC 4.8.1) - x86, debug, static, STL=0
BP> 5. MSYS2/MinGW32 (GCC 5.2.0) - x86, debug, shared, STL=0
BP> 6. Cygwin (GCC 4.9.3) - x86, debug, static, STL=0
Looks like a very reasonable selection.
BP> You can see these builds running here right now:
BP> https://ci.appveyor.com/project/tierra/wxwidgets
I wonder if it would be possible to replace your user name with
"wxwidgets" above to make this URL simpler to remember? Also, on a
tangentially related note, perhaps setting this up would be worth a news
post on the web site main page (on one hand, this is something that is
mostly interested to us ourselves and not our users, but OTOH the last news
is from more than 6 months ago...)?
BP> MinGW: STC, Ribbon, Propgrid, and AUI
BP> MSYS2: XRC (just this alone is the biggest build time factor)
BP> Cygwin: HTML, RichText, and STC
I'm really surprised about XRC, but as long as everything is still built
somewhere, why not.
BP> Here's the main changes I made that make this all possible:
BP> https://github.com/wxWidgets/wxWidgets/commit/209098848
Thanks for writing this PowerShell snippet, I certainly wouldn't have been
capable of this.
BP> If you would prefer though, it's ridiculously easy to pull these new
BP> changes into your own forked repo, and enable AppVeyor on that repo.
BP> Your builds will run completely independently from the main wxWidgets
BP> builds.
Yes, but considering that AppVeyor is a free (and very useful) service, I
think we should try to not waste their resources needlessly, so I'd prefer
to have just one set of wxWidgets builds if possible.
Thanks again for setting this up!
VZ
Bryan Petty
Nov 6, 2015, 2:09:32 PM11/6/15
to wxWidgets Development
On Fri, Nov 6, 2015 at 6:49 AM, Vadim Zeitlin <va...@wxwidgets.org> wrote:
> On Wed, 4 Nov 2015 18:57:24 -0700 Bryan Petty wrote:
>
> BP> You can see these builds running here right now:
> BP> https://ci.appveyor.com/project/tierra/wxwidgets
>
> I wonder if it would be possible to replace your user name with
> "wxwidgets" above to make this URL simpler to remember?
I looked for a way to do this for a while before pushing these changes
> On Wed, 4 Nov 2015 18:57:24 -0700 Bryan Petty wrote:
>
> BP> You can see these builds running here right now:
> BP> https://ci.appveyor.com/project/tierra/wxwidgets
>
> I wonder if it would be possible to replace your user name with
> "wxwidgets" above to make this URL simpler to remember?
up, and I couldn't find any way to setup an "organization-owned"
repo/account. I can actually change my own personal profile to
"wxWidgets" (and have done so now), and it will look like it's
organization owned, but it's actually still my personal account. It
might help to know that in case someone else wants to take it over in
the future, and enable the official repo on their account. Ironically,
this also means that anyone with GitHub commit access also has admin
rights to *all* my personal AppVeyor projects, but that doesn't matter
since wxWidgets is the only one I have (or will likely have).
I'll see about writing up a news post for it. You're right, we're
terrible about posting regularly there.
> BP> Here's the main changes I made that make this all possible:
> BP> https://github.com/wxWidgets/wxWidgets/commit/209098848
>
> Thanks for writing this PowerShell snippet, I certainly wouldn't have been
> capable of this.
PowerShell in there is the snippet that applies custom setup.h
settings for the builds that use it.
Sometimes I think it would be good to learn more PowerShell, but then
I remember that it would be completely useless for me professionally
anyway.
> BP> If you would prefer though, it's ridiculously easy to pull these new
> BP> changes into your own forked repo, and enable AppVeyor on that repo.
> BP> Your builds will run completely independently from the main wxWidgets
> BP> builds.
>
> Yes, but considering that AppVeyor is a free (and very useful) service, I
> think we should try to not waste their resources needlessly, so I'd prefer
> to have just one set of wxWidgets builds if possible.
just meant for purposes of testing your local commits if you wanted
before even opening up a pull request. You're right, I don't want to
abuse the free service.
--
Regards,
Bryan Petty
Bryan Petty
Nov 6, 2015, 2:10:25 PM11/6/15
to wxWidgets Development
On Fri, Nov 6, 2015 at 12:09 PM, Bryan Petty <br...@ibaku.net> wrote:
> I looked for a way to do this for a while before pushing these changes
> up, and I couldn't find any way to setup an "organization-owned"
> repo/account. I can actually change my own personal profile to
> "wxWidgets" (and have done so now)
Oh, and the URL is now:
> I looked for a way to do this for a while before pushing these changes
> up, and I couldn't find any way to setup an "organization-owned"
> repo/account. I can actually change my own personal profile to
> "wxWidgets" (and have done so now)
https://ci.appveyor.com/project/wxWidgets/wxwidgets
--
Regards,
Bryan Petty
Bryan Petty
Nov 6, 2015, 2:13:19 PM11/6/15
to wxWidgets Development
On Fri, Nov 6, 2015 at 12:09 PM, Bryan Petty <br...@ibaku.net> wrote:
> I'll see about writing up a news post for it. You're right, we're
> terrible about posting regularly there.
Before I do this though, could you look into why I'm still having
> terrible about posting regularly there.
trouble with the MinGW build?
Here's the issue:
ar: gcc_mswud\corelib_plitter.o: No such file or directory
It seems like it's supposed to be linking in "corelib_splitter.o"
(missing the "s"), so I suspect something is wrong with the
makefile.gcc (or config.gcc) file, but I can't figure it out.
--
Regards,
Bryan Petty
Vadim Zeitlin
Nov 6, 2015, 2:33:30 PM11/6/15
to wxWidgets Development
On Fri, 6 Nov 2015 12:13:18 -0700 Bryan Petty wrote:
BP> Before I do this though, could you look into why I'm still having
BP> trouble with the MinGW build?
BP>
BP> Here's the issue:
BP> ar: gcc_mswud\corelib_plitter.o: No such file or directory
BP>
BP> It seems like it's supposed to be linking in "corelib_splitter.o"
BP> (missing the "s"), so I suspect something is wrong with the
BP> makefile.gcc (or config.gcc) file, but I can't figure it out.
I've seen it while looking at some PR and thought it was just a fluke,
e.g. data corruption due to some hardware (or software, knowing that all
this is surely running in a virtualized environment) problem, but if this
problem is reproducible then it just might be real. But if so, I have no
idea what could it be, the only similar problems I can remember were due to
truncation of the command line (it's limited to 4096 characters under
Windows, I believe), but then the file name was really just truncated, it
still didn't miss a letter in the middle.
Looking at https://ci.appveyor.com/project/wxWidgets/wxwidgets/build/8/job/g0kgqu6oohygt7b6
(thanks for changing the name!), ar command line contains the correct
"corelib_splitter.o" argument, so the only other explanation I can see is
that this version of "ar" is somehow buggy. It does seem very unlikely, but
I don't see what else could explain it, so I've added "ar --version" to the
batch file, let's see if it shows anything interesting.
Sorry for lack of anything more useful,
VZ
BP> Before I do this though, could you look into why I'm still having
BP> trouble with the MinGW build?
BP>
BP> Here's the issue:
BP> ar: gcc_mswud\corelib_plitter.o: No such file or directory
BP>
BP> It seems like it's supposed to be linking in "corelib_splitter.o"
BP> (missing the "s"), so I suspect something is wrong with the
BP> makefile.gcc (or config.gcc) file, but I can't figure it out.
I've seen it while looking at some PR and thought it was just a fluke,
e.g. data corruption due to some hardware (or software, knowing that all
this is surely running in a virtualized environment) problem, but if this
problem is reproducible then it just might be real. But if so, I have no
idea what could it be, the only similar problems I can remember were due to
truncation of the command line (it's limited to 4096 characters under
Windows, I believe), but then the file name was really just truncated, it
still didn't miss a letter in the middle.
Looking at https://ci.appveyor.com/project/wxWidgets/wxwidgets/build/8/job/g0kgqu6oohygt7b6
(thanks for changing the name!), ar command line contains the correct
"corelib_splitter.o" argument, so the only other explanation I can see is
that this version of "ar" is somehow buggy. It does seem very unlikely, but
I don't see what else could explain it, so I've added "ar --version" to the
batch file, let's see if it shows anything interesting.
Sorry for lack of anything more useful,
VZ
PB
Nov 7, 2015, 1:16:28 PM11/7/15
to wx-dev
Hi,
FWIW I have seen a very similar issue - randomly missing letters in paths when building wxWidgets DLLs with GCC - reported by a user on wxForum several times, E.g. here:
https://forums.wxwidgets.org/viewtopic.php?f=19&t=41240#p166756
https://forums.wxwidgets.org/viewtopic.php?f=1&t=41068&start=15#p165943
Regards,
PB
FWIW I have seen a very similar issue - randomly missing letters in paths when building wxWidgets DLLs with GCC - reported by a user on wxForum several times, E.g. here:
https://forums.wxwidgets.org/viewtopic.php?f=19&t=41240#p166756
https://forums.wxwidgets.org/viewtopic.php?f=1&t=41068&start=15#p165943
Regards,
PB
Vadim Zeitlin
Nov 7, 2015, 5:39:33 PM11/7/15
to wx-...@googlegroups.com
On Sat, 7 Nov 2015 10:16:28 -0800 (PST) PB wrote:
P> FWIW I have seen a very similar issue - randomly missing letters in paths
P> when building wxWidgets DLLs with GCC - reported by a user on wxForum
P> several times, E.g. here:
P> https://forums.wxwidgets.org/viewtopic.php?f=19&t=41240#p166756
P> https://forums.wxwidgets.org/viewtopic.php?f=1&t=41068&start=15#p165943
The only other hypothesis I have is that it could be due to a bug in
mingw32-make itself. But even if it's true, it doesn't help to solve the
problem, of course :-(
Regards,
VZ
P> FWIW I have seen a very similar issue - randomly missing letters in paths
P> when building wxWidgets DLLs with GCC - reported by a user on wxForum
P> several times, E.g. here:
P> https://forums.wxwidgets.org/viewtopic.php?f=19&t=41240#p166756
P> https://forums.wxwidgets.org/viewtopic.php?f=1&t=41068&start=15#p165943
The only other hypothesis I have is that it could be due to a bug in
mingw32-make itself. But even if it's true, it doesn't help to solve the
problem, of course :-(
Regards,
VZ
Bryan Petty
Nov 10, 2015, 9:23:47 AM11/10/15
to wxWidgets Development
still have MinGW, just not building the core GUI library. At least
it's a good test to make sure building just wxBase still works.
If this really is because of command line argument limits, I'm not
sure what could be done with the build files to avoid it. Someone
might be able to confirm by disabling a bunch of core controls in the
build configuration, as long as corelib_splitter.o is still being
linked.
I've also now ported these configurations to the 3.0 branch, and they
seem to be running flawlessly without any modifications from the
master branch scripts. :)
--
Regards,
Bryan Petty
Reply all
Reply to author
Forward
0 new messages