Re: [boost] PSA: Travis OS X bottleneck, <cxxstd> new Boost.Build feature

[Tom Kent]

It seems like this would be a great instance where the steering committee could allocate some of the organization's funds to directly improve the infrastructure being used by our team. I believe that we could upgrade our Travis account to a paid one for a couple hundred a month to get more build resources. 

They also claim on https://travis-ci.org/ that:

> Testing your open source project is 10000% free

> Seriously. Always. We like to think of it as our way of giving back to a community that gives us so much as well.

Maybe we just need to have someone on the project contact them and get our free quota raised? 

Tom

On Thu, Oct 26, 2017 at 11:43 AM, Peter Dimov via Boost <bo...@lists.boost.org> wrote:

Travis's OS X resources for open source projects seem to be insufficient, so OS X jobs are very slow and seem to be falling further and further behind. Therefore, library authors are encouraged to keep the OS X jobs to the minimum necessary. Spawning many OS X jobs takes hours (~25 minutes waiting time per job at a quick estimate).

On a not entirely unrelated note, Rene Rivera has added a new feature <cxxstd> to Boost.Build that controls the C++ standard in use. So for instance, instead of the old

   b2 libs/mylib/test toolset=gcc cxxflags=-std=c++11

one can now use

   b2 libs/mylib/test toolset=gcc cxxstd=11

In addition to being more convenient, this also allows several invocations to be combined into one:

   b2 libs/mylib/test toolset=clang cxxstd=03,11,14,1z

which can be leveraged to cut down on the number of jobs.

An example of using cxxstd in .travis.yml can be seen here:

https://github.com/boostorg/system/blob/develop/.travis.yml

_______________________________________________
Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

[Niall Douglas]

In the past it was felt that the cost benefit wasn't there. But now so many more Boost libraries are hammering Travis, things may be different.

The SC may wish to understand what the difference between Travis and Appveyor and the regression testing machines they've invested in is?

Niall

[Jon Kalb]

Tom,

 

I’m not certain who we’d contact, but I suppose sup...@travis-ci.com is the place to start.

 

I’m willing to contact them on behalf of the project (although I think it would be better if you did this because you know more of what we need).

 

What can you tell me about our OS X resources/requirements/quota or resources/requirements/quota generally that I should know before contacting them? Do we have an account name/number/identifier of some type?

 

Jon

--
The Boost Steering Committee webpage: https://sites.google.com/a/boost.org/steering/
---
You received this message because you are subscribed to the Google Groups "Boost Steering Committee" group.
To unsubscribe from this group and stop receiving emails from it, send an email to boost-steerin...@googlegroups.com.
To post to this group, send email to boost-s...@googlegroups.com.
Visit this group at https://groups.google.com/group/boost-steering.
To view this discussion on the web visit https://groups.google.com/d/msgid/boost-steering/CAArKS8gEiuYw1OUBkFRf3MD6DVkg7GufZR_jpBCgvfLXS8Q6WQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Louis Dionne]

John,

If that's okay with you, I'd like to contact Travis on behalf of the Steering Committee asking them to increase our limits as an open source organization (for free). Our Travis queue is incredibly backed up and it's very crippling for developers.

Louis

[Jon Kalb]

Louis,

 

Sorry that I didn’t see this message until just now.

 

Please do.

 

Jon

 

From: Boost Steering Committee <boost-s...@googlegroups.com> on behalf of Louis Dionne <ldio...@gmail.com>
Reply-To: Boost Steering Committee <boost-s...@googlegroups.com>
Date: Friday, November 10, 2017 at 8:27 PM
To: Boost Steering Committee <boost-s...@googlegroups.com>
Subject: [boost-steering] Re: [boost] PSA: Travis OS X bottleneck, <cxxstd> new Boost.Build feature

 

John,

--

The Boost Steering Committee webpage: https://sites.google.com/a/boost.org/steering/
---
You received this message because you are subscribed to the Google Groups "Boost Steering Committee" group.
To unsubscribe from this group and stop receiving emails from it, send an email to boost-steerin...@googlegroups.com.
To post to this group, send email to boost-s...@googlegroups.com.
Visit this group at https://groups.google.com/group/boost-steering.

To view this discussion on the web visit https://groups.google.com/d/msgid/boost-steering/ef0809d2-bb27-4c42-ad36-6fa9b4ea77db%40googlegroups.com.

[Louis Dionne]

I reached out to Travis and they said they can’t increase our job limits for free. I did argue with them on the basis that we’re just transferring repositories to the Boost organization for ease of maintenance instead of maintaining forks (in which case the limits for individual developers would apply for each library), but no cigar. The essence of their response is two things:

- The smallest package we can get includes 10 extra concurrent jobs, effectively raising the limit to 15, for $500/month, paid annually and in advance.
- They can offer us a 30% discount since we’re a non-profit.

Is there a desire (and sufficient funds) to do this? I suggest we float the idea on the Boost.Dev mailing list to see if the job delays are annoying other developers, and if so, then this avenue can be considered further.

Louis

> To view this discussion on the web visit https://groups.google.com/d/msgid/boost-steering/46A04C7F-0F1A-4890-945B-427BEAA6EE17%40boost.org.

[Jon Kalb]

That is $4200 a year. That is a lot. I’d be interesting in what you hear on the Boost.Dev list. How much pain is there from this?

To view this discussion on the web visit https://groups.google.com/d/msgid/boost-steering/0B2735A4-7FF5-4634-830F-5EC5EAFB9013%40boost.org.

[Boris Kolpackov]

We are dealing with a similar issues as Circle CI (CI builds on Mac OS)
so maybe a bit of background on why they can't offer more of it for free
and why it is expensive would be helpful:

Apple's licensing of Mac OS is pretty hostile to software development:

1. You can only legally run Mac OS on Apple's hardware.

2. You can only legally have/run two Mac OS VMs per physical machine.

Apple's hardware is quite expensive. Plus, because of (2), it is
pointless to, say, purchase a Mac Pro -- with only two VMs at a
time the utilization will be pretty poor. So the current go-to
CI machine for Mac OS is Mac Mini. The problem is, current-
generation Mini's (Late 2014) use mobile processor which are
quite weak for compilation and the best option is actually the
previous-generation (Late 2012) quad-core i7 variant, which are
only available used.

In this light, I think it makes more sense to ask Boost users
who are using Boost on Mac OS to contribute hardware resources
for CI testing.


Best regards,
Boris

To view this discussion on the web visit https://groups.google.com/d/msgid/boost-steering/88F39772-79D4-42EC-A7C7-38E2D2F4E8F5%40boost.org.

[Tom Kent]

On Wed, Nov 29, 2017 at 3:10 AM, Boris Kolpackov <bo...@codesynthesis.com> wrote:

In this light, I think it makes more sense to ask Boost users
who are using Boost on Mac OS to contribute hardware resources
for CI testing.

I think this is a great idea. There is already some Mac OS runners in the testing matrix, but they don't seem to execute that often. Maybe someone who is a Mac user could volunteer to set one up and the steering committee can fund a Mac mini for them?

On the other hand, putting it into the testing matrix doesn't fill the same need as having a Travis/CircleCI runner...neither of those services will allow us to supply build hardware for use within their system. We could stand up a Jenkins instance and get something like this, but then we have to maintain the server side infrastructure as well as the build machine infrastructure. 

Tom 

[Marshall Clow]

On Nov 29, 2017, at 5:10 AM, Tom Kent <t...@teeks99.com> wrote:

On Wed, Nov 29, 2017 at 3:10 AM, Boris Kolpackov <bo...@codesynthesis.com> wrote:

In this light, I think it makes more sense to ask Boost users
who are using Boost on Mac OS to contribute hardware resources
for CI testing.

I think this is a great idea. There is already some Mac OS runners in the testing matrix, but they don't seem to execute that often. Maybe someone who is a Mac user could volunteer to set one up and the steering committee can fund a Mac mini for them?

I have a stack of four minis that are (intermittently) running boost tests.

I’m hoping to make that more consistent next week.

— Marshall

[Robert Ramey]

On 11/29/17 1:10 AM, Boris Kolpackov wrote:

> In this light, I think it makes more sense to ask Boost users
> who are using Boost on Mac OS to contribute hardware resources
> for CI testing.

This is the right idea. Let's go the full monte and take it to it's
logical conclusion.


a) Encourage users to run tests on the libraries they use by making
testing more user friendly.

b) Create a CMake or CMake-like dashboard where users can more or less
automagically post their results to a common database. Presumable boost
regression.py does something similar already.

c) Create a program to permit display of this giant "data cube"
according to the needs of the user/developer.

My personal contribution to moving in this direction can be found at

https://github.com/robertramey/library_status

This permits me to:

a) run the serialization test suite with boost build. This is a single
button which builds all the libraries that the serialization depends
upon, and runs all the tests.

b) The results are displayed on a giant cumulative matrix which permits
me to see which tests are failing on which platforms. Here is a recent
copy of that matrix

http://htmlpreview.github.com/?https://github.com/robertramey/library_status/blob/master/library_status.html

more information is available here:

http://htmlpreview.github.com/?https://github.com/robertramey/library_status/blob/master/index.html

If this idea were implemented we would have:

a) recent tests on all platforms actually being used regardless of how
obscure.
b) an easily browsable data base of test results.
c) less dependency on centralized CI testing resources
d) less pressure on boost testers
e) more people participating and being a member of the boost community -
helping to promote the "Boost Brand (c)"
f) Since Appveyor, Travis, regression.py, and individual users would be
posting all results to a common database, we'd be decoupling test
results presentation from any particular testing platform.
g) Should CMake solution every get accepted, it could easily be adjusted
to post the the database as well.

Boost needs this going back at least a decade.

Robert Ramey