Add gRPC packages - looking for a Jenkins CI

94 views
Skip to first unread message

Lennard Berger

unread,
Jun 21, 2016, 2:59:56 PM6/21/16
to grpc.io
Heya guys. 
We're working on an open-source game called "Inexor" which uses a bunch of technologies from Google

  • gRPC (0.13.1 if I'm not mistaken)
  • protobuf3
  • libCEF (on branch 2526, looking forward to upgrade)
Currently we do need to ship everything pre-compiled in a (we call it) platform-repository, containing executables (...) for the primary distributions we work with.
This is just fine for a bunch of developers, however as we're moving into the beta stage slowly, we'd wish to have a proper architecture behind, including
  • Packages for Debian, Ubuntu (and whatever community-distribution) scripts are going to be shipped 
    • This means we'd like roll the dice and ship our own PPA with up-to-date packages of the mentioned technologies
    • Building on the gRPC, libCEF and protobuf3 packages, we would use those as dependencies for Inexor
  • Use Chocolatery (the windows package manager) and compile binaries for Win7/8/10
Anyhow we've tried this for a couple of times, and it didn't really work out yet, for some reasons
  • We don't have the private capacities (neither are we willing to provide them) to host SUCH a Jenkins instance
  • Compiling libCEF is monstrously, it can take up to 25 GB for each individual build
  • Free, cloud-based hosters (such as CloudBees, who sponsored us previously) and Travis are either too-slow or don't allow container modifications
  • We need those for a decent reason: We build on GCC5+ (more precisely, on it's C++ standard library) and clang 3.2+ (with the clang language frontend), whom are not shipped in older distributions than Ubuntu 15.04 (and this consiquently breaks the entire proccess, since everything is compiled against GCC4)
That's why I'm asking here for a possibility to use the gRPC's Jenkins instance (or some other Google Cloud service) to compile mentioned packages (and Inexor, respectively).
I'd love to hear from ya guys, regards
Lennard "Fohlen" Berger

Nicolas Noble

unread,
Jun 22, 2016, 11:31:30 AM6/22/16
to Lennard Berger, grpc.io

I will have to be a bit blunt, but the answer has to be no. There are many reasons why, but it revolves around the fact that computing resources aren't free. When running arbitrary code on a public facing website, we are constantly running into the risk that someone will spawn bitcoin mining processes on that infrastructure, and we already have to be extra careful about this with grpc. There is a good reason for something like Travis to have hard CPU quotas and strict limitations on what you can do for free. Note that you can give them money to unlock these restrictions. And the grpc team doesn't have the manpower to dedicate to the level of administration required to take additional external customers on our infrastructure - even if they would pay us for it.

I can more than anyone understand your frustration with building a large, resource consuming project for free, because this is exactly what we are doing at the moment with grpc, and we went down the same road as you, realizing that Travis would be enough for us. However, recouping your costs one way or another will unfortunately have to be a necessity I am afraid, be it from sponsorship, or other commercial venues. If you are looking for sponsorship from Google, the grpc team isn't really the appropriate entry point for that, and to be fair, I wouldn't know exactly whom to ask. Maybe try p...@google.com ? At the same time, can I suggest that you use the free trial of Google cloud engine, install Jenkins there, and evaluate the costs of running your project there ?


--
You received this message because you are subscribed to the Google Groups "grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email to grpc-io+u...@googlegroups.com.
To post to this group, send email to grp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/3474049f-5d6c-42ab-8f4e-6cfbef852311%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Lennard Berger

unread,
Jun 27, 2016, 5:39:18 AM6/27/16
to grpc.io
Hey Nicolas
Thank's for your fast response in the first place!
I've been quiet a bit busy lately, which is why I didn't respond as fast as you did.
However, thank you for the insight in your experience. It's interesting to hear that other people alongside struggle to compile the full toolchain..
As for the sponsoring, we don't know yet and will have a look at the suggested address at a later point maybe (with working packages).
For now we'll try to build a infrastructure ahead some VPS's and we'll keep ya updated if we manage to package gRPC for Debian/Ubuntu!
Kind regards, Fohlen
Reply all
Reply to author
Forward
0 new messages