Continuous integration and build system

[Ariya Hidayat]

Folks,

With an increased user base of PhantomJS, more and more people want to
get their hands on the binary packages as early as possible. Some
would like to try the bleeding-edge version but do not have the
time/effort/setup to compile from source. There is also platforms like
Windows and BSD which lack a lot of tests and polishes (or even
compile fixes). Unfortunately, with a small development team and with
a limited set of machines, it is hard to fulfill this skyrocketing
demand. We are doing this in our spare time or as a side project,
there is a practical threshold to our effort.

Like everything in life (especially after the kindergarten), just
asking for it does not make it happen. We are also not in the business
of waving a magical wand. Fortunately, there is a chance for you to
help us moving in the direction everyone would love! If you are an
expert in setting up a continuous integration and build system, then
we love to get some helping hands. The requirement is very simple.
There should be a machine out there (does not matter where, it could
be even Amazon EC2 instance) which can build PhantomJS for every
revision on every supported platforms (Windows, Mac OS X, Linux, ...).
If there is no compile problem and the tests pass, just make the build
available for download. If error occurs, then post the full message
somewhere, preferably straight to the issue tracker.

Dozens of engineering groups are relying on PhantomJS for parts of
their QA workflow. I'm confident there are some organizations out
there in the position to carry out the above important task. In fact,
think of about the improvement cycle there: the more often you
stress-test PhantomJS, the less likely you need to worry about it when
you are about to use it in the later stage. Pardon my abuse of
buzzwords but I believe this scenario falls perfectly into the
"win-win situation" category.

Thus, if someone wants to help, just shout out and we can kick-start
the discussion, refine the steps, and so on.

"Ask not what PhantomJS can do for you - ask what you can do for
PhantomJS" (with an apology to Kennedy).


Thanks a lot!

Regards,

--
Ariya Hidayat, http://ariya.ofilabs.com
http://twitter.com/ariyahidayat

[Gregor Martynus]

Ariya,

I'm not an expert on this field, but could http://travis-ci.org/ help here? Would it fit your technical requirements?

Thanks for the awesome project, I use it heavily for headless browser testing and can't imagine work without anymore.

– Gregor

– 

[Ariya Hidayat]

> I'm not an expert on this field, but could http://travis-ci.org/ help here?
> Would it fit your technical requirements?

Travis CI does not support Mac OS X and Windows (yet).
Also, the free service (at that site) may not permit building
PhantomJS, which is justified (taking too long and too much
resources).

[Nicolas Perriault]

On Sat, Jun 23, 2012 at 11:29 AM, Ariya Hidayat <ariya....@gmail.com> wrote:

> "Ask not what PhantomJS can do for you - ask what you can do for
> PhantomJS" (with an apology to Kennedy).

Main problem is the infrastructure here, launching test suites and
building shouldn't be too much complicated I guess. So… Time for a
Donate button on phantomjs.org homepage?

I'd be the first to give.

++

--
Nicolas Perriault
https://nicolas.perriault.net/ — http://www.akei.com/
Skype: nperriault
Phone: +33 (0) 660 92 08 67

[Ariya Hidayat]

> Main problem is the infrastructure here, launching test suites and
> building shouldn't be too much complicated I guess. So… Time for a
> Donate button on phantomjs.org homepage?

Such a donation, if it is the form of money, will not solve all
problem. For example, currently I have negative spare time. No amount
of donation will clear my "debt".

Of course, we can use the donation to fund the work. However,
originally I thought that if our dark knight finally rises, his line
of thought is more "I don't owe these people anymore, but this stuff
saves my day so let me just setup that damn CI for once and for all"
rather than "Hmm, there's a storm coming. I can use some spare
change". There is nothing wrong with the latter, I just postulate that
the former is more future-proof since it is built on a stronger bond.

If all else fails, then a bounty which kickstarts the CI work can be
still commissioned. Heck, I'll be willing to donate as well.

"This conversation used to end with an unusual request."

[bruce]

Hey Ariya (and others)

I'm getting more into phantom/casper/etc. And I've followed the thread
for awhile now.

What would be your thoughts if someone were to setup a Kickstarter
Project to partially fund this kind of activity??

-bruce

> --
>
>
>

[James Greene]

Disclaimer: I'm not a PhantomJS developer.

I think a Kickstarter campaign would be a great way to get initial funding. However, I'd also be worried about where the future funding would come from... I'd much rather have these guys spending their time evolving PhantomJS than seeking out sponsorship.

I think a much more sustainable model would be setting up appropriate CI servers on Amazon's EC2 free tier... if we keep it to just 1 nightly build (versus CI, though I think that would probably be fine too given the lower frequency of commits), then there shouldn't be any risk of exceeding the limits of the free program (750 hours per OS, I believe). Granted, they are probably weaker servers but I think build speed/duration will be less an obstacle (at least for now) than just having visibility to the current quality of the build each day.

Thoughts?

~~James

Thoughts?

[James Greene]

*750 hours per OS PER MONTH

[Ivan De Marino]

I think we are really facing 2 different issues that could be resolved with 1 solution, but also not.

The first one is producing regularly and timely binary builds.

I noticed recent effort from Jon to make the linux build automated with Vagrant.

I think that is probably the easiest way to go.

With scripts for Mac OS X and Windows, the build could be produced by one of us.

Linux is more "a pain in the ass", but if Vagrant works than the solution is still easy: one vagrant script and we "should be done".

I say should because I'm not an expert of Vagrant in any way.

Of course, we will have to assign roles: we need find volunteer to dedicate their machine CPU overnight.

This should be done based on the hardware we can provide. Pretty sure we can put together all HW for at least the 3 major platforms.

Than there is CI Automation.

Given the skill set of the contributors, I'm confident most of us can sort it out easily.

The issue is the infrastructure cost. Or we need a free solution.

Maybe a EC2 free quota could help, but we would need some planning to know how to use this resource.

Isn't it limited only to the first year (me asks)?

I can definitely volunteer for providing CPU power for 1 or 2 binaries (1 linux + 1 mac / 2 linux).

But we need to finalise our building scripts.

--

--

Ivan De Marino

Coder, Technologist, Cook, Italian

blog.ivandemarino.me | www.linkedin.com/in/ivandemarino | twitter.com/detronizator

[Ivan De Marino]

There you go: http://aws.amazon.com/free/

It's limited to the 1st year.

Even though it sounds good now, we would probably have to pay some money ourselves afterwards.

Not sure if Ariya wants to go there: I think there is always a risk factor when you go that way. Money can turn in serious business all of a sudden.

I think we should stick with "local" hardware: it's not like we need to do it every single night every time.

We could do a weekly build (not a nightly) and publicise stats automatically somewhere free (I'm sure we can write a script to post to a webserver our local-weekly build result - I know of a nice tool that is perfect for that ;) ).

[Ariya Hidayat]

> I think a Kickstarter campaign would be a great way to get initial funding. However, I'd also be worried about where the future funding would come from... I'd much rather have these guys spending their time evolving PhantomJS than seeking out sponsorship.

I'm pretty much in agreement with James here. That kind of bond is
always stronger than any other motives.

[Ariya Hidayat]

> I think we should stick with "local" hardware: it's not like we need to do
> it every single night every time.
> We could do a weekly build (not a nightly) and publicise stats automatically
> somewhere free (I'm sure we can write a script to post to a webserver our
> local-weekly build result - I know of a nice tool that is perfect for that
> ;) ).

True, this can be the fallback solution. The principle remains the
same: if someone wants something, then set it up. Otherwise:

Ain't Gonna Happen (AGH)

(Jon and other English gentlemen might forgive me for my Americanism there)

[Ivan De Marino]

Will try to spend some time on this, but I'm pretty much sure I won't have tons of time: I'm already behind schedule with GhostDriver.

Maybe afterwards.

In the meantime, for mac we have homebrew (who is doing the updates at the recipe?), for linux the scripts seem to work fine.

For windows, we got Alessandro that is taking care of that.

--

[James Greene]

I just threw a pertinent question out on StackOverflow:

     http://stackoverflow.com/questions/11183865/are-there-any-free-open-source-build-systems-to-which-you-can-volunteer-your-own 

No responses yet.

~~James

[Ariya Hidayat]

> In the meantime, for mac we have homebrew (who is doing the updates at the
> recipe?), for linux the scripts seem to work fine.

JFYI Brew just pulls our download and extracts that (it never compiles
anything).

[Ivan De Marino]

:(

Why is that?

The current static build has issues, as we know.

--

[Ariya Hidayat]

> :(
> Why is that?

See https://github.com/mxcl/homebrew/pull/11225.

> The current static build has issues, as we know.

Yeah, but what will be a better alternative?

[Ivan De Marino]

Oh crap, I'm stupid.

We can't just compile PhantomJS against the system/homebrew Qt. What I was thinking?

That's sad though: we are forced to use our own Qt because we need special behaviours, but compiling statically is not fully working.

It looks like the dynamic build with /lib directory should be the way to go on Mac too.

--

[Jon Leighton]

What are the issues with the static build?

On 25/06/12 09:08, Ivan De Marino wrote:
> Oh crap, I'm stupid.
> We can't just compile PhantomJS against the system/homebrew Qt. What I
> was thinking?
>
> That's sad though: we are forced to use our own Qt because we need
> special behaviours, but compiling statically is not fully working.
>
> It looks like the dynamic build with /lib directory should be the way to
> go on Mac too.
>
> On 25 June 2012 01:02, Ariya Hidayat <ariya....@gmail.com

> <mailto:ariya....@gmail.com>> wrote:
>
> > :(
> > Why is that?
>
> See https://github.com/mxcl/homebrew/pull/11225.
>
> > The current static build has issues, as we know.
>
> Yeah, but what will be a better alternative?
>
>
>
> --
> Ariya Hidayat, http://ariya.ofilabs.com
> http://twitter.com/ariyahidayat
>
> --
>
>
>
>
>
>
> --

> *Ivan De Marino*
> Coder, Technologist, Cook, Italian
>
> blog.ivandemarino.me <http://blog.ivandemarino.me> |
> www.linkedin.com/in/ivandemarino
> <http://www.linkedin.com/in/ivandemarino> | twitter.com/detronizator
> <http://twitter.com/detronizator>
>
> --

--
http://jonathanleighton.com/

[Joshua Boyd]

On 6/23/2012 8:01 AM, Ariya Hidayat wrote:
>> I'm not an expert on this field, but could http://travis-ci.org/ help here?
>> Would it fit your technical requirements?
> Travis CI does not support Mac OS X and Windows (yet).
> Also, the free service (at that site) may not permit building
> PhantomJS, which is justified (taking too long and too much
> resources).

I would suggest Jenkins for CI. Installation is very simple.

[Ariya Hidayat]

I could not care less which CI system is used and how it gets
implemented. The point of this "call for action" is for someone to
volunteer and set it up (presumably with a system s/he is an expert
at).


Regards,

[James Greene]

Looks like you could apply for a free Windows build server (using TeamCity) via CodeBetter:

     http://codebetter.com/codebetter-ci/ 

~~James

[Ariya Hidayat]

> Looks like you could apply for a free Windows build server (using TeamCity)
> via CodeBetter:
> http://codebetter.com/codebetter-ci/

Looks interesting, unfortunately I hit the following notice:

************************** NOTE **************************

The build server does NOT have Visual Studio installed on it. There
are several reasons for this, the
major ones being:
- We don't want to pollute the build environment. Builds should not
depend on developer tools
- Licensing
- Maintaining different versions and service packs

As such, if your project depends on Visual Studio, we regret that we
will not be able to support it. We
realize that this eliminates many projects (such as those that use
Windows Phone SDK, XNA,
or MSTest) and we strongly encourage you to either find alternatives
or, better yet, to encourage
Microsoft not to tie their libraries to Visual Studio so closely.

---

[Joshua Boyd]

On 8/10/2012 11:53 PM, Ariya Hidayat wrote:
>> Looks like you could apply for a free Windows build server (using TeamCity)
>> via CodeBetter:
>> http://codebetter.com/codebetter-ci/
> Looks interesting, unfortunately I hit the following notice:
>
> ************************** NOTE **************************
>
> The build server does NOT have Visual Studio installed on it. There

Aside from the work involved, is there any major downside to converting
PhantomJS's Win32 build to be done with mingw?

[Ariya Hidayat]

> Aside from the work involved, is there any major downside to converting
> PhantomJS's Win32 build to be done with mingw?

There is no technical reason why MinGW won't work. I filed it as
http://code.google.com/p/phantomjs/issues/detail?id=711.