round 2: new CI
Ryan Davis
----------------------------------------
cluster_test is a continuous build / test system for ruby
implementations. On the client side, it runs rubyspecs against any
number of ruby versions for a given implementation (mri, rbx,
etc). The server script collects submitted data and produces summaries
of the current results.
If You Want to Contribute:
--------------------------
Please coordinate with the rubinius team! We only need one incremental
and one full per platform. Anything more just gets dumped on the
floor. Save the salmon, etc etc.
Cut to the Chase: How to run a client:
-------------------------------------
0. Install ZenTest at least version 3.11
1. Follow multiruby setup steps below.
* multiruby_setup update:rubygems
* multiruby_setup rbx:git:current
* multiruby -S gem install --no-ri --no-rdoc mspec
2. cd cluster_test
3. mkdir ~/work/rubinius-ci
4. ruby ci.rb -i -d ~/work/rubinius-ci -r rbx
5. Wait. It's now waiting for a commit.
Basic Architecture:
-------------------
ci.rb - client run in various platforms for various ruby
implementations.
ci_submit.cgi - runs on ci.rubini.us and receives test result
payloads and files them via safecat.
ci_cron.rb - runs on ci.rubini.us and collects the results and
generates html files, and updates client trigger files.
Running the Scripts:
--------------------
The package contains the following files:
+ ci.rb - the client side script.
REQUIRES A PROPER MULTIRUBY SETUP. See below for details.
Takes two optional args and one mandatory arg:
ci.rb [-i] [-d dir] -r (mri|rbx)
-i is incremental and builds into base_dir/builds/
incremental.
-d dir changes the base directory to run out of.
-r is the ruby implementation you're running against (mri,
rbx, etc).
The rest is minutia and can be discovered by reading the
source.
+ ci_submit.cgi - server side build result submission CGI.
Tweak BASE_DIR and SAFE_CAT as necessary.
+ ci_cron.rb - server side cron script.
Takes two optional args:
ci_cron.rb [base_dir] [html_dir]
Client Strategy:
----------------
1) start and build necessary dir structure
2) head CI trigger URL to get timestamp
3) sleep and repeat 2 if date unchanged
4) if first time or timestamp has changed, run multiruby_update
5) git pull on rubyspec dir
6) run multiruby on the rubyspecs
7) split results from multiruby output
8) submit each result separately with rubyspec hash & ruby impl id (-v)
Client Dir Structure:
---------------------
root/
latest_hash.txt - file with latest built hash
latest_incremental_hash.txt - file with latest incrementally
built hash
HEAD/... - clone of rubyspecs
builds/$hash/... - clone of HEAD
builds/incremental/... - clone of HEAD
Server Dir Structure:
---------------------
root/
data/
html/
ci/
trigger/
tmp/
Multiruby Suggestions:
----------------------
*** Only have one ruby implementation per user ***
1) for mri, multiruby_setup mri:svn:branches
2) for rbx, multiruby_setup rbx:git:current
3) for other, submit patch to zentest project to teach multiruby_setup
how to work with your impl.
Client Requirements:
--------------------
1) sudo gem install mspec ZenTest
2) multiruby_setup update:rubygems
3) multiruby_setup help and run spec(s) for your test target, e.g.:
multiruby_setup mri:svn:branches - for all MRI active branches
4) multiruby -S gem install mspec
Server Requirements:
--------------------
1) sudo gem install tagz
2) safecat - http://jeenyus.net/~budney/linux/software/safecat.html
Maintenance Details:
--------------------
+ Server is hosted on hoshi.fallingsnow.net in ~ci/
+ CGI is located in ~ci/cgi-bin.
+ Cron is run by user ci and is checked in as "crontab".
+ Results are reported on: http://ci.rubini.us/
The following people have admin access on the server:
+ Evan Phoenix (irc: evan)
+ Ryan Davis (irc: zenspider)
The follow CI clients are setup:
+ elle.fallingsnow.net - linux/x86-32/debian
mode: incremental
admin: Evan Phoenix (irc: evan)
Evan Phoenix
on http://ci.rubini.us. There are number of data points and vectors
within the data, so here's my idea:
* Multiple pages, showing vectors in a simple 2d table
The current page lists simple rubyspec commit, per platform. More
rubyspec commits, the page flows downward, toward older commits. This
vector shows us which rubyspecs commits are working (but not any impl.
data).
* Another page should show a snapshot view:
Impl / Last rubyspec hash / date / good:bad
This page would have a static size, defined by the number of
implementations being running against. At first, this page would just
have one row (Rubinius). This would be a heads up / dashboard page.
* Then there would be a set of pages, one per impl:
Each page would show:
rubyspec hash / date / good:bad
This page wild list all submits against the current impl, allowing
with the rubyspec hashs that were run. This page would be used to
discover when something went wrong. When either the impl. or the
rubyspecs start failing, they'll continue to fail until fixed. You can
use this page to find out where it broke, so that the breakage to be
easily found.
* Compliance page:
A single page, containing:
rubyspec hash / date / rbx / jruby / ...
This page would be used to verify over all compliance among impls.
This page is the least useful right now, with most impls having large
sections of unimplemented features. But it would have a red or green
background in each cell for each impl, depending on if it passed on
failed that rubyspec hash.
So, thats my initial breakdown of some different page views.
- Evan
M. Edward (Ed) Borasky
> Rubinius Cluster Test (bad name--I know)
> ----------------------------------------
>
> cluster_test is a continuous build / test system for ruby
> implementations. On the client side, it runs rubyspecs against any
> number of ruby versions for a given implementation (mri, rbx,
> etc). The server script collects submitted data and produces summaries
> of the current results.
>
> If You Want to Contribute:
> --------------------------
>
> Please coordinate with the rubinius team! We only need one incremental
> and one full per platform. Anything more just gets dumped on the
> floor. Save the salmon, etc etc.
Do you need x86_64 (dual-core SMP!) Gentoo Linux?
>
> Cut to the Chase: How to run a client:
> -------------------------------------
>
> 0. Install ZenTest at least version 3.11
Has that hit the repositories? All I see right now is 3.10.
>
> 1. Follow multiruby setup steps below.
> * multiruby_setup update:rubygems
> * multiruby_setup rbx:git:current
> * multiruby -S gem install --no-ri --no-rdoc mspec
>
> 2. cd cluster_test
> 3. mkdir ~/work/rubinius-ci
> 4. ruby ci.rb -i -d ~/work/rubinius-ci -r rbx
> 5. Wait. It's now waiting for a commit.
This runs non-root, right?
--
M. Edward (Ed) Borasky
http://ruby-perspectives.blogspot.com/
"A mathematician is a machine for turning coffee into theorems." --
Alfréd Rényi via Paul Erdős
jredville
structure morphing over time. All of this of course implemented in
Silverlight!
Glad to help
JD
On Jul 14, 4:49 pm, Evan Phoenix <e...@fallingsnow.net> wrote:
> Something thats not mentioned here is the form that the data appears
> > Maintenance Details:
> > --------------------
>
> > + Server is hosted on hoshi.fallingsnow.net in ~ci/
> > + CGI is located in ~ci/cgi-bin.
> > + Cron is run by user ci and is checked in as "crontab".
> > + Results are reported on:http://ci.rubini.us/
>
> > The following people have admin access on the server:
>
> > + Evan Phoenix (irc: evan)
> > + Ryan Davis (irc: zenspider)
>
> > The follow CI clients are setup:
>
> > + elle.fallingsnow.net - linux/x86-32/debian
> > mode: incremental
>
> - Show quoted text -
Charles Oliver Nutter
> *** Only have one ruby implementation per user ***
>
> 1) for mri, multiruby_setup mri:svn:branches
> 2) for rbx, multiruby_setup rbx:git:current
> 3) for other, submit patch to zentest project to teach multiruby_setup
> how to work with your impl.
Can you push the updated code with rbx stuff somewhere or point me to P4
setup info (pretty sure I still have an account)? The release version
doesn't have rbx stuff in it and I was hoping to base JRuby
modifications off that, since it seems like ZenTest 3.10 has a lot of
MRI-specific logic you've probably reworked for rbx.
Also, is it necessary to have the list of tags and branches in
multiruby.rb? We put out a new release every 1-2 months, so it would be
out of date pretty quickly. Perhaps there could simply be a URL pointing
to a file, probably in SCM, that lists tags and branches to test.
- Charlie