synopses on smoke server
Christopher D. Malon
> if someone can offer regular smoke results (i.e. the tests.yml
> generated by `make smoke'), we can render the Synopses on feather
> with smoke results as well
The obvious way to get this to happen, for all runtimes, is to
integrate the
synopsis-with-smoke production with the existing smoke server
infrastructure.
I've started hacking this in, but have some concerns.
A few are mainly directed to iblech++.
1. Multi-versioning the tests: The inlined tests need to match
the revision at which the smoke was produced. Is it okay to have
the smoke server check out the relevant revision of the pugs/t directory
each time a smoke of a new revision is submitted, or is that too
expensive?
2. Should the synopses just be provided by a third "»" column on
the existing smoke.pugscode.org index page, or should the
synopses-with-smoke have a dedicated index page organized
somewhat differently?
3. What is the full path to smartlinks.pl on the smoke server?
4. Is the smoke server getting refreshed copies of the
synopses somewhere already?
5. Will a commit that changes smokeserv-server.pl be
picked up by the smoke server right away?
I might down the whole smoke system with my commit.
Anyone willing to debug with me when I do?
In summary, the basic pattern would be:
(a) smokeserv-client sends both .html and .yml of smoke
(b) If smokeserv-server gets .yml, it checks out the t/ directory for
that revision, and
(c) smokeserv-server runs smartlinks.pl on the .yml with that t/
directory and indexes it.
Christopher
Yuval Kogman
One is written in Catalyst + DBIC, by yours truly.
http://dev.catalystframework.org/repos/Catalyst/trunk/examples/SmokeServer
I've stopped working on it due to lack of interest - nobody seemed
unhappy enough with the currrent smoke server.
Another is written in Jifty:
http://search.cpan.org/dist/Test-Chimps/
It's further along, but is a little more specific to BP's workflow.
--
Yuval Kogman <nothi...@woobling.org>
http://nothingmuch.woobling.org 0xEBD27418
Jesse
On Tue, Sep 05, 2006 at 07:24:14PM +0300, Yuval Kogman wrote:
> There are two other smoke servers to look into...
>
> One is written in Catalyst + DBIC, by yours truly.
>
> http://dev.catalystframework.org/repos/Catalyst/trunk/examples/SmokeServer
>
> I've stopped working on it due to lack of interest - nobody seemed
> unhappy enough with the currrent smoke server.
>
> Another is written in Jifty:
Actually, it uses Jifty::DBI for its storage, but it's not a jifty app ;)
> http://search.cpan.org/dist/Test-Chimps/
>
> It's further along, but is a little more specific to BP's workflow.
The _intent_ was to make the tool more general than the existing pugs
smoker. If we've somehow messed that up, I'd like to hear about it.
Jesse
> --
> Yuval Kogman <nothi...@woobling.org>
> http://nothingmuch.woobling.org 0xEBD27418
>
--
Jesse
> Actually, it uses Jifty::DBI for its storage, but it's not a jifty app ;)
>
> > http://search.cpan.org/dist/Test-Chimps/
> >
> > It's further along, but is a little more specific to BP's workflow.
>
> The _intent_ was to make the tool more general than the existing pugs
> smoker. If we've somehow messed that up, I'd like to hear about it.
And I should say that if you're interested in hacking on Chimps, we'd
love more hackers.
-j
> Jesse
>
> > --
> > Yuval Kogman <nothi...@woobling.org>
> > http://nothingmuch.woobling.org 0xEBD27418
> >
>
>
>
> --
>
--
Yuval Kogman
> Actually, it uses Jifty::DBI for its storage, but it's not a jifty app ;)
Oh!
> The _intent_ was to make the tool more general than the existing pugs
> smoker. If we've somehow messed that up, I'd like to hear about it.
Oh!
I thought the Anna integration with the repo and all that was the
tailored precisely for you guys.
I stand corrected =)
Agent Zhang
>
> In summary, the basic pattern would be:
> (a) smokeserv-client sends both .html and .yml of smoke
I really really want this feature. I've even added this to Pugs' TASKS file.
> (b) If smokeserv-server gets .yml, it checks out the t/ directory for
> that revision, and
To be honest, at this moment I don't quite care about checking out t/.
Instead I just hope the new smoke server can make the tests.yml with
largest rev number available somewhere in the index page. ;-)
> (c) smokeserv-server runs smartlinks.pl on the .yml with that t/
> directory and indexes it.
>
I don't know if the smoke server has checked out smartlinks.pl like
feather. My current plan is to run smartlinks.pl on feather and render
the synopses using the latest version of pugs smoke results.
I admit your approach is much saner, but apparently needs much more work. :)
I also share a lot of your questions. Hopefully the maintainer of the
current smoke server (ingy or nothingmuch?) can give some guide here.
:)
Cheers,
Agent
Ingo Blechschmidt
Christopher D. Malon wrote:
> Agentz++ writes, in a Pugs blog comment:
>> if someone can offer regular smoke results (i.e. the tests.yml
>> generated by `make smoke'), we can render the Synopses on feather
>> with smoke results as well
>
> The obvious way to get this to happen, for all runtimes, is to
> integrate the
> synopsis-with-smoke production with the existing smoke server
> infrastructure.
>
> I've started hacking this in, but have some concerns.
> A few are mainly directed to iblech++.
>
> 1. Multi-versioning the tests: The inlined tests need to match
> the revision at which the smoke was produced. Is it okay to have
> the smoke server check out the relevant revision of the pugs/t
> directory each time a smoke of a new revision is submitted, or is that
> too expensive?
Currently, the smokeserver does not run smartlinks.pl etc., but
redirects to tests.pugscode.org via .htaccess:
Redirect /iblech/stuff/pugs-smokes/t http://tests.pugscode.org/t
Redirect /iblech/stuff/pugs-smokes/ext http://tests.pugscode.org/ext
(tests.pugscode.org used to be an alias to something under
http://feather.perl6.nl/.)
This way the smokeserver doesn't need to do anything except storing
the .html of smokes; especially it doesn't have to be updated WRT new
synopses/tests/smartlinks.pl.
Accordingly, ...
> 3. What is the full path to smartlinks.pl on the smoke server?
...there is none.
> 5. Will a commit that changes smokeserv-server.pl be
> picked up by the smoke server right away?
No; currently, you'll have to ping me (best by mail).
(Note that testing the smokeserver locally shouldn't be hard, as it
doesn't require setting up a proper database etc. Copying
smokeserv-serv.pl somewhere under cgi-bin/ and adjusting the paths
should suffice.)
> I might down the whole smoke system with my commit.
> Anyone willing to debug with me when I do?
I am, of course. :)
--Ingo
Christopher D. Malon
>
> Currently, the smokeserver does not run smartlinks.pl etc., but
> redirects to tests.pugscode.org via .htaccess:
>
> Redirect /iblech/stuff/pugs-smokes/t http://tests.pugscode.org/t
> Redirect /iblech/stuff/pugs-smokes/ext http://tests.pugscode.org/ext
The multi-versioning of the t/ directory seems critical,
because the tests in the YAML smokes are identified by line number,
and the correspondence will be wrong if the test has changed.
Therefore I'm afraid that a redirect to a single version of t/ won't
suffice.
The way I wrote the patch so far, checkouts of old t/ revisions
would be cleaned up at the same time as the corresponding smoke.
If the load of an SVN checkout (just of t/) upon smoke submit is
too much, maybe we should migrate to a different server.
It seems there are only a couple smokes submitted daily.
Would it be too much load on feather?
Alternatively, we could have the client send the whole t/
directory with the smoke report.
Christopher
Ingo Blechschmidt
Christopher D. Malon wrote:
>> Currently, the smokeserver does not run smartlinks.pl etc., but
>> redirects to tests.pugscode.org via .htaccess:
>>
>> Redirect /iblech/stuff/pugs-smokes/t http://tests.pugscode.org/t
>> Redirect /iblech/stuff/pugs-smokes/ext http://tests.pugscode.org/ext
>
> The multi-versioning of the t/ directory seems critical,
> because the tests in the YAML smokes are identified by line number,
> and the correspondence will be wrong if the test has changed.
> Therefore I'm afraid that a redirect to a single version of t/ won't
> suffice.
ah! I didn't realize that.
> If the load of an SVN checkout (just of t/) upon smoke submit is
> too much, maybe we should migrate to a different server.
Running svn a few times a day is really not an issue :)
--Ingo