Please review: Relax NG schema for bitten recipes
W. Martin Borgert
this is a first draft for a bitten recipe schema. Please
try to validate your recipes using the attached Python
script. Write about any problems to the list, just in case
a bus runs over me and I cannot answer in private.
For most attributes I'm not sure whether they are optional
or not. Some are explicitly described as optional in the
bitten documentation(1), but it's obvious, that many others
are meant to be optional, too.
Furthermore, it seems, that some attributes are meant to
be boolean True/False, i.e. svn:checkout verbose and
svn:update no_auth_cache, but it's not really clear from
the documentation.
Cheers
(1) http://bitten.edgewall.org/wiki/Documentation/commands.html
Olaf Meeuwissen
> this is a first draft for a bitten recipe schema. Please
> try to validate your recipes using the attached Python
> script. Write about any problems to the list, just in case
> a bus runs over me and I cannot answer in private.
Just tried to validate the two recipes we use here most.
$ python bitten-recipe.py step-by-step-make-distcheck.xml
Traceback (most recent call last):
File "bitten-recipe.py", line 10, in <module>
relaxng.assertValid(lxml.etree.parse(recipe))
File "lxml.etree.pyx", line 3017, in lxml.etree._Validator.assertValid (src/lxml/lxml.etree.c:129442)
lxml.etree.DocumentInvalid: Invalid attribute onerror for element step, line 62
The offending line is:
<step id="prep-slave" onerror="fail" description="blah">
Looks like you missed the "onerror"[1] attribute.
[1] http://bitten.edgewall.org/wiki/Documentation/recipes.html
$ python bitten-recipe.py simple-make-distcheck.xml
Traceback (most recent call last):
File "bitten-recipe.py", line 10, in <module>
relaxng.assertValid(lxml.etree.parse(recipe))
File "lxml.etree.pyx", line 3017, in lxml.etree._Validator.assertValid (src/lxml/lxml.etree.c:129442)
lxml.etree.DocumentInvalid: Invalid attribute root for element gcov, line 33
Here the offending bit of XML reads:
<c:gcov include="*.cc" root="_build"/>
Here the "root" attribute is not documented on the wiki but the code has
documentation[2] for it.
[2] http://bitten.edgewall.org/browser/trunk/bitten/build/ctools.py#L286
Hope this helps,
--
Olaf Meeuwissen, LPIC-2 FLOSS Engineer -- AVASYS CORPORATION
FSF Associate Member #1962 Help support software freedom
http://www.fsf.org/jf?referrer=1962
W. Martin Borgert
> Looks like you missed the "onerror"[1] attribute.
> [1] http://bitten.edgewall.org/wiki/Documentation/recipes.html
Indeed, thanks! I added it with the three possible values.
> Here the "root" attribute is not documented on the wiki but the code has
> documentation[2] for it.
> [2] http://bitten.edgewall.org/browser/trunk/bitten/build/ctools.py#L286
OK, added. I used only the wiki, so any further attributes etc.
that are not documented there, are still missing in the Relax NG.
Cheers
Olaf Meeuwissen
Thanks. I've rerun with the updated Relax NG and my
simple-make-distcheck.xml now passes fine. The other,
step-by-step-make-dist.xml recipe does not. This time this is due to a
local extension (I've written an alternative gcov-like command and I'm
using the ohcount command attached to bitten:#453). How would you deal
with these kind of things? Just a thought.
W. Martin Borgert
> Thanks. I've rerun with the updated Relax NG and my
> simple-make-distcheck.xml now passes fine. The other,
> step-by-step-make-dist.xml recipe does not. This time this is due to a
> local extension (I've written an alternative gcov-like command and I'm
> using the ohcount command attached to bitten:#453). How would you deal
> with these kind of things? Just a thought.
Relax NG allows for extensions, but I had to re-arrange the
schema a little bit to make use of this. Take a look at the
attachments and change the Python snippet to use olaf.rng
instead of bitten-recipe.rng.
Olaf Meeuwissen
Cool. That worked, provided I change the xmlns:src value to whatever it
has been set to in the recipe and extend the element definition to cover
any other options that src:ohcount may have grown locally. Of course, I
would also have to add something for that gcov-like addition but simply
commented it out for testing purposes.
So, for any local extensions one may have it would be enough to whip up
your own Relax NG and have it include bitten-recipe.rng.
Thanks,
W. Martin Borgert
> So, for any local extensions one may have it would be enough to whip up
> your own Relax NG and have it include bitten-recipe.rng.
Yes, that's the idea. I will clean up the files a little bit
and contribute them to the bitten sources, if nobody has
further problems or objections.
Cheers
anatoly techtonik
anatoly t.
Simon Cross
> Well, Bitten already had capability to check configs when editing them, and
This is true, but the checks in bitten.recipes are somewhat different.
From the docstring of Recipe.validdate:
"""
This method checks a number of constraints:
- the name of the root element must be "build"
- the only permitted child elements or the root element with the name
"step"
- the recipe must contain at least one step
- step elements must have a unique "id" attribute
- a step must contain at least one nested command
- commands must not have nested content
"""
> it would be more useful if this capability could be exposed to `trac-admin
> bitten config` commands.
I think that would be a fairly easy patch (the Trac admin command just
needs to load the file into a recipe and run validate). I'm not sure
how useful it is while the primary interaction with recipes is via the
Bitten web interface though.
Schiavo
Simon
Simon Cross
On Thu, Feb 2, 2012 at 11:45 AM, W. Martin Borgert <deb...@debian.org> wrote:
> Yes, that's the idea. I will clean up the files a little bit
> and contribute them to the bitten sources, if nobody has
> further problems or objections.
Thank you for all your work on the schema. Do you have some idea of
where in the Bitten sources you'd see them going? Would they just be a
documentation thing? Are you envisioning some sort of tool for
checking recipes built on top of them?
Schiavo
Simon
W. Martin Borgert
> Do you have some idea of
> where in the Bitten sources you'd see them going?
No idea.
> Would they just be a
> documentation thing?
Originally the idea was to have a formal documentation, but...
> Are you envisioning some sort of tool for
> checking recipes built on top of them?
It would be very easy to integrate the check into Bitten. One
would have to allow the user to install or upload their own
schemas, if they use private extensions.
import lxml.etree
schema_fn = "bitten-recipe.rng"
recipe_fn = "myrecipe.build"
if __name__ == "__main__":
with open(schema_fn) as rng:
relaxng = lxml.etree.RelaxNG(lxml.etree.parse(rng))
with open(recipe_fn) as recipe:
relaxng.assertValid(lxml.etree.parse(recipe))
Cheers
Simon Cross
W. Martin Borgert: There's some feedback I need from you on the ticket
before I can land it (just questions about whether you're happy with
the licensing, attribution and copyright assignment).
Schiavo
Simon
W. Martin Borgert
OK with me.
Simon Cross
> OK with me.
Thanks! Patch committed in r1010.
W. Martin Borgert
> Thanks! Patch committed in r1010.
We should also add a description on how to add custom
commands. I sent an example to the list:
http://groups.google.com/group/bitten/msg/02f0d8920b34db9b
Simon Cross
> We should also add a description on how to add custom
> commands. I sent an example to the list:
> http://groups.google.com/group/bitten/msg/02f0d8920b34db9b
Any idea where that description should live? Docstring of the
checkrecipe.py script? Want to submit a patch? :)
Schiavo
Simon