Experimenting with XProc testing

[Jostein Austvik Jacobsen]

FYI: I've been experimenting in my spare time with an XProc testing tool (dubbed XProcSpec):

https://github.com/josteinaj/xprocspec

Jostein

[Romain Deltour]

Wow, cool!

I'm looking forward to playing with it :)

Romain.

--
You received this message because you are subscribed to the Google Groups "DAISY Pipeline Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to daisy-pipeline-...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Javier Asensio]

Oh, looks great! It would be possible to somehow connect it with junit? (maybe xspec can but I know nothing about it)

cheers,

Javi

[Jostein Austvik Jacobsen]

JUnit support would be neat. Would Maven support come for free if it had JUnit support?

Jostein

[Javier Asensio]

right!

cheers,

Javi

[Bert Frees]

Awesome, going to look at it next week. Maybe I'll write a Maven plugin for it.

[Jostein Austvik Jacobsen]

Just note that it's not a complete implementation yet and there's surely going to be a bunch of bugs there. I've only tested the bundled example.

For Maven/JUnit; let me know if there's any XML vocabulary that is more useful to get as output than the current HTML. I haven't decided 100% on the syntax for the machine-readable version of the output yet (I'm certainly having a look at Marisas validation-utils though).

Jostein

[Bert Frees]

You're using XSLT 3.0? That means I need Saxon PE or EE?

2013/4/18 Jostein Austvik Jacobsen <jost...@gmail.com>

[Jostein Austvik Jacobsen]

Yeah I need XSLT 3.0 for its xsl:evaluate. What you need for Saxon versions I don't know.

Jostein

[Bert Frees]

Hi all,

I've started writing a simple Maven plugin that has 2 goals:

1. xproc:xproc: for running any XProc (possibly with inputs, outputs, options and parameters). (Alternative for the Calabash Ant task.)

2. xproc:xprocspec: for running a series of XProcSpec tests and storing the reports.

https://github.com/bertfrees/xproc-maven-plugin

xprocspec doesn't work for me yet because I can't run XSLT 3.0 with Saxon home edition.

Currently the plugin uses Calabash as the XProc engine if you put it on the classpath. The idea is to ultimately support more engines.

As a matter of fact, I also did a proof of concept of an alternative "engine" that is backed by a running instance of the Pipeline. In developer mode, I now have a new "XProc server" module that listens to incoming requests from "XProc clients". I'm using RabbitMQ to pass messages between the client and the server. This approach adds a thin layer of complexity, but the benefits are that URIs from the catalogs work out of the box, and more importantly we are able to use Java functions and steps in our tests.

There's a couple of problems still, but the basic idea is working :) For those who want to check out the code:

https://github.com/bertfrees/pipeline-xproc-connector

https://github.com/daisy-consortium/pipeline-assembly/tree/xproc-server

Installing RabbitMQ is as easy as "brew install rabbitmq" and on Linux it shouldn't be very hard either.

Bert

[Jostein Austvik Jacobsen]

Awesome! I'll try it out when I get some time :)

Btw. xprocspec depends on Calabash because of its cx:eval step. I don't know if any other engines has implemented dynamic evaluation of XProc steps. A sub-optimal workaround for other XProc engines would be a two-step execution process where the first step generates the XProc scripts, and the second step runs them. Anyway, Calabash will be required for now.

(If anyone has a better name than "XProcSpec", let me know. It doesn't really stand for anything, it's just an xprocifization of XSpec. XSpec stands for "XSLT Specifications" I believe, so XProcSpec would be "XProc Specifications".)

Jostein

[Javier Asensio]

Great! is there any possibility to make that play nicely with surefire, so we'll have consistent test reports along the system?

cheers,

Javi

[Jostein Austvik Jacobsen]

As far as I can tell, surefire outputs XML like this[1][2]:

<testsuites>     1 the aggregated result of all junit testfiles
  <testsuite>    * the output from a single TestSuite
    <properties> 1 the defined properties at test execution
      <property> * name/value pair for a single property
    <testcase>   * the results from executing a test method
    <system-out> 1 data written to System.out during the test run
    <system-err> 1 data written to System.err during the test run

So that would be a useful output from xprocspec then I guess?

[1] http://junitpdfreport.sourceforge.net/managedcontent/PdfTranslation

[2] http://windyroad.com.au/dl/Open%20Source/JUnit.xsd

Jostein

[Bert Frees]

I think that would be useful. surefire-report:report could then be used to parse this XML into nice HTML reports.

http://maven.apache.org/surefire/maven-surefire-report-plugin/usage.html

2013/4/22 Jostein Austvik Jacobsen <jost...@gmail.com>

[Jostein Austvik Jacobsen]

junit-style test results hereby added as an output port.

Jostein

[Bert Frees]

I updated xproc-maven-plugin and xproc:xprocspec works now! For an example see https://github.com/bertfrees/xproc-maven-plugin/blob/master/src/it/it-xprocspec-hello-world-calabash/pom.xml. Running mvn surefire-report:report site should generate the HTML report.

[Jostein Austvik Jacobsen]

I'm unable to compile xproc-client and xproc-server.

xproc-client:

[ERROR] /home/jostein/pipeline-xproc-connector/xproc-client/src/main/java/org/daisy/pipeline/xproc/connect/XProcService.java:[7,20] error: generics are not supported in -source 1.3

xproc-server:

[ERROR] /home/jostein/pipeline-xproc-connector/xproc-server/src/main/java/org/daisy/pipeline/xproc/connect/XProcServer.java:[61,19] error: method withInput in class Builder cannot be applied to given types;

Jostein

[Bert Frees]

Ah yes I forgot to push a bugfix for xproc-server. And xproc-client should not be compiled with Java 1.3. Should be fixed now.

2013/4/23 Jostein Austvik Jacobsen <jost...@gmail.com>

[Bert Frees]

Should be fixed now.

2013/4/23 Jostein Austvik Jacobsen <jost...@gmail.com>

I'm unable to compile xproc-client and xproc-server.

[Jostein Austvik Jacobsen]

Great, got both xproc-server and xproc-client compiled.

xproc-maven-plugin also needs its Java version set:

[ERROR] /home/jostein/xproc-maven-plugin/src/main/java/org/daisy/pipeline/maven/xproc/XProcEngine.java:[7,20] error: generics are not supported in -source 1.3

After doing that:

[INFO] --- maven-invoker-plugin:1.8:run (integration-test) @ xproc-maven-plugin ---

[WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent!

[INFO] Building: it-xprocspec-hello-world-dp2/pom.xml

[INFO] ..FAILED (3.2 s)

[INFO]   The build exited with code 1. See /home/jostein/xproc-maven-plugin/target/it/it-xprocspec-hello-world-dp2/build.log for details.

[INFO] Building: it-xprocspec-hello-world-calabash/pom.xml

[INFO] ..FAILED (3.6 s)

[INFO]   The build exited with code 1. See /home/jostein/xproc-maven-plugin/target/it/it-xprocspec-hello-world-calabash/build.log for details.

[INFO] -------------------------------------------------

[INFO] Build Summary:

[INFO]   Passed: 0, Failed: 2, Errors: 0, Skipped: 0

[INFO] -------------------------------------------------

[ERROR] The following builds failed:

[ERROR] *  it-xprocspec-hello-world-dp2/pom.xml

[ERROR] *  it-xprocspec-hello-world-calabash/pom.xml

[INFO] -------------------------------------------------

Jostein

[Jostein Austvik Jacobsen]

Actually, it-xprocspec-hello-world-calabash succeeded once I renamed ~/.calabash to something else.

I guess I have to start Pipeline 2 for it-xprocspec-hello-world-dp2 to succeed.

Jostein

[Bert Frees]

Thanks I'll fix it. I wonder why Maven picks Java 1.3 by default for you.

it-xprocspec-hello-world-dp2 won't work yet because the Pipeline's XProc engine doesn't handle jar: URIs. (Well, actually jar: URI are loaded, but resolving relative URIs against a jar: base URI doesn't work.) I'll have to extract all the xprocspec files prior to doing the conversion.

But yes, you should start the Pipeline for it to work. Check out the xproc-server branch of pipeline-assembly.

2013/4/23 Jostein Austvik Jacobsen <jost...@gmail.com>

Actually, it-xprocspec-hello-world-calabash succeeded once I renamed ~/.calabash to something else.

[Javier Asensio]

if you're copying resources is about too :P

cheers,

Javi

[Bert Frees]

On Debian/Ubuntu:

echo "deb http://www.rabbitmq.com/debian/ testing main" > /etc/apt/sources.list.d/rabbitmq.list

sudo aptitude install rabbitmq-server

2013/4/21 Bert Frees <bert...@gmail.com>

[Bert Frees]

I got everything working now. Both the integration tests succeed.

2013/4/23 Bert Frees <bert...@gmail.com>

[Christian Egli]

Bert Frees <bert...@gmail.com> writes:

> echo "deb http://www.rabbitmq.com/debian/ testing main" >
> /etc/apt/sources.list.d/rabbitmq.list
> sudo aptitude install rabbitmq-server

There is no need to muck around with apt sources. Debian stable has
rabbitmq-server and Ubuntu all the way back to lucid, so simply

sudo aptitude install rabbitmq-server

should do

--
Christian Egli
Swiss Library for the Blind, Visually Impaired and Print Disabled
Grubenstrasse 12, CH-8045 Zürich, Switzerland

-----
Die neue Online-Bibliothek der SBS: Mit wenigen Klicks zum Buch unter http://online.sbs.ch