Build flow plugin: How to copy results from downstream jobs

[Patrick]

Hi All

I've started using the Build Flow plugin to simplify my build pipeline. I've gotten it to work just fine (all the jobs get triggered at the right stage) but I can't seem to copy the generated build results from the downstream jobs. For instance my build produces a CCM log (an XML file) and I want to copy that file to the workspace of the flow job. I've tried the following groovy script (note that I haven't coded in Groovy before so there may well be some silly syntax error in the code. Then again Jenkins runs through it fine ...)

// workspace directory structure should look like:

// WORKSPACE

//     build

//         bin

//         logs

//         temp

//         ...

// ...

workspaceDir = build.environment["WORKSPACE"]

buildDir = workspaceDir + "//build"

buildLogDir = buildDir + "//logs"

new File(buildLogDir).mkdirs()

build("job1")

parallel(

    {         

        build("job2") 

        

        builder = new AntBuilder()

        localBuildLogDir = new File(b.build.environment["WORKSPACE"] + "\\build\\log")

        out.println "Copying CCM files from: " + buildLogDir

        builder.copy(file: localBuildLogDir + "\\nfermi.ccm.xml", tofile: buildLogDir + "\\nfermi.ccm.xml")

    },

    {

        build("job3")

    }.

)

Accoording to this page: http://snipplr.com/view/1932/copy-file/ the file copy should work but it doesn't. Ideally I'd just like to copy the 'build' directory from each job back to the flow job workspace.

Any suggestions?

Thanks

Petrik

[nicolas de loof]

I recommend to avoid such scripting in the flow DSL, that is only designed to manage orchestration, not to replace plugins/job logic

use copy-artifact plugin to get last build / upstream / fingerprint file from upstream job

Future version of build-flow may implement some helper methods to pass fingerprints or copy-artifact plugin, not sure yet about the way to define this in the DSL

2013/1/1 Patrick <petrikva...@gmail.com>

[Patrick van der Velde]

Hi

Mmm ok, I guess I could create a separate job to gather all the test results. Thanks for the advice :)

Regards

Petrik

[Patrick]

Ok I'm going to have to amend this answer. My idea of having a separate job at the end to gather the results would work if it wasn't for the fact that the build flow plugin kills the build as soon as one of the jobs fails. That means I only can get the results if the build works which is not what I want. I want the results gathering to always take place, even if the all the build jobs fail. Any way to achieve this?

Thanks

Petrik

[nicolas de loof]

Guard + rescue

[nicolas de loof]

use gard+rescue so you can execute a post-job even when some jobs are unstable

2013/1/2 Patrick <petrikva...@gmail.com>

[Patrick van der Velde]

Thanks for that suggestion. One question about the guard statement. Can it handle multiple statements? i.e. is the following allowed?

guard {

    build("job1")

    build("job2")

} rescue {

    build("finaljob")

}

or even this

guard {

    parallel(

        { build("job1a") },

        { build("job2a") },

    )

    parallel(

        { build("job1b") },

        { build("job2b") },

    )

} rescue {

    build("finaljob")

}

My script looks a bit like that last one but when I tried putting a guard clause around it I got the following error:

ERROR: Failed to run DSL Script
groovy.lang.MissingMethodException: No signature of method: com.cloudbees.plugins.flow.FlowDelegate.rescue() is applicable for argument types: (Script1$_run_closure1_closure3) values: 

Removing the guard clause made it work. So I'm guessing guard can only handle 1 item?

Thanks

Petrik

[nicolas de loof]

sure, rescue handle whatever happens in gard block, that has no restriction on nested content

2013/1/3 Patrick van der Velde <petrikva...@gmail.com>

[John Russell]

Do you guys have any idea how to pull files, specifically test results, from the jobs started in a build flow up to the build flow job itself so it can be the one that presents all of the test results?

I presume that if I can copy them from the slaves up to the workspace of the build flow build that the post build step of processing the test results will get them all. Any thoughts on how to get those files back to the master?

[Nico dHeureuse]

I have a very similar problem: I'm looking for a way to get the aggregated test results which are published by some of my jobs. However, the main Build Flow job does not seem to correctly identify the downstream tasks it needs to collect the test results from, even though the fingerprinting of all the jobs seems to work fine.

Does anyone have an idea how to accomplish this?

Thanks a lot in advance,
Nico

[teilo]

The BuildFlow doesn't use a workspace anymore[1] - so your workaround most likely won't work as you expect.

I'm not convinced that this is a good thing as like you I would like to show test results in the main flow job - not have another job that is just aggregate & report.

[nicolas de loof]

AggregatedTestResultPublisher should work with build-flow, passing the list of jobs to aggregate. I'm not sure it could automatically support the downstream mode, need to setup a test instance to check what prevent this, but can image few reasons.

2013/8/30 teilo <teilo+...@teilo.net>

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[James Nord (jnord)]

If the things you are aggregating aren’t junit like test reports you’re a bit lost though.

 

E.g. https://github.com/masterthought/jenkins-cucumber-jvm-reports-plugin

 

Whilst cucumber does support JUnit output which could be used by the Aggregated…Publisher – it is exceptionally limited to the point that it is virtually useless.

And whilst this could be a classed as a deficiency in the cucumber plugin or cucumber - mapping cucumber results to Jenkins SuiteResult/CaseResults doesn’t really fit – so you end up trying to put a square peg into a round hole.

 

/James

[John Russell]

Teilo, When did build flow stop having a workspace? I finally got this to work by archiving all of the junit files from the downstream jobs onto the master, running build-flow on the master, and directly copying the files from the archive of the downstream build to build flow and running the junit results post build step.

So this won't work anymore? If there is no workspace how would any junit result post build step work? 

[nicolas de loof]

I indeed removed workspace support especially to discourage such a "write some custom jenkins code using DSL" approach. 

build-flow is about orchestrating jobs, not creating custom plugins

junit aggregator is using the original location for junit results, and dynamically aggregating. It doesn't copy to local project.

2013/8/30 John Russell <jjru...@gmail.com>

[John Russell]

I see the class javadoc for that class but how would I use that to actually aggregate the junit tests to report in the build-flow plugin?

Is it also possible to discourage people in the documentation and yet leave the workspace there so its possible?

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-users/MX29Ld8upCs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-use...@googlegroups.com.

[nicolas de loof]

I don't want to follow this direction, as I know people won't consider the warning on documentation.

AggregatedTestResultPublisher can be configured with a set of jobs, maybe this require some tweaks on UI, not sure (sorry, don't have much time to investigate on this topic today)

2013/8/30 John Russell <jjru...@gmail.com>

[James Nord (jnord)]

The problem with a flow is that the flow can be dynamic – so having to specify the jobs may lead to incorrect jobs being aggregated – or maintinaing duplicate information.

 

It also appears to rely upon getDownStreamRelationship that relies on fingerprints to work unless overridden.

 

Perhaps this should be moved to the dev list…

 

/James

 

From: jenkins...@googlegroups.com [mailto:jenkins...@googlegroups.com] On Behalf Of nicolas de loof
Sent: 30 August 2013 15:52
To: jenkins...@googlegroups.com
Subject: Re: Build flow plugin: How to copy results from downstream jobs

 

I don't want to follow this direction, as I know people won't consider the warning on documentation.

[nicolas de loof]

right, need to programmatically add the AggregatedTestResultAction to current flow as part of the DSL. Probably could be packaged as an extension

2013/8/30 James Nord (jnord) <jn...@cisco.com>

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.