Determining the consequences of starting a job from the REST API

[Salim Fadhley]

Hi Jenkins team,

I'm one of the developers on the Python Jenkins API (https://pypi.python.org/pypi/jenkinsapi)

This project wraps up the Jenkins REST API and is designed to make interacting with Jenkins from Python very easy. Mostly it works well, however there's one feature that our users complain about often: Getting the status of recently started jobs.

It's easy to start a job in Jenkins, you just hit the /build URL. A status code of 200 or 201 indicates whether the request was accepted. So far so good - the problem we have is reporting back to the user in a *timely* and *reliable* manner what just happened (or what may be about to happen) as a consequence of starting a new build. You just cannot tell from an HTTP status code.

We've tried to solve this problem in a goofy way:

* We poll Jenkins to get the status of the builds & queue

* We tell Jenkins to start the build

* We wait a while

* We poll Jenkins again to see what changed.

The assumption is that any difference between the two polls can be accounted for by the job you just started. Quite a lot of the time that's an incorrect. Also, the waiting makes our API wrapper sluggish and less useful.

What'd really want is a change to the /build and /buildWithParmeters so that instead of just returning an HTTP status code, I'd like to see some more detailed status. It ought to be able to identify the queue item or ongoing build that was generated as a result of attempting to start the job.

Does this sound plausible?

Sal

[Jesse Glick]

On Wed, Jul 30, 2014 at 5:15 PM, Salim Fadhley <salimf...@gmail.com> wrote:
> What'd really want is a change to the /build and /buildWithParmeters so that
> instead of just returning an HTTP status code, I'd like to see some more
> detailed status. It ought to be able to identify the queue item or ongoing
> build that was generated as a result of attempting to start the job.

It already returns the queue item information, which allows you to
track the item through to an actual build (if it is ever scheduled).
The “Perform a build” section of the documentation page found from the
“REST API” link on a job’s index page gives details.

[Salim Fadhley]

Thanks Jesse,

According to the docs the /build method should always redirect over to a /queue/item URL:

"To programmatically schedule a new build, post to this URL. If the build has parameters, post to this URL and provide the parameters as form data. Either way, the successful queueing will result in 201 status code with Location HTTP header pointing the URL of the item in the queue. By polling the api/xml sub-URL of the queue item, you can track the status of the queued task. Generally, the task will go through some state transitions, then eventually it becomes either cancelled (look for the "cancelled" boolean property), or gets executed (look for the "executable" property that typically points to the AbstractBuild object.)"

However I've noticed that when the build has parameters it can redirect to Job's URL (not a queue), e.g something that looks like this:

http://localhost:8080/job/my_job

and not like this:

http://localhost:8080/queue/item/1234

I'm using 1.574

Sal

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/_vYBkV3hVtU/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Daniel Beck]

Cannot reproduce the problem.

Two options I could think of:
- You're using the wrong URL. Parameterized jobs need you to use /buildWithParameters, see second sentence you quoted.
- You're using a plugin that implements the QueueDecisionHandler extension point and vetoes scheduling of the task, in which case there is no queue item. Run `Queue.QueueDecisionHandler.all()` in script console to check. This returns an empty list for me.

> You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

[Salim Fadhley]

Thanks for this! It was a case of wrong URL on the build trigger. You guys are awesome.

As a result of your assistance Python Jenkins users will now be able to start builds and know exactly what happened as a consequence.

Sal

[Salim Fadhley]

A few more REST questions:

According to the documentation parameters for builds need to get submitted as form-fields, simple enough but I also notice that the Build with Parameters form submits an additional hidden JSON encoded form field. Is that actually used for anything? (presumably not since the docs make no mention)

Also I'm having some difficulty in the case of submitting files and non-file fields.

I've noticed that when one of the parameters is a file, the form seems to submit to /build rather than /buildWithParameters. As an experiment I set up a script to try to trigger a parameterized build with both files and non-file parameters. If I submitted this to /buildWithParameters results in everything except the file parameters being received. If I submitted to /build then only the file parameters are received. So does Jenkins distinguish between parametrized jobs which do / don't have file parameters? 

Hopefully this is just a case of something a little screwy in my own code!

Sal

[Salim Fadhley]

Actually I think my second question might have been a bit premature - I think I have been misreading things (it's late here in the UK) - from what I can tell every kind of parameter really does go via /buildWithParameters. :-)

[Jesse Glick]

On Sun, Aug 3, 2014 at 2:52 PM, Salim Fadhley <salimf...@gmail.com> wrote:
> According to the documentation parameters for builds need to get submitted
> as form-fields, simple enough but I also notice that the Build with
> Parameters form submits an additional hidden JSON encoded form field. Is
> that actually used for anything?

Probably not. In general, any Jenkins form submits *two* copies of
every field. One is submitted automatically by the browser, according
to the HTML spec; the other is created by JavaScript code in Jenkins
to implement

https://wiki.jenkins-ci.org/display/JENKINS/Structured+Form+Submission

and has the same data but wrapped in JSON and organized according to
logical structure of configuration items. Any given submission handler
pays attention to one of these and ignores the other. Generally
speaking, any /configSubmit or the like which accepts forms created
using tags in the /lib/form namespace uses the JSON data, which is
parsed via methods defined in Stapler. Some miscellaneous form
handlers, such as the one handling parameterized build submissions,
will read parameter definitions straight from the HTTPServletRequest
in the normal way; these could have been URL-encoded POSTed
submissions, or query parameters.

[Salim Fadhley]

Thanks again!

I am still struggling to get parameterized builds that include files and non-file parameters working. Are files handled in a different kind of way to regular parameters?

[Jesse Glick]

On Mon, Aug 4, 2014 at 6:51 AM, Salim Fadhley <salimf...@gmail.com> wrote:
> Are files handled in a different kind of way to regular parameters?

Yes. Best to read the source code for FileParameterValue.