possible new feature development for triggering builds via SNS/SQS

[Jesse Zoldak]

We would like to trigger builds by messages sent to an Amazon SNS topic, which be relayed to a subscribing SQS queue.

The closest plug-in that I have found which does something similar is the github-sqs-plugin. 

As-is however, the plugin has some limitations that would need to be sorted out before it might meet our needs:

* (minor, fixed) Publishing via SNS instead of directly to SQS wasn't working. This turned out to be a bug (JENKINS-19337) for which we just submitted a PR upstream to fix.

* (major, would require development) The plug-in as written expects the format of the json for the payload to be such as would be sent directly from a GitHub webhook. It then pokes any job configured to both use the plugin and also has that particular GitHub repo defined as a Git SCM. This won't meet our needs because we would like to include metadata in the message such as which hash to build. Also I'd prefer that instead of all jobs, you could specify which exact job or jobs to trigger (maybe via regex?), and parameters to pass to those jobs.

* (unknown) I'm not sure exactly but I suspect that this will not trigger workflow jobs. I could be wrong.

I wanted to reach out here before undertaking new feature development.

The direction I would be heading, and am soliciting feedback on the approach, would be to add a global configuration option (checkbox) for the plugin named something like "Trigger builds via metadata in queue message."

* In the (default) case of it being unchecked, the plug-in will operate as it does now.

* If checked, the plugin will parse the json of the message for an object named something like "ci-data" which will in turn contain the regex of job(s) to trigger and the parameter keys and values to pass. Instead of poking the jobs to wake up and decide themselves whether or not to build, it will use ParameterizedJobMixIn's scheduleBuild2 method with a parameters action in which you pass the parameters and their values.

Couple questions:

* Does this seem sane? Any feedback or thoughts on this approach?

* Aaron Walker are you still on this list? Especially interested in your thoughts as the original writer/maintainer.

Thanks.

-- JZ