Jenkins support for uploading multiple files

[Ioannis Moutsatsos]

Greetings to all;

By having a 'file type' parameter in the build submission form you can choose a single file to be uploaded to Jenkins so that it can participate in the build.

I have a requirement to upload multiple (large) image files coming from an automated microscope, so that they can be processed on the Jenkins server (as part of a build). I researched but can't find a Jenkins supported way or plugin that allows the upload of multiple files.

Any suggestions on how to accomplish this would be greatly appreciated. 

Thank you!

Ioannis

[Daniel Beck]

Multiple file parameters should work, as should archiving them and uploading the zip/tar/... file.

> --
> You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/b2a8c7b2-85f6-4aa8-9de1-e5c5794ae5fd%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Ioannis Moutsatsos]

Thanks Daniel;

I'm assuming that by multiple file parameters you mean adding more than one file parameters on the form. The issue is that the number of image files is not always the same, and this number could be in the hundreds!

The zip/tar archive upload makes a lot of sense but it could take a long time on both ends to zip and then unzip. My users would be responsible for archiving on laptops that may not be up to the task, but at the end it may be the idea I will go with.

Best regards

Ioannis

On Sat Feb 21 2015 at 7:03:51 AM Daniel Beck <m...@beckweb.net> wrote:

Multiple file parameters should work, as should archiving them and uploading the zip/tar/... file.

On 21.02.2015, at 12:54, Ioannis Moutsatsos <imout...@gmail.com> wrote:

> Greetings to all;
>
> By having a 'file type' parameter in the build submission form you can choose a single file to be uploaded to Jenkins so that it can participate in the build.
>
> I have a requirement to upload multiple (large) image files coming from an automated microscope, so that they can be processed on the Jenkins server (as part of a build). I researched but can't find a Jenkins supported way or plugin that allows the upload of multiple files.
>
> Any suggestions on how to accomplish this would be greatly appreciated.
>
> Thank you!
> Ioannis
>
> --
> You received this message because you are subscribed to the Google Groups "Jenkins Users" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-users+unsubscribe@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/b2a8c7b2-85f6-4aa8-9de1-e5c5794ae5fd%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-users/rIgxr56PVLA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/74C610B4-80BB-449D-868A-D42020E55198%40beckweb.net.

[Les Mikesell]

On Sat, Feb 21, 2015 at 8:50 AM, Ioannis Moutsatsos
<imout...@gmail.com> wrote:
> Thanks Daniel;
>
> I'm assuming that by multiple file parameters you mean adding more than one
> file parameters on the form. The issue is that the number of image files is
> not always the same, and this number could be in the hundreds!
>
> The zip/tar archive upload makes a lot of sense but it could take a long
> time on both ends to zip and then unzip. My users would be responsible for
> archiving on laptops that may not be up to the task, but at the end it may
> be the idea I will go with.

Depending on what other use you have for these files and whether any
are re-used, you might consider committing to a subversion repository
or similar version control system and letting jenkins check them out
with the job more or less as you would with source code. Or use some
common shared file space.

--
Les Mikesell
lesmi...@gmail.com

[Ioannis Moutsatsos]

@les_mikelsell Thanks for the feedback.

Very good point you are making about considering reuse patterns. 

In our corporate environment (where the images are actually reused) this is already working well using a network accessible file-share. 

Our automated microscopes write to this shared space that is then accessible to Jenkins via NFS.

I'm now trying to create an open source platform for scientists outside our organization to use this functionality. Jenkins will probably be running on the cloud somewhere with limited storage, so I thought users would be uploading the images to be processed just temporarily. It got to be a relatively simple approach as most of these users are not experienced dev-ops folks.

I'm also wondering whether Jenkins could authenticate to specific S3 or other cloud storage on a per build basis. People could maintain their images on these cloud storage platforms and provide authorization via Jenkins to access it for the duration of the build?

Thanks again

Ioannis