[JIRA] (JENKINS-52313) Storing binary file in shared library resources folder

[jakub@pawlinski.pl (JIRA)]

Jakub Pawlinski created an issue

Jenkins / JENKINS-52313 Storing binary file in shared library resources folder Issue Type: Improvement Assignee: Unassigned Components: workflow-cps-global-lib-plugin Created: 2018-07-02 09:25 Priority: Minor Reporter: Jakub Pawlinski Trying to store png file in resources (project logo, common for all pipelines using shared library) and deploy it from there using:   writeFile file: "logo-96x96.png", text: libraryResource("logo-96x96.png")   but the file is no longer readable after this operation. Is there additional parameter I could use (libraryResource has no documented parameters) or is there other suitable way of achieving this functionality?     Add Comment

This message was sent by Atlassian JIRA (v7.10.1#710002-sha1:6efc396)

[dnusbaum@cloudbees.com (JIRA)]

Devin Nusbaum commented on JENKINS-52313

Re: Storing binary file in shared library resources folder It looks like libraryResource reads the contents of the file into a String using the default system encoding, and writeFile encodes the String using the system default encoding, so doing this with binary files totally mangles the data. It doesn't look like there are any options that would make those steps work for your use case. The most obvious workaround that I can think of is if the image is available at some static URL that Jenkins has access to you could download and write it to the desired location in a shell step using curl/wget.

Add Comment

[dnusbaum@cloudbees.com (JIRA)]

Devin Nusbaum edited a comment on JENKINS-52313

Re: Storing binary file in shared library resources folder

It looks like {{libraryResource}} reads the contents of the file into a {{String}} [using the default system encoding|https://github.com/jenkinsci/workflow-cps-global-lib-plugin/blob/aaa7ed1e04ce2ef751b2a770e71f0286c509ddc6/src/main/java/org/jenkinsci/plugins/workflow/libs/LibraryAdder.java#L205], and {{writeFile}} encodes the {{String}} given text using the system default encoding if no other encoding is specified , so doing this with binary files totally mangles the data. It doesn't look like there are any options that would make those steps work for your use case.

The most obvious workaround that I can think of is if the image is available at some static URL that Jenkins has access to you could download and write it to the desired location in a shell step using curl/wget.

Add Comment

[dnusbaum@cloudbees.com (JIRA)]

Devin Nusbaum edited a comment on JENKINS-52313

Re: Storing binary file in shared library resources folder

It looks like {{libraryResource}} reads the contents of the file into a {{String}} [using the default system encoding|https://github.com/jenkinsci/workflow-cps-global-lib-plugin/blob/aaa7ed1e04ce2ef751b2a770e71f0286c509ddc6/src/main/java/org/jenkinsci/plugins/workflow/libs/LibraryAdder.java#L205], and {{writeFile}} encodes the given text using the system default encoding if no other encoding is specified, so doing this with binary files totally mangles the data. It doesn't look like there are any options that would make those steps work for your use case. The most obvious workaround best workarounds that I can think of is are if the image is available at some static URL that Jenkins has access to you could download and write it to the desired location in a shell step using curl/wget , or if the image is in the repo itself you could use a shell step to copy it as desired .

Add Comment

[dnusbaum@cloudbees.com (JIRA)]

Devin Nusbaum commented on JENKINS-52313

Re: Storing binary file in shared library resources folder

If we want to add support for this, the best option to me seems like adding an encoding option to libraryResource, and then updating both libraryResource and writeFile to support Base64 as a valid encoding option to be used for binary data.

Add Comment

[jakub@pawlinski.pl (JIRA)]

Jakub Pawlinski commented on JENKINS-52313

Re: Storing binary file in shared library resources folder

Would be great if we could support this, the resource (png) in my case have to be versioned, I could have separate project for them and expose as artifacts, but its quite an overhead.   Another idea, as I see readFile/writeFile are exposing the encoding parameter, so maybe there is a way to set it in a way to maintain consistency? def fileContents = readFile file: "test.txt", encoding: "UTF-8" fileContents = fileContents.replace("hello", "world") echo fileContents writeFile file: "test.txt", text: fileContents, encoding: "UTF-8"

Add Comment

[dnusbaum@cloudbees.com (JIRA)]

Devin Nusbaum commented on JENKINS-52313

Re: Storing binary file in shared library resources folder

Jakub Pawlinski The problem is that UTF-8 is a variable length encoding so any byte starting with a 1 will not round-trip correctly. You might be able to use a fixed-length encoding like Windows-1252 in your most recent snippet to get things working, but libraryResource currently doesn't take an encoding parameter so you would have to change your system default to use that as a workaround. It seems better to explicitly support this use case.

Add Comment

[dnusbaum@cloudbees.com (JIRA)]

Devin Nusbaum assigned an issue to Devin Nusbaum

Jenkins / JENKINS-52313

Storing binary file in shared library resources folder

Change By: Devin Nusbaum Assignee: Devin Nusbaum

Add Comment

[dnusbaum@cloudbees.com (JIRA)]

Devin Nusbaum started work on JENKINS-52313

Change By: Devin Nusbaum Status: Open In Progress

Add Comment

[dnusbaum@cloudbees.com (JIRA)]

Devin Nusbaum updated JENKINS-52313

Jenkins / JENKINS-52313 Storing binary file in shared library resources folder

Change By: Devin Nusbaum Status: In Progress Review

Add Comment

[dnusbaum@cloudbees.com (JIRA)]

Devin Nusbaum updated JENKINS-52313

Support for Base64 encoding has been added to Pipeline: Shared Groovy Libraries 2.10 and Pipeline: Basic Steps 2.8.1 (or 2.10 if you are on Jenkins 2.121).

Jenkins / JENKINS-52313 Storing binary file in shared library resources folder

Change By: Devin Nusbaum Status: In Review Resolved Resolution: Fixed Released As: workflow-cps-global-lib 2.10, workflow-basic-steps 2.8.1, workflow-basic-steps 2.10

Add Comment

[dnusbaum@cloudbees.com (JIRA)]

Devin Nusbaum edited a comment on JENKINS-52313

Re: Storing binary file in shared library resources folder [~quas] The problem is that {{UTF-8}} is a variable length encoding so any byte starting with a 1 whose highest bit is set will not round-trip correctly. You might be able to use a fixed-length encoding like {{Windows-1252}} in your most recent snippet to get things working, but {{libraryResource}} currently doesn't take an encoding parameter so you would have to change your system default to use that as a workaround. It seems better to explicitly support this use case.

Add Comment