Incrementals error in Jenkins CI for GitLab Branch Source Plugin

[Parichay Barpanda]

Hi all,

GitLab Branch Source Plugin was recently hosted on `jenkinsci` org. But the build is failing on Jenkins CI. I am unable to figure out the error. Can someone help me figure this out? The build information can be found here - https://ci.jenkins.io/blue/organizations/jenkins/Plugins%2Fgitlab-branch-source-plugin/detail/develop/5/pipeline.

Thanks.

[Mark Waite]

That particular error case (` Make sure `git status -s` is empty before using -Dset.changelis  `) likely means that some portion of the build is modifying one or more files in the source code during the build process.  I believe the incrementals build process wants the output of `git status -s` to be empty.

If you run a build yourself and then execute the command `git status -s`, is the output empty?

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/6e9ef5cb-8cf5-4819-bf9d-4931cb0cc480%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Thanks!

Mark Waite

[Baptiste Mathus]

Also possibly related to the fact https://github.com/jenkins-infra/repository-permissions-updater/pull/1218 was not merged, yet. (Now done).

The github field there is used in Incrementals IIRC.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAO49JtG3r34zJZa8ynBUhe8s8OnEkeOHJ-Uz64ikeuBPoreaUQ%40mail.gmail.com.

[Parichay Barpanda]

Thanks Mark and Batmat. I am not sure if it was due to #1218 not being merged. Because soon as it was merged the pipeline still failed with the same results. 

Based on the logs:

[2019-07-14T04:05:22.417Z] [ERROR] Make sure git status -s is empty before using -Dset.changelist: [docs/img/gitlab-credentials.png, docs/img/gitlab-section.png, docs/img/gitlab-server.png, docs/img/gitlab-token-creator.png] (use -Dignore.dirt to make this nonfatal) -> [Help 1]
[2019-07-14T04:05:22.417Z]

we tried to remove the docs dir, it happened to work(build passes now). Earlier it also gave similar errors with webapp dir, so I had to remove it and add to .gitignore. Maybe Mark is right but again I am not sure.

[Mark Waite]

That seems like it would indicate that something is modifying those files in the build process on the CI server.

Any chance there is an image optimization process running in the build, or something else that would change those files?

--

You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/daca3bd2-c550-4cc0-a28d-9fd1742a4220%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Thanks!

Mark Waite

[Parichay Barpanda]

I don't think in our code we are optimisation the docs images or making changes to the docs images. I was thinking the images were indented in the markdown, so the markdown parser could be doing some changes with images. But that is also not the case. 

Does anyone know a plugin with incrementals support and images in the repo?

On Sunday, July 14, 2019 at 8:31:07 PM UTC+5:30, Mark Waite wrote:

That seems like it would indicate that something is modifying those files in the build process on the CI server.

Any chance there is an image optimization process running in the build, or something else that would change those files?

On Sun, Jul 14, 2019 at 8:51 AM Parichay Barpanda <parichay...@gmail.com> wrote:

Thanks Mark and Batmat. I am not sure if it was due to #1218 not being merged. Because soon as it was merged the pipeline still failed with the same results. 

Based on the logs:

[2019-07-14T04:05:22.417Z] [ERROR] Make sure git status -s is empty before using -Dset.changelist: [docs/img/gitlab-credentials.png, docs/img/gitlab-section.png, docs/img/gitlab-server.png, docs/img/gitlab-token-creator.png] (use -Dignore.dirt to make this nonfatal) -> [Help 1]
[2019-07-14T04:05:22.417Z]

we tried to remove the docs dir, it happened to work(build passes now). Earlier it also gave similar errors with webapp dir, so I had to remove it and add to .gitignore. Maybe Mark is right but again I am not sure.

On Sunday, July 14, 2019 at 3:55:32 PM UTC+5:30, Parichay Barpanda wrote:

Hi all,

GitLab Branch Source Plugin was recently hosted on `jenkinsci` org. But the build is failing on Jenkins CI. I am unable to figure out the error. Can someone help me figure this out? The build information can be found here - https://ci.jenkins.io/blue/organizations/jenkins/Plugins%2Fgitlab-branch-source-plugin/detail/develop/5/pipeline.

Thanks.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkin...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/daca3bd2-c550-4cc0-a28d-9fd1742a4220%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Thanks!

Mark Waite

[Parichay Barpanda]

We can also avoid the error by adding `-Dignore.dirt` flag to Maven config as was suggested by Marky Jackson.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/95655c15-e3db-4dfd-b079-e35827cc1aa4%40googlegroups.com.

[Marky Jackson]

That or figure out if the git status -s is in fact empty

[Abhyudaya Sharma]

Hi Parichay!

There's the Role Strategy Plugin that has incrementals enabled and has images in the repository too.

[Parichay Barpanda]

Thank Abyudaya. I think maybe the error is caused to some misconfiguration of incrementals. I am also getting an error message in the deploy stage [2019-07-14T15:58:59.299Z] Invalid archive retrieved from Jenkins, perhaps the plugin is not properly incrementalized? Still I haven't been able figure it out.

[Jesse Glick]

On Sun, Jul 14, 2019 at 11:53 AM Parichay Barpanda
<parichay...@gmail.com> wrote:
> We can also avoid the error by adding `-Dignore.dirt` flag to Maven config as was suggested by Marky Jackson.

No, fix your build process.

https://github.com/jenkinsci/gitlab-branch-source-plugin/compare/master...jenkinsci:develop

is rather suspicious: nothing in `src/` should ever be in `.gitignore`.

[Parichay Barpanda]

Hi Jesse,

I had added the webapp directory to .gitignore to test the build. Now I tried to remove classes that might have affected the images such as GitLabAvatarCache (didn't work), refixed svgs and the pngs (didn't work). The current work can be found in this commit - https://github.com/jenkinsci/gitlab-branch-source-plugin/commit/57c5f1705a78a5a3c6012ebfc0ded91a4c93dcab.

I am unable to understand the issue, if it was bug in the code it would have also returned error about the webapp images but it also returns errors with the docs images. 

[Gavin]

Whats the deploy stage? In the jenkins pipeline? In maven?

Have you tried a clean clone of your develop branch. Running mvn clean install, then checking git status to see what if anything is generated.

Develop seems to be running properly. https://ci.jenkins.io/job/Plugins/job/gitlab-branch-source-plugin/job/develop/ the other branches that still have docs/ in them seem to be failing.

Why would you want the webapp directory ignored? As Jessie said, everything in your src directory should be your actual files which you want committed.

--

You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/8e7cc1c9-5298-461c-afd4-4210153eac19%40googlegroups.com.

[Parichay Barpanda]

Hi Gavin,

I'm trying to fix it in https://github.com/jenkinsci/gitlab-branch-source-plugin/pull/1. I tried to see git status after `mvn clean install`, it says tree is clean and build still fails.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAG%3D_DutSqsVzdw_xfHQvcEpkPV_mhOiCzXRptZtasFoR8w%3DDKw%40mail.gmail.com.

[Gavin]

Oh shot in the dark here.

https://github.com/jenkinsci/gitlab-branch-source-plugin/blob/develop/.gitattributes is set to change the endings of all files. That could potentially mess with binary files.

I'm not certain about this but you might need to say your images are binary. Especially if your committing in windows and the builds that are failing are in Linux.

https://help.github.com/en/articles/configuring-git-to-handle-line-endings

[Parichay Barpanda]

Awesome Gavin. It was .gitattributes that caused the binary files to change. Thanks. 

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAG%3D_Dut1bYFsY6j4urdBU1XxFJxJkzkL-rO_YN6w0Eb%2BKiZLFQ%40mail.gmail.com.

[Parichay Barpanda]

By specifying the binary files in .gitattributes the build passed but the plugin is not deployed incrementally. The deploy stage fails with a message:

Invalid archive retrieved from Jenkins, perhaps the plugin is not properly incrementalized?

The build result for the latest commit can be found here - https://ci.jenkins.io/blue/organizations/jenkins/Plugins%2Fgitlab-branch-source-plugin/detail/develop/9/pipeline

Any idea why it fails? Btw the plugin has github field in `repository-permissions-updater`.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/8e7cc1c9-5298-461c-afd4-4210153eac19%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-dev+unsubscribe@googlegroups.com.

[Jesse Glick]

On Tue, Jul 16, 2019 at 2:47 PM Parichay Barpanda
<parichay...@gmail.com> wrote:
> Invalid archive retrieved from Jenkins, perhaps the plugin is not properly incrementalized?

Well Tyler added this particular error message, perhaps related to INFRA-2125:

https://github.com/jenkins-infra/community-functions/commit/d4bf3d3c68961b6a778325397336bdb7d99fe4a4

Since I lack access to the server logs (INFRA-2094), and the `err`
mentioned there was apparently not included in the HTTP response, I
cannot help further. You can file an `INFRA` ticket to request
assistance.

[Parichay Barpanda]

I raised a ticket here - https://issues.jenkins-ci.org/browse/INFRA-2183. Please let me know if I can assign it to someone or you can ping someone who can help fix it. Thanks in advance.

[Gavin]

Arg. Can't navigate jira well on the phone. I'll comment there tomorrow if I remember.

Unless some weird transpiler, you can't interpolate variables inside double quotes in JavaScript (you could in coffee script). You need backticks or concatenation.

I'll try to submit a PR or something but wanted to write it down incase I forgot

--

You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/7bdd5fad-1804-4234-975a-58f7d891a878%40googlegroups.com.

[Parichay Barpanda]

Hi Gavin,

 

I don't exactly understand what you are saying. It would be great if you send a pr or point to the code where the problem is when you find time. Thanks in advance.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAG%3D_Duvjp8zsX3gVT-JUky0c15vnUJOeQyd%3Db%2BgVgc-aEUfp-Q%40mail.gmail.com.