git checkout fail for patchset

[Arvind Kumar]

Hi team,

I have strange issues where 

We have jenkins gerrit trigger plugin configured and its was working fine except for one branch from where when a patchset is raised, jenkins gerrit event is triggered but it fails with below error

00:00:00.004 [EnvInject] - Inject global passwords.
00:00:00.005 [EnvInject] - Mask passwords passed as build parameters.
00:00:00.005 Retriggered by user user_name for Gerrit: http://gerrit_server:8080/4296
00:00:00.104 [EnvInject] - Loading node environment variables.
00:00:00.121 Building remotely on slave-shared-jenk-02-pao in workspace /apps/paoworkspace/workspace/pao-savings-cwa-patchset
00:00:00.122 [WS-CLEANUP] Deleting project workspace...
00:00:00.130 [WS-CLEANUP] Done
00:00:00.206 Cloning the remote Git repository
00:00:00.209 Cloning repository ssh://jenkins@gerrit_server:29418/pao-savings-cwa
00:00:00.210  > git init /apps/paoworkspace/workspace/pao-savings-cwa-patchset # timeout=10
00:00:00.227 ERROR: Error cloning remote repo 'origin'
00:00:00.227 ERROR: Error cloning remote repo 'origin'

Jenkins : Choosing strategy

Gerrit

Attached are the jenkins debiug logs.

Versions:

-----------------------------

-----------------------------

gerrit version : 2.10.7

Jenkins version: 1.5.9.6 

Jenkins plugin versions:

--------------------------------

Git client plugin: 1.16.1

Git plugin: 2.3.5

Git server plugin: 1.6

GitHub API Plugin: 1.69

GitHub plugin: 1.14.0

Git client version on jenkins server

git version: 1.7.1

[Arvind Kumar]

Retrying my luck again to check if anybody has suggestion/solution to it?

[Alex Blewitt]

What investigation have you done so far?

--
--
To unsubscribe, email repo-discuss...@googlegroups.com
More info at http://groups.google.com/group/repo-discuss?hl=en

---
You received this message because you are subscribed to the Google Groups "Repo and Gerrit Discussion" group.
To unsubscribe from this group and stop receiving emails from it, send an email to repo-discuss...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Arvind Kumar]

First of all, Thanks for the revert.

> We noticed a jenkins plugin changes when the issue was occurred, not sure, if the issue is related to the change. We did install thinbackup jenkins plugin. Will give it a try uninstalling the same to see if it resumes to normal operations.

> Our suspect was gitscm plugin, but it was not changed since a long and with logs, there was nothing useful information available except NullPointerException.  It is able to checkout for most of the patchsets but failing for specific ones. Its a blind guess that it might fail for certain number of files.

> Manual checkout for the gerrit refspec works perfectly fine on the server.

[Alex Blewitt]

Have you looked as to where the NullPointerException is coming from, and have you been able to replicate that by running from the same machine/user with the same version of Git (JGit/CGit)?

Alex

[Arvind Kumar]

Hi Alex,

We have picked it further and tried to run via jenkins shell builder. Surprisingly, It did not reach till shell and failed with error=7, Argument list too long. That lead us to conclude, there is something via gerrit patch only, which is causing the issue. 

Further checking the parameters passed by gerrit was observed and was seen that commit message was 10k+ length. We did reduce and got it worked.

Are you having any clue, how much length is permissible for commit message?

[Alex Blewitt]

10k for a commit message? What are you putting in there?

You’ll have to do some more debugging; the error message sounds like the data is being passed on the command line (or it’s being set via an environment variable on the execution of a command such as a message hook). Neither of these problems/issues are specifically to do with Git or Gerrit; you’re probably limited in terms of the maximum object size that Git has - but you’ll quickly run into other problems if you start creating gigabyte messages.

In any case, you’ll have to dig through the hook(s) and the checkout operations to determine what is happening. Since I know the Git checkout strategy will set things like GERRIT_BRANCH and GERRIT_REFSPEC I’d start looking where they are being set prior to calling an external executable.

I’d suspect it’s coming through here, which looks like it’s populating the GERRIT_CHANGE_COMMIT_MESSAGE environment variable with the the contents of the commit message:

https://github.com/jenkinsci/gerrit-trigger-plugin/blob/64d7d8aa16c193e9f1a051d9996e55221d7edaf1/src/main/java/com/sonyericsson/hudson/plugins/gerrit/trigger/hudsontrigger/GerritTriggerParameters.java#L387

Alex

[Arvind Kumar]

10k+ was hashed code converted by gerrit.

To avoid multiple patchsets for different commits, we did squash all of them and created in a single patchset. To avoid losing history of the branch, we keep on adding the commit messages to a single message.

I hope, it will be helpful, in case someone  face similar issues.