[JIRA] (JENKINS-60213) P4CLIENT environment variable non-reentrant

[kwirth@perforce.com (JIRA)]

Karl Wirth updated an issue

Jenkins / JENKINS-60213 P4CLIENT environment variable non-reentrant Change By: Karl Wirth Summary: P4 P4CLIENT environment variables variable non-reentrant Add Comment

This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)

[msmeeth@perforce.com (JIRA)]

Matthew Smeeth assigned an issue to Matthew Smeeth

Jenkins / JENKINS-60213 P4CLIENT environment variable non-reentrant

Change By: Matthew Smeeth Assignee: Matthew Smeeth

Add Comment

[msmeeth@perforce.com (JIRA)]

Matthew Smeeth commented on JENKINS-60213

Re: P4CLIENT environment variable non-reentrant Hi, I've written an automated test to reproduce this issue, and looking into it further.  It looks to me like the P4_CLIENT variable is actually correct, and it's actually the EXECUTOR NUMBER that is wrong in your script.   I can demonstrate this by putting an extra bit of debugging into the p4 plugin when running jobs. I have modified my build so that it outputs the executor number as the job is executing and this is what I get:   Obtained concurrentBuildsTest/jenkinsfile from p4-brunoCred-//depot/... //jenkins-${NODE_NAME}${JOB_NAME}${EXECUTOR_NUMBER}/... Running in Durability level: MAX_SURVIVABILITY[Pipeline] Start of Pipeline[Pipeline] nodeRunning on Jenkins in /var/jenkins_home/workspace/concurrentBuildsTestProject[Pipeline] {[Pipeline] stage[Pipeline] { (testStage)[Pipeline] script[Pipeline] {[Pipeline] dirRunning in /var/jenkins_home/workspace/concurrentBuildsTestProject/test[Pipeline] {[Pipeline] checkoutExecutor no at runtime:0 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< Extra debugging I've added to the p4 plugin ...... ...... ...... + concurrentBuildsTest/script.sh ================================= WORKSPACE: /var/jenkins_home/workspace/concurrentBuildsTestProject <<<<<<<<<<<<<<<<<<<<<<<<<<<<< CORRECT P4_CLIENT: jenkins-master-concurrentBuildsTestProject-0 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< CORRECT EXECUTOR NUMBER: 1 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< Wrong executor number ================================= ERROR - Bad Workspace ERROR - Bad P4 Client ...... ...... Finished: FAILURE I have run this several times and every time the job fails, the result is the same, the workspace and client are correct, and the executor number outputted by the script is incorrect. I'm theorising that this is because when builds are running concurrently, in between the sync and the running of the script, the executor number gets overridden by the other jobs, hence it ends up incorrect when the script runs. Therefore I don't think the script we're using is suitable for testing this issue.   It's worth noting I am using the latest changes which contains https://github.com/jenkinsci/p4-plugin/pull/113. Which fixes an issue very similar to this, where the workspace was being set wrong. I believe this was why you were getting a different p4 client and workspace.   James Bateman, when available please can you retest your original scenario using a build with https://github.com/jenkinsci/p4-plugin/pull/113 in. As I suspect this may fix your issue.

Add Comment

[msmeeth@perforce.com (JIRA)]

Matthew Smeeth edited a comment on JENKINS-60213

Re: P4CLIENT environment variable non-reentrant

Hi, I've written an automated test to reproduce this issue, Looking into it further, it looks to me like the P4_CLIENT variable is actually correct, and it's actually the EXECUTOR NUMBER that is wrong in your script.

I can demonstrate this by putting an extra bit of debugging into the p4 plugin when running jobs. I have modified my build so that it outputs the executor number as the job is executing and this is what I get:

{code:java} Obtained concurrentBuildsTest/jenkinsfile from p4-brunoCred-//depot/... //jenkins-${NODE_NAME} - ${JOB_NAME} - ${EXECUTOR_NUMBER}/... Running in Durability level: MAX_SURVIVABILITY[Pipeline] Start of Pipeline[Pipeline] nodeRunning on [ Jenkins |http://localhost:5003/computer/(master)/] in /var/jenkins_home/workspace/concurrentBuildsTestProject[Pipeline] {[Pipeline] stage[Pipeline] { (testStage)[Pipeline] script[Pipeline] {[Pipeline] dirRunning in /var/jenkins_home/workspace/concurrentBuildsTestProject/test[Pipeline] {[Pipeline] checkoutExecutor no at runtime:0 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< Extra debugging I've added to the p4 plugin

...... ...... ...... + concurrentBuildsTest/script.sh ================================= WORKSPACE: /var/jenkins_home/workspace/concurrentBuildsTestProject <<<<<<<<<<<<<<<<<<<<<<<<<<<<< CORRECT P4_CLIENT: jenkins-master-concurrentBuildsTestProject-0 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< CORRECT EXECUTOR NUMBER: 1 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< Wrong executor number =================================

* ** *** ERROR - Bad Workspace *** ERROR - Bad P4 Client ...... ...... Finished: FAILURE {code}

I have run this several times and every time the job fails, the result is the same, the workspace and client are correct, and the executor number outputted by the script is incorrect. I'm theorising that this is because when builds are running concurrently, in between the sync and the running of the script, the executor number gets overridden by the other jobs, hence it ends up incorrect when the script runs. Therefore I don't think the script we're using is suitable for testing this issue.

It's worth noting I am using the latest changes which contains [https://github.com/jenkinsci/p4-plugin/pull/113]. Which fixes an issue very similar to this, where the workspace was being set wrong. I believe this was why you were getting a different p4 client and workspace.   [~jbateman], when available please can you retest your original scenario using a build with [https://github.com/jenkinsci/p4-plugin/pull/113] in. As I suspect this may fix your issue.

Add Comment

[msmeeth@perforce.com (JIRA)]

Matthew Smeeth edited a comment on JENKINS-60213

Re: P4CLIENT environment variable non-reentrant

Hi, I've written an automated test to reproduce this issue, and looking Looking into it further .  It , it looks to me like the P4_CLIENT variable is actually correct, and it's actually the EXECUTOR NUMBER that is wrong in your script.

I can demonstrate this by putting an extra bit of debugging into the p4 plugin when running jobs. I have modified my build so that it outputs the executor number as the job is executing and this is what I get:

Obtained concurrentBuildsTest/jenkinsfile from p4-brunoCred-//depot/... //jenkins-${NODE_NAME}-${JOB_NAME}-${EXECUTOR_NUMBER}/...

Running in Durability level: MAX_SURVIVABILITY[Pipeline] Start of Pipeline[Pipeline] nodeRunning on [Jenkins|http://localhost:5003/computer/(master)/] in /var/jenkins_home/workspace/concurrentBuildsTestProject[Pipeline] {[Pipeline] stage[Pipeline] { (testStage)[Pipeline] script[Pipeline] {[Pipeline] dirRunning in /var/jenkins_home/workspace/concurrentBuildsTestProject/test[Pipeline] {[Pipeline] checkoutExecutor no at runtime:0    <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< Extra debugging I've added to the p4 plugin ...... ...... ...... + concurrentBuildsTest/script.sh ================================= WORKSPACE: /var/jenkins_home/workspace/concurrentBuildsTestProject <<<<<<<<<<<<<<<<<<<<<<<<<<<<< CORRECT P4_CLIENT: jenkins-master-concurrentBuildsTestProject-0 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< CORRECT EXECUTOR NUMBER: 1 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< Wrong executor number =================================

* ** * ** ERROR - Bad Workspace

*** ERROR - Bad P4 Client ...... ...... Finished: FAILURE

I have run this several times and every time the job fails, the result is the same, the workspace and client are correct, and the executor number outputted by the script is incorrect. I'm theorising that this is because when builds are running concurrently, in between the sync and the running of the script, the executor number gets overridden by the other jobs, hence it ends up incorrect when the script runs. Therefore I don't think the script we're using is suitable for testing this issue.   It's worth noting I am using the latest changes which contains [https://github.com/jenkinsci/p4-plugin/pull/113]. Which fixes an issue very similar to this, where the workspace was being set wrong. I believe this was why you were getting a different p4 client and workspace.   [~jbateman], when available please can you retest your original scenario using a build with [https://github.com/jenkinsci/p4-plugin/pull/113] in. As I suspect this may fix your issue.

Add Comment

[msmeeth@perforce.com (JIRA)]

Matthew Smeeth edited a comment on JENKINS-60213

Re: P4CLIENT environment variable non-reentrant

Hi, I've written an automated test to reproduce this issue, Looking into it further, it looks to me like the P4_CLIENT variable is actually correct, and it's actually the EXECUTOR NUMBER that is wrong in your script.

I can demonstrate this by putting an extra bit of debugging into the p4 plugin when running jobs. I have modified my build so that it outputs the executor number as the job is executing and this is what I get:

{code:java} Obtained concurrentBuildsTest/jenkinsfile from p4-brunoCred-//depot/... //jenkins-${NODE_NAME}${JOB_NAME}${EXECUTOR_NUMBER}/...

Running in Durability level: MAX_SURVIVABILITY [Pipeline] Start of Pipeline [Pipeline] node

Running on Jenkins in /var/jenkins_home/workspace/concurrentBuildsTestProject

[Pipeline] { [Pipeline] stage [Pipeline] { (testStage) [Pipeline] script [Pipeline] { [Pipeline] dir Running in /var/jenkins_home/workspace/concurrentBuildsTestProject/test [Pipeline] { [Pipeline] checkout Executor no at runtime:0 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< Extra debugging I've added to the p4 plugin ...... ...... ...... + concurrentBuildsTest/script.sh ================================= WORKSPACE: /var/jenkins_home/workspace/concurrentBuildsTestProject <<<<<<<<<<<<<<<<<<<<<<<<<<<<< CORRECT P4_CLIENT: jenkins-master-concurrentBuildsTestProject-0 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< CORRECT EXECUTOR NUMBER: 1 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< Wrong executor number =================================

ERROR - Bad Workspace

ERROR - Bad P4 Client ...... ...... Finished: FAILURE

{code}

I have run this several times and every time the job fails, the result is the same, the workspace and client are correct, and the executor number outputted by the script is incorrect. I'm theorising that this is because when builds are running concurrently, in between the sync and the running of the script, the executor number gets overridden by the other jobs, hence it ends up incorrect when the script runs. Therefore I don't think the script we're using is suitable for testing this issue.   It's worth noting I am using the latest changes which contains [https://github.com/jenkinsci/p4-plugin/pull/113]. Which fixes an issue very similar to this, where the workspace was being set wrong. I believe this was why you were getting a different p4 client and workspace.   [~jbateman], when available please can you retest your original scenario using a build with [https://github.com/jenkinsci/p4-plugin/pull/113] in. As I suspect this may fix your issue.

Add Comment

[msmeeth@perforce.com (JIRA)]

[msmeeth@perforce.com (JIRA)]

Matthew Smeeth edited a comment on JENKINS-60213

Re: P4CLIENT environment variable non-reentrant

Hi, I've written an automated test to reproduce this issue, Looking into it further, it looks to me like the P4_CLIENT variable is actually correct, and it's actually the EXECUTOR NUMBER that is wrong in your script.   I can demonstrate this by putting an extra bit of debugging into the p4 plugin when running jobs. I have modified my build so that it outputs the executor number as the job is executing and this is what I get:   {code:java} Obtained concurrentBuildsTest/jenkinsfile from p4-brunoCred-//depot/... //jenkins-${NODE_NAME}${JOB_NAME}${EXECUTOR_NUMBER}/... Running in Durability level: MAX_SURVIVABILITY [Pipeline] Start of Pipeline

[Pipeline] nodeRunning node Running on Jenkins in /var/jenkins_home/workspace/concurrentBuildsTestProject

[Pipeline] { [Pipeline] stage [Pipeline] { (testStage) [Pipeline] script [Pipeline] {

[Pipeline] dirRunning dir Running in /var/jenkins_home/workspace/concurrentBuildsTestProject/test [Pipeline] { [Pipeline] checkoutExecutor checkout Executor no at runtime:0 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< Extra debugging I've added to the p4 plugin

...... ...... ...... + concurrentBuildsTest/script.sh ================================= WORKSPACE: /var/jenkins_home/workspace/concurrentBuildsTestProject <<<<<<<<<<<<<<<<<<<<<<<<<<<<< CORRECT P4_CLIENT: jenkins-master-concurrentBuildsTestProject-0 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< CORRECT EXECUTOR NUMBER: 1 <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< Wrong executor number ================================= ERROR - Bad Workspace ERROR - Bad P4 Client ...... ...... Finished: FAILURE {code}   I have run this several times and every time the job fails, the result is the same, the workspace and client are correct, and the executor number outputted by the script is incorrect. I'm theorising that this is because when builds are running concurrently, in between the sync and the running of the script, the executor number gets overridden by the other jobs, hence it ends up incorrect when the script runs. Therefore I don't think the script we're using is suitable for testing this issue.   It's worth noting I am using the latest changes which contains [https://github.com/jenkinsci/p4-plugin/pull/113]. Which fixes an issue very similar to this, where the workspace was being set wrong. I believe this was why you were getting a different p4 client and workspace.   [~jbateman], when available please can you retest your original scenario using a build with [https://github.com/jenkinsci/p4-plugin/pull/113] in. As I suspect this may fix your issue.

Add Comment

[ralph@niederloehner.com (JIRA)]

Ralph Ndl updated an issue

Jenkins / JENKINS-60213 P4CLIENT environment variable non-reentrant

Change By: Ralph Ndl Labels: P4_A P4_CLIENT P4_VERIFY

Add Comment

[msmeeth@perforce.com (JIRA)]

Matthew Smeeth closed an issue as Fixed

Jenkins / JENKINS-60213 P4CLIENT environment variable non-reentrant

Change By: Matthew Smeeth Status: Open Closed Resolution: Fixed

Add Comment

This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38)

[msmeeth@perforce.com (JIRA)]

Matthew Smeeth commented on JENKINS-60213

Re: P4CLIENT environment variable non-reentrant

There seems to be underlying issues with the executor number in Jenkins core.  JENKINS-48882, JENKINS-24679, JENKINS-7357, JENKINS-4756. Our testing confirms that we calculate the correct executor number for the client, however the environment variable EXECUTOR_NUMBER set by Jenkins (hudson.model.Computer) seems always be incorrect with concurrent execution. Closing this issue as we have resolved the client name part of the issue. Please raise this against Jenkins core if you require the EXECUTOR_NUMBER to be fixed.

Add Comment

[tomreed81@gmail.com (JIRA)]

Tom Reed commented on JENKINS-60213

Re: P4CLIENT environment variable non-reentrant

I have been battling a similar issue. My situation is that the P4_CLIENT I build uses the variables "jenkins-${JOB_NAME}-{NODE_NAME}" The NODE_NAME is wrong a few times a night with 6 parallel builds on 6 different nodes. About 40-050 builds are queued a night, and the wrong P4_CLIENT is causing the wrong P$_CHANGELIST to be reported.

Add Comment

[tomreed81@gmail.com (JIRA)]

Tom Reed edited a comment on JENKINS-60213

Re: P4CLIENT environment variable non-reentrant

I have been battling a similar issue. My situation is that the P4_CLIENT I build uses the variables "jenkins-${JOB_NAME}-\{NODE_NAME}"

The NODE_NAME is wrong a few times a night with 6 parallel builds on 6 different nodes. About 40-050 builds are queued a night, and the wrong P4_CLIENT is causing the wrong P$_CHANGELIST P4_CHANGELIST to be reported.

Add Comment