[JIRA] (JENKINS-55315) Obtain target changelist from PerforceScm object in pipeline

6 views
Skip to first unread message

alisdair.robertson@seeingmachines.com (JIRA)

unread,
Dec 22, 2018, 10:27:02 PM12/22/18
to jenkinsc...@googlegroups.com
Alisdair Robertson created an issue
 
Jenkins / New Feature JENKINS-55315
Obtain target changelist from PerforceScm object in pipeline
Issue Type: New Feature New Feature
Assignee: Unassigned
Components: p4-plugin
Created: 2018-12-23 03:26
Priority: Major Major
Reporter: Alisdair Robertson

We have a multibranch pipeline which after being triggered immediately splits into 6 stages for various linux and windows platforms. The sequence of each of the pipelines goes like this:

  • Obtain branch source (checkout scm)
  • Obtain platform specific thirdparties (p4sync)
  • Obtain common thirdparties (p4sync)
  • Build/Test/Archive/Etc

The problem I am having is pinning the thirdparty syncs to the same changelist as the branch, because each instance of a sync can set the P4_CHANGELIST environment variable (which seems to be updated globally) to a different value, so unless I perform a single sync ahead of all the others just to get the changelist and store it, I have a race condition.

What I would like to do is be able to put this at the start of my script, before I enter the pipeline block:

env.GLOBAL_P4_CHANGELIST = scm.getChangelist()

Which would ensure I have a global reference to the changelist that triggered the build to use for pinning the subsequent thirdparty syncs,  that isn't overridden by any subsequent uses of p4sync.

I see that PerforceScm has a property P4Ref revision. Can this be exposed in the manner I'd like?

 

My current workaround consists of either wrapping an initial checkout in a lock step so that I can set the global variable after a first sync has completed (this delays all other parallel stages by the time it takes to sync, which can be up to 10 minutes currently)

lock("$env.JOB_NAME-GLOBAL_P4_CHANGELIST"){
    if ( !env.GLOBAL_P4_CHANGELIST ){
        checkout source_control_management
        env.GLOBAL_P4_CHANGELIST = "$env.P4_CHANGELIST"
        env.GLOBAL_P4_CHANGELIST_STAGE = "$env.STAGE_NAME"
    }
    echo "GLOBAL_P4_CHANGELIST = $env.GLOBAL_P4_CHANGELIST"
    echo "GLOBAL_P4_CHANGELIST_STAGE = $env.GLOBAL_P4_CHANGELIST_STAGE"
}
if ( ! (env.STAGE_NAME == $env.GLOBAL_P4_CHANGELIST_STAGE) ){
    checkout source_control_management
}

or by trying to use a lock on updating the variable immediately after a checkout scm, which still has a race condition.

Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

alisdair.robertson@seeingmachines.com (JIRA)

unread,
Dec 23, 2018, 5:31:02 PM12/23/18
to jenkinsc...@googlegroups.com
Alisdair Robertson updated an issue
Change By: Alisdair Robertson
We have a multibranch pipeline which after being triggered immediately splits into 6 stages for various linux and windows platforms. The sequence of each of the pipelines goes like this:
- Obtain branch source ({{checkout scm}})
- Obtain platform specific thirdparties ({{p4sync}})
- Obtain common thirdparties ({{p4sync}})
- Build/Test/Archive/Etc


The problem I am having is pinning the thirdparty syncs to the same changelist as the branch, because each instance of a sync can set the {{P4_CHANGELIST}} environment variable (which seems to be updated globally) to a different value, so unless I perform a single sync ahead of all the others just to get the changelist and store it, I have a race condition.

What I would like to do is be able to put this at the start of my script, before I enter the pipeline block:
{code :java }
env.GLOBAL_P4_CHANGELIST = scm.getChangelist()
{code}
Which would ensure I have a global reference to the changelist that triggered the build to use for pinning the subsequent thirdparty syncs,  that isn't overridden by any subsequent uses of {{p4sync}}.

I see that {{PerforceScm}} has a property {{P4Ref revision}}. Can this be exposed in the manner I'd like?


 

My current workaround consists of either wrapping an initial checkout in a {{lock}} step so that I can set the global variable after a first sync has completed (this delays all other parallel stages by the time it takes to sync, which can be up to 10 minutes currently)
{code :groovy }

lock("$env.JOB_NAME-GLOBAL_P4_CHANGELIST"){
    if ( !env.GLOBAL_P4_CHANGELIST ){
        checkout source_control_management scm
        env.GLOBAL_P4_CHANGELIST = "$env.P4_CHANGELIST"
        env.GLOBAL_P4_CHANGELIST_STAGE = "$env.STAGE_NAME"
    }
    echo "GLOBAL_P4_CHANGELIST = $env.GLOBAL_P4_CHANGELIST"
    echo "GLOBAL_P4_CHANGELIST_STAGE = $env.GLOBAL_P4_CHANGELIST_STAGE"
}
if ( ! (env.STAGE_NAME == $env.GLOBAL_P4_CHANGELIST_STAGE) ){
    checkout source_control_management scm
}
{code}
or by trying to use a lock on updating the variable immediately after a {{checkout scm}}, which still has a race condition.

kwirth@perforce.com (JIRA)

unread,
Jan 2, 2019, 11:58:02 AM1/2/19
to jenkinsc...@googlegroups.com

kwirth@perforce.com (JIRA)

unread,
Jan 2, 2019, 12:05:01 PM1/2/19
to jenkinsc...@googlegroups.com
Karl Wirth commented on New Feature JENKINS-55315
 
Re: Obtain target changelist from PerforceScm object in pipeline

Hi Alisdair Robertson,

Thanks for the suggestion. I can see that this would be useful so I will discuss with the developers.

kwirth@perforce.com (JIRA)

unread,
Jan 2, 2019, 12:07:02 PM1/2/19
to jenkinsc...@googlegroups.com

kwirth@perforce.com (JIRA)

unread,
Feb 6, 2019, 11:00:02 AM2/6/19
to jenkinsc...@googlegroups.com
 
Re: Obtain target changelist from PerforceScm object in pipeline

There are a couple of ways to approach this. A temporary workaround for now is to cache the highest changelist across the depot (or part) instead of the sync mentioned above.

For example:

  def ws = [$class: 'StreamWorkspaceImpl', 
        charset: 'none', format: env.JOB_NAME + '-get-CL', pinHost: false, 
        streamName: '//streams/main']

    // Create P4 object
    def p4 = p4(credential: 'JenkinsMaster', workspace: ws)

    def Result = p4.run('changes','-m1','//...')
    GLOBAL_CHANGELIST = Result[0].change;

However we also see merrit in the plugin doing this so I'm passing it onto the developers.

kwirth@perforce.com (JIRA)

unread,
Feb 6, 2019, 11:00:02 AM2/6/19
to jenkinsc...@googlegroups.com
Karl Wirth assigned an issue to Unassigned
 
Change By: Karl Wirth
Assignee: Karl Wirth

kwirth@perforce.com (JIRA)

unread,
Feb 6, 2019, 11:00:03 AM2/6/19
to jenkinsc...@googlegroups.com

pallen@perforce.com (JIRA)

unread,
Jun 4, 2019, 10:38:01 AM6/4/19
to jenkinsc...@googlegroups.com

pallen@perforce.com (JIRA)

unread,
Jun 4, 2019, 10:39:02 AM6/4/19
to jenkinsc...@googlegroups.com

pallen@perforce.com (JIRA)

unread,
Jun 4, 2019, 10:39:02 AM6/4/19
to jenkinsc...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages