[JIRA] (JENKINS-60337) Can't use image gallery with managed identity

timjacomb1@gmail.com (JIRA)

unread,

Dec 1, 2019, 6:17:03 PM12/1/19

to jenkinsc...@googlegroups.com

Tim Jacomb created an issue

Jenkins /

JENKINS-60337

Can't use image gallery with managed identity

Issue Type:	Bug
Assignee:	Jie Shen
Components:	azure-vm-agents-plugin
Created:	2019-12-01 23:16
Environment:	Jenkins ver. 2.190.3 azure-vm-agents-1.3.0 https://github.com/hmcts/cnp-jenkins-docker/blob/master/plugins.txt https://github.com/hmcts/cnp-jenkins-docker/blob/master/Dockerfile https://github.com/hmcts/cnp-flux-config/blob/master/k8s/admin/jenkins/patches/mgmt-sandbox/cluster-00/jenkins.yaml#L101-L180 https://github.com/hmcts/cnp-flux-config/blob/master/k8s/admin/jenkins/jenkins.yaml#L344-L348
Priority:	Minor
Reporter:	Tim Jacomb

Hi

I'm trying to use a "User assigned managed identity" for all authentication with Jenkins.

This plugin appears to support it, I even borrowed code from it for the azure-keyvault-plugin: https://github.com/jenkinsci/azure-keyvault-plugin/pull/27

But when I try use my image gallery (in a different subscription if that matters but it has contributor on that subscription), I get a not found error:

"The target gallery image does not exist"

I added debug code to the "AzureVMManagementServiceDelegate"

It's failing with this exception:

 
                                                                2019-12-01 22:56:18.358+0000 [id=130]	INFO	c.m.a.v.AzureVMManagementServiceDelegate#verifyVirtualMachineImage: Exception when looking up gallery
rx.exceptions.OnErrorThrowable$OnNextValue: OnError while emitting onNext value: null
	at rx.exceptions.OnErrorThrowable.addValueAsLastCause(OnErrorThrowable.java:118)
	at rx.internal.operators.OnSubscribeMap$MapSubscriber.onNext(OnSubscribeMap.java:73)
Caused: java.lang.NullPointerException
	at com.microsoft.azure.management.compute.implementation.GalleryImageVersionImpl.<init>(GalleryImageVersionImpl.java:50)
	at com.microsoft.azure.management.compute.implementation.GalleryImageVersionsImpl.wrapModel(GalleryImageVersionsImpl.java:42)
	at com.microsoft.azure.management.compute.implementation.GalleryImageVersionsImpl.access$000(GalleryImageVersionsImpl.java:24)
	at com.microsoft.azure.management.compute.implementation.GalleryImageVersionsImpl$4.call(GalleryImageVersionsImpl.java:84)
	at com.microsoft.azure.management.compute.implementation.GalleryImageVersionsImpl$4.call(GalleryImageVersionsImpl.java:81)
	at rx.internal.operators.OnSubscribeMap$MapSubscriber.onNext(OnSubscribeMap.java:69)
	at rx.internal.operators.OnSubscribeMap$MapSubscriber.onNext(OnSubscribeMap.java:77)
	at rx.internal.operators.OperatorMerge$MergeSubscriber.emitScalar(OperatorMerge.java:511)
	at rx.internal.operators.OperatorMerge$MergeSubscriber.tryEmit(OperatorMerge.java:466)
	at rx.internal.operators.OperatorMerge$MergeSubscriber.onNext(OperatorMerge.java:244)
	at rx.internal.operators.OperatorMerge$MergeSubscriber.onNext(OperatorMerge.java:148)
	at rx.internal.operators.OnSubscribeMap$MapSubscriber.onNext(OnSubscribeMap.java:77)
	at retrofit2.adapter.rxjava.CallArbiter.deliverResponse(CallArbiter.java:120)
	at retrofit2.adapter.rxjava.CallArbiter.emitResponse(CallArbiter.java:102)
	at retrofit2.adapter.rxjava.CallExecuteOnSubscribe.call(CallExecuteOnSubscribe.java:46)
	at retrofit2.adapter.rxjava.CallExecuteOnSubscribe.call(CallExecuteOnSubscribe.java:24)
	at rx.Observable.unsafeSubscribe(Observable.java:10327)
	at rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:48)
	at rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:33)
	at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48)
	at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30)
	at rx.Observable.unsafeSubscribe(Observable.java:10327)
	at rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:48)
	at rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:33)
	at rx.Observable.unsafeSubscribe(Observable.java:10327)
	at rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:48)
	at rx.internal.operators.OnSubscribeMap.call(OnSubscribeMap.java:33)
	at rx.Observable.unsafeSubscribe(Observable.java:10327)
	at rx.internal.operators.DeferredScalarSubscriber.subscribeTo(DeferredScalarSubscriber.java:153)
	at rx.internal.operators.OnSubscribeTakeLastOne.call(OnSubscribeTakeLastOne.java:32)
	at rx.internal.operators.OnSubscribeTakeLastOne.call(OnSubscribeTakeLastOne.java:22)
	at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:48)
	at rx.internal.operators.OnSubscribeLift.call(OnSubscribeLift.java:30)
	at rx.Observable.subscribe(Observable.java:10423)
	at rx.Observable.subscribe(Observable.java:10390)
	at rx.observables.BlockingObservable.blockForSingle(BlockingObservable.java:443)
	at rx.observables.BlockingObservable.last(BlockingObservable.java:226)
	at com.microsoft.azure.management.compute.implementation.GalleryImageVersionsImpl.getByGalleryImage(GalleryImageVersionsImpl.java:91)
	at com.microsoft.azure.vmagent.AzureVMManagementServiceDelegate.verifyVirtualMachineImage(AzureVMManagementServiceDelegate.java:2294)
	at com.microsoft.azure.vmagent.AzureVMManagementServiceDelegate$4.call(AzureVMManagementServiceDelegate.java:2101)
	at com.microsoft.azure.vmagent.AzureVMManagementServiceDelegate$4.call(AzureVMManagementServiceDelegate.java:2097)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
 
                                                            

In the vm agents plugin it is this line that triggers the exception:
https://github.com/jenkinsci/azure-vm-agents-plugin/blob/8bb8638abbf257824afdf13a9d7b5d3d15bf7347/src/main/java/com/microsoft/azure/vmagent/AzureVMManagementServiceDelegate.java#L2293

I've manually ran the API calls that the java sdk is using and it works fine:

 
                                                                az login --identity
TOKEN=$(az account get-access-token -o tsv --query accessToken)
curl -H "Authorization: Bearer ${TOKEN}" "https://management.azure.com/subscriptions/<subscription-id>/resourceGroups/<rg>/providers/Microsoft.Compute/galleries/cnpimagegallery/images/jenkins-agent/versions/1.2.1?api-version=2018-06-01"

Add Comment

This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)

jieshe@microsoft.com (JIRA)

unread,

Dec 2, 2019, 2:28:02 AM12/2/19

to jenkinsc...@googlegroups.com

Jie Shen commented on

JENKINS-60337

Re: Can't use image gallery with managed identity

Thanks for reporting this issue. I will check it this week.

Add Comment

jieshe@microsoft.com (JIRA)

unread,

Dec 5, 2019, 9:49:02 PM12/5/19

to jenkinsc...@googlegroups.com

Jie Shen commented on

JENKINS-60337

Re: Can't use image gallery with managed identity

 
                                                                      - agentLaunchMethod: "SSH"
        builtInImage: "Windows Server 2016"
        credentialsId: "agent_admin_account"
        diskType: "managed"
        doNotUseMachineIfInitFails: true
        enableMSI: false
        enableUAMI: true
        ephemeralOSDisk: false
        executeInitScriptAsRoot: true
        existingStorageAccountName: "***"
        imageReference:
          galleryImageDefinition: "imageDefinition"
          galleryImageVersion: "0.0.1"
          galleryName: "imageGallery"
          galleryResourceGroup: "myGalleryRG"
          gallerySubscriptionId: "***"
        imageTopLevelType: "advanced"
        initScript: |-
          sudo add-apt-repository ppa:openjdk-r/ppa -y
          sudo apt-get -y update
          sudo apt-get install openjdk-8-jre openjdk-8-jre-headless openjdk-8-jdk -y
        installDocker: false
        installGit: false
        installMaven: false
        labels: "gallery"
        location: "East US"
        noOfParallelJobs: 1
        osDiskSize: 0
        osType: "Linux"
        preInstallSsh: true
        retentionStrategy:
          azureVMCloudRetentionStrategy:
            idleTerminationMinutes: 60
        shutdownOnIdle: false
        storageAccountNameReferenceType: "existing"
        storageAccountType: "Standard_LRS"
        templateDisabled: false
        templateName: "gallery"
        uamiID: "/subscriptions/***/resourceGroups/myGalleryRG/providers/Microsoft.ManagedIdentity/userAssignedIdentities/js"
        usageMode: "Use this node as much as possible"
        usePrivateIP: false
        virtualMachineSize: "Standard_F2"
 
                                                            

Above is my configuration, but I can successfully provision the agents. Do I miss anything?

Add Comment

timjacomb1@gmail.com (JIRA)

unread,

Dec 6, 2019, 2:19:02 AM12/6/19

to jenkinsc...@googlegroups.com

Tim Jacomb commented on

JENKINS-60337

Re: Can't use image gallery with managed identity

You didn’t include the credential config for agent_admin_account and is the gallery in the same subscription as your managed identity or a different one?

My Jenkins and managed identity are in one subscription and the gallery is in a different one

Add Comment

Reply all

Reply to author

Forward