Skupper Version upgrade procedure.

[Mangesh Shirke]

Hi Team, 

Currently we have Skupper 1.4 installed in the production environment. Skupper is installed using a YAML based installation method. Now we have a requirement to upgrade Skupper to the latest version. I am not sure about the upgrade path and procedure. Kindly help me to understand the below mentioned concerns. What is the procedure to upgrade Skupper version. 

1. Current Skupper Site Controller version: 1.4.x

2. What is the detailed procedure to update Skupper version. 

3. What is the Skupper upgrade path? 

4. If I upgrade Skupper site controller, will it upgrade other skupper components automatically?

5. What is the procedure to upgrade Skupper to the specific version rather than the latest version. 

Thanks & Regards.......
Mangesh Shirke

[Gordon Sim]

On Wednesday, November 29, 2023 at 7:11:44 AM UTC mangesh...@gmail.com wrote:

1. Current Skupper Site Controller version: 1.4.x

2. What is the detailed procedure to update Skupper version. 

Just update the skupper site controller. Often that will only be an image change, but some releases may alter the example yaml, e.g. to add permissions.

 

3. What is the Skupper upgrade path? 

I'm not sure what you are asking here. Generally you don't need to update separately to each incremental version, i.e. you can go from 1.4.1 to 1.4.3 without first upgrading to 1.4.2.

 

4. If I upgrade Skupper site controller, will it upgrade other skupper components automatically?

Yes

 

5. What is the procedure to upgrade Skupper to the specific version rather than the latest version. 

You can control exactly which version of the site controller you are using and it will update the versions of the other skupper deployed containers to match. 

[vishavjeet somwanshi]

Hell Team,

As you mentioned in [2], [4] & [5] we would just need to update the skupper-site-controller image version skupper-site-controller deployment and it will automatically upgrade other components.

In this case skupper-site-controller container image having below default image version and i assume it will automatically pull these images with respective image tags from quay.io and upgrade the components to the latest build ( main version ).

client version 1.4.0

transport version quay.io/skupper/skupper-router:main (sha256:23ce799fbb34)

controller version quay.io/skupper/service-controller:main (sha256:0e175024f480)

config-sync version quay.io/skupper/config-sync:main (sha256:33189aa085d5)

flow-collector version quay.io/skupper/flow-collector:main (sha256:c461b962710c

There was one more case raised from our end to understand on how to provide the specific versions and install these components with it. so Engineer responded with statement i.e. we can specifcally

set the environment variables in skupper-site-controller deployment yaml configuration so it could pull specific images

Here are the environment variables that can be used to customize it:

QDROUTERD_IMAGE
SKUPPER_SERVICE_CONTROLLER_IMAGE
SKUPPER_CONFIG_SYNC_IMAGE
SKUPPER_FLOW_COLLECTOR_IMAGE
PROMETHEUS_SERVER_IMAGE

we tried this somehow due to some image repository token mismatch it is failing, we are closely working on this testing in our environment.

In [5] you mentioned, "You can control exactly which version of the site controller you are using and it will update the versions of the other skupper deployed containers to match. "

On this point, How we can identify which customized container image versions are supported with skupper-site-controller.

e.g we are having skupper-site-controller 1.4.0 then how we can identify which customized container image tag/version is supported for other below components.

transport version quay.io/skupper/skupper-router:main (sha256:23ce799fbb34)

controller version quay.io/skupper/service-controller:main (sha256:0e175024f480)

config-sync version quay.io/skupper/config-sync:main (sha256:33189aa085d5)

flow-collector version quay.io/skupper/flow-collector:main (sha256:c461b962710c

Suppose we are going to upgrade with some version e.g 1.4.3 then how we can identify which customized images version is supported here.

Note: Above line is just as an example.

Thanks,

Vishavjeet S

[Gordon Sim]

By default, the site controller will pull images for the other
components that match the same release, except for the router which is
released on a separate schedule. So site-controller 1.4.3 will use
service-controller 1.4.3.

You can as you mention set environment variables to override this.
That however would not be the general recommendation and would only be
for cases where either you cannot use the quay.io repo - in which case
you would still be using the same images, just from a different repo -
or where you need a specific patch or fix or enhancement for one or
other component - this would hopefully be a temporary position until a
release was available that included the necessary change.

> --
> You received this message because you are subscribed to the Google Groups "Skupper" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to skupper+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/skupper/2baa3dd5-0703-4d71-af5a-ae415307cedan%40googlegroups.com.

[vishavjeet somwanshi]

Hi Gordon and Team,

As mentioned i.e if site-controller is 1.4.3 then it will pull the same release for other components e.g service-controller will be 1.4.3.

1] If you see the attached image .. the latest build for service-controller was just 20 hours ago.

2] I am not sure but I assume other components' images are tagged as below in site-controller container image for other components.

transport version quay.io/skupper/skupper-router:main (sha256:23ce799fbb34)

controller version quay.io/skupper/service-controller:main (sha256:0e175024f480)

config-sync version quay.io/skupper/config-sync:main (sha256:33189aa085d5)

flow-collector version quay.io/skupper/flow-collector:main (sha256:c461b962710c

3] So according to [1], it will pull the latest image/build, will it be compatible with site-controller 1.4.3 image ?

4] Any procedure to identify these image versions.

Regards,

Vishavjeet S

[Gordon Sim]

sorry, adding back group

On Fri, Dec 1, 2023 at 11:52 AM vishavjeet somwanshi
<xxxxxxxxxxxxxx> wrote:
> As mentioned i.e if site-controller is 1.4.3 then it will pull the same release for other components e.g service-controller will be 1.4.3.
>
> 1] If you see the attached image .. the latest build for service-controller was just 20 hours ago.

That is not the default image for 1.4.3 site controller. I would check
which version of the site controller you are running and whether you
have overridden the images it is using.

> 2] I am not sure but I assume other components' images are tagged as below in site-controller container image for other components.
>
> transport version quay.io/skupper/skupper-router:main (sha256:23ce799fbb34)
> controller version quay.io/skupper/service-controller:main (sha256:0e175024f480)
> config-sync version quay.io/skupper/config-sync:main (sha256:33189aa085d5)
> flow-collector version quay.io/skupper/flow-collector:main (sha256:c461b962710c
>
> 3] So according to [1], it will pull the latest image/build, will it be compatible with site-controller 1.4.3 image ?

I would recommend using the same versions of components (i.e. not
overriding images) unless there is a clear reason to deviate from
that.

> 4] Any procedure to identify these image versions.

The image tag (e.g. main or 1.4.3) is the first thing to use; the sha
can then be compared with what is in quay for that tag.

[vishavjeet somwanshi]

Hi Team,

Suppose currently, I'm running environment where we have deployed skupper using YAML configuration followed by Continuous Deployment method via ArgoCD.

In our environment, skupper-site-controller is on 1.4.2 version and skupper-site configmap has deployed all other dependent components by pulling the images which are compatible to 1.4.2 version as below.

```

client version                 1.4.2

transport version              quay.io/skupper/skupper-router:2.4.2 (sha256:c0dfccf2c269)

controller version             quay.io/skupper/service-controller:1.4.2 (sha256:06f3ef0047c6)

config-sync version            quay.io/skupper/config-sync:1.4.2 (sha256:2a350062450a)

flow-collector version         quay.io/skupper/flow-collector:1.4.2 (sha256:c3b1e43cce4a)

```

Suppose, we need to upgrade the skupper components to 1.5.4 near in future, so what are the necessary steps we need to perform to achieve this upgrade.

is it something like, just update the skupper-site-controller to 1.5.4 and delete other pods `skupper-router, service-controller, config-sync, flow-collector` and it will pull come up with latest compatible images for 1.5.4.

Do we need to recreate the links in that case ?

Thanks,

Vishavjeet S

[Gordon Sim]

On Wed, Jul 3, 2024 at 3:58 PM vishavjeet somwanshi
<vishavjeet...@gmail.com> wrote:
> Suppose, we need to upgrade the skupper components to 1.5.4 near in future, so what are the necessary steps we need to perform to achieve this upgrade.
>
> is it something like, just update the skupper-site-controller to 1.5.4 and delete other pods `skupper-router, service-controller, config-sync, flow-collector` and it will pull come up with latest compatible images for 1.5.4.

You should only need to update the skupper-site-controller deployment.
The other pods should be restarted automatically after that.

> Do we need to recreate the links in that case ?

No, they should re-establish themselves once the pods restart.

[vishavjeet somwanshi]

Thank you for the confirmation.

I've one more question, suppose in one of the environment i've installed the skupper using CLI  "skupper init" and running with 1.4.2 version later i deployed the skupper-site-controller 1.4.2 and adopted the environment in the ArgoCD. 

If i change the skupper-site-controller version to some other version and restarted the pods, will it work and upgrade to perticular version ?

Thanks,

Vishavjeet S

[Gordon Sim]

On Thu, Jul 4, 2024 at 8:18 AM vishavjeet somwanshi
<vishavjeet...@gmail.com> wrote:
>
> Thank you for the confirmation.
>
> I've one more question, suppose in one of the environment i've installed the skupper using CLI "skupper init" and running with 1.4.2 version later i deployed the skupper-site-controller 1.4.2 and adopted the environment in the ArgoCD.
>
> If i change the skupper-site-controller version to some other version and restarted the pods, will it work and upgrade to perticular version ?

No. Sites created using the cli need to be updated with the cli
(skupper update). The site controller only updates the sites it
created.

[vishavjeet somwanshi]

I've deployed skupper-site-controller and skupper configuration using YAML method and we have CD using ArgoCD in place to deploy these configuration, only skupper token creation and linking we do it manually.

Initially I deployed skupper-site-controller version 1.4.2 and when deployed skupper-site configmap it deployed other dependent component which are compatible with skupper version 1.4.2.

client version                 1.4.2

transport version              quay.io/skupper/skupper-router:2.4.2 (sha256:c0dfccf2c269)

controller version             quay.io/skupper/service-controller:1.4.2 (sha256:06f3ef0047c6)

config-sync version            quay.io/skupper/config-sync:1.4.2 (sha256:2a350062450a)

Now, i am trying to upgrade skupper to 1.5.4 version hence i changed the skupper-site-controller image version to 1.5.4 and synced my app using ArgoCD, skupper-site-controller updated to 1.5.4 however other components are still equivalent to 1.4.2.

I restarted skupper-router and skupper-service-controller as well to see whether it gets updated post rolling restart but still showing the same version.

client version                 1.5.4

transport version              quay.io/skupper/skupper-router:2.4.2 (sha256:c0dfccf2c269)

controller version             quay.io/skupper/service-controller:1.4.2 (sha256:06f3ef0047c6)

config-sync version            quay.io/skupper/config-sync:1.4.2 (sha256:2a350062450a)

Anything else that we need to take care of ?

Post upgrading of skupper version e.g from 1.4.2 to 1.5.4 or any later version do we need to work on relinking the skupper ?

For the site on which we have installed skupper using CLI and which are running on 1.4.2, how to upgrade them in place to 1.5.4 , any specific document we have ?

We have a huge skupper configuration where in Site A we have one namespace where we do create a token and in site B there are more than 30 namespaces which we have linked with site A namespace and exposed several number of services, that's why we are more worried about it. Thanks in advance !!

Thanks,

Vishavjeet S

[Fernando Giorgetti]

Now, i am trying to upgrade skupper to 1.5.4 version hence i changed the skupper-site-controller image version to 1.5.4 and synced my app using ArgoCD, skupper-site-controller updated to 1.5.4 however other components are still equivalent to 1.4.2.
I restarted skupper-router and skupper-service-controller as well to see whether it gets updated post rolling restart but still showing the same version.

If you're using the YAML to deploy the skupper-site-controller, an important thing to mention is that

not only the image version has changed, but there are some role changes as well (comparing 1.4.2 to 1.5.4).

See: https://gist.github.com/fgiorgetti/5f816040ebc78846b195f03bc6ac38ca

Anything else that we need to take care of ?
Post upgrading of skupper version e.g from 1.4.2 to 1.5.4 or any later version do we need to work on relinking the skupper ?

Once the site-controller is upgraded, it should update the running sites for all owned namespaces and

existing links should remain operational (this should be true for updates within 1.x).

Here are some snippets from the skupper-site-controller logs once the new version is running that gives you some evidences

that you have the desired version running and that the update process was executed successfully:

$ kubectl -n skupper-site-controller logs deployments/skupper-site-controller
2025/04/09 18:39:07 Skupper site controller watching all namespaces
2025/04/09 18:39:07 Version: 1.5.4
2025/04/09 18:39:07 Deleted old cluster role "skupper-service-controller"
2025/04/09 18:39:07 Cluster role "skupper-service-controller-basic" already exists
2025/04/09 18:39:07 Starting the Skupper site controller informers
2025/04/09 18:39:07 Waiting for informer caches to sync
2025/04/09 18:39:07 Checking if sites need updates (1.5.4)
2025/04/09 18:39:10 Updated version for namespace "my-namespace"
2025/04/09 18:39:10 Starting workers
2025/04/09 18:39:10 Started workers
2025/04/09 18:39:11 Skupper site exists my-namespace/skupper-site
2025/04/09 18:39:11 Checking token requests...
2025/04/09 18:39:11 Done.

For the site on which we have installed skupper using CLI and which are running on 1.4.2, how to upgrade them in place to 1.5.4 , any specific document we have ?

All you gotta do is to install the desired CLI version (e.g. 1.5.4) and run skupper update against the desired namespace.

https://skupper.io/docs/kubernetes-reference/skupper_update.html

--
You received this message because you are subscribed to the Google Groups "Skupper" group.
To unsubscribe from this group and stop receiving emails from it, send an email to skupper+u...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/skupper/48fbecbf-67f7-435a-a55a-17f4043dfbedn%40googlegroups.com.

--

Fernando Giorgetti

Red Hat

fgio...@redhat.com   

[vishavjeet somwanshi]

Hi Fernando, Thanks for the information.

1) I tried skupper update in the site where i had installed skupper using CLI and that went through. 

skupper-1.5.4 version

client version                 1.5.4

transport version              quay.io/skupper/skupper-router:2.5.3 (sha256:51b3a4549b7b)

controller version             quay.io/skupper/service-controller:1.5.4 (sha256:682db207ba2b)

config-sync version            quay.io/skupper/config-sync:1.5.4 (sha256:d10ff29148c5)

I have one ask we have higher environments where we have ingress controller and we configured or install skupper using below commands followed by passing the ingress-host parameter ( Ingress controller details ).

skupper init --ingress-host siteA-cluster-url.xyz.containers.domain.cloud --enable-console --enable-flow-collector

In that case do we need to provide the ingress-host URL again during upgrading skupper using CLI ?

2) While upgrading using YAML method, I could see that skupper-site-controller is updated to 1.5.4 but he has some failure logs as below and may be due to which skupper-router and skupper-service-controller failed to update.

2025/04/09 14:20:30 Skupper site controller watching current namespace skupper2

22025/04/09 14:20:30 Version: 1.5.4

32025/04/09 14:20:30 Unable to delete old cluster role "skupper-service-controller": clusterroles.rbac.authorization.k8s.io "skupper-service-controller" is forbidden: User "system:serviceaccount:skupper2:skupper-site-controller" cannot delete resource "clusterroles" in API group "rbac.authorization.k8s.io" at the cluster scope

42025/04/09 14:20:30 Unable to create new cluster role "skupper-service-controller-basic": clusterroles.rbac.authorization.k8s.io is forbidden: User "system:serviceaccount:skupper2:skupper-site-controller" cannot create resource "clusterroles" in API group "rbac.authorization.k8s.io" at the cluster scope

52025/04/09 14:20:30 Starting the Skupper site controller informers

62025/04/09 14:20:30 Waiting for informer caches to sync

72025/04/09 14:20:30 Checking if sites need updates (1.5.4)

82025/04/09 14:20:30 Version update check failed for namespace "skupper2": roles.rbac.authorization.k8s.io "skupper-router" is forbidden: User "system:serviceaccount:skupper2:skupper-site-controller" cannot update resource "roles" in API group "rbac.authorization.k8s.io" in the namespace "skupper2"

92025/04/09 14:20:30 Starting workers

102025/04/09 14:20:30 Started workers

112025/04/09 14:20:31 Skupper site exists skupper2/skupper-site

122025/04/09 14:20:31 Error checking annotations: deployments.apps "skupper-prometheus" not found

132025/04/09 14:20:31 Checking token requests...

142025/04/09 14:20:31 Done.

152025/04/09 14:20:31 Handling token for skupper2/skupper1-skupper2

162025/04/09 14:20:32 Router updated and restarted due to changes in token skupper2/skupper1-skupper2

So to address this issue, do i need to explicitly grant the permission to "skupper-site-controller" service account and bind with cluster role ?

Regards,

Vishavjeet N. Somwanshi

+91 8888202485

[Fernando Giorgetti]

In that case do we need to provide the ingress-host URL again during upgrading skupper using CLI ?

The update process does not require you to enter any argument.

It should be able to preserve your previously defined settings.

So to address this issue, do i need to explicitly grant the permission to "skupper-site-controller" service account and bind with cluster role ?

Exactly. Look at that gist link I sent you, it highlights the new permissions needed.

Here are the links to the original deployments, if you want to compare them yourself.

https://github.com/skupperproject/skupper/blob/1.4.2/cmd/site-controller/deploy-watch-all-ns.yaml

https://github.com/skupperproject/skupper/blob/1.5.4/cmd/site-controller/deploy-watch-all-ns.yaml

[vishavjeet somwanshi]

Thank you very much Fernando !!

The link you shared are pretty helpful, earlier I created a role for 1.4.2 version https://github.com/skupperproject/skupper/blob/1.4.2/cmd/site-controller/deploy-watch-all-ns.yaml instead of clusterroles since I needed it in multiple namespaces. Now for upgrade to 1.5.4 i've just updated the role and rolebinding followed by this link https://github.com/skupperproject/skupper/blob/1.5.4/cmd/site-controller/deploy-watch-all-ns.yaml. 

1. Upgrade from 1.4.2 to 1.5.4 worked smooth for YAML configuration method as well however i still could see this error in skupper-site-controller pod the 2nd and 3rd line in logs.

2025/04/10 15:08:58 Version: 1.5.4

32025/04/10 15:08:58 Unable to delete old cluster role "skupper-service-controller": clusterroles.rbac.authorization.k8s.io "skupper-service-controller" is forbidden: User "system:serviceaccount:skupper1:skupper-site-controller" cannot delete resource "clusterroles" in API group "rbac.authorization.k8s.io" at the cluster scope

42025/04/10 15:08:58 Unable to create new cluster role "skupper-service-controller-basic": clusterroles.rbac.authorization.k8s.io is forbidden: User "system:serviceaccount:skupper1:skupper-site-controller" cannot create resource "clusterroles" in API group "rbac.authorization.k8s.io" at the cluster scope

52025/04/10 15:08:58 Starting the Skupper site controller informers

62025/04/10 15:08:58 Waiting for informer caches to sync

72025/04/10 15:08:58 Checking if sites need updates (1.5.4)

82025/04/10 15:09:01 Updated version for namespace "skupper1"

92025/04/10 15:09:01 Starting workers

102025/04/10 15:09:01 Started workers

112025/04/10 15:09:02 Skupper site exists skupper1/skupper-site

122025/04/10 15:09:02 Error checking annotations: deployments.apps "skupper-prometheus" not found

132025/04/10 15:09:02 Checking token requests...

142025/04/10 15:09:02 Done.

152025/04/10 15:09:02 Handling token for skupper1/skupper2-skupper1

162025/04/10 15:09:02 Router updated and restarted due to changes in token skupper1/skupper2-skupper1

But if you see the very last time of the logs, it states the router pod restarted due to changes in token, will it just refresh the token or create a new one and autolink ?

After upgrade i checked skupper link status and which gives below o/p

$ skupper-1.5.4 link status

Links created from this site:

Link skupper2-skupper1 is connected

Current links from other sites that are connected:

There are no connected links

2. Upgrade to 1.5.4 from CLI method also went through. 

Whatever I tested is just in one of the test clusters, I will test this soon in another lower environment.

Regards,

Vishavjeet N. Somwanshi

+91 8888202485

[Fernando Giorgetti]

1. Upgrade from 1.4.2 to 1.5.4 worked smooth for YAML configuration method as well however i still could see this error in skupper-site-controller pod the 2nd and 3rd line in logs.

In case you are deploying the site-controller to watch a specific namespace, check out these files below,

as the ones I sent you earlier are for installations watching all namespaces:

https://github.com/skupperproject/skupper/blob/1.4.2/cmd/site-controller/deploy-watch-current-ns.yaml

https://github.com/skupperproject/skupper/blob/1.5.4/cmd/site-controller/deploy-watch-current-ns.yaml

The restart seems to have been caused by host aliases being detected on an existing token, but probably

not defined as part of the existing router deployment. This check was added as part of 1.5.

To view this discussion visit https://groups.google.com/d/msgid/skupper/CAH4aTUy4JZwoPs3%3D8nnSjiMMZ3ex1Hxm28mAMD6xcpeE7%2BUicA%40mail.gmail.com.