Quarkus kubernetes deployment

298 views
Skip to first unread message

Vishal Goel

unread,
Jun 22, 2020, 12:13:30 PM6/22/20
to Quarkus Development mailing list
Hi All,

I am trying to deploy a quarkus application in kubernetes cluster (EKS) using quarkus kubernetes extension.

There are two issues:

1) During mvnw package, image is built fine, pushed to ECR. But it is not deployed on my k8s cluster.
I have given below options in application.properties:
quarkus.container-image.build=true
quarkus.container-image.push=true
quarkus.kubernetes.deploy=true
Also, i tried running mvnw clean package -Dquarkus.kubernetes.deploy=true, but still it did not deploy the app in k8s.

I see below error in mvn output (though build is shown as success). I have "aws" in my path and even on same console, I am able to run aws command. Can below error be cause of pods not getting deployed? If yes, how can I solve it?

[INFO] --- quarkus-maven-plugin:1.4.2.Final:build (default) @ security-keycloak-authorization-quickstart ---
[WARNING] [io.quarkus.deployment.QuarkusAugmentor] Using Java versions older than 11 to build Quarkus applications is deprecated and will be disallowed in a future release!
[ERROR] Failed to parse the kubeconfig.
java.io.IOException: Cannot run program "aws": CreateProcess error=2, The system cannot find the file specified
    at java.lang.ProcessBuilder.start (ProcessBuilder.java:1048)
    at io.fabric8.kubernetes.client.Config.loadFromKubeconfig (Config.java:570)
    at io.fabric8.kubernetes.client.Config.tryKubeConfig (Config.java:491)
    at io.fabric8.kubernetes.client.Config.autoConfigure (Config.java:230)

2) Second issue: Is there any way of providing the k8s namespace for pod deployment? Anyway via props, env vars etc.? 

Georgios Andrianakis

unread,
Jun 22, 2020, 12:59:03 PM6/22/20
to vishalg...@gmail.com, Quarkus Development mailing list
Hi,

I don't understand what the purpose of "aws" is. Can you explain?
Also, can you try with Quarkus 1.5.2 to see if it makes a difference?

2) Second issue: Is there any way of providing the k8s namespace for pod deployment? Anyway via props, env vars etc.? 

You aren't the first person to ask this, but I have been hesitant to add it. We don't reference a namespace, which means that all manifests are applied to whatever namespace your kubectl context refers to

--
You received this message because you are subscribed to the Google Groups "Quarkus Development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to quarkus-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/1a626eee-03f7-4e7b-9fc7-4ba96f66062en%40googlegroups.com.

Vishal Goel

unread,
Jun 22, 2020, 2:46:38 PM6/22/20
to Quarkus Development mailing list
Hi Andrian,

"aws" is aws cli, not sure why quarkus is picking it. I have some aws configs in my .kube/config as I am using aWS EKS>
Same error is there with 1.5.2.

Georgios Andrianakis

unread,
Jun 22, 2020, 2:51:57 PM6/22/20
to vishalg...@gmail.com, Quarkus Development mailing list
Can you with Quarkus 1.5.2 execute the maven build with the addition of the -X flag and paste the stacktrace?

Also, is the any mention of "aws" in ~/.kube/config?


To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/daeae86c-8221-4b70-9025-3472ba6710e7n%40googlegroups.com.

William Burke

unread,
Jun 22, 2020, 4:24:56 PM6/22/20
to Georgios Andrianakis, vishalg...@gmail.com, Quarkus Development mailing list
Vishal,

Tangent question:  Do you think it would be better to have some sort of "deploy" phase instead of creating and pushing the image at build?

i.e.

$mvn quarkus:deploy

I've been thinking that during development you would not normally want to build an image and you definitely would not want to push it until you're ready for your testbed to take advantage of it.

To view this discussion on the web visit https://groups.google.com/d/msgid/quarkus-dev/CALeTM-nXva%2BCuB_LGEmH%2B5uVzYaxXLYrH-gKjfcOWkbOGsE04Q%40mail.gmail.com.


--
Bill Burke
Red Hat

Vishal Goel

unread,
Jun 22, 2020, 4:38:43 PM6/22/20
to Quarkus Development mailing list
Hi Bill,

It depends. I think it is better to govern it by application.properties or by options like  mvnw clean package -Dquarkus.kubernetes.deploy. It can be useful if in one step, someone wants to cleanly build image and deploy it for quick testing.
I checked google cloud code. It uses skaffold and it has deploy section to deploy the image on k8s. This is supported during build time. 

does quarkus leverages skaffold to deploy or it has its own impl? skaffold seems to me quite useful for full lifecycle including app build, image build, push and deploy. Also, you can do the remote debugging on k8s cluster (quite helpful) 

Georgios Andrianakis

unread,
Jun 23, 2020, 1:42:09 AM6/23/20
to vishalg...@gmail.com, Quarkus Development mailing list
On Mon, Jun 22, 2020 at 11:38 PM Vishal Goel <vishalg...@gmail.com> wrote:
Hi Bill,

It depends. I think it is better to govern it by application.properties or by options like  mvnw clean package -Dquarkus.kubernetes.deploy. It can be useful if in one step, someone wants to cleanly build image and deploy it for quick testing.
I checked google cloud code. It uses skaffold and it has deploy section to deploy the image on k8s. This is supported during build time. 

does quarkus leverages skaffold to deploy or it has its own impl? skaffold seems to me quite useful for full lifecycle including app build, image build, push and deploy. Also, you can do the remote debugging on k8s cluster (quite helpful) 

We don't have integration with Scaffold at this point

Vishal Goel

unread,
Jun 23, 2020, 7:17:46 AM6/23/20
to Quarkus Development mailing list
Hi,

I had aws as command in my .kube/config which mvnw was picking during build. Strangely, it could not resolve aws (though it is there in PATH), so I provided full path in .kube/config and it worked out correctly.
Quarkus is pushing images to ECR and even deploying it on k8s cluster.

Namespace is being picked from kube context. I think it would be better and more flexible if we can have it as property also in application.properties file.

Brett Tofel

unread,
Jun 23, 2020, 8:43:22 AM6/23/20
to Quarkus Development mailing list
I have an example that pushes a native Quarkus built operator to a local cluster registry (CRC's in this case) and deploys it to the cluster, perhaps it might help you. Most important docs are at the end of the Readme.md:


-Brett

Max Rydahl Andersen

unread,
Jun 23, 2020, 9:21:20 AM6/23/20
to Vishal Goel, Quarkus Development mailing list

Hi,

I had aws as command in my .kube/config which mvnw was picking during
build. Strangely, it could not resolve aws (though it is there in PATH), so
I provided full path in .kube/config and it worked out correctly.
Quarkus is pushing images to ECR and even deploying it on k8s cluster.

can you share what you exactly had in your config where aws gets added to the mix ?

It sounds like the parsing/processing of .kube/config might not be done with the proper environment (i.e. it uses the global system env instead of your terminals current context).

Having an example to reproduce would make it easier for us to fix :)

/max

You received this message because you are subscribed to the Google
Groups "Quarkus Development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to quarkus-dev...@googlegroups.com.
To view this discussion on the web visit

--
You received this message because you are subscribed to the Google
Groups "Quarkus Development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to quarkus-dev...@googlegroups.com.

To view this discussion on the web visit

--
Bill Burke
Red Hat

--
You received this message because you are subscribed to the Google Groups
"Quarkus Development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to quarkus-dev...@googlegroups.com.

To view this discussion on the web visit

--
You received this message because you are subscribed to the Google Groups "Quarkus Development mailing list" group.
To unsubscribe from this group and stop receiving emails from it, send an email to quarkus-dev...@googlegroups.com.

Vishal Goel

unread,
Jun 23, 2020, 9:27:56 AM6/23/20
to Quarkus Development mailing list
In users section of .kube/config, I have below:

users:
- name: <eks_cluster_arn>
  user:
    exec:
      args:
      - --region
      - us-west-2
      - eks
      - get-token
      - --cluster-name
      - <cluster_name>
      command: aws
      env: null

Max Rydahl Andersen

unread,
Jun 23, 2020, 9:43:59 AM6/23/20
to Vishal Goel, Quarkus Development mailing list


> In users section of .kube/config, I have below:
>
> users:
> - name: <eks_cluster_arn>
> user:
> exec:
> apiVersion: client.authentication.k8s.io/v1alpha1
> args:
> - --region
> - us-west-2
> - eks
> - get-token
> - --cluster-name
> - <cluster_name>
> *command: aws*
> env: null

hey - I learned something new :) kubernetes cli can use external
authentication.
https://kubernetes.io/docs/reference/access-authn-authz/authentication/

I believe this is a bug - can you open issue for it please ?

/max
> https://groups.google.com/d/msgid/quarkus-dev/e2521142-3d29-49c6-9cbb-3ac87dcc623dn%40googlegroups.com.


Georgios Andrianakis

unread,
Jun 23, 2020, 9:46:11 AM6/23/20
to Max Andersen, Vishal Goel, Quarkus Development mailing list
On Tue, Jun 23, 2020 at 4:44 PM Max Rydahl Andersen <mand...@redhat.com> wrote:


> In users section of .kube/config, I have below:
>
> users:
> - name: <eks_cluster_arn>
>   user:
>     exec:
>       apiVersion: client.authentication.k8s.io/v1alpha1
>       args:
>       - --region
>       - us-west-2
>       - eks
>       - get-token
>       - --cluster-name
>       - <cluster_name>
>       *command: aws*
>       env: null

hey - I learned something new :) kubernetes cli can use external
authentication.
https://kubernetes.io/docs/reference/access-authn-authz/authentication/

I believe this is a bug - can you open issue for it please ?

I was not aware of this either. What you are seeing is most likely a bug in the Kubernetes Client, but please open an issue in Quarkus and we can explore

Vishal Goel

unread,
Jun 23, 2020, 10:01:26 AM6/23/20
to Quarkus Development mailing list
Hi,

Opened below:
Thanks 

Reply all
Reply to author
Forward
0 new messages