stuck with Auth Fail when deployer attempts to clone from authoring

[bitsofinfo]

following: https://docs.craftercms.org/en/3.1/system-administrators/activities/kubernetes/simple-kubernetes-deployment.html?highlight=kubernetes

Getting this error: (i've verified the contents of /opt/crafter/.ssh/id_rsa* RSA keys match on both the authoring/deployer nodes)

I can ping authoring-ssh-service from inside the deployer and services shows the following

$ kubectl get services

NAME                    TYPE           CLUSTER-IP       EXTERNAL-IP      PORT(S)          AGE

authoring-service       LoadBalancer   10.108.202.158   192.168.99.103   8080:30615/TCP   3h25m

authoring-ssh-service   ClusterIP      10.105.58.217    <none>           22/TCP           3h25m

delivery-service        LoadBalancer   10.96.78.196     192.168.99.103   9080:31526/TCP   3h11m

elasticsearch-service   ClusterIP      10.97.141.211    <none>           9200/TCP         3h27m

kubernetes              ClusterIP      10.96.0.1        <none>           443/TCP          3h29m

root@delivery-deployment-7dd8978bf5-tcqfs:/opt/crafter/.ssh# cat config

Host authoring-ssh-service

  StrictHostKeyChecking no

Anything else I should be looking for?

2019-12-09 18:36:01.165 ERROR 24 --- [deployment-2] l.processors.AbstractDeploymentProcessor : Processor 'gitPullProcessor' for target 'bitsofinfo-default' failed

org.craftercms.deployer.api.exceptions.DeployerException: Failed to clone Git remote repository ssh://authoring-ssh-service/opt/crafter/data/repos/sites/bitsofinfo/published into /opt/crafter/data/repos/sites/bitsofinfo; nested exception is org.eclipse.jgit.api.errors.TransportException: ssh://authoring-ssh-service/opt/crafter/data/repos/sites/bitsofinfo/published: Auth fail

        at org.craftercms.deployer.impl.processors.GitPullProcessor.cloneRemoteRepository(GitPullProcessor.java:173)

        at org.craftercms.deployer.impl.processors.GitPullProcessor.doClone(GitPullProcessor.java:143)

        at org.craftercms.deployer.impl.processors.GitPullProcessor.doMainProcess(GitPullProcessor.java:92)

        at org.craftercms.deployer.impl.processors.AbstractMainDeploymentProcessor.doExecute(AbstractMainDeploymentProcessor.java:39)

        at org.craftercms.deployer.impl.processors.AbstractDeploymentProcessor.execute(AbstractDeploymentProcessor.java:142)

        at org.craftercms.deployer.impl.DeploymentPipelineImpl.executeProcessors(DeploymentPipelineImpl.java:82)

        at org.craftercms.deployer.impl.DeploymentPipelineImpl.execute(DeploymentPipelineImpl.java:68)

        at org.craftercms.deployer.impl.TargetImpl$DeploymentTask.run(TargetImpl.java:397)

        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

        at java.util.concurrent.FutureTask.run(FutureTask.java:266)

        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

        at java.lang.Thread.run(Thread.java:748)

Caused by: org.eclipse.jgit.api.errors.TransportException: ssh://authoring-ssh-service/opt/crafter/data/repos/sites/bitsofinfo/published: Auth fail

        at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:255)

        at org.eclipse.jgit.api.CloneCommand.fetch(CloneCommand.java:302)

        at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:200)

        at org.craftercms.deployer.utils.GitUtils.cloneRemoteRepository(GitUtils.java:105)

        at org.craftercms.deployer.impl.processors.GitPullProcessor.cloneRemoteRepository(GitPullProcessor.java:166)

        ... 12 more

Caused by: org.eclipse.jgit.errors.TransportException: ssh://authoring-ssh-service/opt/crafter/data/repos/sites/bitsofinfo/published: Auth fail

        at org.eclipse.jgit.transport.JschConfigSessionFactory.getSession(JschConfigSessionFactory.java:191)

        at org.eclipse.jgit.transport.SshTransport.getSession(SshTransport.java:140)

        at org.eclipse.jgit.transport.TransportGitSsh$SshFetchConnection.<init>(TransportGitSsh.java:280)

        at org.eclipse.jgit.transport.TransportGitSsh.openFetch(TransportGitSsh.java:170)

        at org.eclipse.jgit.transport.FetchProcess.executeImp(FetchProcess.java:138)

        at org.eclipse.jgit.transport.FetchProcess.execute(FetchProcess.java:124)

        at org.eclipse.jgit.transport.Transport.fetch(Transport.java:1292)

        at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:244)

        ... 16 more

Caused by: com.jcraft.jsch.JSchException: Auth fail

        at com.jcraft.jsch.Session.connect(Session.java:519)

        at org.eclipse.jgit.transport.JschConfigSessionFactory.getSession(JschConfigSessionFactory.java:145)

        ... 23 more

[Sumer Jabri]

On the Studio machine, look at the contents of `~/.ssh/known_hosts` are the known hosts stored with RSA or ECDSA? You can post a snippet here to review.

JGit is very sensitive to that stuff.

--sumer

[bitsofinfo]

No known_hosts file exist on either the authoring/tomcat under /home/crafter/.ssh nor /opt/crafter/.ssh (neither exist on the deployer side either)

I just deployed all this via the k8s example link mentioned previously

[avasquez]

Whats appears in the header of your private key? Make sure it begins with

-----BEGIN RSA PRIVATE KEY-----

[bitsofinfo]

Yes thats what it starts with

[Alfonso Vásquez]

Everything seems fine so not sure what could be your issue. Did you try to open a shell in the Deployer container and then ssh from there to the authoring-ssh-service?

--
You received this message because you are subscribed to a topic in the Google Groups "CrafterCMS" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/craftercms/mYxnfsDO6r4/unsubscribe.
To unsubscribe from this group and all its topics, send an email to craftercms+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/craftercms/72b69075-e124-434e-a94d-9a11da459b56%40googlegroups.com.

[bitsofinfo]

inside authoring tomcat container:

root@authoring-deployment-85dccfcd7b-km42h:/home/crafter/.ssh# pwd

/home/crafter/.ssh

root@authoring-deployment-85dccfcd7b-km42h:/home/crafter/.ssh# ls -al id_rsa

-rw------- 1 crafter crafter 3243 Dec 10 15:28 id_rsa

root@authoring-deployment-85dccfcd7b-km42h:/home/crafter/.ssh# md5sum id_rsa

d08aac9a0ce97e7f55633a54a6ef7b0e  id_rsa

root@authoring-deployment-85dccfcd7b-km42h:/home/crafter/.ssh# head -n 2 id_rsa

-----BEGIN RSA PRIVATE KEY-----

MIIJJwIBAAKCAgEAq2QmgUlFxCWdPRAvJLexq4QCD8UzFCbo/F2gqh4LGSBmsE5u

root@authoring-deployment-85dccfcd7b-km42h:/home/crafter/.ssh#

Inside delivery pod, deployer container:

crafter@delivery-deployment-7dd8978bf5-pkx8h:~/.ssh$ pwd

/home/crafter/.ssh

crafter@delivery-deployment-7dd8978bf5-pkx8h:~/.ssh$ ls -al id_rsa

-rw------- 1 crafter crafter 3243 Dec 10 15:28 id_rsa

crafter@delivery-deployment-7dd8978bf5-pkx8h:~/.ssh$ md5sum id_rsa

d08aac9a0ce97e7f55633a54a6ef7b0e  id_rsa

crafter@delivery-deployment-7dd8978bf5-pkx8h:~/.ssh$ head -n 2 id_rsa

-----BEGIN RSA PRIVATE KEY-----

MIIJJwIBAAKCAgEAq2QmgUlFxCWdPRAvJLexq4QCD8UzFCbo/F2gqh4LGSBmsE5u

crafter@delivery-deployment-7dd8978bf5-pkx8h:~/.ssh$ telnet authoring-ssh-service 22

Trying 10.105.58.217...

Connected to authoring-ssh-service.default.svc.cluster.local.

Escape character is '^]'.

SSH-2.0-OpenSSH_7.9

^^^]

telnet> quit

Connection closed.

crafter@delivery-deployment-7dd8978bf5-pkx8h:~/.ssh$ ssh -i id_rsa crafter@authoring-ssh-service

crafter@authoring-ssh-service's password:

On Monday, December 9, 2019 at 5:04:19 PM UTC-7, avasquez wrote:

Everything seems fine so not sure what could be your issue. Did you try to open a shell in the Deployer container and then ssh from there to the authoring-ssh-service?

To unsubscribe from this group and all its topics, send an email to craft...@googlegroups.com.

[bitsofinfo]

an authorized_keys everywhere only contains the public key that I configured in the GUI for the remote github origin.... not the public key generated and put into ssh-keys secret in k8s.

[Sumer Jabri]

As you continue to figure out the SSH issue, which you should since it's a good path, you may also consider a different path altogether.

Crafter is very lego-ish. You can do the following:

- Configure the preview deployer (runs alongside every Studio instance) to have a new target to push from the published repository to an external service like GitHub or GitLab etc. You can set that to be time-based (default 60 seconds) to push from `published/live` to the same on the remote (using the Git push processor). If many deployers do that, no problem (it's git).

- Configure the regular deployer to pull from that remote git repo (over HTTPS)

This is similar to the regular deployer pulling from Studio/published/live to push into S3 and Elasticsearch, except you added GitHub/similar in the middle. Many actual deployments like this path because they want to never allow anything to dip into Studio's disk. We don't favor it because it's: a) slower, and b) relies on the uptime of an external entity. You can probably enumerate many other pros and cons.

Ideally, you'd make SSH keys work (those can be tricky) but at least now you may start to visualize other paths (yes, there can be more).

--sumer

[bitsofinfo]

ssh thing fixed, the authorized keys I was provisioning in the secret had an old pub key in it.