[JIRA] (JENKINS-55655) BlueOcean Pipeline creation does not respect ssh ProxyCommand even though classic project creation does

[bulb+jenkins@ucw.cz (JIRA)]

Jan Hudec created an issue

Jenkins / JENKINS-55655 BlueOcean Pipeline creation does not respect ssh ProxyCommand even though classic project creation does Issue Type: Bug Assignee: Mark Waite Attachments: jenkins-git-exception.txt Components: blueocean-plugin, git-client-plugin Created: 2019-01-17 15:28 Environment: Docker image jenkins/jenkins:lte, Linux-amd64, Jenkins ver. 2.150.1, BlueOcean 1.10.1, Git plugin 3.9.1, Git client plugin 2.7.6. Priority: Minor Reporter: Jan Hudec I have built a docker container, based on jenkins/jenkins:lte, where I configured appropriate ProxyCommand for our convoluted intranet setup in /etc/ssh/ssh_config, and preinstalled the blueocean plugin using the /usr/local/bin/install-plugins.sh script. With this setup, I can create a project referring to our internal git repository as the Git client plugin uses the git binary (as is by default), which uses ssh, which reads the configuration and correctly tunnels through the proxy. However when I switch to BlueOcean, and try to create the pipeline from there, it fails to connect. The log says: SEVERE: Error running git remote-ls: org.eclipse.jgit.api.errors.TransportException: ssh://g...@internal.git.server.com:4567/dep/Project.git: Failed to connect hudson.plugins.git.GitException: org.eclipse.jgit.api.errors.TransportException: ssh://g...@internal.git.server.com:4567/dep/Project.git: Failed to connect Caused by: org.eclipse.jgit.api.errors.TransportException: ssh://g...@internal.git.server.com:4567/dep/Project.git: Failed to connect Caused by: org.eclipse.jgit.errors.TransportException: ssh://g...@internal.git.server.com:4567/dep/Project.git: Failed to connect Caused by: java.io.IOException: There was a problem while connecting to internal.git.server.com:4567 Caused by: java.net.UnknownHostException: internal.git.server.com: Name or service not known (anonymised the server and project name, otherwise exact log except the backtraces; full log including the backtraces attached) The server name is not known, because only the proxy server can resolve it. The configuration in /etc/ssh/ssh_config is like: Host internal.git.server.com ProxyCommand socat - proxy:10.20.30.40:%h:%p,proxyport=8080,proxyauth=user:password The %h, which gets expanded to internal.git.server.com, is resolved by the proxy, so it does not matter than the Jenkins host does not have the DNS for the land behind the proxy. However, the blueocean connection manages to do a request that goes through the JGit and the Java ssh2 library (com.trilead.ssh2), even when they are not configured in the git-client-plugin options, and this does not honor that configuration and fails. Workaround I haven't tested it yet, but I realised a workaround would be to: Add internal.git.server.com to /etc/hosts, pointing to localhost and Get socat tcp-listen:4567,reuseaddr,fork proxy:1020.30.40:internal.git.server.com:4567,proxyport=8080,proxyauth=user:password running in the background—that is, create kind of transparent proxy of it. Then no ssh configuration is needed. I still believe that either the access should consistently honour the settings. Add Comment

This message was sent by Atlassian Jira (v7.11.2#711002-sha1:fdc329d)

[mark.earl.waite@gmail.com (JIRA)]

Mark Waite assigned an issue to Unassigned

Jenkins / JENKINS-55655 BlueOcean Pipeline creation does not respect ssh ProxyCommand even though classic project creation does

Change By: Mark Waite Assignee: Mark Waite

Add Comment

[mark.earl.waite@gmail.com (JIRA)]

Mark Waite updated an issue

Jenkins / JENKINS-55655 BlueOcean Pipeline creation does not respect ssh ProxyCommand even though classic project creation does

Change By: Mark Waite Component/s: blueocean-plugin

Add Comment

[mark.earl.waite@gmail.com (JIRA)]

Mark Waite commented on JENKINS-55655

Re: BlueOcean Pipeline creation does not respect ssh ProxyCommand even though classic project creation does Thanks for the bug report. The stack trace indicates that Blue Ocean is generating it, but Blue Ocean is doing exactly the right thing. It asks for a git client implementation and then calls it. I think the bug is that the default for git client seems to be to return a JGit instance instead of a CLI git instance when it is called with all defaults. That's a mistake, since CLI git is the canonical implementation of the Jenkins git client. Unfortunately, changing the default behavior like that is also a significant change to compatibility. It will need more thought before I dare make that change. In the interim, you might experiment with different settings of git implementations in the Jenkins global configuration page. It may change behavior if you explicitly declare a git implementation and use "git" as the executable. I haven't tried it myself, so that is just a wild guess. I have written a unit test which shows the problem and placed it on my check-default-git-client branch. If you'd like to experiment with changing the default setting in the git client plugin, you could take that branch and its failing unit test as a starting point.

Add Comment

[mark.earl.waite@gmail.com (JIRA)]

Mark Waite edited a comment on JENKINS-55655

Re: BlueOcean Pipeline creation does not respect ssh ProxyCommand even though classic project creation does

Thanks for the bug report.  The stack trace indicates that Blue Ocean is generating it, but Blue Ocean is doing exactly the right thing.  It asks for a git client implementation and then calls it.

I think the bug is that the default for git client seems to be to return a JGit instance instead of a CLI git instance when it is called with all defaults .  That's a mistake, since CLI git is the canonical implementation of the Jenkins git client.

Unfortunately, changing the default behavior like that is also a significant change to compatibility.  It will need more thought before I dare make that change. In the interim, you might experiment with different settings of git implementations in the Jenkins global configuration page.  It may change behavior if you explicitly declare a git implementation and use "git" as the executable.  I haven't tried it myself, so that is just a wild guess.

I have written a unit test which shows the problem and placed it on my [check-default-git-client branch|https://github.com/MarkEWaite/git-client-plugin/commits/check-default-git-client].  If you'd like to experiment with changing the default setting in the git client plugin, you could take that branch and its failing unit test as a starting point.

Add Comment

[bulb+jenkins@ucw.cz (JIRA)]

Jan Hudec commented on JENKINS-55655

Re: BlueOcean Pipeline creation does not respect ssh ProxyCommand even though classic project creation does

It is more strange than that, because the git client does use the CLI instance when called by the normal Job setup. It only returns the JGit instance, which it does not even have configured, only when called from Blue Ocean (and maybe some other contexts).

Add Comment

[mark.earl.waite@gmail.com (JIRA)]

Mark Waite commented on JENKINS-55655

Re: BlueOcean Pipeline creation does not respect ssh ProxyCommand even though classic project creation does

Those other cases that you note are using more arguments to the Git object before they request the GitClient object. The extra arguments are a good thing in those cases because they tell the Git object the repository directory and the implementation to be used. The Blue Ocean plugin doesn't need to provide those extra arguments because it is performing an early permission check. It doesn't need a repository directory and shouldn't need to pass an implementation to be used. I suspect the specific issue you're seeing could also be solved by changing Blue Ocean to have it read the list of available git implementations and pass the first implementation as the implementation for that call. However, that is more work for Blue Ocean and gives it knowledge of the git plugin implementation that it really should not need.

Add Comment

[mark.earl.waite@gmail.com (JIRA)]

Mark Waite started work on JENKINS-55655

Change By: Mark Waite Status: Open In Progress

Add Comment

[mark.earl.waite@gmail.com (JIRA)]

Mark Waite updated an issue

Jenkins / JENKINS-55655

BlueOcean Pipeline creation does not respect ssh ProxyCommand even though classic project creation does

Change By: Mark Waite Environment: Docker image jenkins/jenkins: lte lts , Linux-amd64, Jenkins ver. 2.150.1, BlueOcean 1.10.1, Git plugin 3.9.1, Git client plugin 2.7.6.

Add Comment

[mark.earl.waite@gmail.com (JIRA)]

Mark Waite updated an issue

Jenkins / JENKINS-55655 BlueOcean Pipeline creation does not respect ssh ProxyCommand even though classic project creation does Change By: Mark Waite

I have built a docker container, based on {{jenkins/jenkins: lte lts }}, where I configured appropriate {{ProxyCommand}} for our convoluted intranet setup in {{/etc/ssh/ssh_config}}, and preinstalled the {{blueocean}} plugin using the {{/usr/local/bin/install-plugins.sh}} script.

With this setup, I can create a project referring to our internal git repository as the Git client plugin uses the git binary (as is by default), which uses ssh, which reads the configuration and correctly tunnels through the proxy. However when I switch to BlueOcean, and try to create the pipeline from there, it fails to connect. The log says:

{noformat}

SEVERE: Error running git remote-ls: org.eclipse.jgit.api.errors.TransportException: ssh://g...@internal.git.server.com:4567/dep/Project.git: Failed to connect hudson.plugins.git.GitException: org.eclipse.jgit.api.errors.TransportException: ssh://g...@internal.git.server.com:4567/dep/Project.git: Failed to connect Caused by: org.eclipse.jgit.api.errors.TransportException: ssh://g...@internal.git.server.com:4567/dep/Project.git: Failed to connect Caused by: org.eclipse.jgit.errors.TransportException: ssh://g...@internal.git.server.com:4567/dep/Project.git: Failed to connect Caused by: java.io.IOException: There was a problem while connecting to internal.git.server.com:4567 Caused by: java.net.UnknownHostException: internal.git.server.com: Name or service not known

{noformat}

(anonymised the server and project name, otherwise exact log except the backtraces; full log including the backtraces attached) The server name is not known, because only the proxy server can resolve it. The configuration in {{/etc/ssh/ssh_config}} is like:

{noformat} Host internal.git.server.com ProxyCommand socat - proxy:10.20.30.40:%h:%p,proxyport=8080,proxyauth=user:password {noformat} The {{%h}}, which gets expanded to {{internal.git.server.com}}, is resolved by the proxy, so it does not matter than the Jenkins host does not have the DNS for the land behind the proxy. However, the blueocean connection manages to do a request that goes through the JGit and the Java ssh2 library (com.trilead.ssh2), even when they are not configured in the git-client-plugin options, and this does not honor that configuration and fails. h4. Workaround

I haven't tested it yet, but I realised a workaround would be to:

# Add internal.git.server.com to /etc/hosts, pointing to localhost and # Get socat tcp-listen:4567,reuseaddr,fork proxy:1020.30.40:internal.git.server.com:4567,proxyport=8080,proxyauth=user:password running in the background—that is, create kind of transparent proxy of it.

Then no ssh configuration is needed. I still believe that either the access should consistently honour the settings.

Add Comment

[mark.earl.waite@gmail.com (JIRA)]

Mark Waite commented on JENKINS-55655

Re: BlueOcean Pipeline creation does not respect ssh ProxyCommand even though classic project creation does Jan Hudec I've created tests and an implementation to pass the tests in PR 421. I would love to have a code review of that PR and any interactive testing of the PR that you are able to provide.

Add Comment

[mark.earl.waite@gmail.com (JIRA)]

Mark Waite edited a comment on JENKINS-55655

Re: BlueOcean Pipeline creation does not respect ssh ProxyCommand even though classic project creation does

[~bulb] I've created tests and an implementation to pass the tests in [PR 421|https://github.com/jenkinsci/git-client-plugin/pull/421].  I would love to have a code review of that PR and any interactive testing of the PR that you are able to provide. The automated tests are failing and I don't have a reasonable explanation why they are failing.  I don't have time to spend on deeper investigation, so I'll close the pull request until I can take more time to investigate.

Add Comment

[rene.scheibe@gmail.com (JIRA)]

René Scheibe stopped work on JENKINS-55655

Change By: René Scheibe Status: In Progress Open

Add Comment

This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)