[JIRA] (JENKINS-62208) sshget is showing the entire contents of the file in console output log

2 views
Skip to first unread message

craig.webster@td.com (JIRA)

unread,
May 7, 2020, 2:23:03 PM5/7/20
to jenkinsc...@googlegroups.com
Craig Webster created an issue
 
Jenkins / Bug JENKINS-62208
sshget is showing the entire contents of the file in console output log
Issue Type: Bug Bug
Assignee: Naresh Rayapati
Components: ssh-steps-plugin
Created: 2020-05-07 18:22
Environment: Jenkins ver. 2.190.3 on Windows 2012 with ssh versions below.
ssh 2.6.1
ssh-agent 1.17
ssh-credentials 1.18
ssh-steps 2.0.0
Priority: Minor Minor
Reporter: Craig Webster

I sent this to the mailing list and was advised that it might be a bug and to open a Jira ticket:

When executing the following in a scripted pipeline:
 
sshGet remote: soc_remote, from: "/srv/jboss/server/${MAL}/log/${sourcefile}", into: FileName, override: true
 
The file is downloaded successfully from the remote server into the local directory on the Jenkins server.  However, the console output shows the contents of that file in its entirety, and the logs files in the build folder also contains the contents of the file.  The file is embedded within the rest of the log entries, and is not usable.  Therefore when I am downloading large files with sshGet, I end up consuming twice the space I should be, since I have a log file roughly the size of the file, and then the file itself which was downloaded.  
 
This is not only a space issue, but also a security issue, since sometimes I am downloading files with sensitive information in them and don't want those details available in the jenkins logs.

Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.13.12#713012-sha1:6e07c38)
Atlassian logo

naresh.rayapati@gmail.com (JIRA)

unread,
May 7, 2020, 2:54:02 PM5/7/20
to jenkinsc...@googlegroups.com
Naresh Rayapati commented on Bug JENKINS-62208
 
Re: sshget is showing the entire contents of the file in console output log

A bit tricky one to reproduce, we are actively using this plugin and don't see this on our Jenkins servers, and probably could be due to Jenkins and plugin combination.

Can you provide more information? like what is FileName is? Nothing pops up to me, can you make sure into: FileName is string vs a variable ? like into: "abc.txt"

craig.webster@td.com (JIRA)

unread,
May 7, 2020, 3:09:05 PM5/7/20
to jenkinsc...@googlegroups.com

Not sure if this will format correctly, but hopefully gives you enough of an idea:

import java.text.SimpleDateFormat

node {
def dateFormat = new SimpleDateFormat("yyyyMMddHHmm")
sourcefile="server.log"
String partialFileName = "
" + sourcefile + "${MAL}" + dateFormat.format(new Date()) + "soc"

servers = ['10.xxx.xxx.xxx',]

def soc_remote = [:]
soc_remote.name = "${MAL} SOC"
soc_remote.allowAnyHosts = true
soc_remote.fileTransfer = 'scp'

withCredentials([usernamePassword(credentialsId: 'Credentials', usernameVariable: 'USER_ID', passwordVariable: 'USER_PASSWORD')]) {
soc_remote.user = USER_ID
soc_remote.password = USER_PASSWORD

servers.each { server ->
String FileName = partialFileName + server + ".txt"
echo "${FileName}"
soc_remote.host = server

stage("Run log gather script on ${server}") {
sshCommand remote: soc_remote, command: "ls -ltr /srv/jboss/server/${MAL}/log/${sourcefile}"
}
stage("Download ${MAL} logs from ${server}") {
dir("D:\\Logs\\${MAL}\\logs
${server}") {
try {
def commandResult = sshGet remote: soc_remote, from: "/srv/jboss/server/${MAL}/log/${sourcefile}", into: FileName, override: true
} catch (err) {
echo "something failed"
echo "${err}"
}
}
}
}
}
}

naresh.rayapati@gmail.com (JIRA)

unread,
May 7, 2020, 4:47:02 PM5/7/20
to jenkinsc...@googlegroups.com

you could use the code block in JIRA for formatting. Like

import java.text.SimpleDateFormat

node {
    def dateFormat = new SimpleDateFormat("yyyyMMddHHmm")
    sourcefile="server.log"
    String partialFileName = "" + sourcefile + "${MAL}" + dateFormat.format(new Date()) + "soc"
    servers = ['10.xxx.xxx.xxx',]
    def soc_remote = [:]
    soc_remote.name = "${MAL} SOC"
    soc_remote.allowAnyHosts = true
    soc_remote.fileTransfer = 'scp'

    withCredentials([usernamePassword(credentialsId: 'Credentials', usernameVariable: 'USER_ID', passwordVariable: 'USER_PASSWORD'
)]) {
        soc _remote.user = USER_ID
        soc_remote.password = USER_PASSWORD
        servers.each { server ->
            
String FileName = partialFileName + server + ".txt"
            echo "${FileName}"
            soc_remote.host = server
            stage("Run log gather script on ${server}") {
                sshCommand remote: soc_remote, command: "ls -ltr /srv/jboss/server/${MAL}/log/${sourcefile}"
            }
            stage("Download ${MAL} logs from ${server}"
) {
                dir("D:\\Logs\\${MAL}\\logs${server}") {
                    
try {
                        def commandResult = sshGet remote: soc_remote, from: "/srv/jboss/server/${MAL}/log/${sourcefile}", into: FileName, override: true
                    } catch (err) {
                        echo "something failed"
                        echo "${err}"
                    }
                }
            }
        }
    }
}

naresh.rayapati@gmail.com (JIRA)

unread,
May 7, 2020, 4:51:02 PM5/7/20
to jenkinsc...@googlegroups.com

Okay couple of things: 

  • is the actual server that this step is trying to get the file from linux machine? 
  • Not sure def CommandResult is causing this issue, can we delete that assignment and see if that is the cause.
  • Looks like you are running Jenkins on Windows, unfortunately I didn't test this plugin on windows server, can you able test it on linux agent to see if that is the problem 

craig.webster@td.com (JIRA)

unread,
May 7, 2020, 5:06:02 PM5/7/20
to jenkinsc...@googlegroups.com

Yes, the remote server that it is getting the file from is a linux server, the source Jenkins server is Windows 2012
I actually added the def CommandResult as part of troubleshooting but i took it out again and get the same results
I am using a work provided Jenkins instance; not sure i have access to a linux box to install Jenkins on.

naresh.rayapati@gmail.com (JIRA)

unread,
May 7, 2020, 5:26:02 PM5/7/20
to jenkinsc...@googlegroups.com

I don't have windows machine to test this right now, probably it might take more time for to me troubleshoot on this issue.

craig.webster@td.com (JIRA)

unread,
May 7, 2020, 9:25:03 PM5/7/20
to jenkinsc...@googlegroups.com

craig.webster@td.com (JIRA)

unread,
May 7, 2020, 9:27:04 PM5/7/20
to jenkinsc...@googlegroups.com
Craig Webster commented on Bug JENKINS-62208
 
Re: sshget is showing the entire contents of the file in console output log

Ok, I spun up a micro instance on Google Cloud with Ubuntu and installed a fresh copy of Jenkins. I then took my pipeline and had it scp a file from that Jenkins server to that Jenkins server (didn't have anything else I could connect to). I attached the console log - it is still putting the contents of the file into the log itself. Code I used below, log is attached.

node {
 [^consoletext.txt]     sourcefile="test.txt"

servers = ['localhost',]

	def soc_remote = [:]
	soc_remote.name = "SOC"
	soc_remote.allowAnyHosts = true
	soc_remote.fileTransfer = 'scp'
   
	withCredentials([usernamePassword(credentialsId: 'Credentials', usernameVariable: 'USER_ID', passwordVariable: 'USER_PASSWORD'
)]) {
		soc_remote.user = USER_ID
		soc_remote.password = USER_PASSWORD
			
		servers.each { server ->
            String FileName = "test.txt"
            echo "${FileName}"
            soc_remote.host = server
				
				stage("Run log gather script on ${server}"
) {
					sshCommand remote: soc_remote, command: "ls -ltr /tmp/test/${sourcefile}"
				}
				stage("Download logs from ${server}") {
					dir("/tmp/") {
                        try {
                        sshGet remote: soc_remote, from: "/tmp/test/${sourcefile}", into: FileName, override: true
                        } catch (err) {
                          echo "something failed"
                          echo "${err}"
                          }
					}
				}
        }
	}
}

naresh.rayapati@gmail.com (JIRA)

unread,
May 7, 2020, 10:03:05 PM5/7/20
to jenkinsc...@googlegroups.com

Apparently I didn't read your code fully before sorry about that.
It is probably you are using the 'scp' as a fileTransfer mode, try to remove that line so that it is going to fall back to sftp or set it to 'sftp'

...
soc_remote.fileTransfer = 'scp'
...

craig.webster@td.com (JIRA)

unread,
May 8, 2020, 9:26:03 AM5/8/20
to jenkinsc...@googlegroups.com

Thank you, i removed that line and it fixed my issue. scp vs sftp still confuses me; when i want to copy a file between linux servers the command I use is scp.... so probably that's why I had that in there.

Curious though, why does the plugin echo the file contents to console output when using scp as transfer mode? Honestly in some ways it is handy because there are times when i actually do want the file echoed to the console.

naresh.rayapati@gmail.com (JIRA)

unread,
May 8, 2020, 9:47:03 AM5/8/20
to jenkinsc...@googlegroups.com

I found a bug in the code, going to fix it soon to not to print the file in scp mode.

But in cases like where you wanted to print the file contents explicitly, there are couple of other steps around reading file contents and printing those to console.
https://www.jenkins.io/doc/pipeline/steps/workflow-basic-steps/#readfile-read-file-from-workspace

def data = readFile(file: FileName)
println(data)

Few advanced steps to read specific files like yaml,csv or json here: https://www.jenkins.io/doc/pipeline/steps/pipeline-utility-steps/

Reply all
Reply to author
Forward
0 new messages