Running Java ProcessBuilder on Agent

[Tal Yanai]

Hi,

I'm trying to execute a simple /bin/csh command from within Java (I know I can use the Shell exec Step, but want to try and parse some data comes in from the stdout of the Linux command using Java.

I'm using ProcessBuilder for that, however, Jenkins always running this on the controller environment, and not on the Agent. (I have controller on Windows and Agent on Linux)

Anything I can do to make sure Runtime.exec (or ProcessBuilder) is running on the Agent and not on the Controller?

Also, and proven way to parse the output coming from a Linux shell execution using Java?

Thanks,

Tal.

[Ullrich Hafner]

Am 15.11.2020 um 10:29 schrieb Tal Yanai <t...@yanai.org.il>:

Hi,

I'm trying to execute a simple /bin/csh command from within Java (I know I can use the Shell exec Step, but want to try and parse some data comes in from the stdout of the Linux command using Java.

Why do you need a shell script to parse something in Java?

I'm using ProcessBuilder for that, however, Jenkins always running this on the controller environment, and not on the Agent. (I have controller on Windows and Agent on Linux)

Anything I can do to make sure Runtime.exec (or ProcessBuilder) is running on the Agent and not on the Controller?

Where do you invoke the call? In a publisher? Then you have the workspace of the agent and can call the code on the agent as well. 

Also, and proven way to parse the output coming from a Linux shell execution using Java?

Yes, this is something the warnings plugin is doing.

Thanks,

Tal.

--
You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/073df562-c83b-4787-afcc-fcb377b3fd03n%40googlegroups.com.

[Tal Yanai]

There’s a process user must execute in shell, that produce some lines of code I want to parse.  I’m trying to do the all thing from the Java plugin to spare the shell usage to begin with. 

 

The call is made from hudson.tasks.Builder (perform), and is always running on the Controller, not on the agent. 

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/c7Z3OFLyzx0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/937E1434-52AA-489D-A610-E732CB7B882D%40gmail.com.

[Jesse Glick]

Use the `Launcher` you are provided.

[Tal Yanai]

I see ProcLauncher using the Launcher.  How can I use ProcessBuilder, or what is the difference?

 

From: jenkin...@googlegroups.com <jenkin...@googlegroups.com> On Behalf Of Jesse Glick
Sent: יום א 15 נובמבר 2020 16:57
To: Jenkins Dev <jenkin...@googlegroups.com>
Subject: Re: Running Java ProcessBuilder on Agent

 

Use the `Launcher` you are provided.

--

You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/c7Z3OFLyzx0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr3GKnFRoCNQRFeGZHtbExZuPy5ncYvhuX24LhtXqJ1f4Q%40mail.gmail.com.

[Tal Yanai]

Thanks Jesse,

 

I’m getting closer, but fail to understand how to get back the stdout of the process:

 

-----------------------------------------------------------------------------------

ProcStarter ps = launcher.new ProcStarter();

ps.cmds(command); //.stdout(listener);

Proc proc = launcher.launch(ps); 

 

BufferedReader in = new BufferedReader(new InputStreamReader(ps.stdin(), StandardCharsets.UTF_8));

String s = null;

while ((s = in.readLine()) != null) {

          this.jobListener.getLogger().print(s);

}

 

 

I see the process runs OK, but nothing is being returned back.

 

Thanks,

 

Tal.

 

 

From: jenkin...@googlegroups.com <jenkin...@googlegroups.com> On Behalf Of Jesse Glick
Sent: יום א 15 נובמבר 2020 16:57
To: Jenkins Dev <jenkin...@googlegroups.com>
Subject: Re: Running Java ProcessBuilder on Agent

 

Use the `Launcher` you are provided.

--

You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/c7Z3OFLyzx0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr3GKnFRoCNQRFeGZHtbExZuPy5ncYvhuX24LhtXqJ1f4Q%40mail.gmail.com.

[Gavin Mogan]

Why would stdin return anything? Stdin is what gets sent into the application.

You want to be reading stdout or stderr depending on what the app is outputting

I'm not sure why your going this route instead of just doing a sh step. Piping or redirecting to a file and reading that 

You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/007a01d6bb6c%24babdeec0%243039cc40%24%40yanai.org.il.

[Tal Yanai]

Yes, I need to read the STDOUT, but using an InputStream.  Doing proc.getStdout() throws a NullPointer and no other way that I can see that returns InputStream.

 

If I can’t get the InputStream, there’s something I don’t do right here.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAG%3D_DusaX7LL%2Bi7S0CK48FRt4dBrZ0%3Dcz0-XeSHyET3KB7MewQ%40mail.gmail.com.

[Tal Yanai]

This is how I solved it eventually:

 

public void execBatchCommand(FilePath filePath) throws IOException, InterruptedException {

 

        String[] command = {“/bin/csh”, “/home/tyanai/run.sh”, “input_param”};

 

        filePath.act(new FileCallable<Void>() {

 

            private static final long serialVersionUID = 6166111757469534436L;

 

            @Override

            public void checkRoles(final RoleChecker checker) throws SecurityException {

            }

 

            @Override

            public Void invoke(final File workspace, final VirtualChannel channel) throws IOException {  

                String sessionNameToMonitor = null;

                boolean foundGoodVSIF = false;

                try {

                    ProcessBuilder builder = new ProcessBuilder(command);

                    builder.redirectErrorStream(true);

                    Process proc = builder.start();

                    BufferedReader in = new BufferedReader(new InputStreamReader(proc.getInputStream()));

                   

                    String s = null;

                    while ((s = in.readLine()) != null) {

                        jobListener.getLogger().print(s + "\n");

                    }

                } catch (IOException e) {

                    jobListener.getLogger().println(e.getMessage());

                    for (StackTraceElement ste : e.getStackTrace()) {

                        jobListener.getLogger().println(" " + ste);

                    }

                    jobListener.getLogger().println(ExceptionUtils.getFullStackTrace(e));

                }

                return null;

            }

        });

}

 

From: ga...@gavinmogan.com <ga...@gavinmogan.com> On Behalf Of 'Gavin Mogan' via Jenkins Developers
Sent: יום א 15 נובמבר 2020 18:36

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CAG%3D_DusaX7LL%2Bi7S0CK48FRt4dBrZ0%3Dcz0-XeSHyET3KB7MewQ%40mail.gmail.com.

[Jesse Glick]

On Sun, Nov 15, 2020 at 4:31 PM Tal Yanai <t...@yanai.org.il> wrote:
> filePath.act(new FileCallable<Void>() {

If you need to run `act` (which you do not here), use
`MasterToSlaveFileCallable`, do not override `checkRoles`, and use a
`static` nested class, not inner.

> ProcessBuilder builder = new ProcessBuilder(command);

No, as above, use `Launcher`, not `ProcessBuilder`.

[Tal Yanai]

Thanks,

I'm having issues getting back the stdout for parsing using Launcher. With ProcessBuilder I don't. Anmy reason why to use Launcher?

-----Original Message-----
From: jenkin...@googlegroups.com <jenkin...@googlegroups.com> On Behalf Of Jesse Glick
Sent: יום ב 16 נובמבר 2020 17:21
To: Jenkins Dev <jenkin...@googlegroups.com>
Subject: Re: Running Java ProcessBuilder on Agent

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/c7Z3OFLyzx0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr14EyUPyj_UL-KuUTPR1Mewr%2B6tXGX67kya9bw-QQNGTQ%40mail.gmail.com.

[Jesse Glick]

On Mon, Nov 16, 2020 at 10:30 AM Tal Yanai <t...@yanai.org.il> wrote:
> I'm having issues getting back the stdout for parsing using Launcher.

https://javadoc.jenkins.io/hudson/Proc.html#getStdout-- works.

[Jesse Glick]

On Mon, Nov 16, 2020 at 11:17 AM Jesse Glick <jgl...@cloudbees.com> wrote:
> https://javadoc.jenkins.io/hudson/Proc.html#getStdout-- works.

…if you actually need to _parse_ stdout. If you merely want to mirror
it to the build log, just use

https://javadoc.jenkins.io/hudson/Launcher.ProcStarter.html#stdout-hudson.model.TaskListener-

[Tal Yanai]

Launcher is not Serializable.

-----Original Message-----
From: jenkin...@googlegroups.com <jenkin...@googlegroups.com> On Behalf Of Jesse Glick
Sent: יום ב 16 נובמבר 2020 18:18
To: Jenkins Dev <jenkin...@googlegroups.com>
Subject: Re: Running Java ProcessBuilder on Agent

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/c7Z3OFLyzx0/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr2uaCfcr%3DOBpoMiq73Rp2nMkkdT3Tn_%3DKqy0fKVuofsKQ%40mail.gmail.com.

[Jesse Glick]

On Mon, Nov 16, 2020 at 1:03 PM Tal Yanai <t...@yanai.org.il> wrote:
> Launcher is not Serializable.

Correct. You run it from the controller side. You do not need to use
any `MasterToSlaveCallable`.