Writing from Slave To Master

[Tal Yanai]

Hello,

Trying to write a code that let me dump new file with content over to the master workspace (while running on a remote Node).

2 things I'm struggling with:

1.  How to find out the Master 'FilePath' while running within the Node.

2.  How to make sure the code is written such that it will not require the administrator to change the master security policy (to accept the Node writing over into it's directoris)

I have the following code I made for this function:

public void writeToMaster(FilePath workspace, OutputStream fileOutput ) throws IOException, InterruptedException {

        try (final OutputStream out = new RemoteOutputStream(fileOutput)) {

            workspace.act(new Callable<Void,IOException>() {

                @Override

                public Void call() throws IOException {

                    // agent is just writing to a pipe to the master. Quite safe

                    out.flush();

                    out.close();

                   return null;

                }

                @Override

                public void checkRoles(RoleChecker checker) throws SecurityException {

                    throw new UnsupportedOperationException("Not supported yet."); //To change body of generated methods, choose Tools | Templates.

                }

            });

        }

    }

I want to think this address my second concern, but still I don't know how to extract the Master's PathFile from the Run class (or otherwise).

Thanks,

Tal.

[Daniel Beck]

> On 11. Nov 2020, at 12:58, Tal Yanai <t...@yanai.org.il> wrote:
>
> Trying to write a code that let me dump new file with content over to the master workspace (while running on a remote Node).
>
> 2 things I'm struggling with:
>
> 1. How to find out the Master 'FilePath' while running within the Node.
> 2. How to make sure the code is written such that it will not require the administrator to change the master security policy (to accept the Node writing over into it's directoris)
>

This looks like a potential XY problem. What are you really trying to accomplish?

It should be possible to restructure your code to make the controller read from the agent, rather than the agent writing to the controller.

[Tal Yanai]

Hi Daniel,

There are two things I'm trying, both are related for the ability to show data within Jenkins after the build is over. I have a dashboard plugin and another plugin within the job itself that also shows a report for that job. Both of these two web capabilities relies on an artifact that is being retrieved by two another plugins, one is a step (and pipeline as well) and one is a post build plugin (and pipeline as well).

I was thinking on copying over the artifacts to the master's location as soon as they are ready within the job itself. Are you saying I can pull them inside the master instead?

Thanks,

Tal.

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/I0WuyxdA9Nc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/9705606D-9A93-4B3E-AA76-ADF78277E1F2%40beckweb.net.

[Ullrich Hafner]

Am 11.11.2020 um 14:25 schrieb Tal Yanai <t...@yanai.org.il>:

Hi Daniel,

There are two things I'm trying, both are related for the ability to show data within Jenkins after the build is over.  I have a dashboard plugin and another plugin within the job itself that also shows a report for that job.  Both of these two web capabilities relies on an artifact that is being retrieved by two another plugins, one is a step (and pipeline as well) and  one is a post build plugin (and pipeline as well).

I was thinking on copying over the artifacts to the master's location as soon as they are ready within the job itself. Are you saying I can pull them inside the master instead?

Yes. 

Do you already have a publisher that can do the job?

Thanks,

Tal.

-----Original Message-----
From: jenkin...@googlegroups.com <jenkin...@googlegroups.com> On Behalf Of Daniel Beck
Sent: יום ד 11 נובמבר 2020 14:14
To: Jenkins Developers <jenkin...@googlegroups.com>
Subject: Re: Writing from Slave To Master

On 11. Nov 2020, at 12:58, Tal Yanai <t...@yanai.org.il> wrote:

Trying to write a code that let me dump new file with content over to the master workspace (while running on a remote Node).

2 things I'm struggling with:

1.  How to find out the Master 'FilePath' while running within the Node.
2.  How to make sure the code is written such that it will not require the administrator to change the master security policy (to accept the Node writing over into it's directoris)

This looks like a potential XY problem. What are you really trying to accomplish?

It should be possible to restructure your code to make the controller read from the agent, rather than the agent writing to the controller.

-- 
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/I0WuyxdA9Nc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/9705606D-9A93-4B3E-AA76-ADF78277E1F2%40beckweb.net.

-- 

You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/001c01d6b82e%2424f8a650%246ee9f2f0%24%40yanai.org.il.

[Tal Yanai]

I suppose.

 

I have a publisher that is responsible for creating the report during the post job step.  It extends Recorder and implements SimpleBuildStep, Serializable.

 

In its perform function I have this:

 

@Override

    public void perform(@Nonnull Run<?, ?> run, @Nonnull FilePath fp, @Nonnull Launcher lnchr, @Nonnull TaskListener tl) throws InterruptedException, IOException {

 

        this.build = run;

        DSLBuildAction buildAction = new DSLBuildAction("NA", run);

        run.addAction(buildAction);

 

        if (advancedFunctions) {

            if (retrieveSummaryReport) {

                summaryReportParams.noneSharedNFS = false;

                ReportBuildAction reportAction = new ReportBuildAction(run, summaryReportParams, vAPIConnectionParam, tl,run.getExecutor().getCurrentWorkspace());  //This line creates the report on the Node’s disk space

                run.addAction(reportAction);

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/EA913530-C672-4F36-836B-2E8497C17EF2%40gmail.com.

[Jesse Glick]

On Wed, Nov 11, 2020 at 11:24 AM Tal Yanai <t...@yanai.org.il> wrote:
> ReportBuildAction reportAction = new ReportBuildAction(run, summaryReportParams, vAPIConnectionParam, tl,run.getExecutor().getCurrentWorkspace());

Use the `fp` parameter you were given, not
`run.executor.currentWorkspace`, and just use `FilePath` methods to
copy stuff from the agent to the controller.

[Tal Yanai]

Thanks,

I managed to make it working when both master and slave are running on Windows. However, when Master is on Windows and Slave on Linux, I can't see the files being transferred over.

This is my code:

String buildDir = build.getRootDir().getAbsolutePath();
FilePath masterDirectory = new FilePath(new File(buildDir));
launcher.getChannel().call(new vManagerWriteToSlave(content, new FilePath(masterDirectory, fileName)));


public class vManagerWriteToSlave extends MasterToSlaveCallable<Void, IOException> {

private String message;
private FilePath outputFile;

public vManagerWriteToSlave(String message, FilePath outputFile) {
this.message = message;
this.outputFile = outputFile;

}

@Override
public Void call() throws IOException {

SimpleVManagerIO sio = new SimpleVManagerIO();
sio.writeToFile(message,outputFile.getRemote());
return null;

}
}

-----Original Message-----
From: jenkin...@googlegroups.com <jenkin...@googlegroups.com> On Behalf Of Jesse Glick
Sent: יום ה 12 נובמבר 2020 20:16
To: Jenkins Dev <jenkin...@googlegroups.com>
Subject: Re: Writing from Slave To Master

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/I0WuyxdA9Nc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr1y5F2iM4ogWCuWtwDKR-uGWFh%2BxFw9Hy42drgWsdisCA%40mail.gmail.com.

[Tal Yanai]

I think I got it:

String buildDir = build.getRootDir().getAbsolutePath();
FilePath masterDirectory = new FilePath(new File(buildDir + File.separator + fileName));
this.filePath.list(fileName)[0].copyTo(masterDirectory);

Thanks,

Tal.

-----Original Message-----
From: jenkin...@googlegroups.com <jenkin...@googlegroups.com> On Behalf Of Jesse Glick
Sent: יום ה 12 נובמבר 2020 20:16

To: Jenkins Dev <jenkin...@googlegroups.com>
Subject: Re: Writing from Slave To Master

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/I0WuyxdA9Nc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr1y5F2iM4ogWCuWtwDKR-uGWFh%2BxFw9Hy42drgWsdisCA%40mail.gmail.com.

[Jesse Glick]

On Thu, Nov 12, 2020 at 4:41 PM Tal Yanai <t...@yanai.org.il> wrote:
> FilePath masterDirectory = new FilePath(new File(buildDir + File.separator + fileName));

Just be careful. If `fileName` is user-controlled, this is an easy
attack vector (pass `../../../secrets/master.key` for example).

[Tal Yanai]

Thanks,

No risk, the plugin creates these files in the workspace hardcoded, an than copy them over to the master.

-----Original Message-----
From: jenkin...@googlegroups.com <jenkin...@googlegroups.com> On Behalf Of Jesse Glick
Sent: יום ו 13 נובמבר 2020 00:03
To: Jenkins Dev <jenkin...@googlegroups.com>
Subject: Re: Writing from Slave To Master

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/I0WuyxdA9Nc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr2iBozf8BskdL-nvkq_ACT1r0wJ9N9MQre6hKc6qKDDnw%40mail.gmail.com.

[Daniel Beck]

> On 12. Nov 2020, at 22:40, Tal Yanai <t...@yanai.org.il> wrote:
>
> String buildDir = build.getRootDir().getAbsolutePath();
> FilePath masterDirectory = new FilePath(new File(buildDir + File.separator + fileName));

As all paths here are on the master, is this the same as the following?

FilePath masterDirectory = build.getRootDir().child(fileName)

Would still not recommend storing files directly in the build dir; most plugins create subdirectories and store their build data there. Otherwise your files may collide with others; and if user controlled file names are involved, even without path traversal, replace build.xml etc.

[Tal Yanai]

Thanks,

All these files have the build_number + build_id as part of their names.

-----Original Message-----
From: jenkin...@googlegroups.com <jenkin...@googlegroups.com> On Behalf Of Daniel Beck
Sent: יום ו 13 נובמבר 2020 00:28
To: Jenkins Developers <jenkin...@googlegroups.com>
Subject: Re: Writing from Slave To Master

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/I0WuyxdA9Nc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/E5C82C3F-99A4-4557-BBD4-E534658AF557%40beckweb.net.

[Jesse Glick]

On Thu, Nov 12, 2020 at 5:35 PM Tal Yanai <t...@yanai.org.il> wrote:
> All these files have the build_number + build_id as part of their names.

Then you can predict the filename, and must not call `FilePath.list`.
You cannot trust anything that happens on the agent side.

[Tal Yanai]

Using FilePath.child?

--
You received this message because you are subscribed to a topic in the Google Groups "Jenkins Developers" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-dev/I0WuyxdA9Nc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to jenkinsci-de...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr1t7cJKvZXCdEE8isN7x%2BFs9bjMxsBmWRedx20tKgLKig%40mail.gmail.com.

--

Thanks, Tal.

[Jesse Glick]

On Fri, Nov 13, 2020 at 3:12 PM Tal Yanai <t...@yanai.org.il> wrote:
> Using FilePath.child?

Yes.

[Gavin Mogan]

Also remember the new terms.

Slave was renamed to agent many years ago

Master was renamed to controller (I have trouble remembering this one but getting there)

--

You received this message because you are subscribed to the Google Groups "Jenkins Developers" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-de...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-dev/CANfRfr1b0%3DTj2Pp%2Bv9C97%2B8VW_Kkm5ieCrWzO4af4yv3ihnTuw%40mail.gmail.com.