Transferring files from node to node

[Gin]

Hello Everyone,

I want to share a file across several nodes using exported resources. The example configuration that I use looks like this: http://pastebin.com/ZMEbBAW4

I've tried both, with source = file://XXX and source = XXX.

I have a /tmp/input file on the "source_node" and expect it to be recreated as /tmp/output on the "target_node".

The operation consists of 2 steps. 

• Firstly I run "sudo puppet agent -td" on the "source_node". 
  
• The only file-related evidence is two uninformative lines in puppetdb.log with a few md5 entries.
• Then I trigger the "target_node" and get such an output:
  
• "Error: /Stage[main]/Main/Node[source_node]/File[/tmp/output]: Could not evaluate: Could not retrieve information from environment production source(s) /tmp/input"

This is a follow-up of https://ask.puppetlabs.com/question/14891/transferring-files-from-node-to-node/

I'm stuck with this for quite a while now hence thought to try my luck here.

Thanks,

Gin

[Felix Frank]

Hi,

I assume that you're hoping for the export to pick up file contents on the exporting agent node and distribute it to all collecting nodes, yes?

This cannot work. Exported resources are basically just key/value containers. The collecting catalog will just receive a resource with a `source` parameter that references a file that is local to the agent that will apply this new catalog.

To transfer data from an agent to others in this fashion, you will have to
1. Create a custom fact that takes the desired file content as its value.
2. Export a file resource that passes this value to the agents through the `content` attribute

Your collectors will work just as expected then.

Please note that the whole file content needs to be kept in PuppetDB, and is included verbatim in each involved catalog. For large files, this can be rather expensive.

HTH,
Felix

[Gin]

Thanks, Felix, for clarifying.

As far as I know, facts are collected *before* the execution.

therefore the following workflow will not work: 1. generate certificates 2. distribute them across the odes (including their contents).

Any workaround for this?

One idea I could come up is to export the file only if it exists and is not empty. 

That leads to a double run. On the first run the certificates would be generated. Then, on the second run the facter would be able to pick up the file contents.

Cheers,

Gin

[Felix Frank]

On 12/29/2014 06:40 PM, Gin wrote:
> As far as I know, facts are collected *before* the execution.
> therefore the following workflow will not work: 1. generate
> certificates 2. distribute them across the odes (including their
> contents).
>
> Any workaround for this?
> One idea I could come up is to export the file only if it exists and
> is not empty.
> That leads to a double run. On the first run the certificates would be
> generated. Then, on the second run the facter would be able to pick up
> the file contents.

Hi,

well, yes - if the information that you want to export comes available
*during* a Puppet run, you cannot export it until the next run. This
cannot be worked around, because the compiling phase is disjoint from
the catalog application phase, and exporting happens only during the former.

How is the source file generated? If the master creates it, say, using a
template, it would be much easier to export the target file with a
`content` attribute that carries the expanded template to all receiving
agents.

HTH,
Felix

[Gin]

Unfortunately, certificates are generated with exec type (shell script).

As far as I know, there is no way to cat file contents into a puppet variable, hence the only option is to do two runs.

Thank you, Felix!

Cheers,

Gin