Package Module Materialization

[Bryan Burke]

Greetings,

I'm updating a small installation I use from 3.12 => 3.18 (also CentOS 7 => 8-stream), and I had some trouble with the package modules.

They used to be packaged in masterfiles directly, but it looks like a few months ago, if my searching taught me anything, they were converted to mustache templates. Try as I might, however, I was unable to find where in the policies/updates the package modules got materialized. I'm assuming this is done for you somehow during bootstrap or update, but I couldn't figure it out.

Mitigation: since it doesn't look like they actually have any interpolations in them, for now, I just copied the one I needed (yum) into place and distribute that, but I assume this will break at some point.

Question: I want to make sure I'm materializing these correctly, so can someone advise me on how to do that?

Context: why am I trying to materialize them directly? Well, I've got a pretty custom setup in Google Cloud. Cloud Storage is my distribution method, with service account credentials for authentication (more below, if you're interested). So, I've got a completely custom update.cf/bootstrap.cf that can authenticate and speak to Google Cloud Storage, do some basic setup, and install the latest version of my policy files.

The policy files themselves are also from scratch (honestly I forgot why I did that at this point, but there was a reason, probably related to not having a policy server). That is, I define my own body common control/bundlesequence/inputs.

Secondary question: it looks like the update mechanism is a lot more flexible these days, and I'm wondering if I really need to do what I'm doing anymore. Is that possible, and are there any pointers to doing so?

Additional context: I didn't want to run a policy server (costs $$$, and trying to make that accessible to different networks I manage, either with VPC peering or making the policy server public, were not desireable), so I made a custom distribution system.

Thanks for any help you can provide! :)

Bryan

[Nick Anderson]

Is this the policy your looking for?

https://github.com/cfengine/masterfiles/search?q=modules_presence

--
You received this message because you are subscribed to the Google Groups "help-cfengine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to help-cfengin...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/help-cfengine/1952fc38-63cb-42c4-8279-1a1ac940616en%40googlegroups.com.

[Bryan Burke]

Thank you, Nick! Yes, this appears to be what I was looking for; not sure why I couldn't find it before.

As for the second question: looking at that file, I get some hints on how to hook in my own update mechanism. Seems that if it's defined, it is used completely in place of the normal update mechanism? Still not sure how well the whole thing would work without a policy_hub defined, but I can maybe try it out sometime soon. Any docs that describe this, or should I just be looking at the masterfiles source?

Bryan

[Nick Anderson]

Bryan Burke <bbu...@baburke.net> writes:

Thank you, Nick! Yes, this appears to be what I was looking for; not sure why I couldn't find it before.

I added nodes for modules to the rendered MPF docs.

If you would like to contribute to the modules docs please make PRs here.

As for the second question: "it looks like the update mechanism is a lot more flexible these days, and I'm wondering if I really need to do what I'm doing anymore. Is that possible, and are there any pointers to doing so?"

looking at that file, I get some hints on how to hook in my own update mechanism. Seems that if it's defined, it is used completely in place of the normal update mechanism? Still not sure how well the whole thing would work without a policy_hub defined, but I can maybe try it out sometime soon. Any docs that describe this, or should I just be looking at the masterfiles source?

Yes, we have really tried to make it easy to hook into the MPF and change it's behavior without having to resort to modifying the vendored policy. Instrumenting a custom update policy is pretty straight forward, if you have some suggestions for where docs for this need to be improved feel free to open a pr or at least share a link to where in the docs and then some suggested verbiage so someone else can get it added.

You can define your own update mechanism by:

• Adding your custom update policy to update inputs
  • From the root of the MPF docs in the Reference manual: Append to inputs used by update policy
• Setting the name of the bundle to use for policy update
  • From the root of the MPF docs in the Reference manual: Specify the agent bundle used for policy update

That should be all you need to do.

[Bryan Burke]

Awesome, thanks for all the information! Everything is currently working for me so there's no rush for me to change it, but I plan to move more hosts to this in the near future and may revisit. I'll reference this information, and if I have any suggestions, etc. I'll submit a PR.

Cheers,

Bryan

--
Nick Anderson | Doer of Things | (+1) 785-550-1767 | https://northern.tech

[Nick Anderson]

\( ﾟ◡ﾟ)/ https://cfengine.com/blog/2022/example-custom-policy-update-cfbs-module/