Using Salt to update TemplateVMs

[Andrew David Wong]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

1. When using the Qubes Update widget, a mgmt DisposableVM is started.
Why is that? Is it just for executing Salt commands so that they're
not executed in dom0?

2. How can one update a TemplateVM the way the Qubes Update widget does?
For example, when I update a Fedora TemplateVM myself, I just execute
`dnf update` in the template. I don't start any DisposableVMs, so
clearly my method of updating is different from what the Qubes Update
widget does. Is there some kind of scriptable qubesctl command I can
issue from dom0 that does the same thing as the Qubes Update widget?

- --
Andrew David Wong (Axon)
Community Manager, Qubes OS
https://www.qubes-os.org

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEZQ7rCYX0j3henGH1203TvDlQMDAFAl0qa8sACgkQ203TvDlQ
MDDEgxAAtTn7AdUjlxM/e5I+JZO5qwAZy3f8D3motU15GWOIqc2DCiCX+lbxpiru
S/H+DtAZ4Fh7rh+thdEGbJtIxosTl+2C4tKx3a0U7zxLcm1CWs4kkE8PAYScmhPK
j3nhLfbcHixegURFVKl2Qg0Tp5NdhJIESa7FVUjtWUpBAO1Glz3nbjSyfsPxZ78f
ow400/9O9dC8cQnj+AcapJZkG07XNnRsYCiP7CZJAI38jZjuMH+p+vAwPTWeFJ9Z
te263p39+Apj3WwoeO5zC511Kt79OGhWHGvo1I85FvSrLJosvH9QttFcBumuAjNF
0sE/I7jdEuYD3dj5qZsD96PJ2kFHERfLfTa6Gz6ONIBzgd4CwAdwIc+XV7enP/q1
5xultiA3bJ56V9zfJYGTiUDiocQJ/ISrTWi0g84hl2jSzp8olGk8bQTI+10cy9Z8
q12hYwxtQftmeF0HFfeXHjREf6HVdDv5kqMQWfQrd8p3St56MqV4MIrKV2STNJW3
6CkZ6X/wfq5pe1P2uk8oEeC+XjfSiTYQZhNFNQKRFZfaKqqk2QbR6hVjVyIpLDEq
f2fKhUVzgi8tPeXWnn9lF7eWDMiZbmsl1X+duu7gtNWcBPAd/+Kd1nwD95vvI1Vs
USugKd2yEfFmJs6GumWgFXO1T/KGm0wb6ItdZuw5k028YWtuvdE=
=RjqF
-----END PGP SIGNATURE-----

[unman]

On Sat, Jul 13, 2019 at 06:40:00PM -0500, Andrew David Wong wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
>
> 1. When using the Qubes Update widget, a mgmt DisposableVM is started.
> Why is that? Is it just for executing Salt commands so that they're
> not executed in dom0?

Yes, this is standard in Qubes.

>
> 2. How can one update a TemplateVM the way the Qubes Update widget does?
> For example, when I update a Fedora TemplateVM myself, I just execute
> `dnf update` in the template. I don't start any DisposableVMs, so
> clearly my method of updating is different from what the Qubes Update
> widget does. Is there some kind of scriptable qubesctl command I can
> issue from dom0 that does the same thing as the Qubes Update widget?
>

The update widget calls qubesctl and runs the state file in
/srv/formuals/base/update-formula/update/qubes-vm.sls

You can run this yourself by:
qubesctl --skip-dom0 --targets=<targets> --show-output state.sls update.qubes-vm

Skip the "show-output" option if you want to script.

It's a wrapper to salts pkg.uptodate call, so you could put that in a
state file yourself.

[Andrew David Wong]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Thanks, unman. I'm not quite sure what the last sentence means. Why
would one want to put that in a state file oneself?

- --
Andrew David Wong (Axon)
Community Manager, Qubes OS
https://www.qubes-os.org

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEZQ7rCYX0j3henGH1203TvDlQMDAFAl0r4A8ACgkQ203TvDlQ
MDBOqxAAzI9YGaQJmxMZDOKDajwCLUjftm91R+cEEKKSf+0xVc8TL0muZOnYuT7u
Oo70ZY5S1b9XMi2r2RQo6VsqZeeCjT/06P3cfQY9IXmtMT6NteXtGN9SsDKz6PBZ
OMUtYrJTgUlBeqaWIL4XjrIQClmnhP4UXDPfB02bFlE/1B+JOaArRskl5qb7OTfl
n5utGepbJ5uyiPjlkA+Bbng2KkLAL3h0HJCQvMqVbwSUWT0VemPFlDpwl2amqr1B
/xrrdkAG6npiT3Z2235bbhCxydxJxOghvFQgFfpdGA/sKiJhju23QRpwmXB5WUEu
VVEOGpgVVopfzlLEWNRJBDjbFn3BDkQbMvvfPbJCpmYRVbC6xRerVw+AhQfIs1Un
ZqWbZSYFdpN3PZ7Cv1g8/YPZYXjz5NL90p8mxfGQ2wHAudKTCLXIvVoyMRSv/93H
EAtdPBJFWFI6LXxinaKrEc3aFz5V9c5ff2Zb4fGxSpMDw6ZL5G1bDoQcIE3KwFxx
IQGAvipdZwFjYrIbOJrKKHX2dckc9h0PL5UsI+C8Nm5SzWiUm13vFL8aABdXkC4Z
M7Q+eYnIo/4XcvzJGHkftpGOot22r8Odfcjh2JmfshCU01JH9xb/CRd+wXQTm3VH
h49czYZOT0q5WYWF4SwunDT7CUHigCk3vsMjIjuMwHwiaDyA8vg=
=MZtR
-----END PGP SIGNATURE-----

[Andrew David Wong]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 14/07/2019 9.08 PM, Andrew David Wong wrote:
> On 14/07/2019 8.19 AM, unman wrote:
>> On Sat, Jul 13, 2019 at 06:40:00PM -0500, Andrew David Wong
>> wrote:
>>>
>>> 1. When using the Qubes Update widget, a mgmt DisposableVM is
>>> started. Why is that? Is it just for executing Salt commands so
>>> that they're not executed in dom0?
>
>> Yes, this is standard in Qubes.
>
>>>
>>> 2. How can one update a TemplateVM the way the Qubes Update
>>> widget does? For example, when I update a Fedora TemplateVM
>>> myself, I just execute `dnf update` in the template. I don't
>>> start any DisposableVMs, so clearly my method of updating is
>>> different from what the Qubes Update widget does. Is there some
>>> kind of scriptable qubesctl command I can issue from dom0 that
>>> does the same thing as the Qubes Update widget?
>>>
>
>> The update widget calls qubesctl and runs the state file in
>> /srv/formuals/base/update-formula/update/qubes-vm.sls
>
>> You can run this yourself by: qubesctl --skip-dom0
>> --targets=<targets> --show-output state.sls update.qubes-vm
>
>> Skip the "show-output" option if you want to script.
>
>> It's a wrapper to salts pkg.uptodate call, so you could put that
>> in a state file yourself.
>
>
> Thanks, unman. I'm not quite sure what the last sentence means.
> Why would one want to put that in a state file oneself?
>

Could you explain what these options mean?

--skip-dom0 -- The documentation doesn't really explain this.
--targets -- Is this the qube to be updated in this case?

The reason I'm asking: I've just been updating via `dnf update` (and
similar) for a long time now, but I'm noticing that certain bug fixes
are being implemented via Salt, and I'm worried that I might skip
these fixes if I never update via Salt. Do you think that updating via
qubesctl is a better idea than updating "manually," or does it not
matter?

- --
Andrew David Wong (Axon)
Community Manager, Qubes OS
https://www.qubes-os.org

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEZQ7rCYX0j3henGH1203TvDlQMDAFAl0r4TgACgkQ203TvDlQ
MDBq0w/8DmHxyfuwQXgW7jOOhkbeK3HbO0L0Yw9samgGe14kmFUy6XBag9Ek3uNU
U7qDigdon4hALBLF0SHXW8csw/nnFv6y9DlARVwLvjeyZJETALWd4LYK5v4GndC2
2Smrgj5sZNvLgNhkaSfFuAqWw1NC1z2lYMtroDMiy9dHdKCJn0wfOfS4IZN24pqm
XU9yiDS/QdYjETX2Z0lNCxfskBGzqrtXNSdLC5bUH96/6lY+Yqxcx4NCVpxtJZsB
U/WO7eMh12h9GVDtwxQl7LCo9eBiQhj5QdCp40Zx42BBeCT4nAcu2MbvmXVMRQnh
9JQNMtu76zZ35TqEnp5pbzGhsLpo7d9RMYjPT/wCJWwZsAQpOYyDyMzIGN9bXz/8
9N5A0mXWcey/5AFS+kvgnJMpBLgh96hWHNFJK++G4+3LFDWMWCdbxMyBOAttKEv9
ea9SGiDYHHeVmoy5E2GzPYMQEkz3voYWR7BvRs8ilCMaAETdtorYyAy59s8Y4t0P
1zT6IFgnD48p9pjLBUxbkegsCWPbCnFKBWUBLTaR8lMYbykvCkdEkT43A1+FMVDJ
4yuUeIwDcPElzlDZzHh7UuBgkJHrcglfEobsWCPgXwI3Cr56eBwq90Sm5iX/e2iL
J9x9tiiYZ/DhZaAr0uyhv8IZJkXLHY2Y6c3uVA58RQ+xtAxN4mE=
=VhP6
-----END PGP SIGNATURE-----

[unman]

--skip-dom0 -- Doesnt try to action state in dom0.

--targets -- You can give list of qubes to use as targets, (comma
delimited) or use keywords. 'qubesctl --templates'.

> The reason I'm asking: I've just been updating via `dnf update` (and
> similar) for a long time now, but I'm noticing that certain bug fixes
> are being implemented via Salt, and I'm worried that I might skip
> these fixes if I never update via Salt. Do you think that updating via
> qubesctl is a better idea than updating "manually," or does it not
> matter?

I really do recommend using qubesctl for almost all system
configuration. If only because it makes recovery so much easier.
I see people saying "keep a list of packages you've installed" - if you
keep state and use salt you can rebuild your system (almost) completely
automatically.

I think there are some cases where a configuration fix may be pushed via
salt, but in most I would expect changes to be incorporated in to an
updated package, so you would get those using a manual update.

[brenda...@gmail.com]

On Tuesday, July 16, 2019 at 10:35:11 AM UTC-4, unman wrote:
> I really do recommend using qubesctl for almost all system
> configuration. If only because it makes recovery so much easier.
> I see people saying "keep a list of packages you've installed" - if you
> keep state and use salt you can rebuild your system (almost) completely
> automatically.

Do you happen to have some example "personalized" salt scripts you use (or a pointer to where someone has posted some)?

I was planning to put together some bash scripts to push configuration into my templates (90% repo adjustments and specific packages to download), but your comment above is intriguing.

B

[Qux Bar]

Hi,

I was also interested and found some documentation on the Qubes website: https://www.qubes-os.org/doc/salt/

Hope it helps!

++

--
You received this message because you are subscribed to the Google Groups "qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qubes-users...@googlegroups.com.
To post to this group, send email to qubes...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/qubes-users/46f4a28d-fe95-4ce3-abad-162ccd8d5a4f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Kushal Das]

There is also https://qubes-ansible.readthedocs.io/en/latest/ if you
like Ansible.

Kushal
--
Public Interest Technologist, Freedom of the Press Foundation
CPython Core Developer
Director, Python Software Foundation
https://kushaldas.in

[Johannes Graumann]

On Wed, 2019-07-17 at 18:56 +0530, Kushal Das wrote:
> On Tue, Jul 16, 2019 at 11:26 PM <brenda...@gmail.com> wrote:
> > On Tuesday, July 16, 2019 at 10:35:11 AM UTC-4, unman wrote:
> > > I really do recommend using qubesctl for almost all system
> > > configuration. If only because it makes recovery so much easier.
> > > I see people saying "keep a list of packages you've installed" -
> > > if you
> > > keep state and use salt you can rebuild your system (almost)
> > > completely
> > > automatically.
> >
> > Do you happen to have some example "personalized" salt scripts you
> > use (or a pointer to where someone has posted some)?
> >
> > I was planning to put together some bash scripts to push
> > configuration into my templates (90% repo adjustments and specific
> > packages to download), but your comment above is intriguing.
> >
> There is also https://qubes-ansible.readthedocs.io/en/latest/ if you
> like Ansible.

What's the relationship/comparison to
https://github.com/Rudd-O/ansible-qubes?

Sincerely, Joh

[Kushal Das]

https://qubes-ansible.readthedocs.io/en/latest/ is a pure Python implementation
and does not use Salt anywhere. Also, the plugin is already merged in upstream
Ansible project.

[Johannes Graumann]

That is cool. Is it using the same security mechanisms that salt in
Qubes utilizes (like executing in a dispVM)?

Sincerely, Joh

[unman]

I cant post my files, but I've put up an example which shows how to
create a qube for building Qubes.
Here: - https://github.com/unman/notes/tree/master/config/build

There are some notes I used in training which are a very basic
hands on intro to salt in Qubes:
https://github.com/unman/notes/tree/master/salt

In the build example,you'll see:
1. Create.sls - Create a new qube: installing fedora-30-minimal if not already
there, cloning to new template, using new template to create qube,
configure the new qube, and configure dom0.
2. install.sls - installs required software in template.
3. config.sls - Configures new qube as needed.

I've broken this down to make it as clear as possible, and kept it
simple.
You could run each section like:
qubesctl state.sls build.create
qubesctl --skip-dom0 --targets=template-builder state.sls build.install
qubesctl --skip-dom0 --targets=builder state.sls build.config

Of course, you can do everything here using scripting. But for some
things, (like targeting packages and configuration at distro and version),
salt is somewhat easier.

unman

[Andrew David Wong]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Thanks, unman. This is helpful, but I'm still unclear on --skip-dom0.

Looking at the command you provided:

qubesctl --skip-dom0 --targets=<targets> --show-output state.sls update.qubes-vm

What is the difference between including --skip-dom0 and excluding that
option?

It's counterintuitive to me that, if I *specify* a target to be acted
upon using --targets, I *also* have to specify *not* to take action on a
*different* target (namely dom0).

>> The reason I'm asking: I've just been updating via `dnf update` (and
>> similar) for a long time now, but I'm noticing that certain bug fixes
>> are being implemented via Salt, and I'm worried that I might skip
>> these fixes if I never update via Salt. Do you think that updating via
>> qubesctl is a better idea than updating "manually," or does it not
>> matter?
>
> I really do recommend using qubesctl for almost all system
> configuration. If only because it makes recovery so much easier.
> I see people saying "keep a list of packages you've installed" - if you
> keep state and use salt you can rebuild your system (almost) completely
> automatically.
>
> I think there are some cases where a configuration fix may be pushed via
> salt, but in most I would expect changes to be incorporated in to an
> updated package, so you would get those using a manual update.
>

Good to know. Thank you.

- --
Andrew David Wong (Axon)
Community Manager, Qubes OS
https://www.qubes-os.org

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEZQ7rCYX0j3henGH1203TvDlQMDAFAl000X0ACgkQ203TvDlQ
MDD+4RAAjYpeinjlq4LRr6x6TlWW+e0fjB9XAQhlMUJrA4op1AzABy5csN7L5bHr
n5+z78CbWMkHQA9LmNHa49Es939jCItk+RH8PEyVSTSsGWvZHp8dX+9kaoIbQ8cH
5ihNUIXwlglKzeKbGTTE2DgI2S6GY3pr8aO2VwOMXilygIhsWE2cygR2DI2jXlme
haWIF/soHxrqGUHiXvnbhd2nwTI4Mt0RCjPct1R5L5Hq4cSHhDxioRke4htFl1zB
vXRfe2NpJ0i2gUKesKqo55qo3NqKS19H9XFiHuBxasybFuXJyqKT8uhV05nwlCvb
IjRgoPxzC3X6hilA0zYMQsWc7RxcZ0VfYArYuwaI0nwCLDUC8RxhtmzTj261If0i
/hcHSvj2QpNtxfNnavF9ck8+8R9bn+MJdNLCEJj4BWtqwOhddOMhTwkcE9goziwx
Tw8EusMQvePdBk1Bf7qqzPFLHA6P8Lr775ZRaZq+k7j8afW/VGLRH7J6Db/f0Se+
YxjWI/SoW6Mzjy8UJXg8e6vyImriBk99tDypNf2RA1QYbCIINiVRBHfnFhrDFBcX
EPrCRrvDIQTyewQIx3X/uZxYjnZ7wN2xHv4rYcx7y5kCr71SVJkt6xO4qIdD3nQW
33K44GsxJ9FgAtg3dhP/PYmE8ShPtHD0VKTZRPOj8XlviWP5LtM=
=aoC0
-----END PGP SIGNATURE-----

[unman]

Depending on how you structure your states, and interdependencies, this
is actually quite useful.
For example, you can create a qube, configured as you wish, and then
manipulate the files on that qube. Including dom0 allows you to do
equivalent of qvm-create and qvm-prefs.
If you subsequently change the preferences, you don't want to run the same
state against dom0, because that would revert the changes you have made,
so you can specify --skip-dom0 but apply the same state in the qube
filesystem.
For quick and dirty work that's fine.

[Andrew David Wong]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Sorry, I'm not following your example. What does it mean to "do the
equivalent of qvm-create and qvm-prefs"? I had no idea you could create
the effects of those commands without those commands. This sounds like a
rather arcane area of Qubes wizardry into which I have not ventured.

- --
Andrew David Wong (Axon)
Community Manager, Qubes OS
https://www.qubes-os.org

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEZQ7rCYX0j3henGH1203TvDlQMDAFAl02VvkACgkQ203TvDlQ
MDDKFg//WmbUD+TwUf6z0MxAVwV1UbrJapCL0KD2TuD0jW2KAtwGAVe24+3BsTOt
gK2Gufuct2cWZ19Ljg2HQu/ZVnEd7ndq60IcIFDyJ7dCwzSjFZeeiQ4hDNc0xWlY
nR4HID+tdDg52Ao7HTHK8fmuBB4607/JjHxeg95N4LqvFLkaCgm9rSwv8hT+L9+y
weGgV/xTdlJ+S1Nzgpp68ZRC+xQ7A/dYY0IJAqKhqoPF/zxSTSrgrmR/NjNnOry9
WnWrmF38rluHrlDRZZIyvTCPmuDKMJ7KkRx0CB0VvE5t7iibwyxU64RRKhyD5NTF
QXexsL/OAay7I2V78HiZ/y+y9AL2nYAd6drq7DCZYbwnWw0LQzwpH/qRJJH/j9Ix
2ED9kUY+ojUBmg77hSUz7svf6XETG/RtM92AORgVB2eDgi4KAuZkOR2NbVTUW/q6
4DrpKdO22Hag0Z87MKokTGPF8N837TtPd6471Tqucgv0TDkPgar5fe1l2L6y5904
FvY83Cv2GuqXJVZXLYpNaF+5UjmWHYmRFzsq2NGR+iMpaJlea4nOHJytMbZgBRQf
/cGeOjLN+vXzOMNZSHvkvx8QYEs7Iksn02rT1/4dmXBaVmDIZRzZ7zx6fqJjfKMD
qAxSNSpHigK+GASbZ4R48rlfNxxzeD8JtTl7LChOa2dnwRgz+Uk=
=lCnl
-----END PGP SIGNATURE-----

[Johannes Graumann]

Can you comment of whether the ansible implementation is using (or can
be made to do so) something equivalent to what is described for salt in
https://github.com/QubesOS/qubes-issues/issues/1541#issuecomment-
187482786 :
1) For every VM managed by Salt (from dom0):
- Start target VM.
- Have dom0 to create DispVM.
- Send all the Salt configuration there.
- Grant it qubes.VMShell access to that selected VM only
- Run salt-ssh (over qrexec) from the DispVM, targeting that
single VM. Do not filter return channel there - so for example
all the grains will be available to salt-ssh during state
rendering.
- Collect output back to dom0 (success/failure flag, optionally
logging full output to some file)
- Destroy DispVM
- Shutdown target VM (opt-out? only when wasn't running at the
start?).

Joh

[unman]

On Mon, Jul 22, 2019 at 07:38:27PM -0500, Andrew David Wong wrote:
>
> Sorry, I'm not following your example. What does it mean to "do the
> equivalent of qvm-create and qvm-prefs"? I had no idea you could create
> the effects of those commands without those commands. This sounds like a
> rather arcane area of Qubes wizardry into which I have not ventured.
>

Actually you probably have ventured, but not realised it.
When you run the installer and configure the system, the various qubes
are created with salt.
The state files are in /srv/formulas/base/virtual-machines-formula/qvm
and you can inspect them there.
These files use templating.

For example the sys-net.sls file uses:
name: sys-net
present:
- label: red
prefs:
- netvm: ''
- virt_mode: hvm

etc etc

You could get the same effect by calling the qvm functions directly:

-----------------------
sys-net:
qvm.present:
-name: sys-net
-template: debian-10
-label: red

sys-net-prefs:
qvm.prefs:
-netvm: ''
-virt_mode: hvm
-----------------------

If you look at the example I linked to you can see a breakdown of how to
create a Qubes builder qube.
The steps are: clone template and create qube, install necessary software in
template,configure qube. I've left that as simple as I can.

By keeping these formulae, you can (re)create any aspects of your Qubes
system in a few commands.
Of course, you can do this using batch files calling qvm-create,
qvm-prefs, qvm-copy-to-vm, and assorted calls to qvm-run. imo the salt
formulae are clearer and easier to maintain.

Also salt handles very well cases where (e.g) package names differ
between distributions, or you want the configuration to change between
debian-9 and debian-10, or between individual qubes,

I'm happy to post simple examples if anyone wants to give a specification.

unman

[Andrew David Wong]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

This is very instructive! Thank you, unman!

- --
Andrew David Wong (Axon)
Community Manager, Qubes OS
https://www.qubes-os.org

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEZQ7rCYX0j3henGH1203TvDlQMDAFAl05KccACgkQ203TvDlQ
MDC7lw//aAODJ4O990eexosaf9fihzizjysgIlIhv/00gaROt40+7fHbhE/AWnOz
qhdzO6TCVp2NF6GFSlTWQDxNUErD2JuY7fitIPBFqeB50XR9MjtZKhPY6alGz9hY
cyVy0wMvk1hd3rHUuyPzARsP1rK5rpc7nSTKR+T1v9t+yjG8OFIFw9hVvgUX0MZ9
A6qYsHnVonNBNh1LQy2yEd5MjkZbQKzJUkpq3/Ip16qQOsSrfy0tBjmGHyQq4U2c
mfRQpobxWbMPJlgy0oqKQVPNe0fZQiOJWdXf2v5nFc64olNF01oBjFdGQxdrbnRS
ytP0tndlDQ5fhItoJ993wP+vrkYopXgbKSUYnXDIxgtP07bxaEzIrkJq2ay5pmCG
xxKtEhidKBgjUmV9bjpV50MT7fT+YNg6vIVtgGIlFHblzopE2rEiU4wjV5FsGNiJ
bVGjfGkSni+uHeplJfPB+T8rBhRnKZV9qQK4noKTgOScn3lenweqUA7OfTRR1WNh
9j8r9z9PGMHw/qz1FIKhpoPQApuwF8BYj3L1XUrLc2pLkSAt5Codjlh7cZZm2p4q
xmXmJmgj0sZiCzqz0q5X1s5YoCAbCpGla21VlxNGcxyB51BtIR3oW/YwTywsRxJR
L52hoiT2Nvt5ywlaDKJxQnZ9YcVw1/oDeG7h8zkbKfFqA/mQaBU=
=Ts5B
-----END PGP SIGNATURE-----

[Brian C. Duggan]

On 7/18/19 10:40 AM, unman wrote:
> I cant post my files, but I've put up an example which shows how to
> create a qube for building Qubes.
> Here: - https://github.com/unman/notes/tree/master/config/build
>
> There are some notes I used in training which are a very basic
> hands on intro to salt in Qubes:
> https://github.com/unman/notes/tree/master/salt
>
> In the build example,you'll see:
> 1. Create.sls - Create a new qube: installing fedora-30-minimal if not already
> there, cloning to new template, using new template to create qube,
> configure the new qube, and configure dom0.
> 2. install.sls - installs required software in template.
> 3. config.sls - Configures new qube as needed.
>
> I've broken this down to make it as clear as possible, and kept it
> simple.
> You could run each section like:
> qubesctl state.sls build.create
> qubesctl --skip-dom0 --targets=template-builder state.sls build.install
> qubesctl --skip-dom0 --targets=builder state.sls build.config
>
> Of course, you can do everything here using scripting. But for some
> things, (like targeting packages and configuration at distro and version),
> salt is somewhat easier.
>
> unman
>

Thanks for these great resources, unman.

I would like to synchronize Salt formulas or entire Salt configurations
between Qubes machines.

What does that process look like? Should users edit and sign Salt
configurations in dom0? Where should users keep those configurations
under version control? How should they import them in to another dom0
and verify their integrity?

In an earlier thread on the topic, Marek mentioned that he synchronizes
Salt configurations by transferring signed tarballs. But it's unclear to
me where those tarballs are created and signed.

Brian

--
Brian C. Duggan
he/him/his

[Brian C. Duggan]

On 7/18/19 10:40 AM, unman wrote:

> I cant post my files, but I've put up an example which shows how to
> create a qube for building Qubes.
> Here: - https://github.com/unman/notes/tree/master/config/build
>
> There are some notes I used in training which are a very basic
> hands on intro to salt in Qubes:
> https://github.com/unman/notes/tree/master/salt
>
> In the build example,you'll see:
> 1. Create.sls - Create a new qube: installing fedora-30-minimal if not already
> there, cloning to new template, using new template to create qube,
> configure the new qube, and configure dom0.
> 2. install.sls - installs required software in template.
> 3. config.sls - Configures new qube as needed.
>
> I've broken this down to make it as clear as possible, and kept it
> simple.
> You could run each section like:
> qubesctl state.sls build.create
> qubesctl --skip-dom0 --targets=template-builder state.sls build.install
> qubesctl --skip-dom0 --targets=builder state.sls build.config
>
> Of course, you can do everything here using scripting. But for some
> things, (like targeting packages and configuration at distro and version),
> salt is somewhat easier.
>
> unman
>

Thanks for these great resources, unman. Wish I had known about them when I got started.

Where do you edit the salt files and how do you keep them under version control? Earlier, Marek said he synchronized his configuration using
signed tarballs, manually:

https://groups.google.com/d/msg/qubes-users/PtzhBZ8pT4w/8hyG1KWiCAAJ

But it's unclear to me whether he edits, signs, and tars in dom0 and transfers those *out* of dom0, or does those things in a VM and transfers
them *in* to dom0.

I ask because it's obviously much more convenient to edit, sign, and version control those files in a VM with the latest editors, gnupg, and
git. But copying data in to dom0 is generally undesirable and slows salt config iteration.

[unman]

On Thu, Sep 12, 2019 at 10:41:35PM -0400, Brian C. Duggan wrote:
> Thanks for these great resources, unman. Wish I had known about them when I got started.
>
> Where do you edit the salt files and how do you keep them under version control? Earlier, Marek said he synchronized his configuration using
> signed tarballs, manually:
>
> https://groups.google.com/d/msg/qubes-users/PtzhBZ8pT4w/8hyG1KWiCAAJ
>
> But it's unclear to me whether he edits, signs, and tars in dom0 and transfers those *out* of dom0, or does those things in a VM and transfers
> them *in* to dom0.
>
> I ask because it's obviously much more convenient to edit, sign, and version control those files in a VM with the latest editors, gnupg, and
> git. But copying data in to dom0 is generally undesirable and slows salt config iteration.
>
> Brian
>

I don't know what Marek does.
I edit files in dom0, manually copy them to offline dispVM , gpg sign,
copy them to online dispVM and store in git. This sounds long winded:
it's not. It's automated in script.

On a new machine, just reverse the process. Pull down the files you
want - check sig - copy to offline - check - transfer to dom0.
Because salt files are so straightforward it's possible to audit
before use. You could also store checksums from dom0.

[Brian C. Duggan]

On 9/17/19 9:49 PM, unman wrote:
> I don't know what Marek does.

Right, but does this mean there's no best practice or documented process
for synchronizing or sharing salt configurations? I understand your
process now that you've described it, but it wasn't obvious to me.

> I edit files in dom0, manually copy them to offline dispVM , gpg sign,
> copy them to online dispVM and store in git. This sounds long winded:
> it's not. It's automated in script.
>

This makes sense, thank you.

> On a new machine, just reverse the process. Pull down the files you
> want - check sig - copy to offline - check - transfer to dom0.
> Because salt files are so straightforward it's possible to audit
> before use. You could also store checksums from dom0.
>

What's the purpose of storing checksums from the source dom0? To check
whether files have changed on the destination dom0 since the last sync?

[Johannes Graumann]

Do you have any demonstration solution that would do something along
the lines of:
a) query templates-itl, templates-itl-testing, templates-community for
the highest fedora-X-minimal version (X) available
b) proceed with installing that version

As far as I know there's no "latest" metapackages that would render
this trivial ...

Sincerely, Joh