Reinitialize Templates

[Jasper Weiss]

I've been messing around with the templates for a bit and I'd like to delete and re install them.
(I know, duplicate templates before you mess with them)
Obviously one can't simply delete the default templates so how would you go about re-installing the default debian-8 and fedora-23 templates without re-installing Qubes altogether?

[Jasper Weiss]

I figured it out.

If you want to reinitialize the Debian-8 template, for example, you'll have to remove all VM's that are based on the Debian-8 template first.
Then, open up a terminal in Dom0 and use 'sudo yum erase qubes-template-debian-8' to delete it. When that's done use 'sudo qubes-dom0-update qubes-template-debian8' to install the template again.

For fedora it's a little trickier. You'll still need the Update- and Net-VM to download te template.
You will have to make a clone of the current fedora template first and set that as the template for all VM's currently using the fedora VM.
You can then delete the default fedora template using 'sudo yum erase qubes-template-fedora-23' and reinstall using 'sudo qubes-dom0-update qubes-template-fedora-23' after that, you can set that as the template for the VM's currently using the cloned fedora template (which you can delete afterwards)

I'd be happy to write some prettier documentation for the official website

[Unman]

This is more suitable for qubes-users.

There's no need to "remove all VMs" - just change their template to
something other then the one you are replacing.
And you could do it in one step with :
sudo qubes-dom0-update --action=reinstall qubes-template-debian-8

[Ivan]

I thought the same thing but 'reinstall' doesn't work, at least on my
setup. The command above gives:

[...]
ERROR: yum version installed in VM sys-firewall does not suppport
--downloadonly option
ERROR: only 'install' and 'upgrade' actions supported (reinstall not)

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Indeed, currently --action=reinstall (or any other than listed above)
requires Fedora-based updatevm. Debian has too old yum version for
that...

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJXXSDkAAoJENuP0xzK19cs7pgH/jX3Bq6zmCYZzNRbSI+Qemno
7TZWYH7vuY1ACOIOmVNG/h2o9HZG/LBOTTjEDrmPg3PwbTMaU10ChH2o9sGTLj4I
dTyhl9CCd0nH46jmecJD10h1BYkV/w+Q0rc9O1LBHAjdUu0n/0/p/+BdzbRgExef
umdeFnsLQqppim3v8hDEpB+9l9Tdl5Tjx9CwKAIxqEMrnX5M6ElP35DX7ThG9P+6
jFhJW07V+d4v7oJE/rkS6L+vquRv/kgmUdp5s/KZVImZ/y+QjqFEgTi6kHyZ6BTH
0/SpUEcjY8NXGvX33lJIKmVGaUsc91ha9FoyF4g1PD8nAyH6McsAIykWMf81ORY=
=X0p7
-----END PGP SIGNATURE-----

[Holger Levsen]

On Sun, Jun 12, 2016 at 10:44:20AM +0200, Marek Marczykowski-Górecki wrote:
> > ERROR: yum version installed in VM sys-firewall does not suppport
> > --downloadonly option
> > ERROR: only 'install' and 'upgrade' actions supported (reinstall not)
> Indeed, currently --action=reinstall (or any other than listed above)
> requires Fedora-based updatevm. Debian has too old yum version for
> that...

what version of yum is needed? Debian indeed has only 3.4.3-3, even in
unstable :/

also: will this in future (eg for 3.2) require dnf or will yum still be
supported for some time?


--
cheers,
Holger

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Sun, Jun 12, 2016 at 10:22:54AM +0000, Holger Levsen wrote:
> On Sun, Jun 12, 2016 at 10:44:20AM +0200, Marek Marczykowski-Górecki wrote:
> > > ERROR: yum version installed in VM sys-firewall does not suppport
> > > --downloadonly option
> > > ERROR: only 'install' and 'upgrade' actions supported (reinstall not)
> > Indeed, currently --action=reinstall (or any other than listed above)
> > requires Fedora-based updatevm. Debian has too old yum version for
> > that...
>
> what version of yum is needed? Debian indeed has only 3.4.3-3, even in
> unstable :/

Hmm, Fedora has 3.4.3-507, so I'm afraid --downloadonly option is added
only in some Fedora local patch :/

> also: will this in future (eg for 3.2) require dnf or will yum still be
> supported for some time?

yum is enough. In fact anything that can parse yum repo metadata,
resolve dependencies and download rpms is enough. So maybe yum/dnf
python API is robust enough to write a _simple_ tool which would
download requested packages?

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJXXUDtAAoJENuP0xzK19csYYUH/1KV43heeajzHgcravprZTWh
8t3hCfKsU6pGgHByzaHBWfvRmSyVUf0qW59jyhq3dWwZ8iVkA0uuwvo//xI6bZGp
OsoDtQ/+Tkf1bKxMVKQm3A6+a9OjzciIWb7k8Mp3ERjAS+2t90jNyJ2Vwcb7/GwV
/M3VNqkRhsOCxAqs2INtTeCjCtfaGhS2S/f5gAQkKVcgQBsld+Wh8OiqP3xoMrHu
8P2ZzgtP8pAW5iCiqlTzAcV6qJvZkb8PakUtnlVcetowzvG7pAmaYOEYQmmWAP8c
jxCCdv8Wy1tCRn2i0YOZ0j1OBRBkpVqPPNC0qip40fpk2p1qj7Fj8Hw0yjg9qXo=
=o3ss
-----END PGP SIGNATURE-----

[Holger Levsen]

On Sun, Jun 12, 2016 at 01:01:00PM +0200, Marek Marczykowski-Górecki wrote:
> > what version of yum is needed? Debian indeed has only 3.4.3-3, even in
> > unstable :/
> Hmm, Fedora has 3.4.3-507, so I'm afraid --downloadonly option is added
> only in some Fedora local patch :/

hmpf. if that patch were identified one could file a wishlist bug
against the Debian package to include it…

> > also: will this in future (eg for 3.2) require dnf or will yum still be
> > supported for some time?
> yum is enough. In fact anything that can parse yum repo metadata,
> resolve dependencies and download rpms is enough. So maybe yum/dnf
> python API is robust enough to write a _simple_ tool which would
> download requested packages?

does dnf have --downloadonly? Cause I suppose it would be more useful to
just package dnf for Debian instead of writing yet another tool to do
that job… but maybe then that tool would have less dependencies and
features… hm.


--
cheers,
Holger

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Sun, Jun 12, 2016 at 11:21:59AM +0000, Holger Levsen wrote:
> On Sun, Jun 12, 2016 at 01:01:00PM +0200, Marek Marczykowski-Górecki wrote:
> > > what version of yum is needed? Debian indeed has only 3.4.3-3, even in
> > > unstable :/
> > Hmm, Fedora has 3.4.3-507, so I'm afraid --downloadonly option is added
> > only in some Fedora local patch :/
>
> hmpf. if that patch were identified one could file a wishlist bug
> against the Debian package to include it…

Sigh.., it's part of a giant patch 3.4.3..HEAD:
http://pkgs.fedoraproject.org/cgit/rpms/yum.git/tree/yum-HEAD.patch

Anyway it looks like the right commit is here:
http://yum.baseurl.org/gitweb?p=yum.git;a=commit;h=639a768c93d8c18a1906bb22840d4acb5efa7ac4

But there were few modifications later. Generally this should be
helpful:
http://yum.baseurl.org/gitweb?p=yum.git&a=search&h=HEAD&st=commit&s=downloadonly

Also it looks like downloadonly option was implemented as a plugin the
past. So maybe cleaner solution would be to package that plugin on
Debian?
It used to be part of yum-utils repository, at least at the time of its
last release (1.1.31, 5 years ago):
http://yum.baseurl.org/gitweb?p=yum-utils.git;a=tree;f=plugins/downloadonly;h=b8442914b20623015a87ba30b4ae63e20e5c5e53;hb=ccf29c9525c1b39498d37384ede1a12bec766a1b

Generally it looks like yum people abandoned releasing new versions and
simply package "master" when convenient...

> > > also: will this in future (eg for 3.2) require dnf or will yum still be
> > > supported for some time?
> > yum is enough. In fact anything that can parse yum repo metadata,
> > resolve dependencies and download rpms is enough. So maybe yum/dnf
> > python API is robust enough to write a _simple_ tool which would
> > download requested packages?
>
> does dnf have --downloadonly?

Yes, it works slightly differently than in yum, but is good enough.

> Cause I suppose it would be more useful to
> just package dnf for Debian instead of writing yet another tool to do
> that job… but maybe then that tool would have less dependencies and
> features… hm.

Exactly why I'm considering this idea at all. But still it should handle
options like --enablerepo, --exclude and few more... So maybe better try
other options first.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJXXVEWAAoJENuP0xzK19csIfcH/j+plcgPdDrzsWB4QUmCJTt5
C7O/1Xa5tKPVe9GpfDkmJ66VtyxyxTp3yC810D883pvh30Q9A4hbbqJ2VrhlIubH
dpHDPiWfii9Zt0BZcBBc20Ci3CNS/ivCxx2s7/ZWJeQ7wUGwFv5jjyXCxrhByTig
nKCqJfvG083ri3SnxB5YVLYf9JlUjz4hxGcRhFWIprcYu/CNmq2Xq2eLeYPhXJte
Ch2Hy2RCePGCxNSD+uH3e5du6uRPyMx09aoPxyQtClEeKjYVmktcFmJSxINjDFAM
SviZ6DOiy0ac4C2MVJQiPrnY1C/SOe8N58gRQ0gSnoAXpDX516W5EXSn6QYmnRQ=
=PYMh
-----END PGP SIGNATURE-----

[Jasper Weiss]

There's some new documentation added to the website on reinstalling templates here

[Ivan]

Hi,

On 06/14/2016 12:34 PM, Jasper Weiss wrote:
> There's some new documentation added to the website on reinstalling

> templates here <https://www.qubes-os.org/doc/reinstall-template/>
>

I guess many qubes users have a shared template they use for both sys-*
vms and app vms (usually fedora23), so maybe there should be some notes
about reinstalling the template sys-* vms (or other vms needed for
network access) are based on. For instance:

- obviously there will be no network once you shutdown the sys-* vms, so
how to download the template rpm and perform an offline installation ?

- Or, do vms really need to be stopped when updating the template's
root.img ? IIRC in linux one can open() a file, delete it, create
another file with the same name, but continue to read/write from the
deleted file until a close(). A subsequent open() would then open the
new file/inode. Maybe that'd work too with XEN or are there some caveats ?

Comments in previous posts also suggest using the --downloadonly option
(currently only in fedora templates) in order to be able to use
--action=reinstall in dom0. Would that allow skipping the "dummy
template" steps ?. If yes, maybe make a note about that too ?

cheers
Ivan

[Jasper Weiss]

On Tuesday, June 14, 2016 at 12:20:19 PM UTC+2, Ivan wrote:

I guess many qubes users have a shared template they use for both sys-*
vms and app vms (usually fedora23), so maybe there should be some notes
about reinstalling the template sys-* vms (or other vms needed for
network access) are based on. For instance:

- obviously there will be no network once you shutdown the sys-* vms, so
how to download the template rpm and perform an offline installation ?

You'll want to make a dummy template. Just make a clone of the currently used template and set that as the template for the sys-* VM's. You can delete it afterwards.
 

- Or, do vms really need to be stopped when updating the template's
root.img ? IIRC in linux one can open() a file, delete it, create
another file with the same name, but continue to read/write from the
deleted file until a close(). A subsequent open() would then open the
new file/inode. Maybe that'd work too with XEN or are there some caveats ?

Comments in previous posts also suggest using the --downloadonly option
(currently only in fedora templates) in order to be able to use
--action=reinstall in dom0. Would that allow skipping the "dummy
template" steps ?. If yes, maybe make a note about that too ?

Perhaps these steps are too complicated for most users. It might be nice to have a simple "reinstall/reinitialize template" option in the Qubes VM Manager.
It would run through the following steps in an automated manner:

-Check if the template is used by any VM's
-If so, create a clone and set that as the template for these VM's
-Delete the the template
-Install the template
-If applicable, replace the clone template with the newly installed template and delete it.

Add in a fancy gui with a progress bar and we're set.
 

[Chris Laprise]

I have a patch for that in progress. Currently it checks for
--action=reinstall and includes the template package in that case. Your
guess is half right about file deletion: The setup scripts cause
root.img to be updated seamlessly even when dependent vms are running,
but they leave private.img untouched--that needs to change.

https://github.com/QubesOS/qubes-core-admin-linux/compare/master...ttasket:patch-1

Chris

>
> cheers
> Ivan
>

[Ivan]

Great.

May I suggest having an option to keep private.img untouched then ? I'm
thinking of some cases where I'll want to only update the root image,
and keep custom configurations I have in private.img (default options
for various apps like firefox, gnome-terminal, ...)
Alternatively, a warning message warning the user that private.img will
be replaced would be enough to hint people to do a backup before
replacing the template.

[Ivan]

On 06/14/2016 01:45 PM, Jasper Weiss wrote:
>
>
> On Tuesday, June 14, 2016 at 12:20:19 PM UTC+2, Ivan wrote:
>>
>>
>>
>> I guess many qubes users have a shared template they use for both sys-*
>> vms and app vms (usually fedora23), so maybe there should be some notes
>> about reinstalling the template sys-* vms (or other vms needed for
>> network access) are based on. For instance:
>>
>> - obviously there will be no network once you shutdown the sys-* vms, so
>> how to download the template rpm and perform an offline installation ?
>>
>
> You'll want to make a dummy template. Just make a clone of the currently
> used template and set that as the template for the sys-* VM's. You can
> delete it afterwards.

Yes I know - my question was rhetorical. More experienced users will
probably figure this on themselves, but the instructions are quite
detailed so they seem to be intended for people without a lot of
knowledge of yum/qubes templates. So IMO it would have made sense to
document that kind of details too.
Anyway, the instructions will probably change now that Chris L. is
working on a user-friendly solution.

>
>> - Or, do vms really need to be stopped when updating the template's
>> root.img ? IIRC in linux one can open() a file, delete it, create
>> another file with the same name, but continue to read/write from the
>> deleted file until a close(). A subsequent open() would then open the
>> new file/inode. Maybe that'd work too with XEN or are there some caveats ?
>>
>> Comments in previous posts also suggest using the --downloadonly option
>> (currently only in fedora templates) in order to be able to use
>> --action=reinstall in dom0. Would that allow skipping the "dummy
>> template" steps ?. If yes, maybe make a note about that too ?
>>
>

> Perhaps these steps are too complicated for most users.It might be nice to

[Chris Laprise]

From a security standpoint, the default behavior should be to erase the
old private.img. I'll look into adding a warning.