Query on upgrade process - what next after downloading the fedora23 template?

[eoba...@gmail.com]

Hi qubes people,

I have just completed the Qubes 3.0 to 3.1 upgrade process, ending up, as suggested in the upgrade doc, with getting the new Fedora 23 template.

However, I'm not sure where to go from here. All my existing VMs (which is just the standard out-of-the-box set) are still using fedora 21, obviously - what is the next step meant to be? Can I update my TemplateBasedVMs from 21 to 23 without losing my files?

Thanks

Emma

[Andrew David Wong]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Yes, you can upgrade your existing templates using this procedure:

https://www.qubes-os.org/doc/upgrade-to-r3.1/

As always, please make sure to back everything up before attempting this.

- --
Andrew David Wong (Axon)
Community Manager, Qubes OS
https://www.qubes-os.org
-----BEGIN PGP SIGNATURE-----

iQIcBAEBCgAGBQJXsoDjAAoJENtN07w5UDAwWvYP+QEu6OwMm+aqDvsmujHCtdzW
kKGkxTRi/6hjPSr9kz5ijAzwvcQAROEtIBaxMncfpsn0Lr/+iScIyNYKNEARcz+L
SwfuB/crbar0yJNdU+RP8ZeaD03yQOfjvk7QwaC9odgnNG7KIGPjyMWwYCN55+fc
p7RPetRJsEyHXas51ZF11ENpFtgGORcasOSwHwKn0QE9V+eLNjPSc8DnklQWiKvv
eomSLTjp1j5f+lba5m7ZD+2G44Brvc+hH8zaZKnPH4KsLmYUNGcq4QHsAi8jsJJA
XJ9EVXrERjVXK3jA0+Z2n8MEBFw0pyAezZAJb8OLnVlV1cduUMYACtbmPaERdK6l
teQS3yyEIaShulIotxZHL8L3dZjkRCLQW8dp+hBMQXnoZ3SmQW1zfe/KhZgxbDkg
hoEBP+IdFX+5mkJ5UCDdsbsZssuAXEJceN1TJxFvHZmVklfqbFjWpACcPstNfjyR
0Fb0L+5WOjjUjupXKv0om3HHAHW8mnTxJwPrp9duW/mubDwnkTUgqOeCqkRz75/L
Vq6Htys/oNlTSHEYoPeJh3n+kuWEZbRCjlA7tfcvZ1hiPsDhTsvOZOZ92sCsZvpL
JDPjbmIBQ3pMg66jCkJaf7h6KihiYrkrpLTqbkRbojbUNi7ZnoXK+Dgyd0nxUmpl
X51K+RESghLQbut4iRLl
=70x1
-----END PGP SIGNATURE-----

[Andrew David Wong]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 2016-08-15 20:10, Emma Baillie wrote:
> No, you misunderstand me. I already did all the things in that document.

Sorry for the misunderstanding. You do not have to (and, indeed, cannot)
update your TemplateBasedVMs from Fedora 21 to Fedora 23, since they're
Template-based. This means that they don't have a root filesystem (where the OS
and programs are stored) of their own. Instead, they get their root filesystem
from the TemplateVM on which they're based. As long as your TemplateVMs have
been upgraded correctly, your TemplateBasedVMs should work with the upgraded
TemplateVMs normally.

This page contains more information about how TemplateVMs work:

https://www.qubes-os.org/doc/templates/

> I'm asking what to do _next_. Just shut down all my VMs one by one
> including sys_net and sys_firewall and tell them in VM-manager to be
> fedora-23 not fedora-21 VMs? Or is there a better way?
>

Hm. Your questions here suggest that you may not have followed the
instructions correctly. The procedure, in outline, is this:

1. Upgrade all TemplateVMs and StandaloneVMs.
2. Upgrade dom0.
3. Reboot dom0.

(The details of how to perform each step are in the linked documentation page.)

The fact that you're now asking whether you should shut down all of your VMs
one by one suggests that you may not have followed the above procedure, since
rebooting dom0 entails shutting down all of those VMs (but it's also possible
that this is just a miscommunication).

It's also not clear to me what you mean by "tell them in VM-manager to be
fedora-23 not fedora-21 VMs." There isn't really any way to "tell" a
TemplateBasedVM to "be" a fedora-21 or fedora-23 VM. The closest thing would
be to set a TemplateBasedVM's template to either a fedora-21 or fedora-23
template. If you've just performed an in-place upgrade on a template (from
Fedora 21 to Fedora 23) with the intention of using it, then it would indeed
make sense to base your AppVM on this newly upgraded template (if it wasn't
already). Again, as long as the upgrade has been done correctly, everything
should work normally.

P.S. - Please keep the list CCed, and please don't top-post.

> On Tue, Aug 16, 2016 at 12:56 PM, Andrew David Wong <a...@qubes-os.org>

> wrote: On 2016-08-15 19:33, eoba...@gmail.com wrote:
>>>> Hi qubes people,
>>>>
>>>> I have just completed the Qubes 3.0 to 3.1 upgrade process, ending
>>>> up, as suggested in the upgrade doc, with getting the new Fedora 23
>>>> template.
>>>>
>>>> However, I'm not sure where to go from here. All my existing VMs
>>>> (which is just the standard out-of-the-box set) are still using
>>>> fedora 21, obviously - what is the next step meant to be? Can I
>>>> update my TemplateBasedVMs from 21 to 23 without losing my files?
>>>>
>>>> Thanks
>>>>
>>>> Emma
>>>>
>
> Yes, you can upgrade your existing templates using this procedure:
>
> https://www.qubes-os.org/doc/upgrade-to-r3.1/
>
> As always, please make sure to back everything up before attempting this.
>

- --
Andrew David Wong (Axon)
Community Manager, Qubes OS
https://www.qubes-os.org
-----BEGIN PGP SIGNATURE-----

iQIcBAEBCgAGBQJXsozCAAoJENtN07w5UDAwXBkQALAnAOtJHJ1e1oH+oLAHjZl2
srv0HoyXF5t25qhKCGBNa3xfFhZgyCbskXhjsr1SFqnLOe1i26DXvk/EgQRr4rsT
Csz6SHjl47pkYrHWEBfoHrA7G1FWqVjGnJ4ZdDtgFeBN/1hMRENAFW8UHvHOudlb
lHFdzBSshTu7exByIk/zshW8WJFPESAAtDVZsc88GWsvPCcqaZsoTVQibb7tJuCt
2bgzGMmH2i1f1vJBoy25wFXw9rxxEBRyma09hH5e9mh9a1Ve8psHQQw9Lo825KZE
M770oUYIhTR06SuhfBkziOHqOsHmULoyZooYoBF1dp6hb6aIPzTX6RujZLUpyCuU
gWv6RyAa1QdkINLUEdA5WLbELlVAquUHCRH86IBfvqhSBStt/0KhjR5Cz41ljc/H
OYDKUbJL583kiZxz9ayb8xg8wSuXkeSZEufEOiOB2dccSoHzXX5GC+xqRpTlVhM4
0DTsfn+5seYU0Zk4SrwN/xk6o3fYIiEpQeoXK41LwQr3O0ww2symuH/x+YEoFvK4
XPNK5TaOj1fQDO7j1Iu1jNhftx/2lWn4VZH96bE7KjLKJnv8lQGYAiQmKk+c/N5V
GrPRE4r/pB9sZyww6pt6XHGgDLrh+A2KMH0LmwPn7HrBavRpyGUCWLM8S9jZt22c
HPJEP3AfMNbgOJQ1rED1
=+28m
-----END PGP SIGNATURE-----

[eoba...@gmail.com]

> Sorry for the misunderstanding. You do not have to (and, indeed, cannot)
> update your TemplateBasedVMs from Fedora 21 to Fedora 23, since they're
> Template-based.

Normally speaking, though, if I'm running a computer with a particular operating system, I can upgrade that operating system to a new version. So are you saying I can't do this with the various VMs run by Qubes?

This means that they don't have a root filesystem (where the OS
> and programs are stored) of their own. Instead, they get their root filesystem
> from the TemplateVM on which they're based. As long as your TemplateVMs have
> been upgraded correctly, your TemplateBasedVMs should work with the upgraded
> TemplateVMs normally.

The result of running "sudo qubes-dom0-update qubes-template-fedora-23" was not to update a TemplateVM, though. It created a new TemplateVM, so now I have 2: fedora-21 and fedora-23 - the second of which doesn't yet seem to have a purpose unless or until I create new VMs

> The fact that you're now asking whether you should shut down all of your VMs
> one by one suggests that you may not have followed the above procedure, since
> rebooting dom0 entails shutting down all of those VMs (but it's also possible
> that this is just a miscommunication).

no, it just means I'm doing this process gradually, leaving space for things like, eg, restarting a VM in order to post things on the Internet

>
> It's also not clear to me what you mean by "tell them in VM-manager to be
> fedora-23 not fedora-21 VMs."

"upgrade the operating system the VMs are using to version 23 of fedora rather than version 21"

> If you've just performed an in-place upgrade on a template (from
> Fedora 21 to Fedora 23) with the intention of using it, then it would indeed
> make sense to base your AppVM on this newly upgraded template (if it wasn't
> already). Again, as long as the upgrade has been done correctly, everything
> should work normally.

based on that, it doesn't seem to be - I don't think my system is currently in the state you describe

It is possible that the step that went wrong is step one - "Upgrade fedora templates". Would you normally expect that after that first step, the default TemplateVM would have upgraded itself to the most recent available version of Fedora? Because that didn't happen.

Bearing in mind, in all of this my ultimate purpose is "make my Qubes installation indistinguishable from a fresh installation of version 3.1". Which is not yet the case!

[Andrew David Wong]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 2016-08-16 00:18, eoba...@gmail.com wrote:
>
>> Sorry for the misunderstanding. You do not have to (and, indeed, cannot)
>> update your TemplateBasedVMs from Fedora 21 to Fedora 23, since they're
>> Template-based.
>
> Normally speaking, though, if I'm running a computer with a particular
> operating system, I can upgrade that operating system to a new version. So
> are you saying I can't do this with the various VMs run by Qubes?
>

No, you certainly can. However, the way to do it in Qubes (due to the way the
Template system works) is to upgrade the operating system in the TemplateVM,
not the TemplateBasedVM. This is because the TemplateVM is where the operating
system files reside. They do not reside in the TemplateBasedVM -- only user
data resides there.

>> This means that they don't have a root filesystem (where the OS and
>> programs are stored) of their own. Instead, they get their root
>> filesystem from the TemplateVM on which they're based. As long as your
>> TemplateVMs have been upgraded correctly, your TemplateBasedVMs should
>> work with the upgraded TemplateVMs normally.
>
> The result of running "sudo qubes-dom0-update qubes-template-fedora-23"
> was not to update a TemplateVM, though. It created a new TemplateVM, so now
> I have 2: fedora-21 and fedora-23 - the second of which doesn't yet seem
> to have a purpose unless or until I create new VMs
>

That command ("sudo qubes-dom0-update qubes-template-fedora-23") installs a
new fedora-23 template from the repo. It is the preferred way to obtain a
fresh fedora-23 template. However, if you instead want to upgrade an existing
fedora-21 template to fedora-23 (e.g., to preserve any customizations you've
made), then you must use the in-place upgrade procedure:

https://www.qubes-os.org/doc/fedora-template-upgrade-21/

>> The fact that you're now asking whether you should shut down all of your
>> VMs one by one suggests that you may not have followed the above
>> procedure, since rebooting dom0 entails shutting down all of those VMs
>> (but it's also possible that this is just a miscommunication).
>
> no, it just means I'm doing this process gradually, leaving space for
> things like, eg, restarting a VM in order to post things on the Internet

Ok, understood.

>>
>> It's also not clear to me what you mean by "tell them in VM-manager to be
>> fedora-23 not fedora-21 VMs."
>
> "upgrade the operating system the VMs are using to version 23 of fedora
> rather than version 21"
>

Ok, got it.

>> If you've just performed an in-place upgrade on a template (from Fedora
>> 21 to Fedora 23) with the intention of using it, then it would indeed
>> make sense to base your AppVM on this newly upgraded template (if it
>> wasn't already). Again, as long as the upgrade has been done correctly,
>> everything should work normally.
>
> based on that, it doesn't seem to be - I don't think my system is
> currently in the state you describe
>
> It is possible that the step that went wrong is step one - "Upgrade fedora
> templates". Would you normally expect that after that first step, the
> default TemplateVM would have upgraded itself to the most recent available
> version of Fedora? Because that didn't happen.
>

Ah, sorry for the misunderstanding. No, there are additional steps required to
upgrade an existing Fedora 21 template to Fedora 23 that are not listed on
that page. They are instead listed on this page (same as linked above):

https://www.qubes-os.org/doc/fedora-template-upgrade-21/

>
> Bearing in mind, in all of this my ultimate purpose is "make my Qubes
> installation indistinguishable from a fresh installation of version 3.1".
> Which is not yet the case!
>

If this is your goal, then as noted above, it may be easier and more
preferable to obtain a fresh fedora-23 template from the repo (as you've
already done). Either way works, and which is preferable depends on things
like the modifications (if any) you've made to your template(s) and how
acceptable you find the results of the in-place upgrade procedure for your
individual purposes.

- --
Andrew David Wong (Axon)
Community Manager, Qubes OS
https://www.qubes-os.org
-----BEGIN PGP SIGNATURE-----

iQIcBAEBCgAGBQJXszo7AAoJENtN07w5UDAwvzsP/Ay2mCUF6WdEmAkpESHtJMj7
R+s3tHarYYS1MnEFAt/kTziT1iOi6a6eiO90Xt1Zh9BqGeUvR6y4cYPE8D21fNVY
AEcmd/k2zI3mkQYRQNEK4PxbdCE3nPvWX7RdXNtmb/lloM6ol5oCB+5XTymYUyq+
J8WShC1PF/Kl2GghDeHHgOf4QHEBZZsQKpBTlUjK3pCbFNI6GwWfjh8hFqNlA6Fm
MDKn9x/rrNcGg1zQ96pMjq31Pb9oxB13o09Rjq67VTCnbWoCvxCw/Y2Ayn5QVDuh
6Eb5ydKs2UMT8mwx+Tqsk+Yna77K3SyRaXRYpxklKMFzrw+7P29eaEW4uCpvhxH7
8e/NPnk5YYDkiAbrDDI6ut5mKfT/KWZvWeBHakCiIWhib8RJGBQg1sUDNjvzmRBW
BCa9APvrz3irisUJUckBm2Ru/DYSUkAH05YSvtTjnV0/zTRhUYIKMRMundzOUJ08
DVGlHN85diLgER4bZsFBTJWl/pFs6TxlmFAJZFY1FwOTAJArNBh0u0c6mwW77k7N
TBZqhrbwQib5g8ITTJNaD86yoYvfcrnF7uKy4BWpRoOUIa6fuGR/E+pu0UdqaRBc
VdrC4Ui1Fr2V4ZWjsDrRB2KiJEKubOW0+RJ88g8GIjYVyrlGFgYnZBRuiuq2Yapv
ie3T8oYIrdKOcwiV83He
=LZn4
-----END PGP SIGNATURE-----

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

I'm not sure if I understand the intention here, but isn't this all
thread about just switching AppVMs to the new template (either upgraded
in-place, or downloaded fresh one). You can do this in VM settings -
just change "fedora-21" to "fedora-23" in template selection drop-down.
You need to do this for every VM (unless you want to keep some of VMs
using old template for some reason).

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJXszwcAAoJENuP0xzK19cskVcIAJOaeXxNsd4DYH1IowqIb9Au
SivB8hnI40ZOaE/3vV0CBRSPJJjc4X8sE0xGosz+eyCkkVA9z/UICvG1V1d9dIB5
WLhV/UZEeiNBcqZyn8C/x8MBx76OzRYhm8g7FigSlDY0WtVPBNOce0i3hxtJED30
nFHBOT6O2wWmky1MFXj4JuuosUD25SjXXuitbpdulGo4KR7MJBtE0lDApFw4R4tw
5bHT+L/wN4OWbNK5tBM/YtSJ2rSQkwkpE+ZE/iJ6t2Vsl1/2ggC6IidROk7ohlPf
MpUZ74qgQj3mygcilO0XxlZSyQL5Rfm6Cmmmhki36S+YHWXTGq/QEJLAQYlryqI=
=rszm
-----END PGP SIGNATURE-----

[eoba...@gmail.com]

On Wednesday, 17 August 2016 02:15:32 UTC+10, Marek Marczykowski-Górecki wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> On Tue, Aug 16, 2016 at 09:07:24AM -0700, Andrew David Wong wrote:

Yes, that's exactly right. Basically, what you just posted is the answer to my original question :) Confusion arose because the upgrade instructions stop at "install new templates" - and didn't continue to "how do you then _use_ them" which is possibly assumed knowledge for people who've been using Qubes for a bit (which I haven't).

Another question then - is there a particular reason for upgrading the default fedora and/or other installed TemplateVMs as a part of the R3.0 -> R3.1 upgrade? Because it seems from all this that that is actually quite an independent process - that there's no particular point in both updating the fedora-21 template and also downloading fedora-23, (unless for some reason you wanted some VMs to be upgraded to 23 and some not) and that in any case this process could go on before, after, or at some quite unrelated time to, upgrading dom0 from 3.0 to 3.1

Thanks!

E

[Andrew David Wong]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

I think there's some terminological confusion here and in the docs due to the
existence of three possible senses of the word "upgrade" when applied to a
template:

1. Upgrading a template to be compatible with a new Qubes OS release:
https://www.qubes-os.org/doc/upgrade-to-r3.1/#tocAnchor-1-1-1

2. Upgrading a template "internally," e.g., Fedora 21 to Fedora 23:
https://www.qubes-os.org/doc/fedora-template-upgrade-21/

3. "Effectively" upgrading by replacing an existing template with a newer one
from the repo.

The first one can be important because if you don't upgrade all of your
TemplateVMs and StandaloneVMs before you upgrade dom0, it will be much more
difficult (or impossible) to upgrade them afterward.

You're right about the second one. It's an independent process. For example,
IIRC, I upgraded my Fedora templates from 21 to 23 (due to 21's EOL) well
before upgrading to Qubes R3.1.

The first two are in-place upgrades. The third one is not. The third, like the
second, is also an independent process. You can decide to obtain a fresh
template from the repo any time you like and use it to replace an existing
template, and this may effectively constitute upgrading the template for you.

- --
Andrew David Wong (Axon)
Community Manager, Qubes OS
https://www.qubes-os.org
-----BEGIN PGP SIGNATURE-----

iQIcBAEBCgAGBQJXs734AAoJENtN07w5UDAwod0QAKQLEbkzx1XcOK6DUelmVJIS
TJAyPKfsSB2MCt2T0LI+FJu5qBjsSIDTlNwHndmL6w7V7xqanA1GH6hQTH6c4iIL
ls/f/GXMYzuN8DXu8Z5SMz+v3rIh5bHgNsvDTTjMBnAijLmq2B/9YSSi4/BjTDle
5Pu5sgjL+0leBJ6Ms69JXIwIBrDnVv7DD/wnOQ52KutgrcG4t3zRJXq0K84y1stn
87Sdir98o7thJmKybiscpH0xDIz7E1mt20EHnHKzJEB1E0YRF0hhP/9T5pVeJ0Vf
l2lhcEvVV483VWzWFpoGUJpJNIKpd8df1EJzcsPhRUF0GakE/iR2Pj17tduGIx+6
nQib3RcukeGh2y8YwT17We66cke1MOyVTNsgXyWWKrwo0BwHxeZgR7Uh6yU6H8J3
uz83Rp2FlEHxgYZeBvClV22FaaEGIF2yvXW106klqHHc56bHQ1K3ITn8XEG9x2sU
KkHvKxOtg4sPDNrEsLLXd1xbIJILm+NwVIw6rxstppbAavrg68I+ld7hfrnqRFr6
OjeCfWFJZiLHvVNX5SMqC8qx382w/uswu0uGlsCqGsssJVz+cAp4BdlAd5bb8PVD
GvBpY4x+gMwkoPx7ctkGSg8RpkdwTe13h6wuLxYC4DPCK5+lcevpNs31YKSq3Ndu
uII9Wo4veKFrRcIUfi2y
=vUSz
-----END PGP SIGNATURE-----