Qubes Security Bulletin #2

[Joanna Rutkowska]

---===[ Qubes Security Bulletin #2 ]===---


Problem description
--------------------

Today Xen.org has announced a critical bug in the Xen hypervisor [1],
that allows for PV DomU->hypervisor escalation. The bug has been found,
as usual, by Rafal Wojtczuk, Qubes co-author, and is a beautiful
demonstration of incorrect design decisions made by Intel, specifically
in the behavior of the SYSRET instruction (AMD processors are not
vulnerable).

So, while the bug has been patched in software by adding some extra
checks to the hypervisor (Xen.org just released a patch), one should
still consider it a CPU-level bug. The issue also affects other systems,
not just Xen (more details in the original advisory). Congrats to Rafal
for coming unprecedentedly close to "The Holly Grail" of system-level
exploitation!

Patching
----------

We have uploaded the patched Xen packages and users of current Qubes
Beta 3 should upgrade immediately, by running (in Dom0 console):

sudo qubes-dom0-updates

... which should bring Xen v4.1.2-4 packages that are immune to the
attack. Please reboot your systems afterwards.

References
------------

1) The official Xen.org advisory:
http://lists.xen.org/archives/html/xen-devel/2012-06/msg00670.html

2) A complete list of all Qubes Security bulletins (2010-2012):
http://wiki.qubes-os.org/trac/wiki/SecurityBulletins

Thanks,
joanna.

[Gynvael Coldwind]

Great find - good work! :)

Are you going to be releasing more details about this?

--
gynvael.coldwind//vx

[Zrubecz Laszlo]

On 12 June 2012 14:13, Joanna Rutkowska <joa...@invisiblethingslab.com> wrote:
> Patching
> ----------
>
> We have uploaded the patched Xen packages and users of current Qubes
> Beta 3 should upgrade immediately, by running (in Dom0 console):
>
> sudo qubes-dom0-updates

Am I missed something?:
"bash: qubes-dom0-updates: command not found"

Instead I have qvm-dom0-update - but it is said: "No updates avaliable"



--
Zrubi

[Joanna Rutkowska]

On 06/12/12 19:21, Zrubecz Laszlo wrote:
> On 12 June 2012 14:13, Joanna Rutkowska <joa...@invisiblethingslab.com> wrote:
>> > Patching
>> > ----------
>> >
>> > We have uploaded the patched Xen packages and users of current Qubes
>> > Beta 3 should upgrade immediately, by running (in Dom0 console):
>> >
>> > sudo qubes-dom0-updates
> Am I missed something?:
> "bash: qubes-dom0-updates: command not found"
>

Ah, indeed, qubes-dom0-update has been introduced after Beta3 (and is
available only in the current-testing branch), so, yes, please use
qvm-dom0-update instead...

> Instead I have qvm-dom0-update - but it is said: "No updates avaliable"

Hm... that's strange. What qubes-* packages versions do you have?

rpm -qa | grep ^qubes

?

[Marek Marczykowski]

On 12.06.2012 22:12, Joanna Rutkowska wrote:
> On 06/12/12 19:21, Zrubecz Laszlo wrote:
>> On 12 June 2012 14:13, Joanna Rutkowska <joa...@invisiblethingslab.com> wrote:
>>>> Patching
>>>> ----------
>>>>
>>>> We have uploaded the patched Xen packages and users of current Qubes
>>>> Beta 3 should upgrade immediately, by running (in Dom0 console):
>>>>
>>>> sudo qubes-dom0-updates
>> Am I missed something?:
>> "bash: qubes-dom0-updates: command not found"
>>
> Ah, indeed, qubes-dom0-update has been introduced after Beta3 (and is
> available only in the current-testing branch), so, yes, please use
> qvm-dom0-update instead...
>
>> Instead I have qvm-dom0-update - but it is said: "No updates avaliable"

Perhaps you have cached metadata from some time before publishing fix. You can
try:
qvm-dom0-update --clean

Or just wait some time (for qubes-dom0-current repo, metadata_expire is set to
7 days...).

--
Best Regards / Pozdrawiam,
Marek Marczykowski
Invisible Things Lab

[Zrubecz Laszlo]

On 12 June 2012 22:31, Marek Marczykowski

<marm...@invisiblethingslab.com> wrote:
> Perhaps you have cached metadata from some time before publishing fix. You can
> try:
> qvm-dom0-update --clean

It is not helped... still: "No updates avaliable"

--
Zrubi

[Marek Marczykowski]

Ah, in the meantime our release signing key expired (in March), we now use new
key. Updates signatures are verified while fetched to dom0, so because you
haven't new key, verification fails.
You need to update qubes-release package first (signed still by old key) and
then the rest of updates:

sudo qvm-dom0-update --clean qubes-release
sudo qvm-dom0-update --clean

[Zrubecz Laszlo]

On 13 June 2012 02:34, Marek Marczykowski

<marm...@invisiblethingslab.com> wrote:
> sudo qvm-dom0-update --clean qubes-release
> sudo qvm-dom0-update --clean

Now that's working... Update was successfull :)

Maybe we need some update on related the wiki pagea s well:
http://wiki.qubes-os.org/trac/wiki/SoftwareUpdateDom0


Thanks.



--
Zrubi

[Daniel Cegiełka]

http://www.kb.cert.org/vuls/id/649219

Daniel

[Zrubecz Laszlo]

On 13 June 2012 09:18, Zrubecz Laszlo <ma...@zrubi.hu> wrote:
> On 13 June 2012 02:34, Marek Marczykowski
> <marm...@invisiblethingslab.com> wrote:
>> sudo qvm-dom0-update --clean qubes-release
>> sudo qvm-dom0-update --clean
>
> Now that's working... Update was successfull :)

Somthing wrong... after Updated my system already got a full system
freeze 3 times now :(

I had 2 kind of freeze:

* instant hard reset
No logs, no error messages just a hard reset and qubes reboots...

* screen freeze
I can see my last 'screenshot' but my system not respondst to any kind
of input - just the power button (4 sec and hard off)
No logs, no error messages.

all these are random, and I can't reproduce it... Ther was no problem
at all befor the upgrade...

Any idea how can I solve this?

--
Zrubi

[Joanna Rutkowska]

And which packages the update brought to your system? Perhaps also some
new Dom0 kernel(s)? Are you sure you still use the same Dom0 kernel as
before?

joanna.

[Zrubecz Laszlo]

On 14 June 2012 13:46, Joanna Rutkowska <joa...@invisiblethingslab.com> wrote:
> And which packages the update brought to your system? Perhaps also some
> new Dom0 kernel(s)? Are you sure you still use the same Dom0 kernel as
> before?

not sure... but also not sure which packages contains the dom0 kernel...

These packages was updated:

Jun 13 07:39:22 Updated: qubes-release-1-3.noarch
Jun 13 07:41:14 Updated: 1000:xen-hypervisor-4.1.2-4.qubes.x86_64
Jun 13 07:41:15 Updated: 1000:xen-licenses-4.1.2-4.qubes.x86_64
Jun 13 07:41:17 Updated: 1000:xen-libs-4.1.2-4.qubes.x86_64
Jun 13 07:41:21 Updated: 1000:xen-runtime-4.1.2-4.qubes.x86_64
Jun 13 07:41:23 Updated: 1000:xen-4.1.2-4.qubes.x86_64


--
Zrubi

[Zrubecz Laszlo]

> These packages was updated:
>
> Jun 13 07:39:22 Updated: qubes-release-1-3.noarch
> Jun 13 07:41:14 Updated: 1000:xen-hypervisor-4.1.2-4.qubes.x86_64
> Jun 13 07:41:15 Updated: 1000:xen-licenses-4.1.2-4.qubes.x86_64
> Jun 13 07:41:17 Updated: 1000:xen-libs-4.1.2-4.qubes.x86_64
> Jun 13 07:41:21 Updated: 1000:xen-runtime-4.1.2-4.qubes.x86_64
> Jun 13 07:41:23 Updated: 1000:xen-4.1.2-4.qubes.x86_64

And it seems the kernel is the same:
2.6.38.3-10.xenlinux.qubes.x86_64



--
Zrubi

[Joanna Rutkowska]

Perhaps the problems were introduced by this commit:

http://git.qubes-os.org/?p=joanna/xen.git;a=commit;h=c83b66efe6b7380946794681a9a3ee5f78430de6

In order to rule this out, can you please try to downgrade your xen
packages to 4.1.2-3? See the wiki[1] for howto on downgrading.

[1] http://wiki.qubes-os.org/trac/wiki/SoftwareUpdateDom0

If that doesn't help, then the problem is most likely caused by switch
from Xen 4.1.1 (that was default in Beta 3) and 4.1.2 that we switched
to later, as can be seen in the repo. In that case, well... you're out
of luck, and you would need to debug the issue with the help of
xen-devel people... :/

joanna.

[Zrubecz Laszlo]

On 14 June 2012 14:57, Joanna Rutkowska <joa...@invisiblethingslab.com> wrote:
> In order to rule this out, can you please try to downgrade your xen
> packages to 4.1.2-3? See the wiki[1] for howto on downgrading.

Just did it...

now waiting for the crash - hopefully this wil not happen :)



--
Zrubi

[Marek Marczykowski]

You can also compare dom0 kernel output from different xen versions (original
4.1.1, new 4.1.2 and old 4.1.2) - perhaps it differs somehow in active kernel
features. You can find it in /var/log/dmesg (current), dmesg.old (previous)
and /var/log/messages (all together).

[Zrubecz Laszlo]

On 14 June 2012 14:57, Joanna Rutkowska <joa...@invisiblethingslab.com> wrote:

If that doesn't help, then the problem is most likely caused by switch
from Xen 4.1.1 (that was default in Beta 3) and 4.1.2 that we switched
to later, as can be seen in the repo. In that case, well... you're out
of luck, and you would need to debug the issue with the help of
xen-devel people... :/

Well, It is happened again with 4.1.2-3 as well :(

Now reverted back to 4.1.1 to be sure this was the reason... Unfortunatelly i do not have time for such debugging right now :(

--
Zrubi

[Zrubecz Laszlo]

On 14 June 2012 09:59, Marek Marczykowski <marm...@invisiblethingslab.com> wrote:

You can also compare dom0 kernel output from different xen versions (original
4.1.1, new 4.1.2 and old 4.1.2) - perhaps it differs somehow in active kernel
features. You can find it in /var/log/dmesg (current), dmesg.old (previous)
and /var/log/messages (all together).

Just diffed the 3 xen version dmesg's - but no significant diff beside the memory mappings...



--
Zrubi