work on vt-d support in libreboot for Intel GM45/Penryn yet?

736 views
Skip to first unread message

Pudding4Brains

unread,
Dec 6, 2015, 5:58:47 AM12/6/15
to qubes-users
Howdy,

As I'm looking to get a ThinkPad T400 (or some such) and flash it with libreboot (out of technical curiousity and also princple to have at least one unblobbed system available), I was wondering if anyone knows if there is any work being done on vt-d support for this architecture yet?

Two fairly recent posts by Thierry Laurion suggest it wasn't/isn't available on the libreboot for his x200 (similar arch.):
https://groups.google.com/d/msg/qubes-users/ty7EsA5xBb4/B5PbNg7QDgAJ
https://groups.google.com/d/msg/qubes-devel/044FDrqJDPc/ooFd1g-uBAAJ

Searching the coreboot site only rendered one message, mentioning some support of IOMMU tables for ACPI for the newer Sandy Bridge and Ivy Bridge architectures.
http://blogs.coreboot.org/blog/2015/11/10/coreboot-changelog-5/

My understanding of what is needed for full vt-d support (for OSes such as Qubes) is rudimentary to non-existant (sorry :-/ )

Is there any hope at all of getting this done at all, especially with the ME cut out of the loop the way it is done for libreboot?

Historically I have worked quite a bit with low level firmware/software (PC-BIOS, chip level / device driver assembly programming) but that is all some time ago (8088-486 era). What I've read so far the whole setup as run on "modern" Intel platforms seems quite complex/daunting - but hey: never stop learning :o)

Is there any point at all in setting myself up with some stuff (laptop, current ISP SOIC-programmer etc) and any chance at all of getting my hands on some usefull documentation to maybe help with some work (programming/testing?) in this direction?

Or is it a total nonstarter to begin with?

I realize, there would still be the microcode in the CPU as a "kinda blob" to keep some sort of uncomfortability going, that won't easily go away, so maybe efforts would be put to better use in the ARM camp, but I'm way more familiar with x86 stuff than all that, so this seemed to be a nice hook to start some tinkering with low level stuff again :o)

So, in short:
- Could it theoretically be done, or is true vt-d support for libreboot a nonstarter due to issues with the architecture (ME etc)
- Anyone working on this yet?

Cheers!

Tim W

unread,
Dec 6, 2015, 4:30:06 PM12/6/15
to qubes-users



I think the issue has to do with that to get VT-D working requires the proprietary Intel blobs.  I could be wrong on this but IIRC that is what I read.  Now the not quite as good option is coreboot which of course  use those blobs.  Its sort ot a half way ground in terms of control as I see it.   Intels pro feature set in there processors creeps me out.  You can not even be sure that processors do not have the hardware functionality there that are listed to not have Pro with it simply being disabled where it could be enabled remotely.   

Pudding4Brains

unread,
Dec 9, 2015, 5:45:45 AM12/9/15
to qubes-users
Thanks Tim - even if somewhat disappointing ;o)

That would be really messed up.

Am I correct in understanding that this means we have no hope whatsoever in bringing a reasonably "modern" (well, some of you may already disagree on that bit) x86 platform under control of the user?

- If I forget about coreboot/librebooot and leave the firmware (on any machine) under control by Intel (or worse, the system mfg) I can install Qubes and let it use vt-d to isolate untrusted DMA-capable interface (and anti-evil-maid booting), but there is not much sense as the backdoored/buggy/broken firmware with ME and all that is still in place.

- If I remove/disable the original firmware/ME I can't use AEM nor can Qubes protect me runtime from DMA attacks through the NIC or other DMA devices with possible compromised/backdoored firmware.

Catch22

Is the reason that coreboot uses Intel-blobs to get systems running really that it can't be done - in principle - to properly set up the IOMMU without total control of the ICH(firmware) or some such, or is it just darn complex to set this up in a stable fashion and it hasn't been done yet? (and hence using the blob is a "temporary" workaround)

I probably should be asking this in the coreboot/libreboot communities, but I figured there are some very, very IOMMU-savvy folks here and I'm already here, so hey ... ;o)

Cheers!

Tim W

unread,
Dec 10, 2015, 12:37:34 AM12/10/15
to qubes-users


On Wednesday, December 9, 2015 at 5:45:45 AM UTC-5, Pudding4Brains wrote:
Thanks Tim - even if somewhat disappointing ;o)

That would be really messed up.

Am I correct in understanding that this means we have no hope whatsoever in bringing a reasonably "modern" (well, some of you may already disagree on that bit) x86 platform under control of the user?

- If I forget about coreboot/librebooot and leave the firmware (on any machine) under control by Intel (or worse, the system mfg) I can install Qubes and let it use vt-d to isolate untrusted DMA-capable interface (and anti-evil-maid booting), but there is not much sense as the backdoored/buggy/broken firmware with ME and all that is still in place.

- If I remove/disable the original firmware/ME I can't use AEM nor can Qubes protect me runtime from DMA attacks through the NIC or other DMA devices with possible compromised/backdoored firmware.

Catch22

Is the reason that coreboot uses Intel-blobs to get systems running really that it can't be done - in principle - to properly set up the IOMMU without total control of the ICH(firmware) or some such, or is it just darn complex to set this up in a stable fashion and it hasn't been done yet? (and hence using the blob is a "temporary" workaround)

I probably should be asking this in the coreboot/libreboot communities, but I figured there are some very, very IOMMU-savvy folks here and I'm already here, so hey ... ;o)

Cheers!



This is why you see Joanna stating repeatedly how we are stuck in a position where we have to trust the HW/firmware.  Its not that we actually do TRUST it but that we have no choice thus until something can be done about it we move on to areas we can do tighten security.

I guess one possibility would be a AMD IOMMU laptop processor then you could possibly have the open motherboard firmware and full virtualization performance.

I would be more than willing to take a step back of one or two gens of processors from newest from Intel using a AMD if it meant the above could be accomplished. Honestly in terms of laptop processor perf very little has been gained over the last few gens of CPU board.   I would be fine with Haswell equivalent from AMD with open firmware and all vt working.
 

Thierry Laurion

unread,
Dec 10, 2015, 11:48:11 AM12/10/15
to qubes-users
Hey. I'm still on that case, since I want low budget people to still have access to security and privacy if needed.

There is 5 distinct problems to seperate here (maybe more, i'm trying to figure out the chain myself)

Bottom-up possible problems:
-Latest CPU microcode version present in the CPU. (no need for microcode update in that case. 1067a seems to be the latest version available. I have some models to test with. )
-Corrupted DMAR tables in GM45 (For graphic card initialisation only on latest BIOS. There seems to be workaround to test: "intel_iommu=igfx_off" i stumbled upon. Will test it ASAP. Here is what was outputed without it.)
-Bios initialisation of it. 
-Libreboot/Coreboot initialisation of it.
-XEN usage of it (Qubes released 3.1 rc1 yesterday with newer xen 4.6)


Until proven otherwise, it seems that this hardware is supposed to support vt-d.
It it is the case, I can confirm that with 8GB or ram, this laptop doesn't make it's age and will provide needed security for most use cases.

I continue to think that this laptop requires a little more love from XEN/Libreboot guys.

Regards,
Thierry

timet...@gmail.com

unread,
Dec 10, 2015, 4:36:33 PM12/10/15
to qubes-users
On Wednesday, December 9, 2015 at 10:37:34 PM UTC-7, Tim W wrote:
> On Wednesday, December 9, 2015 at 5:45:45 AM UTC-5, Pudding4Brains wrote:
> Thanks Tim - even if somewhat disappointing ;o)

> This is why you see Joanna stating repeatedly how we are stuck in a
> position where we have to trust the HW/firmware.  Its not that we
> actually do TRUST it but that we have no choice thus until something can be done about it we move on to areas we can do tighten security.
>

I've been tossing this problem around in the back of my mind and I have a question. It seems that IME is one huge backdoor. The question is not whether there is a way to get rid of it but is there any way to block it without destroying the processor? Once one gets past the hardware then it is all code on top of code on top of code. So is there any way for the code higher up on the food chain to block code on the lower levels? Or would such an attempt at either the BIOS or driver level makes the rest of the processor unusable?

Pudding4Brains

unread,
Dec 12, 2015, 6:57:50 AM12/12/15
to qubes-users
 Howdy,

Thanks for thinking along all of you :o)

So, I've decided I'll get myself some T400 for tinkering, regardless of chances of succes. Started downloading some documentation to begin with and obviously I have some catching up to do reading/learning about newer technologies than the ones I used to be familiar with. The ICH9 datasheet alone is some 1000 pages, family 4 chipset datasheet is less, but probably too shallow, so I will still need to get proper info for a better basic understanding of IOMMU, VT-d and Intel implementations of TPM, their ME and all that jazz. Not to be done in a week or so *rolleyes*

Obviously, at the very least, microcode will remain a blob and probably more :o(  but at the very least it will be an educational road trip to at least try and get as far as I can. Will share here of course if any usable results come of it, but don't hold your breath ;o)


> It seems that IME is one huge backdoor. The question is not whether there is a way to get rid of it but is there any way to block it without destroying the processor?

Yes, the buggyness or possible/probable backdoors are my "concern" with the whole management setup too. I don't have much to hide, but out of very principle all that stuff should not be under anyone's control but mine. The industry (hardware, software and online "services") has taken way too much control away from the (average) user already. It has to stop somewhere.

That said, from what I read it is quite possible to disable the ME - or cut it out of the loop anyway, or at least that is my understandig of what libreboot is already doing on these laptops. It's just that this "solution" also seems to implicate that we have to do without VT-d ... which isn't a good thing. So it would be nice to find a way to do all this and still setup the IOMMU and what else is needed to properly support VT-d for the likes of Qubes.

I'm not hopeful that it can be done, but it'll be a trip to find out _exactly_ why not anyway.

Cheers

Marek Marczykowski-Górecki

unread,
Dec 12, 2015, 2:40:19 PM12/12/15
to Thierry Laurion, qubes-users
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Thu, Dec 10, 2015 at 08:48:11AM -0800, Thierry Laurion wrote:
> Hey. I'm still on that case, since I want low budget people to still have
> access to security and privacy if needed.
>
> There is 5 distinct problems to seperate here (maybe more, i'm trying to
> figure out the chain myself)
>
> Bottom-up possible problems:
> -Latest CPU microcode version present in the CPU. (no need for microcode
> update in that case. 1067a seems to be the latest version available. I have
> some models to test with. )
> -Corrupted DMAR tables in GM45 (For graphic card initialisation only on
> latest BIOS. There seems to be workaround to test
> <https://bugzilla.redhat.com/show_bug.cgi?id=538163>: "intel_iommu=igfx_off"
> i stumbled upon. Will test it ASAP. Here is what was outputed without it.
> <https://paste.fedoraproject.org/298016/>)

I don't think it will change anything. When Xen is in use, Linux kernel
cannot configure VT-d for it's own purposes (at least not yet - there is
some ongoing work for "nested VT-d"). So IOMMU is not configured by dom0 kernel
at all - only at Xen level. And according to this log, it's disabled
even there because of corrupted DMAR tables. So the workaround, if any,
would be some Xen command line option. But of course the better would be
to have DMAR tables fixed.
- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJWbHgaAAoJENuP0xzK19cs4E8H/05yBIwJRyNM0s9OA2nZ9uqG
kV4vo8zncIJaDiqe2F9Yj1hT7iOnx+HHijgusF8xvyaFgdezLdhCd1oP4B/DxNP6
Tthlo6USg87D0cO/pC2HlQ2pGInavTT9FCjjLO8QFkEg9jfMcbXIheH0X0GtJgO0
NxsskG2azthNdmam0SixlPM5dfsKwhNUQxUpTLZLKKPjZvdD2P3r/EKJglAdaKBe
bwimNMyHj5Xj8S2gMUmOLNhsa3BiNz9ekKDFQHERHxnOFczww8V6gdTOc0hxE7dQ
GEExll4yXiDb2RFlSXYvVwAQJP4dWDFjTrCw2ZcUbqeroIIFpriguAF/NeorzqY=
=ulnf
-----END PGP SIGNATURE-----

Pudding4Brains

unread,
Dec 12, 2015, 3:51:08 PM12/12/15
to qubes-users
Hi Marek,

Thanks for that. Yes, I had stumbled on the corrupted DMAR tables issue. Still trying to get my head around all this - what is setup by which firmware in what chip etc etc during initialisation of the system, so forgive me if I make a silly "logic" mistake, but I somewhat hoped that with ample research and hardware/software hacking it might be possible to setup the tables from the likes of libreboot or some other boot time firmware under user/owner control anyway (and at the same time fix the corrupted tables issue).

The corrupted tables seem to be the result of the setup arranged by the various blobs of Intel (or 3rd party code) that are running in the various modules/chips involved in system startup. Right(?).

Libreboot seems to have found a way around executing some/all of those blobs (except microcode update I presume?), but that doesn't give us VT-d as I presume that by cutting the ME out of the loop the tables don't get setup properly either and the coreboot/libreboot project have no real "replacement" for the blobs they disabled - just happy (enough) that they succeeded in disabling them. Correct?

So the work I was asking about, and presumably might want to look into if not done already, is exactly that level of startup initialization needed to properly setup all the chips (and memory) so that the VT-d functionality might be used (at all). Good to know that it would be Xen handling it on a higher level (which makes perfect sense of course), but with that "Qubes" (a full Qubes installation) would also have VT-d available to have more secure compartementalization. I guess that is what I was talking about when mentioning Qubes before ;o)

So yes, I realize that it is the low level stuff that needs work - I'm just not up to speed (yet) with all the nitty gritty of the workings of all the Intel ME/TPM/DRM stuff involved in the mobo initialization (IOMMU setup) before *anything* is booted from disk, to be able to assess if it is a hopeless endeavour (as in: can't be done as it is done by code we can't *possibly* change, like stuff running in the ICH9 or some such).

Hence my initial question.

Currently I'm running some i3 laptop (newer than te T400) that has served me well for years, but it's slowly falling apart. This laptop has no VT-d either and I still used Qubes on it. But as I was looking around for a cheap replacement (I don't do much CPU-heavy stuff) I figured it would be neat to go for "libre" or as libre as I can practically afford anyway. Currently the T400 or x200 seem to be the easiest budget way to go, so I figured I'd get two T400 - one for (hardware)hacking/testing and one for reference/usage and give it a go. Somehow hoping that by the time my i3 gives up completely I might have a working T400 with libreboot and probably Qubes (even if no VT-d) or <dreammode>even a T400 with working VT-d and Qubes</dreammode> :o)

I'll just try and get started on it, regardless of chances of success. But if anyone has good pointers as to where the biggest showstopper issues are at, I would still be thankful for pointers like that :o)

Also any (mention of) good documentation on all the technology involved, if not readily found online, would be accepted in gratitude :o)

Cheers

Thierry Laurion

unread,
Dec 12, 2015, 8:27:01 PM12/12/15
to qubes-users
Hey all!
Goooooood news for today!

I've managed to have vt-d supported (kind of) on Qubes 3.1 RC1 through Xen 4.6 newly provided iommu workaround no-igfx option.

To keep in mind:
-CPU microcode: 1067a

What got changed:
-Latest BIOS update from lenovo got updated (It is a good idea to have a working system before going the Libreboot way and fix things up knowing the system can be fixed on a firmware level)
-iommu=no-igfx got passed to xen

Result:
-basic vt-d support got recognised and HCL file can be produced.

Still bogus:
-The system boots until netvm tries to use iommu features. Have not played around iommu features to activate and deactivate.
Any idea of how to troubleshoot this further?
x200_vtd_works_on_latest_bios_with_no-igfx
Qubes-HCL-LENOVO-745434U-20151212-193925.yml

Thierry Laurion

unread,
Dec 12, 2015, 10:10:32 PM12/12/15
to qubes-users
Note that using the following parameters makes TPM support appear in HCL report:
iommu=pv iommu=no-igfx iommu=pass-through

So:
-vt-x activated
-vt-d activated
-tpm present

Thierry

Marek Marczykowski-Górecki

unread,
Dec 13, 2015, 6:53:20 AM12/13/15
to Thierry Laurion, qubes-users
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

I don't think those options are valid. Only iommu=no-igfx appears to do
anything. I think the other ones were valid for some old Xen version
(4.1? 3.4?).
List of currently supported options is here:
http://xenbits.xen.org/docs/unstable/misc/xen-command-line.html

And just for double check - code which really parses those options:
http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/drivers/passthrough/iommu.c;h=d513773347d92db0c40c12552d8d9959bc0794d0;hb=refs/heads/stable-4.6#l70

Anyway - iommu=dom0-passthrough (which is what you've probably meant by
iommu=pass-through) affects only dom0 - basically disabling IOMMU for
it. So doesn't do anything about NetVM.

Try adding iommu=verbose and/or iommu=debug. It would be also a good
idea to check netvm (sys-net) console outout -
/var/log/xen/console/guest-sys-net.log.

> So:
> -vt-x activated
> -vt-d activated
> -tpm present


- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJWbVwlAAoJENuP0xzK19csO3oH/05xed0++1QpS0JRez4Wue6o
MpWBrKgedv/R0yX4mb7MrgtPgu4ioGQAiKkLn2y7GomWjFP5hlUODE5hQUxrhE9V
xafVrysLWP9h+/dRbkgSUqwAYz1hAAKP7g7tRUQQlUIh6P8Vmg8FGxWS02k5QhbF
Jq/D9iFKG/JTxAbUMxpn1Y+WZPQWsqhSltUe0SVZSJ9itjdsZGQ0bN3WpqyU+Q0k
UD1zVSeIUmwQmNoUsXPFyw+SxHT19K4T3/6t+rL6KXCZcvRq1AAfpv76f720M/lP
2q1eSTHJAMSsHWq98481cFi/P/Ucd+/7+u3xwMT8Ebc0XMgGqV/coMoJOiyoBRA=
=4eRj
-----END PGP SIGNATURE-----

Thierry Laurion

unread,
Dec 13, 2015, 1:36:11 PM12/13/15
to Marek Marczykowski-Górecki, qubes-users
Marek, you are absolutely right.

xen options to make it boot with vt-d when no netvm or firewall vm configured:
iommu=no-igfx
added option:
sync_console loglvl=all iommu=debug [removed console=none]


Kernel option:
intel_iommu=igfx_off debug [removed rhgb quiet]

I am not sure if it is actually netvm that freezes the system or firewallvm.
Here are collected logs with netvm and firewallvm booting until frozen state.

Note that i'm actually writing to you from that computer, so the last
run of logs are not valid (xen started with iommu=0)
So the last lines linked to the freezing are:

guest-netvm.log
Last line:
501 [ 34.497887] IPv6: ADDRCONF(NETDEV_UP): wlp0s0: link is not ready
When booting with iommu=no-igfx:
Next coming lines are:
997 [ 35.436878] IPv6: ADDRCONF(NETDEV_UP): wlp0s0: link is not ready
998 [ 82.329087] wlp0s0: authenticate with 20:0c:c8:3c:47:89
999 [ 82.339195] wlp0s0: send auth to 20:0c:c8:3c:47:89 (try 1/3)
1000 [ 82.341959] wlp0s0: authenticated
1001 [ 82.346318] wlp0s0: associate with 20:0c:c8:3c:47:89 (try 1/3)
1002 [ 82.350821] wlp0s0: RX AssocResp from 20:0c:c8:3c:47:89
(capab=0x431 1002 status=0 aid=2)
1003 [ 82.353901] wlp0s0: associated
1004 [ 82.353952] IPv6: ADDRCONF(NETDEV_CHANGE): wlp0s0: link becomes ready

As it is seen, a log of things seem to happen between second 35 and 82
but are not reported at kernel level.

xl_dmesg file:
was launched with: iommu=no-igfx iommu=debug sync_console loglvl=all
iommu=intremap iommu_inclusive_mapping

But no iommu option other then iommu=no-igfx had impact, as seen here:
(XEN) Intel VT-d iommu 2 supported page sizes: 4kB.
(XEN) Intel VT-d iommu 1 supported page sizes: 4kB.
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB.
(XEN) Intel VT-d iommu 3 supported page sizes: 4kB.
(XEN) Intel VT-d Snoop Control not enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation not enabled.
(XEN) Intel VT-d Interrupt Remapping not enabled.
(XEN) Intel VT-d Shared EPT tables not enabled.
(XEN) I/O virtualisation enabled
(XEN) - Dom0 mode: Relaxed
(XEN) Interrupt remapping disabled
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using new ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1

If anything else is needed or any insight pops in your head, please
advise accordingly.
Any insight to debug the freeze?

Thierry
--
Thierry Laurion
guest-netvm.log
guest-firewallvm.log
Qubes-HCL-LENOVO-745434U-20151213-111829.yml
xl_dmesg

Marek Marczykowski-Górecki

unread,
Dec 13, 2015, 2:21:58 PM12/13/15
to Thierry Laurion, qubes-users
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Sun, Dec 13, 2015 at 01:36:09PM -0500, Thierry Laurion wrote:
> Marek, you are absolutely right.
>
> xen options to make it boot with vt-d when no netvm or firewall vm configured:
> iommu=no-igfx
> added option:
> sync_console loglvl=all iommu=debug [removed console=none]
>
>
> Kernel option:
> intel_iommu=igfx_off debug [removed rhgb quiet]
>
> I am not sure if it is actually netvm that freezes the system or firewallvm.
> Here are collected logs with netvm and firewallvm booting until frozen state.

Probably netvm. Or USB vm (if you have one). Generally only VMs with PCI
devices are affected by IOMMU in any way.

> Note that i'm actually writing to you from that computer, so the last
> run of logs are not valid (xen started with iommu=0)
> So the last lines linked to the freezing are:
>
> guest-netvm.log
> Last line:
> 501 [ 34.497887] IPv6: ADDRCONF(NETDEV_UP): wlp0s0: link is not ready
> When booting with iommu=no-igfx:
> Next coming lines are:
> 997 [ 35.436878] IPv6: ADDRCONF(NETDEV_UP): wlp0s0: link is not ready
> 998 [ 82.329087] wlp0s0: authenticate with 20:0c:c8:3c:47:89
> 999 [ 82.339195] wlp0s0: send auth to 20:0c:c8:3c:47:89 (try 1/3)
> 1000 [ 82.341959] wlp0s0: authenticated
> 1001 [ 82.346318] wlp0s0: associate with 20:0c:c8:3c:47:89 (try 1/3)
> 1002 [ 82.350821] wlp0s0: RX AssocResp from 20:0c:c8:3c:47:89
> (capab=0x431 1002 status=0 aid=2)
> 1003 [ 82.353901] wlp0s0: associated
> 1004 [ 82.353952] IPv6: ADDRCONF(NETDEV_CHANGE): wlp0s0: link becomes ready
>
> As it is seen, a log of things seem to happen between second 35 and 82
> but are not reported at kernel level.

Probably time between starting a VM and starting nm-applet, which would
would instruct NetworkManager to actually connect to the network. Or sth
like this...

> xl_dmesg file:
> was launched with: iommu=no-igfx iommu=debug sync_console loglvl=all
> iommu=intremap iommu_inclusive_mapping
>
> But no iommu option other then iommu=no-igfx had impact, as seen here:
> (XEN) Intel VT-d iommu 2 supported page sizes: 4kB.
> (XEN) Intel VT-d iommu 1 supported page sizes: 4kB.
> (XEN) Intel VT-d iommu 0 supported page sizes: 4kB.
> (XEN) Intel VT-d iommu 3 supported page sizes: 4kB.
> (XEN) Intel VT-d Snoop Control not enabled.
> (XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
> (XEN) Intel VT-d Queued Invalidation not enabled.
> (XEN) Intel VT-d Interrupt Remapping not enabled.
> (XEN) Intel VT-d Shared EPT tables not enabled.
> (XEN) I/O virtualisation enabled
> (XEN) - Dom0 mode: Relaxed
> (XEN) Interrupt remapping disabled
> (XEN) ENABLING IO-APIC IRQs
> (XEN) -> Using new ACK method
> (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
>
> If anything else is needed or any insight pops in your head, please
> advise accordingly.
> Any insight to debug the freeze?

If you have real serial console (not USB one), this would give reliable
log output, because in cause of freeze the most important log lines
probably didn't make it to the disk... In case of laptop, it worth
checking the port on docking station (if you have one).

You can also try to detach selected device(s) from your netvm to check
which one cause the freeze.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJWbcVQAAoJENuP0xzK19cse+oH/0I2nrRHCpcWa4VjVKh+LXlm
8w3bXlAndv1hPr6JKZQN7JOYXKe37299HwVNaI+uvdHsUZw0vmiL1fn+/4DnnL2o
hNQhoFTr465cSe0hjiHNK5ruca8dmMQSIoLrTUBP341Yl6B5fyODEILpi5uEVn49
nDqPm+LGBScNazZRNZ73fnNC/Uj7gn5h8w2WC23OcAEEQwFJWZQcZqRPuOfYG8Ls
JCw7TcgEeWVyMCz4yvPnOBo0ILShaKVVzsxGD+JqQQ/R8pDA1XA5JuXc9rBXE0hz
Ceg2emlmDx87PVjpO7YfiNG9MEZrDInANF128dYy4d6TE7HVsVqaTwn0W+cNUxQ=
=CgkJ
-----END PGP SIGNATURE-----

Thierry Laurion

unread,
Dec 13, 2015, 3:44:45 PM12/13/15
to Marek Marczykowski-Górecki, qubes-users

Thanks, I'll check it out.

David Schissler

unread,
Dec 15, 2015, 6:43:23 AM12/15/15
to qubes-users, marm...@invisiblethingslab.com
If you become very ambitious the entire Ivy Bridge Thinkpad lineup has some basic Coreboot support but hasn't disabled the ME stuff.  They say that Ivy Bridge is the last possible CPU line that has the possibility of being opened up.  The new Purism notebook that has partnered with Qubes has I guess chosen to not flick some code signing switch on the CPU and so in theory it can be opened up.  I've been following the trail for a bit and it seems like the best hopes are the current (very old) Libreboot endorsed models, the Ivy Bridge n*30 Thinkpad models and then the Purism models.

Thierry Laurion

unread,
Dec 22, 2015, 1:59:51 PM12/22/15
to qubes-users
David : coreboot is different then libreboot in the sense then it ships with binary blobs, including me and AMT.

Libreboot says that intel always was reluctant to disable such technologies, and purism is still pretty vague on the subject, specifying that no binary blog will be present in software, excluding bios, firmwares and the like by what seems to be a necessity.

Penryn and gm45, with p8xxx cpus coming with microcode 1067a seems to be, from my testing, the last intel laptops (t400, x200) that will be supported by libreboot, and as a consequence, have RYF certification.

I've posted on xen-devel today. Let's hope for the best. I'm waiting for my pcmcia to serial adapter to troubleshoot iommu netvm problem deeper.

I still believe that console serial output will be the same that wirh sync_console switch passed to xen.

We seem to be not so far from a x200 supported laptop by xen, and as a result, Qubes :)

Thierry Laurion

unread,
Dec 22, 2015, 2:03:35 PM12/22/15
to qubes-users
Sorry. Forgot to share xen-devel link.
https://www.mail-archive.com/xen-...@lists.xen.org/msg50200.html

Happy holidays!
This is what I wish for Christmas :)

Thierrt

Thierry Laurion

unread,
Jan 3, 2016, 5:33:45 PM1/3/16
to qubes-users
Hi all.


Last week, I realised I had one of my x200 that had one vpro sticker on it, and realised that the bios was offering amt option.
I flashed latest BIOS, verified microcode was 1067a(latest), swapped hard drives, configured amt to have a static IP on the network through RJ45 connection and logged everything I could through SOL connection with amtterm.

The laptop suffers from random hang that i am still not able to troubleshoot the cause, which disappears when iommu=0 is given to xen at boot.
qubes-hcl reports vt-d and vt-x and tpm. Unfortunately, there seems to be a problem with the i915 driver with iommu, but I can't get any output from the hang to make a proper bug report to Xen.



Here is some differential output from different options given at boot:

Longest boot before hang: 3400 seconds
file: x200_iommu-no-igfx_no-iommu_inclusive_mapping-intel_iommu-igfx_on-drm.debug.gz
Everything worked until hang: both RJ45 and wifi were connected, usbvm had flash drive connected and both whonix and feroda templates were updating themselves. As you can see, there is no onteresting output from drm.debug nor from xen. How can I get it more verbose?
( only iommu=no-igfx is active)
Boot options:
Xen:
(XEN) Command line: placeholder console=none dom0_mem=min:1024M dom0_mem=max:4096M console_timestamps=datems loglvl=all guest_loglvl=all sync_console console_to_ring com1=115200,8n1,amt console=com1 lapic=debug apic_verbosity=debug apic=debug iommu=no-igfx iommu=debug
Kernel:
[    0.000000] Command line: placeholder root=/dev/mapper/qubes_dom0-root ro rd.lvm.lv=qubes_dom0/root vconsole.font=latarcyrheb-sun16 rd.lvm.lv=qubes_dom0/swap rd.luks.uuid=luks-17c811ab-60ef-49d5-a017-8605d9bb433e earlyprintk=xen initcall_debug debug loglevel=10 console=hvc0 drm.debug=255


Shortest hang after boot: 111seconds (xen: iommu_inclusive_mapping=true)
file:x200_iommu-no-igfx_iommu_inclusive_mapping-intel_iommu-igfx_on-drm.debug.gz
(XEN) Command line: placeholder console=none dom0_mem=min:1024M dom0_mem=max:4096M console_timestamps=datems loglvl=all guest_loglvl=all sync_console console_to_ring com1=115200,8n1,amt console=com1 lapic=debug apic_verbosity=debug apic=debug iommu=no-igfx iommu_inclusive_mapping=true iommu=debug
[    0.000000] Command line: placeholder root=/dev/mapper/qubes_dom0-root ro rd.lvm.lv=qubes_dom0/root vconsole.font=latarcyrheb-sun16 rd.lvm.lv=qubes_dom0/swap rd.luks.uuid=luks-17c811ab-60ef-49d5-a017-8605d9bb433e earlyprintk=xen initcall_debug debug loglevel=10 console=hvc0 drm.debug=255
Hang at netvm startup.


Short hang after boot: 185seconds (xen iommu_inclusive_mapping=true, kernel intel_iommu=igfx_off)
file: x200_iommu-no-igfx_iommu_inclusive_mapping-intel_iommu-igfx_off-drm.debug.gz
(XEN) Command line: placeholder console=none dom0_mem=min:1024M dom0_mem=max:4096M console_timestamps=datems loglvl=all guest_loglvl=all sync_console console_to_ring com1=115200,8n1,amt console=com1 lapic=debug apic_verbosity=debug apic=debug iommu=no-igfx iommu_inclusive_mapping=true iommu=debug
[    0.000000] Command line: placeholder root=/dev/mapper/qubes_dom0-root ro rd.lvm.lv=qubes_dom0/root vconsole.font=latarcyrheb-sun16 rd.lvm.lv=qubes_dom0/swap rd.luks.uuid=luks-17c811ab-60ef-49d5-a017-8605d9bb433e earlyprintk=xen initcall_debug debug loglevel=10 intel_iommu=igfx_off console=hvc0 drm.debug=255
Hang at i915 randering


Observations:
-The hang is not specific to having usbvm booted on or not.
-The hang is not specific to wlan being connected or not.
-The hang is not linked directly to drm observed freeze.
-Having xen iommu_inclusive_mapping=true option makes the hang come quicker.
-Having kernel intel_iommu=igfx_off option makes the hang come quicker.

My debugging conclusions to date:
-sync_console seems to have the exact same logging functionnality (no output difference) then over serial.



Any other idea of troubleshoot I could do?

Thierry
x200_iommu-no-igfx_no-iommu_inclusive_mapping-intel_iommu-igfx_on-drm.debug.gz
x200_iommu-no-igfx_iommu_inclusive_mapping-intel_iommu-igfx_on-drm.debug.gz
x200_iommu-no-igfx_iommu_inclusive_mapping-intel_iommu-igfx_off-drm.debug.gz
Reply all
Reply to author
Forward
0 new messages