assigning a controller when they are used by sys-usb

[Franz]

Finally upgraded from R2 to R3 rc1 (only because it was impossible to upgrade fedora 21) and during installation selected the flag to use the new experimental sys-usb service. However I found no explanation how it works, so tried the following:

1. Restored a multimediavm to which one of the two controllers of Lenovo x230 was assigned. After restore the multimediavm does not start claiming that pci device 00:1a.0  is in use by  driver xenlight. domain sys-usb.

2. Looking into qvm-pci -l sys-usb it shows that the two controllers are both assigned to sys-usb, so removed one with
qvm-pci -d sys-usb 00:1a.0,
rebooted the computer, but again multimediavm does not start claiming that pci device 00:1a.0  is in use by  driver xenlight. domain multimediavm.

It seems a bug, but I do not know how this new arrangement works.

Best

Fran

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Is the multimediavm set to autostart? If so, maybe it fails to start
for some reason, but the device was already marked as used by
multimediavm. Xen 4.6 used in R3.1rc1 is must stricter about checking
device capabilities required to securely assign the device. For example
it checks if it is possible to reset the device before assigning (FLR or
other methods), or if any memory area is shared with other devices.
Check Xen logs for that (`xl dmesg`).

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJWeUePAAoJENuP0xzK19csJjgH/jDhnR45jqBOmwyFYKwGrK77
agaJeaxQsgyC9XpyCD6FjIREpNSQscCw6l8dVW09Zuoij832KiAMSEPbMWcInA1q
ocZhhRCr3KIRMjs6AHWJJn21dab3HA9kVVmeLg4ucJFP0jU5CWyGoOFHMLrJJ3Oe
TcNstXb4WxcrXgTYgET/gASFea6cykmDptVn9yCom3IRIoCdaApo428PkK72Fqxg
rdAjZZIS/J7pOPVevHCJfVEaal4cdqSdkoPZV4hOErTupiPW+iiAbqhAyEdTFqg8
vqn1zyJBMpLY6b5AcyuuHzDLM+cp0LLOaEOwELWvDtPCFnXA+V23A+rNTKF3Tp4=
=ArNU
-----END PGP SIGNATURE-----

[Franz]

On Tue, Dec 22, 2015 at 9:52 AM, Marek Marczykowski-Górecki <marm...@invisiblethingslab.com> wrote:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On Tue, Dec 22, 2015 at 09:40:49AM -0300, Franz wrote:
> Finally upgraded from R2 to R3 rc1 (only because it was impossible to
> upgrade fedora 21) and during installation selected the flag to use the new
> experimental sys-usb service. However I found no explanation how it works,
> so tried the following:
>
> 1. Restored a multimediavm to which one of the two controllers of Lenovo
> x230 was assigned. After restore the multimediavm does not start claiming
> that pci device 00:1a.0  is in use by  driver xenlight. domain sys-usb.
> 2. Looking into qvm-pci -l sys-usb it shows that the two controllers are
> both assigned to sys-usb, so removed one with
> qvm-pci -d sys-usb 00:1a.0,
> rebooted the computer, but again multimediavm does not start claiming that
> pci device 00:1a.0  is in use by  driver xenlight. domain multimediavm.
>
> It seems a bug, but I do not know how this new arrangement works.

Is the  multimediavm set to autostart?

no
 

If so, maybe it fails to start
for some reason, but the device was already marked as used by
multimediavm. Xen 4.6 used in R3.1rc1 is must stricter about checking
device capabilities required to securely assign the device. For example
it checks if it is possible to reset the device before assigning (FLR or
other methods), or if any memory area is shared with other devices.
Check Xen logs for that (`xl dmesg`).
 

(XEN) [VT-D) It's risky to assign 0000:00:1d.0 with shared RMRR at da2ba000 for dom3.

(XEN) [VT-D) It's disallowed to assign 0000:00:1a.0 with shared RMRR at da2ba000 for Dom5.

(XEN) XEN_DOMCTL_assign_device: assign 0000:00:1a.0 to dom5 failed (-1)

My note: 00:1d.0 is the other PCI device which remains assigned to sys-usb after I removed 00:1a.0 I suppose it is the other USB controller. But if it tells that it is risky is it worth to do it the same?

The other two lines detail what is not working.

Best

Fran

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

This basically means that 0000:00:1d.0 and 0000:00:1a.0 use some shared
resources, so shouldn't be assigned to different domains. Xen currently
doesn't detect whether you are trying to assign them to the same domain,
or a different ones, so you'll get that message in all the cases. But
you can override it with pci_strictreset=false VM property (this changes
the message from "disallowed" to "risky" and doesn't fail the
assignment).

In practice this means that you should assign both devices to the same
VM (either sys-usb or multimediavm). And set pci_strictreset=false there.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJWeVECAAoJENuP0xzK19csYwwH/19jyETmuYUA7PwJiGRyvvYm
lYim8NiQNq+7nlHlQOkRnjKoqEljdirPbOJZ7SHRiZpAjjNUnz/CR2KtNwbUUFS5
i6G77Y9wOVEkFbuHrSqHiOAgJHzN0x1hw5SmNWJd3jB0BeNnEZDzOzVbpuYZlBm6
z3LCPecAyU/fAlyk5jK7ZC6fYh+XIwuE14CJ2HvDIAYXHwcfuhlz2/ODCWkmV0IT
6LI8+5Gssivt4FERhgHcjqWWMVLi8mFgAIXaUpvk2vC9d0UBhyCiZj2pENHIlulE
IeucpwIn0jafP6DdxZlWmKsXCVMgG6hfRMVPN3F2ZGrMTiE0bbcaGbWb0XaRt3g=
=LgbH
-----END PGP SIGNATURE-----

[Franz]

Very interesting. I imagine that even Xen 4.6 does not allow to attach (as block devices) usb devices that are not disk, such as usb-to-serial or audio-card, so sometimes it may be necessary to assign both usb controllers to a VM different from sys-usb. In that case where is the proper place to put "pci_strictreset=false"? I looked into VMManager and was unable to identify this place. Where is it?

Additionally it may be interesting to add to the HCL a reference to the independence of USB controllers. It seems that if the US controllers share some resouces, there is a limitation to the possible uses of them.

Best

Fran

[Marek Marczykowski-Górecki]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

You can do that using qvm-prefs tool. It isn't available in Qubes
Manager (yet).

> Additionally it may be interesting to add to the HCL a reference to the
> independence of USB controllers. It seems that if the US controllers share
> some resouces, there is a limitation to the possible uses of them.

Yes, you're right. Generally it would be much better if/when we'll get
proper PVUSB support. But unfortunately it is still far from being
stable.

- --
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCAAGBQJWeXWyAAoJENuP0xzK19cs1YgH/RuEG3mQWLPi7wGVT7j3w0+c
EZ6B1NyqfyoWndS6u6Z4Hz1cJGRnzL/bunKM4tKJRogwY6Ct4NaZrpy7mAEHxtZ4
ZqaF4XlZIDYPibokhGhaGt7rt7kqpV9KMN9qlk3tUdzsU4jkIVAQ6JTYaCJV2Swc
YYjn/t3SKMF96Us+zVhANbBcf/SDM64glVeJiQgnP8U8qgHjGXAE7/uidjVHFBkO
ibZJziKW72hRHD8MkdiKaMWZCv5Pg81QmPNhmnWrYVWGcUcw+19OLScJ1mGVfQgr
uxu2lLAnmWeHjfA7Fx9kPJ0AmOKrGEFt4SJ+uSSF7hhhe3g8U0CiuzEcYBpLeMU=
=1sa3
-----END PGP SIGNATURE-----

[Matt McCutchen]

On Tuesday, December 22, 2015 at 11:09:27 AM UTC-5, Marek Marczykowski-Górecki wrote:

> Additionally it may be interesting to add to the HCL a reference to the
> independence of USB controllers. It seems that if the US controllers share
> some resouces, there is a limitation to the possible uses of them.

Yes, you're right. Generally it would be much better if/when we'll get
proper PVUSB support. But unfortunately it is still far from being
stable.

I just ran into this.  I have a Lenovo ThinkPad L430 with USB controllers 00:14.0, 00:1a.0, and 00:1d.0, all of which share an RMRR at d9cd8000 according to the Xen logs I got when I assigned them to VMs.  But before assuming it was safe to attach these three controllers as a group to an untrusted VM, I wanted to confirm that they don't share an RMRR with any other devices.  Maybe it's common sense to hardware folks that they wouldn't, but I didn't know.  I didn't want to actually attach all of the other devices to VMs to check for Xen log messages, so the best method I came up with was:

- In dom0: sudo qubes-dom0-update acpica-tools && acpidump >dump && acpixtract -s RMAD dump
(To all appearances, this is the DMAR table; I don't know why the name is RMAD here.  Some strange byte order?)
- Open rmad.dat in a hex editor and decode it manually according to the definitions in http://www.intel.com/content/dam/www/public/us/en/documents/product-specifications/vt-directed-io-spec.pdf sections 8.1 and 8.4.

I guess no one yet saw a need to write user-friendly tools to view DMAR/RMRR information. :(

And indeed, the RMRR at d9cd8000 is not shared with any other devices, and there are no other shared RMRRs; my "VGA compatible controller", 00:02.0, has a unique RMRR at db800000.  I suppose I could submit this information to the HCL, but I don't want to spend any more time right now.