Clevo P750DM2-G and Thunderbolt 3: Potential bug with daisy chained docks and a workaround (Might apply to other notebooks as well) + news on AMD's Ryzen NPT performance fix for KVM and VFIO.

[Dennis Mungai]

Hello guys,

I've been using an Origin Eon 15-X based on the Clevo P750DM2-G, with the following specs:

Processor: Intel Core i7 7700k @ 4.2 Ghz base, 4.5 GHz Turbo.
GPU: GeForce GTX 1070 8 GB MXM 3.0b
RAM: 64 GB 2400MHz SODIMMs (4 * 16 GB sticks)
Storage: 2 * 2 TB Samsung 960 Pro NVMe SSDs + 2 * 2 TB Crucial MX300 SATA III SSDs.
OS: Windows 10 Pro, Build version 1709 (16229.19)

I always plug my accessories to a Cablematters Thunderbolt 3 dock that's also daisy-chained to an older Caldigit Thunderbolt 2 docking station and an Akitio Thunder 2 Quad Mini storage enclosure, as shown below from AIDA64's Motherboard > Chipset information tab:




​

Device chain as shown on Windows:


​

Connection: PCI-E 2.0 x4 port #1 In Use @ x4 (Intel DSL6540 (Alpine Ridge 4C 2015) Thunderbolt 3 Bridge --> Intel DSL6540 (Alpine Ridge 4C 2015) Thunderbolt 3 Bridge --> Intel DSL6540 (Alpine Ridge 4C 2015) Thunderbolt 3 NHI, Intel DSL6540 (Alpine Ridge 4C 2015) Thunderbolt 3 Bridge --> Intel JHL6540 (Alpine Ridge 4C 2016) Thunderbolt 3 Bridge (C step) --> Intel JHL6540 (Alpine Ridge 4C 2016) Thunderbolt 3 Bridge (C step) --> Fresco Logic USB 3.0 xHCI Controller, Intel JHL6540 (Alpine Ridge 4C 2016) Thunderbolt 3 Bridge (C step) --> Intel I210 Gigabit Network Connection): In Use @ x4 (Intel DSL6540 (Alpine Ridge 4C 2015) Thunderbolt 3 Bridge --> Intel DSL6540 (Alpine Ridge 4C 2015) Thunderbolt 3 Bridge --> Intel DSL6540 (Alpine Ridge 4C 2015) Thunderbolt 3 NHI, Intel DSL6540 (Alpine Ridge 4C 2015) Thunderbolt 3 Bridge --> Intel JHL6540 (Alpine Ridge 4C 2016) Thunderbolt 3 Bridge (C step) --> Intel JHL6540 (Alpine Ridge 4C 2016) Thunderbolt 3 Bridge (C step) --> Fresco Logic USB 3.0 xHCI Controller, Intel JHL6540 (Alpine Ridge 4C 2016) Thunderbolt 3 Bridge (C step) --> Intel I210 Gigabit Network Connection)

Using the default BIOS settings that have VT-d enabled (under Advanced Chipset Control menu), connecting that setup results in the onboard SATA controller being "unplugged" automatically. Windows makes the device disconnected sound and all the SATA drives disappear from the system. Only a reboot with the Thunderbolt dock unplugged fixes this.

Contacting Origin support to raise the issue didn't bear much fruit, so I gave up on it, till yesterday.

As I was setting up KVM on Ubuntu 16.04LTS and migrating older VMS, I took a look at the IOMMU groups and immediately observed a disturbing issue: They were messed up. For one, the GPU's audio bus was mangled with the dock's USB ports! That gave me a hint: What if I disabled VT-d in the BIOS? Would that fix what I had observed on Windows?

So, turns out that disabling the offending setting does fix the issue, and now, as I speak, the dock is plugged in with the storage array daisy-chained and fully functional. And of course, the onboard SATA controller isn't being disabled anymore.


TL-DR: If you encounter a Thunderbolt dock related issue on this machine or a similar setup, disable VT-d in the BIOS first.

Remember, VT-d for directed I/O is not the same as VT-x extensions.

And unrelated, there's a fix for AMD's Ryzen NPT (Nested Paging Tables) acceleration performance with GPU passthrough, as detailed in this patchwork: https://lists.linuxfoundation.org/pipermail/iommu/2017-October/024822.html

So, if you're on Ryzen running a similar workload, take note.

 

[Kennedy Mwenja]

Damn dude! *clap* *clap* *clap*

And it ends up being a BIOS setting?! Assuming I got the gist of this (most likely don't), is this a bug in the chipset supporting VT-d or a standard breaking implementation from the Thunderbolt dock?

[Dennis Mungai]

Yes, Kevin.

It's a BIOS setting, which is set by default, and their implementation of it is horridly broken.

They contacted me a while back to confirm that the issue lies on their side. So far, only one OEM has fixed this , the Portuguese OBSIDIAN PC systems. Clevo is also aware of the issue and they're pushing out firmware updates as we speak.

--
You received this message because you are subscribed to the Google Groups "Nairobi GNU/Linux User Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nairobi-gnu+unsubscribe@googlegroups.com.
To post to this group, send email to nairo...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.