Sm Bus Controller Driver Asrock

[Caleb Nelands]

Usecase for the build: I work and game and run various projects from my home office, my daily drivers were two Core i5 6th and 7th gen running OSX plus Virtualbox and Windows exclusively for gaming.

The server will run these two workloads virtualized, plus a bunch of accessory VMs I use daily for y work/experiments

The only issue I have encountered so far (other than spending way too much money for parts that are not readily available here in southern europe) is the fact that the BIOS sees only my 32GB DIMMS, while the 16GB DIMMS are not showing either in the BMC inventory or in the BIOS, but are seen and work in the OS

In order to get maximum perfomance and minimum latency out of the OSX and Windows VMs I have adapted some existing Proxmox init scripts to make sure the following happens when a VM that needs low latency is started:

So far so good, perfomance is more than good, I get about 3GB/S in sequential read and write on the OSX VM using a 2TB MZ-V7S2T0 970 EVO Plus , and decent performance out of the 5700XT in windows (mainly Apex Legends)

I had to try a bunch of USB PCI cards and settled for the non SATA powered version of these cards: Inateck KTU3FR-4P 4 porte PCI Express card desktop usb Controller, Fresco FL1100 chipset, they are connected to a USB hub each on one port, and a Zoom audio interface on the other (Livetrak L-12 on the OSX side, Zoom U-24 on the windows side)

Hi, the main reasons were:

1 - I already had the micro ATX cases

2 - The Romed6 has two more network interfaces, I thought they would come in handy for passtrough

3- I thought 3x slim PCI ports would give me more flexibility than two additional 16x slots

4- I am not planning on running a CPU that would effectively make use of the additional 2x dimms, if my understanding of the mapping between chillers and access to memory is correct

The OSX VM causing Latency spikes on the Windows VM when gaming/streaming audio: I have solved this by shuffling around the PCI cards in order for them to be in separate physical groups/slots, if that makes sense. Originally the OSX NVME was in the same Root Complex as the Windows GPU

The Windows GPU (XFX 5700XT 8GB) was mostly working, but if it crashed for any reason while gaming, it could not be reset other than with a full power down of the Proxmox host (not even a reset would suffice). This was really annoying, I tried multiple possible workarounds/fixes, but then just gave up and exchanged it with a 3070ti. The behaviour is the same (some games still crash, sometimes) but a shutdown and restart of the VM fixes it, the only quirk I still have is that when this happens the reboot takes anywhere from 1 to 2 minutes for the bios screen to show up, and I have to power off and on any usb device attached to the passed through USB pci card

The 3070ti properly supports PCIe Active State Power Management, the 5700XT could not, and enabling it would cause endless pci bus resets: this means that when the WIndows GPU is not in use it can be actually set to a lower power state, saving some 20-30W when idling (finger in the air measurement)

This is my current layout for the pci bus, In the end I had to make use of all three slim line pci extension boards to be able to move stuff in a way there were no sharing between passed through pci cards and different VMs

I have also played around a little with airflow, mostly because of my case not really being suited for this type of motherboard.

Room temperature today is on the warm side (26C), and these are ths temps registered with my main OSX VM running :

What I need is two (or more) IOMMU-separable USB-controllers, so that one can be given to a VM and the other can stay in the host. Just to be sure that the problem is what I think: can you confirm that you could not find two IOMMU-separable USB ports on the ROMED6U motherboard, that also map to different ports (i.e. so that one becomes available for passthrough)?

Because I need only two. It is a bit confusing, as there seems to be two IOMMU-separable controllers on the EPYC IO die (your 05:00.3 and 44:00.3 if I read your lspci -tv right). I have the corresponding two controllers on my Supermicro H12SSL-base EPYC system, and they end up in different IOMMU groups, but both seem to map to the same couple of physical ports.

Another question - when you replaced the TT cases, did you look for a new mATX case first, without finding a good alternative for your combination of hardware? Or did you decide from start to go with ATX?

Hi, no particular issues with the GPUs, followed the proxmox-vfio docs at:

_passthrough

and had no issues with either AMD or Nvidia based cards

I am not passing through, and have never tried to, Sata controllers.

I used to pass through to my Osx VM and Windows VM dedicated NVME drives, off a 2 slot Nvme adapter, using bifurcation, but moved to passing through normal virtual disks so that I could manage/protect them with ZFS snapshots

3a8082e126