[PATCHES] MSI, MSI-X and other

[Alexander Kudryavtsev]

Hello everyone!

I represent a team working in the virtualization area at ISP RAS
(http://ispras.ru/en/). Since August 2011 we started investigating
Palacios. Our first goal was to reproduce Palacios authors results.

While working with Palacios, we needed some features. We implemented
these features, here is the list (not complete):

1. PCI Capabilities passthrough support (patch 0002). It is required by
many devices to see their capabilities to work correctly. Only PCI (256
bytes) capabilities are supported.
2. PCI MSI (message-signalled interrupts) support (patches 3, 4, 5, 18,
20, 28, 32). It includes MSI interrupt router and support for MSI
capability passthrough. We tested MSI with 2.6.18 and 2.6.32 guests on
different hardware. MSI capability passthrough required real APIC ID
retrieving, so we changed OS interface. Patch for Kitten OS is attached.
3. PCI MSI-X support (patches 12, 29, 31). It allows many interrupts per
device and is required on some server hardware. MSI-X support is
included inside MSI router. Code for MSI-X passthrough is present, but
currently it is not working.
4. Support for deferred EOI (End of interrupt) for passed through
devices (patches 6-7). It is required for some devices to work. Support
from host OS is also required, patch for Kitten is attached.
5. VMX, EPT related fixes (patches 8, 9, 10, 13, 22, 23, 24, 26). Some
of them became not relevant. EPT fix enables caching of guest memory and
of EPT tables, highly increasing guest speed.
6. PCI-2-PCI bridge support (patch 15), SMBIOS device support (patch 17)
- these two patches were required by one of our test Infiniband card
driver to work inside guest.

Kitten patches are the following:
1. Report real APIC ID to Palacios. It is required for MSI/MSI-X
passthrough handling. (palacios-get-apic-id)
2. Do not perform EOI for devices owned by Palacios. (no-palacios-eoi)

We propose these patches with the hope that they may help developing
Palacios. These patches could be applied onto commit
840a7c8a40cbb875c1c748845f093ff898e99766 and represent our local branch.
Some patches contain the same fixes which can be found in Palacios
development branch. We do not rebase our changes on Release-1.3 branch
since we don't know which of these changes you would like to use and our
resources are limited. We will be happy to answer your questions.

[Peter Dinda]

The V3VEE team and I would like to express our appreciation for this
extensive set of patches, as well as for the previous patches for MSI
and QUIX86 that your team previously provided to our project. Our
handling of the patches has delayed due to the 1.3 release effort (and
now the holidays), but we are in the process of reviewing them
carefully. We have also added your patches to the contributed code
section of our web site, with full attribution. We fully understand
the effort involved in your team's work and are thankful for it.

Peter Dinda

On Dec 12, 6:12 am, Alexander Kudryavtsev <a.o.kudryavt...@gmail.com>
wrote:

>  0032-Fixed-bug-with-spoiled-MSI-address-and-data-cache.patch
> 1KViewDownload
>
>  0031-Fixed-bug-with-using-of-not-fixed-src-value-in-MSI-X.patch
> 2KViewDownload
>
>  0030-Minor-fixes.patch
> 4KViewDownload
>
>  0029-Added-MSI-X-BAR-update-for-32-bit-BARs.patch
> 2KViewDownload
>
>  0028-May-be-fix-for-MSI-data-not-updated.patch
> < 1KViewDownload
>
>  0027-Patches-for-making-progress-on-ProLiant-node.patch
> 3KViewDownload
>
>  0026-Improved-vpid-and-invept-support-cosmetic-changes.patch
> 22KViewDownload
>
>  0025-Fixed-AMD-booting.patch
> 2KViewDownload
>
>  0024-Added-invept-and-do-not-ignore-PAT.patch
> 2KViewDownload
>
>  0023-Fixes-for-UG-and-for-building-w-o-SVM-support.patch
> 60KViewDownload
>
>  0022-Enable-VPID-for-EPT-XXX-check-VPID-is-supported.patch
> < 1KViewDownload
>
>  0021-irq-balance-patch.patch
> 12KViewDownload
>
>  0020-Fix-for-MSI-pending-support.patch
> 1KViewDownload
>
>  0019-Hide-some-annoying-messages.patch
> 2KViewDownload
>
>  0018-Fix-for-passed-through-MSI-support.patch
> 3KViewDownload
>
>  0017-Add-virtual-smbios-device-support.patch
> 15KViewDownload
>
>  0016-VMM-Emulator-SCAS-CMP.patch
> 6KViewDownload
>
>  0015-PCI-2-PCI-Bridge-support.patch
> 8KViewDownload
>
>  0014-Experimental-serial-passthrough-feature.patch
> 3KViewDownload
>
>  0013-VMX-MSR-temporary-fix.-XXX-do-something-with-it.patch
> 1KViewDownload
>
>  0012-PCI-MSI-X-initial-support.patch
> 64KViewDownload
>
>  0011-Allow-to-specify-function-number-for-passed-through-.patch
> 1KViewDownload
>
>  0010-Enable-caching-in-EPT-tables.-XXX-hooks-are-not-work.patch
> 3KViewDownload
>
>  0009-EPT-fixes.patch
> 13KViewDownload
>
>  0008-Configuration-entry-to-deny-using-VMX-unrestricted-g.patch
> 1KViewDownload
>
>  0007-Support-for-deferred-EOI-of-passed-through-interrupt.patch
> 6KViewDownload
>
>  0006-Update-for-v3_pci_raise_irq-v3_raise_irq-router_ops-.patch
> 19KViewDownload
>
>  0005-PCI-MSI-passthrough-support.patch
> 16KViewDownload
>
>  0004-PCI-MSI-router-support.patch
> 27KViewDownload
>
>  0003-Support-for-real-APIC-ID-retrieving-from-host-OS.-No.patch
> 3KViewDownload
>
>  0002-PCI-Capabilities-passthrough-support.patch
> 28KViewDownload
>
>  palacios-get-apic-id
> < 1KViewDownload
>
>  no-palacios-eoi
> 2KViewDownload

[Alexander Kudryavtsev]

Thanks for your reply! It is nice to see our code in the contributed
section of Palacios web site!

Today we fixed a bug which caused MSI-X passthrough not to work; patch
is attached. Now MSI-X passthrough seems to be working, at least we can
ping MSI-X-enabled Ethernet (Intel Corporation 82576) and Infiniband
(Mellanox Technologies MT26438) cards and run some MPI tests.

[Peter Dinda]

Thank you again. I've placed this new patch in the contributions
section of the web site.

Peter


On Dec 20 2011, 9:53 am, Alexander Kudryavtsev <al...@ispras.ru>
wrote:

>  0033-Minor-fixes-found-during-MSI-X-debugging.patch
> 6KViewDownload