Re: [qubes-devel] Re: 2012 Macbook Air -- Solved Booting Pains -- First Impressions

[Marek Marczykowski]

On 22.03.2013 19:48, knock...@gmail.com wrote:
> I just got a MacBook Air 2012 (version 5,1) up and running. It supports
> Vt-d (although because Apple goofed a bit of their ACPI tables, io-apic
> interrupt remapping does not work, although I plan on fixing this up with a
> minor Xen patch). My only regret now is getting a system with 4GB of RAM,
> as it is not upgradable.
>
> In the end, I had better luck installing from a USB flash drive. However,
> you cannot use a USB 3 drive - it must be USB 2.0 due to what appears to be
> issues in this machine's xHCI implementation. A simple 'dd' (which I
> performed via a live FC18 boot stick) worked just fine. I relied on rEFInd
> for all booting, including the Qubes installer from USB flash, as it seems
> to be more robust that the ALT key boot method.
>
> Another source of difficulty was figuring out why Qubes would not boot
> after the installer appeared to run correctly. Booting was not an "out of
> the box" experience for me - perhaps because I had never used Bootcamp or
> previously run gptsync. Even with rEFInd installed, Qubes would not boot.
> Ultimately, what was needed was to run gdisk (available for use on OS X),
> and setting up the "hybrid MBR" that the MacBook appears to use as a signal
> to provide BIOS emulation (I believe this is essentially the same as the
> "gptsync /dev/sda" mentioned above). Figuring out how and why things boot
> on the MacBook, especially when relying on BIOS emulation, is a real pain.
> It does not help that grub2 is orders of magnitude more complicated than
> legacy grub. As I understand it, once Qubes updates to Xen 4.2 or 4.3,
> booting on Macs will become tremendously easier, as a pure EFI boot chain
> is available in the form of xen.efi. With an appropriate configuration,
> rEFInd will be able to avoid grub entirely.
>
> A high-level overview of my install process (ignoring many missteps along
> the way):
> - shrink OS X via Disk Utility
> - [optional] run the Recovery Assistant Backup, to have the recovery
> partition available on a bootable USB flash drive
> - install rEFInd
> - install gdisk
> - [optional] with gdisk, delete the recovery partition (I only have a 128GB
> drive, so I wanted to reclaim this space)
> - with gdisk, create a 128MB type EF02 partition after the OS X partition
> (I gave it partition # 99), as the Mac firmware apparently wants to see
> this much space after the OS X partition. The remaining space can be left
> unpartitioned.
> - boot via rEFInd into the Qubes installer on USB flash
> - install (should just use the available space)
> - reboot into OS X
> - run gdisk, and create a hybrid MBR. Specifically, the Qubes boot
> partition needs to be included in the hybrid MBR. To be safe, you might
> want to make sure the partition numbers for the Qubes boot partition are
> the same on GPT and MBR. For example, they are both /dev/sda3 under both
> partitioning schemes on my system. Only tag the Qubes boot partition as
> bootable. I looks like you can safely use the first two letters of the GPT
> type as the MBR type (e.g., the OS X partition is 'AF', and the Qubes boot
> partition is '07').
> - boot into Qubes. It should be trivial to dual boot via rEFInd at this
> point.

Thanks for detailed instruction. I will link your post to our wiki.

> The BIG problem, as noted above, is that the modules for the Broadcom
> BCM43224 are not included in Qubes, either with the original 3.7.6 kernel
> or in the later 3.7.4 kernel. Now I have a chicken & egg problem: I could
> respin the kernel to build the needed modules, but I don't have the network
> access needed to pull in the needed RPMs with GCC, etc. for building.
> Although I can reboot into OS X and do one-off downloads, then mount the OS
> X FS in Qubes to get these files, yum has a lot of RPM dependencies in
> connection with the needed RPMs that would seem to make downloading them
> impractical.
>
> It would be greatly appreciated if you would PLEASE respin the 3.7.4 kernel
> with the config update noted by Steve. Specifically, include the following
> in the kernel config:
>
> CONFIG_BRCMUTIL=m
> CONFIG_BRCMFMAC=m
> CONFIG_BRCMFMAC_SDIO=y
> CONFIG_BRCMFMAC_SDIO_OOB=y
> CONFIG_BRCMFMAC_USB=y

Sure, can do. Just compiled kernel-3.7.4-3, now it is uploading to unstable repo.

--
Best Regards / Pozdrawiam,
Marek Marczykowski
Invisible Things Lab

[Alex Dubois]

Hi Eric,

I am interested in this. I have a MacBook Pro with a CPU that support VT-D but the ChipSet didn't.
Could you confirm which ChipSet is in your MacBook Air.

Thanks,
Alex

[Joanna Rutkowska]

On 03/23/13 10:59, Alex Dubois wrote:
> Hi Eric,
>
> I am interested in this. I have a MacBook Pro with a CPU that support VT-D but the ChipSet didn't.
> Could you confirm which ChipSet is in your MacBook Air.
>
> Thanks,
> Alex
>

Perhaps Eric can just run qubes-hcl-report and send us the output? (Note
that you might need to update to latest core to have this tool).

joanna.

[Joanna Rutkowska]

On 03/23/13 13:12, knock...@gmail.com wrote:
>
> PS: Given the hit-or-miss nature of laptop Vt-d support, maybe it is worth
> rolling up a small bootable USB drive program that can be used to run an
> in-store boot test of Xen to see what virtualization features are available.
>

You can easily create such USB stick -- just proceed with Qubes
installation using your stick as the destination drive. You should then
be able to boot into full Qubes in a shop, so not only check VT-d but
most other things -- video, networking, sleep, etc.

joanna.

>
> On Saturday, March 23, 2013 6:33:37 AM UTC-4, joanna wrote:
>>
>> On 03/23/13 10:59, Alex Dubois wrote:
>>> Hi Eric,
>>>
>>> I am interested in this. I have a MacBook Pro with a CPU that support
>> VT-D but the ChipSet didn't.
>>> Could you confirm which ChipSet is in your MacBook Air.
>>>
>>> Thanks,
>>> Alex
>>>
>>
>> Perhaps Eric can just run qubes-hcl-report and send us the output? (Note
>> that you might need to update to latest core to have this tool).
>>
>> joanna.
>>
>>> On 22 Mar 2013, at 21:30, Marek Marczykowski <

>> marm...@invisiblethingslab.com <javascript:>> wrote:

[7v5w7go9ub0o]

On 03/23/13 12:33, Joanna Rutkowska wrote:
> On 03/23/13 13:12, knock...@gmail.com wrote:
>>
>> PS: Given the hit-or-miss nature of laptop Vt-d support, maybe it
>> is worth rolling up a small bootable USB drive program that can be
>> used to run an in-store boot test of Xen to see what
>> virtualization features are available.
>>
>
> You can easily create such USB stick -- just proceed with Qubes
> installation using your stick as the destination drive. You should
> then be able to boot into full Qubes in a shop, so not only check
> VT-d but most other things -- video, networking, sleep, etc.

Perhaps a copy of the HCL analysis script could be conveniently included
in the Qubes installation - facilitating a quick analysis for us newbies!?

[Joanna Rutkowska]

All you need to do is to update core packages in Dom0. Unfortunately we
merged the Zrubi's script only after releasing R2B2, so it didn't make
it there. But again, you just need to install Qubes to a USB stick and
then, once running it from the stick, perform Dom0 update via
qubes-dom0-update.

j.

[Eric Shelton]

An update on where things stand with my present configuration on the
MacBook Air 2012.

Based on the information here:
http://people.canonical.com/~sforshee/enablement/quantal/macbookair5.1/apic-notes.txt
and ACPI tables for other Ivybridge systems, it is clear that there is
a minor bug in the MacBook Air 2012 ACPI tables. Specifically, the
single DRHD for an IOAPIC in the DMAR table incorrectly identifies an
IOAPIC with an ID of 0, when it should be 0x02 (to correspond to the
IOAPIC identified in the APIC table). In other words, offset of 0x5C
in the DMAR table should be 0x02, not 0x00.

The attached xen-dmar.patch is a complete hack that forces the ID to
0x02, which depends on there being only one IOAPIC. However, I think
it is much better to get interrupt remapping and proper use of the
IOAPIC, rather than just disabling it as was done in August 2012. A
copy of the output from 'xl dmesg' is attached.

In my quest to get WIFI working, I build the 3.8.4 kernel for this
system. A copy of my .config file is attached. Note that in the 3.8
kernel series, ehci-hcd has split out some of the code into a new
ehci-pci module. If initramfs does not include this module, you will
not have access to the built-in keyboard and not be able to type in
your encryption password (although you can get around the issue by
plugging in an external USB keyboard). You can make sure the module
is included by adding the attached file ehci-missing-driver.conf to
/etc/dracut.conf.d/ Some other patches have to be made to the
Qubes-provided patches to get 3.8.4 to build, but I have not fished
those out.

WIFI requires use of the brcmsmac module. Additionally, if you do not
use permissive mode for PCI passthrough, the system will hang when
netvm loads the WIFI driver (is a domU supposed to be able to cause a
hang, assumedly because of an improper IO access)? The attachment
qubes-permissive-netvm.service, which is to be put in
/etc/systemd/system (or where the links in there point to), enables
permissive mode for the WIFI PCI device. The attached
qubes_netvm.service was modified to make sure netvm is started after
permissive mode is set.

Only one thing remains at this time - suspend to RAM will not wake up
successfully with the WIFI module loaded (suspend works fine
otherwise). My initial attempt with /lib/qubes/prepare-suspend has
not yet worked, and I have not yet dug into it. If anyone has
something like this working, I would love to see it.

Anyways, this system is pretty awesome now. The only thing I want to
change, as noted before, is to have 8GB of RAM, instead of 4GB.

Hopefully this provides ample information for anyone else giving this a go.


On Sat, Mar 23, 2013 at 8:12 AM, <knock...@gmail.com> wrote:
> I am happy to run the report, but I will have to get to it a little bit
> later.
>
> Alex - I saw your earlier report on the MacBook Pro, and was worried that
> the Air would not likewise not provide Vt-d. As I think you know, its
> availability depends on not just the CPU, but also having southbridge (or at
> least chipset) support. As it appears Apple is not making use of Vt-d at
> this time, its availability on their hardware is matter of luck.
>
> Working from an iFixIt-type teardown (for 13 inch, although I have an 11
> inch model), indicating an "Intel E201B953 SLJ8B" chip, a search for "Intel
> SJJ8B" points you to:
>
> http://ark.intel.com/products/64336/Intel-BD82QS77-PCH
>
> Which indicates Vt-d support. Somewhere else I came across a page showing
> the particular CPU supports Vt-d (again, luck, as I think many i5 CPUs do
> not support Vt-d). I recall the CPU does not support TXT (although the
> chipset does), and that you have to step up to the i7 CPU to get that
> support (although I suspect that the Mac firmware breaks any chain of trust
> before you get to a Qubes boot). Somewhere I saw a matrix showing the
> various combinations, probably in connection with Intel's announcement of
> the above chipset.
>
> For what it is worth, I think the 13 inch retina MacBook Pro uses the same
> chipset.
>
> Best,
> Eric

>
> PS: Given the hit-or-miss nature of laptop Vt-d support, maybe it is worth
> rolling up a small bootable USB drive program that can be used to run an
> in-store boot test of Xen to see what virtualization features are available.
>
>
>

> On Saturday, March 23, 2013 6:33:37 AM UTC-4, joanna wrote:
>>

[Joanna Rutkowska]

On 04/02/13 04:01, Eric Shelton wrote:
> An update on where things stand with my present configuration on the
> MacBook Air 2012.
>
> Based on the information here:
> http://people.canonical.com/~sforshee/enablement/quantal/macbookair5.1/apic-notes.txt
> and ACPI tables for other Ivybridge systems, it is clear that there is
> a minor bug in the MacBook Air 2012 ACPI tables. Specifically, the
> single DRHD for an IOAPIC in the DMAR table incorrectly identifies an
> IOAPIC with an ID of 0, when it should be 0x02 (to correspond to the
> IOAPIC identified in the APIC table). In other words, offset of 0x5C
> in the DMAR table should be 0x02, not 0x00.
>
> The attached xen-dmar.patch is a complete hack that forces the ID to
> 0x02, which depends on there being only one IOAPIC. However, I think
> it is much better to get interrupt remapping and proper use of the
> IOAPIC, rather than just disabling it as was done in August 2012. A
> copy of the output from 'xl dmesg' is attached.
>

Have you tried to send this patch to xen-devel. I think it might be
useful for Xen to have a collection of DMAR workarounds for various
machines, as I expect there are lots of hardware with broken ACPI
tables, specifically with regard to VT-d, out there?

> In my quest to get WIFI working, I build the 3.8.4 kernel for this
> system. A copy of my .config file is attached. Note that in the 3.8
> kernel series, ehci-hcd has split out some of the code into a new
> ehci-pci module. If initramfs does not include this module, you will
> not have access to the built-in keyboard and not be able to type in
> your encryption password (although you can get around the issue by
> plugging in an external USB keyboard). You can make sure the module
> is included by adding the attached file ehci-missing-driver.conf to
> /etc/dracut.conf.d/ Some other patches have to be made to the
> Qubes-provided patches to get 3.8.4 to build, but I have not fished
> those out.
>
> WIFI requires use of the brcmsmac module. Additionally, if you do not
> use permissive mode for PCI passthrough, the system will hang when
> netvm loads the WIFI driver (is a domU supposed to be able to cause a
> hang, assumedly because of an improper IO access)?

Well, no, it shouldn't! :) I would encourage you to report it and
discuss at xen-devel list.

Also, one should not really use the permissive mode, but ideally find
out which specific PCIe config register the device wanted to access (and
why?) and enable this access in the pcback quirks config file...

> The attachment
> qubes-permissive-netvm.service, which is to be put in
> /etc/systemd/system (or where the links in there point to), enables
> permissive mode for the WIFI PCI device. The attached
> qubes_netvm.service was modified to make sure netvm is started after
> permissive mode is set.
>
> Only one thing remains at this time - suspend to RAM will not wake up
> successfully with the WIFI module loaded (suspend works fine
> otherwise). My initial attempt with /lib/qubes/prepare-suspend has
> not yet worked, and I have not yet dug into it. If anyone has
> something like this working, I would love to see it.
>

So if you manually rmmod your wifi module in the netvm, then S3 works
fine, and if you let the script do it, it wont? Sounds fishy...

> Anyways, this system is pretty awesome now. The only thing I want to
> change, as noted before, is to have 8GB of RAM, instead of 4GB.
>
> Hopefully this provides ample information for anyone else giving this a go.
>

Seems like you put lots of efforts into getting Qubes working on your
MacBook, thanks! And hope you will find the effort worthwhile.

joanna.

[Eric Shelton]

Responses inline...

On Apr 2, 2013, at 5:09 AM, Joanna Rutkowska

<joa...@invisiblethingslab.com> wrote:

> On 04/02/13 04:01, Eric Shelton wrote:
>> An update on where things stand with my present configuration on the
>> MacBook Air 2012.
>>
>> Based on the information here:
>> http://people.canonical.com/~sforshee/enablement/quantal/macbookair5.1/apic-notes.txt
>> and ACPI tables for other Ivybridge systems, it is clear that there is
>> a minor bug in the MacBook Air 2012 ACPI tables. Specifically, the
>> single DRHD for an IOAPIC in the DMAR table incorrectly identifies an
>> IOAPIC with an ID of 0, when it should be 0x02 (to correspond to the
>> IOAPIC identified in the APIC table). In other words, offset of 0x5C
>> in the DMAR table should be 0x02, not 0x00.
>>
>> The attached xen-dmar.patch is a complete hack that forces the ID to
>> 0x02, which depends on there being only one IOAPIC. However, I think
>> it is much better to get interrupt remapping and proper use of the
>> IOAPIC, rather than just disabling it as was done in August 2012. A
>> copy of the output from 'xl dmesg' is attached.
>>
>
> Have you tried to send this patch to xen-devel. I think it might be
> useful for Xen to have a collection of DMAR workarounds for various
> machines, as I expect there are lots of hardware with broken ACPI
> tables, specifically with regard to VT-d, out there?
>

I can imagine several mechanisms for working with broken ACPI tables
where a fix can be identified:

1) Recent Linux kernels have added a mechanism for loading replacement
ACPI tables via initramfs. Perhaps something like this might be done.

2) A boot command line parameter might be added such as "acpi-patch 8388205c 02"

3) In terms of user friendliness, ACPI quirks code is probably best,
to save people from having to scour the Internet to identify and
remedy system issues.

>> In my quest to get WIFI working, I build the 3.8.4 kernel for this
>> system. A copy of my .config file is attached. Note that in the 3.8
>> kernel series, ehci-hcd has split out some of the code into a new
>> ehci-pci module. If initramfs does not include this module, you will
>> not have access to the built-in keyboard and not be able to type in
>> your encryption password (although you can get around the issue by
>> plugging in an external USB keyboard). You can make sure the module
>> is included by adding the attached file ehci-missing-driver.conf to
>> /etc/dracut.conf.d/ Some other patches have to be made to the
>> Qubes-provided patches to get 3.8.4 to build, but I have not fished
>> those out.
>>
>> WIFI requires use of the brcmsmac module. Additionally, if you do not
>> use permissive mode for PCI passthrough, the system will hang when
>> netvm loads the WIFI driver (is a domU supposed to be able to cause a
>> hang, assumedly because of an improper IO access)?
>
> Well, no, it shouldn't! :) I would encourage you to report it and
> discuss at xen-devel list.
>
> Also, one should not really use the permissive mode, but ideally find
> out which specific PCIe config register the device wanted to access (and
> why?) and enable this access in the pcback quirks config file...
>

That's what I thought. I suspect there will not be any Xen developers
with this particular wifi card. If so, I'll have to identify the
quirked registers. Any idea how this is done? One possible solution
would be to run netvm in an HVM, and have qemu log out-of-bounds IO
accesses. Unfortunately, I suppose the hang suggests there is a bug
tied to this.

It would be helpful to test this out on a version of Xen more recent
than 4.1.2 (although I recognize MANY patches are applied for the
Qubes Xen) before turning to xen-devel. Is there any reason I cannot
run Qubes R2B2 with Xen 4.1.5 pre? Even better, is there a Qubes dev
code base that would allow me to use 4.2.1 or 4.3 unstable?

>> The attachment
>> qubes-permissive-netvm.service, which is to be put in
>> /etc/systemd/system (or where the links in there point to), enables
>> permissive mode for the WIFI PCI device. The attached
>> qubes_netvm.service was modified to make sure netvm is started after
>> permissive mode is set.
>>
>> Only one thing remains at this time - suspend to RAM will not wake up
>> successfully with the WIFI module loaded (suspend works fine
>> otherwise). My initial attempt with /lib/qubes/prepare-suspend has
>> not yet worked, and I have not yet dug into it. If anyone has
>> something like this working, I would love to see it.
>>
>
> So if you manually rmmod your wifi module in the netvm, then S3 works
> fine, and if you let the script do it, it wont? Sounds fishy...
>

I spent way too much time messing around with kernel & module builds
before arriving at the solution of using permissive mode, so I haven't
gotten around to this. So far I only know:
1) S3 works if the module has never been loaded
2) My initial attempt at using prepare-suspend resulted in it entering
and exiting S3, but in short order hanging in pretty much the same way
I saw before enabling permissive mode. pciback reports via sysfs it
is still in permissive mode after S3.

I suspect a well-placed sleep might resolve this, assuming the bug
discussed above is not a factor.

>> Anyways, this system is pretty awesome now. The only thing I want to
>> change, as noted before, is to have 8GB of RAM, instead of 4GB.
>>
>> Hopefully this provides ample information for anyone else giving this a go.
>>
>
> Seems like you put lots of efforts into getting Qubes working on your
> MacBook, thanks! And hope you will find the effort worthwhile.
>
> joanna.
>

I think I will. It gives me an entirely different way to look at at
handle data security. Unfortunately, 4GB is a bit tight for running
OS X in an HVM. I'd also be interested in getting a ZFS-based storage
domain up and running, as snapshotting, etc. seems superior to LVM.
However, ZFS is somewhat notorious for its memory consumption
(although it looks like it might be done with 800-1000 MB of RAM). I
also gather that the utility of a separate storage domain is reduced
on a system lacking TXT (although it still might be useful to isolate
bug-prone fs code from a domain managing encryption keys).

On the issue of memory consumption, might any substantial reduction be
made by running HVM-based domUs, and enabling memory page
deduplication? I have heard that on 64-bit systems, there is no
meaningful performance difference from PV. On the other hand, qemu
may significantly increase the attack surface.

[Marek Marczykowski]

There is r2-xen-4.1-devel branch in my vmm-xen git repo with xen 4.1.4. W
stick with 4.1.2 because of some regressions, related tickets:
http://wiki.qubes-os.org/trac/ticket/688
http://wiki.qubes-os.org/trac/ticket/679
http://wiki.qubes-os.org/trac/ticket/689

All our patches are for tools, not hypervisor itself (with exception for
security fixes, which are taken from in upstream version anyway). You can
safely compile hypervisor yourself (from upstream sources) and replace only
this file in /boot to test it (do not install vanilla tools, as it will break
many Qubes-specific code).
Any newer (4.2 or 4.3-unstable) require much more work because of API differences.

>
>>> The attachment
>>> qubes-permissive-netvm.service, which is to be put in
>>> /etc/systemd/system (or where the links in there point to), enables
>>> permissive mode for the WIFI PCI device. The attached
>>> qubes_netvm.service was modified to make sure netvm is started after
>>> permissive mode is set.
>>>
>>> Only one thing remains at this time - suspend to RAM will not wake up
>>> successfully with the WIFI module loaded (suspend works fine
>>> otherwise). My initial attempt with /lib/qubes/prepare-suspend has
>>> not yet worked, and I have not yet dug into it. If anyone has
>>> something like this working, I would love to see it.
>>>
>>
>> So if you manually rmmod your wifi module in the netvm, then S3 works
>> fine, and if you let the script do it, it wont? Sounds fishy...
>>
>
> I spent way too much time messing around with kernel & module builds
> before arriving at the solution of using permissive mode, so I haven't
> gotten around to this. So far I only know:
> 1) S3 works if the module has never been loaded
> 2) My initial attempt at using prepare-suspend resulted in it entering
> and exiting S3, but in short order hanging in pretty much the same way
> I saw before enabling permissive mode. pciback reports via sysfs it
> is still in permissive mode after S3.
>
> I suspect a well-placed sleep might resolve this, assuming the bug
> discussed above is not a factor.

I have work-in-progress set of patches to fix some more general S3 issues
(regarding IRQ delivery, scheduler state). When get it to work, will commit
patches to above mentioned repository. For now you can collect hints from this
thread:
http://xen.markmail.org/thread/rhvon54gvq2gimdi

Anyway this can be completely separate issue.

>>> Anyways, this system is pretty awesome now. The only thing I want to
>>> change, as noted before, is to have 8GB of RAM, instead of 4GB.
>>>
>>> Hopefully this provides ample information for anyone else giving this a go.
>>>
>>
>> Seems like you put lots of efforts into getting Qubes working on your
>> MacBook, thanks! And hope you will find the effort worthwhile.
>>
>> joanna.
>>
>
> I think I will. It gives me an entirely different way to look at at
> handle data security. Unfortunately, 4GB is a bit tight for running
> OS X in an HVM. I'd also be interested in getting a ZFS-based storage
> domain up and running, as snapshotting, etc. seems superior to LVM.
> However, ZFS is somewhat notorious for its memory consumption
> (although it looks like it might be done with 800-1000 MB of RAM). I
> also gather that the utility of a separate storage domain is reduced
> on a system lacking TXT (although it still might be useful to isolate
> bug-prone fs code from a domain managing encryption keys).
>
> On the issue of memory consumption, might any substantial reduction be
> made by running HVM-based domUs, and enabling memory page
> deduplication?

It isn't available in Xen, at least not in 4.1. AFAIR it is currently
available only in KVM. Xen have support for "tmem", but it isn't exactly
deduplication.

> I have heard that on 64-bit systems, there is no
> meaningful performance difference from PV. On the other hand, qemu
> may significantly increase the attack surface.

This is the reason why we do not keep qemu in dom0 (keyword: stubdomain).

[Joanna Rutkowska]

Besides, AFAIK, tmem works only for PV Linux DomUs.

>> I have heard that on 64-bit systems, there is no
>> meaningful performance difference from PV. On the other hand, qemu
>> may significantly increase the attack surface.
>
> This is the reason why we do not keep qemu in dom0 (keyword: stubdomain).
>

qemu has nothing to do with memory virtualization, so I would be
surprised if it could deliver memory deduplication.

Anyway, perhaps we're missing some new feature in Xen? Do you have a
link to this mysterious Xen's deduplication option?

joanna.

[Joanna Rutkowska]

> On Tue, Apr 2, 2013 at 7:09 PM, Joanna Rutkowska

On 04/03/13 01:46, Eric Shelton wrote:> It would appear that I
misunderstood what KSM provides: process-level
> memory deduplication...
>
>
http://blog.scriptkiddie.org/2010/06/27/kvm-vs-xenserver-vs-vmware-memory-overcommitment/
> "Page Sharing or Memory De-Duplication
>
> VMware patented this process. The base Linux kernel in 2.6.32 has
> added a similar feature in KSM, and the ksmd daemon, which runs in
> user-space and can de-duplicate memory across different processes. As
> KVM VMs run as processes, KVM immediately benefits from this.
>
> Again it looks like the community Xen 4.0 hypervisor supports KSM for
> HVM guests, but not PVs and this feature is not present in XenServer
> 5.6."
>

What is "community Xen hypervisor"? Where is it written that it supports
"KSM for HVM guests"? KSM is a Linux kernel feature, right? Or, are you
talking about something else? Links, links, not words, pls!

j.

[Eric Shelton]

It would appear that I misunderstood what KSM provides: process-level
memory deduplication...

http://blog.scriptkiddie.org/2010/06/27/kvm-vs-xenserver-vs-vmware-memory-overcommitment/
"Page Sharing or Memory De-Duplication

VMware patented this process. The base Linux kernel in 2.6.32 has
added a similar feature in KSM, and the ksmd daemon, which runs in
user-space and can de-duplicate memory across different processes. As
KVM VMs run as processes, KVM immediately benefits from this.

Again it looks like the community Xen 4.0 hypervisor supports KSM for
HVM guests, but not PVs and this feature is not present in XenServer
5.6."

On Tue, Apr 2, 2013 at 7:09 PM, Joanna Rutkowska

[Eric Shelton]

Actually, one more link that looks interesting:

http://www.gossamer-threads.com/lists/xen/devel/246460

Assuming the code is still functional, and the same kernel is used
across guests, maybe a reasonable set of shared pages could be
identified. I have no idea how much address space layout
randomization might diminish its utility.

On Tue, Apr 2, 2013 at 8:09 PM, Eric Shelton <eshe...@pobox.com> wrote:
> I suspect the comments in that article are from the following portion
> of the release notes for Xen 4.0 (which should be a bit more
> authoritative than the previously linked article):
>
> http://wiki.xensource.com/xenwiki/Xen4.0
>
> "Xen 4.0 New Features
> ...
> Memory page sharing and page-to-disc for HVM guests: Copy-on-Write
> sharing of identical memory pages between VMs. This is an initial
> implementation (tech preview) and will be improved in upcoming
> releases."
>
> I do not know any more, and there is a fair chance I misunderstood
> something I read a while back...
>
> - Eric
>
> On Tue, Apr 2, 2013 at 7:50 PM, Joanna Rutkowska

[Eric Shelton]

I suspect the comments in that article are from the following portion
of the release notes for Xen 4.0 (which should be a bit more
authoritative than the previously linked article):

http://wiki.xensource.com/xenwiki/Xen4.0

"Xen 4.0 New Features
...
Memory page sharing and page-to-disc for HVM guests: Copy-on-Write
sharing of identical memory pages between VMs. This is an initial
implementation (tech preview) and will be improved in upcoming
releases."

I do not know any more, and there is a fair chance I misunderstood
something I read a while back...

- Eric

On Tue, Apr 2, 2013 at 7:50 PM, Joanna Rutkowska