"Exploiting the DRAM rowhammer bug to gain kernel privileges"

[Pedro Martins]

"Exploiting the DRAM rowhammer bug to gain kernel privileges"

http://googleprojectzero.blogspot.com/2015/03/exploiting-dram-rowhammer-bug-to-gain.html

"“Rowhammer” is a problem with some recent DRAM devices in which
repeatedly accessing a row of memory can cause bit flips in adjacent
rows. We tested a selection of laptops and found that a subset of them
exhibited the problem. We built two working privilege escalation
exploits that use this effect. One exploit uses rowhammer-induced bit
flips to gain kernel privileges on x86-64 Linux when run as an
unprivileged userland process. When run on a machine vulnerable to the
rowhammer problem, the process was able to induce bit flips in page
table entries (PTEs). It was able to use this to gain write access to
its own page table, and hence gain read-write access to all of physical
memory."

Time to start to look for laptops with ECC RAM ;)

Maybe also shutdown all unnecessary VMs before performing sensitive ops?

Game over if dom0 gets compromised?

--
Pedro Martins

[WhonixQubes]

On 2015-03-09 9:24 pm, Pedro Martins wrote:
> "...and hence gain read-write access to all of physical memory."

>
> Time to start to look for laptops with ECC RAM ;)
>
> Maybe also shutdown all unnecessary VMs before performing sensitive
> ops?
>
> Game over if dom0 gets compromised?

So in the Qubes model, if this doesn't just compromise the AppVM kernel,
but gives access to all of the physical machine's memory, then that
sounds like a total game over.

I'm guessing if one can write to all of the physical memory with this,
then they can then easily manipulate and control Xen or Dom0 and the
total machine?


WhonixQubes

[Zrubi]

On 03/10/15 01:34, WhonixQubes wrote:
> On 2015-03-09 9:24 pm, Pedro Martins wrote:
>> "...and hence gain read-write access to all of physical memory."
>>
>> Time to start to look for laptops with ECC RAM ;)
>>
>> Maybe also shutdown all unnecessary VMs before performing sensitive ops?
>>
>> Game over if dom0 gets compromised?
>
>
> So in the Qubes model, if this doesn't just compromise the AppVM kernel,
> but gives access to all of the physical machine's memory, then that
> sounds like a total game over.

Sure thing - but I don't see how can this exploit jump trough xen the level?

Correct me if I'm wrong - but a VM kernel only see xen provided virtual
memory, not the physical pages.



--
Zrubi

[WhonixQubes]

That's what I'm wondering, and naturally would hope that physical memory
"rowhammering" would not be achievable through Xen/Qubes virtual memory.


But these blurbs from it make me wonder...



"We learned about the rowhammer problem from Yoongu Kim et al’s paper,
“Flipping Bits in Memory Without Accessing Them: An Experimental Study
of DRAM Disturbance Errors” (Yoongu Kim, Ross Daly, Jeremie Kim, Chris
Fallin, Ji Hye Lee, Donghyuk Lee, Chris Wilkerson, Konrad Lai, Onur
Mutlu).

They demonstrate that, by repeatedly accessing two “aggressor” memory
locations within the process’s virtual address space, they can cause bit
flips in a third, “victim” location. The victim location is potentially
outside the virtual address space of the process it is in a different
DRAM row from the aggressor locations, and hence in a different 4k page
(since rows are larger than 4k in modern systems)."

...

"To many security researchers, especially those who practice writing
proof-of-concept exploits, it is well known that bit flips can be
exploitable. For example, a 2003 paper explains how to use random bit
flips to escape from a Java VM. (See “Using Memory Errors to Attack a
Virtual Machine” by Sudhakar Govindavajhala and Andrew W. Appel.)"

...

"Our proof-of-concept exploits use the x86 CLFLUSH instruction, because
it’s the easiest way to force memory accesses to be sent to the
underlying DRAM and thus cause row hammering.

The fact that CLFLUSH is usable from unprivileged code is surprising,
because the number of legitimate uses for it outside of a kernel or
device driver is probably very small. For comparison, ARM doesn’t have
an unprivileged cache-flush instruction. (ARM Linux does have a
cacheflush() syscall, used by JITs, for synchronising instruction and
data caches. On x86, the i-cache and d-cache are synchronised
automatically, so CLFLUSH isn’t needed for this purpose.)

We have changed NaCl’s x86 validator to disallow CLFLUSH. Unfortunately,
kernels can’t disable CLFLUSH for normal userland code. Currently,
CLFLUSH can’t be intercepted or disabled, even using VMX (x86
virtualisation). (For example, RDTSC can be intercepted without VMX
support. VMX allows intercepting more instructions, including WBINVD and
CPUID, but not CLFLUSH.) There might be a case for changing the x86
architecture to allow CLFLUSH to be intercepted. From a security
engineering point of view, removing unnecessary attack surface is good
practice.

However, there might be ways of causing row hammering without CLFLUSH,
which might work on non-x86 architectures too..."

...

Comment: "This exploit technology can be definitely applied to
hypervisor, and cause privilege escalation to VMX root mode."



WhonixQubes

[WhonixQubes]

Looks like our friend HW42 is also on the case... :)

https://groups.google.com/d/topic/rowhammer-discuss/YeGtBkpfUoQ

"double side rowhammer in virtualized environments"

> does the double side rowhammer test program work in virtualized
> environments. Especially Xen PVM, Xen HVM and Qemu/KVM?


WhonixQubes

[cprise]

I posted a reply showing my result from a Qubes R2 PVM (bit flipped
after 240s).

I think any hope of mitigation for us depends on hypervisor memory
allocation patterns.

[WhonixQubes]

On 2015-03-10 8:18 am, cprise wrote:
> I posted a reply showing my result from a Qubes R2 PVM (bit flipped
> after 240s).
>
> I think any hope of mitigation for us depends on hypervisor memory
> allocation patterns.

Too bad. Right. Or potentially just using ECC memory, it seems.


WhonixQubes

[Zrubi]

On 03/10/15 09:18, cprise wrote:

> I posted a reply showing my result from a Qubes R2 PVM (bit flipped
> after 240s).
>
> I think any hope of mitigation for us depends on hypervisor memory
> allocation patterns.
>

Then it seems I was not wait enough - canceled after ~150 iterations.

running the test again...

--
Zrubi

[cprise]

> .
>

Joanna,

Do you think Xen's memory allocation policies allow rowhammer
exploitation across VM boundaries?

And if so, what are the chances for a patch that prevents different
domains from getting too close to each other?

[Andrew]

cprise:

Very interesting. I've been unable to trigger this bug from within a
regular PV AppVM after ~17k seconds.

Can you provide more information about your hardware?

My memory is G.Skill 2x8GB SODIMM F3-1600C10D-16GSQ, DDR3-1600
CL10-10-10 1.5v.

DRAM chips have:

ELPIDA TWN
J4208EASE
-DJ-F
1138092DN10

silkscreened on them.


Andrew

[Andrew]

Andrew:

I should note that this is particularly interesting to me as the rest of
my platform is very similar to yours: X230 vs T430s, but identical CPU
and chipset (even down to revision).

Andrew

[cprise]

I haven't yet removed/replaced my RAM modules to look at them, but I do
know they are Kingston HyperX DDR3L 1600 (1.35V) kit model#
KHX16LS9P1K2/8 (2x4GB). Dmidecode output says they are KHX1600C9S3L/4G.
The CAS latency is 9.

I don't know how to access the full DRAM timing info in Linux/Xen.

BIOS version is G7ETA1WW (2.61) from Oct. 2014. What is your BIOS version?

[Andrew]

cprise:

Right, I found your response in rowhammer-discuss afterwards. The data
sheet I found was not helpful at all ;).

I'd rather not say which BIOS version I'm currently using, but I don't
think it's relevant, anyway.

You asked a great question, though. I just opened up my laptop and
looked at the modules themselves. I did find out how to get more
information via software, though, and this information looks very useful.

In Dom0:
$ sudo qubes-dom0-update i2c-tools i2c-tools-perl
$ sudo modprobe eeprom
$ decode-dimms

I'm attaching my results. It appears my system is not vulnerable: I
haven't detected an error in ~36k seconds. Comparing the information
attached with systems that are almost immediately shown vulnerable might
give some insight.

I would guess (I'm no expert) a few things might be key variables:
1) memory module manufacturer (the on-board controller appears to be
responsible for refresh, though refresh time might be a JEDEC standard?)
2) module voltage -- lower voltage means less charge in each cell, so
more susceptible to interference
3) tRC -- according to the authors, this becomes the bottleneck [1], see
also [3]
4) module frequency -- more opportunities to strobe wordline[2] per
refresh period (?)
...and probably more


Andrew

-----

[1] From the original paper:
http://users.ece.cmu.edu/~yoonguk/papers/kim-isca14.pdf

"When trying to open and close the same row as quickly as possible, tRC
becomes the bottleneck -- limiting the maximum rate to once every tRC."

-----

[2] From the original paper:
http://users.ece.cmu.edu/~yoonguk/papers/kim-isca14.pdf

"We identify the root cause of DRAM disturbance errors as voltage
fluctuations on an internal wire called the wordline. DRAM comprises a
two-dimensional array of cells, where each row of cells has its own
wordline. To access a cell within a particular row, the row's wordline
must be enabled by raising its voltage -- i.e., the row must be
activated. When there are many activations to the same row, they force
the wordline to toggle on and off repeatedly. According to our
observations, such voltage fluctuations on a row’s wordline have a
disturbance effect on nearby rows, inducing some of their cells to leak
charge at an accelerated rate. If such a cell loses too much charge
before it is restored to its original value (i.e., refreshed), it
experiences a disturbance error."

-----

[3] https://www.ece.cmu.edu/~ece548/localcpy/dramop.pdf ("Understanding
DRAM Operation")

[Pedro Martins]

In that group there is someone hinting that memtest86+ might trigger bit
flips. [0][1]

[0] https://groups.google.com/forum/#!topic/rowhammer-discuss/t6UCrTD-of8
[1] https://lkml.org/lkml/2012/10/14/13

--
Pedro Martins

[2452345234523452345234523452354]

The circle of trust get closer and closer...

Can the CVE-2015-0565 security gap get a Qubes OS countermeasure like in this way:

- the hammering: cachflushings and rewriting in high frequencies get monitored and/or forbidden (the best is that some customizing-switch can allow or hinder this kind of tech-feature)
- the system prevent to take ofer user-processes
https://grsecurity.net/
- the memory-pages get an integrity-check - like a hash (which at least get checked by the system after some cache-flashing happens)
- VM have some RAM-adress-Spaches seperated by white pages
- If someone writes into this white pages, some white space rollup-function generate an allert: CVE-2015-0565, is the rollup is ne 0

[xet7]

Can anvil kernel module protections for rowhammer be added to Qubes?

https://news.ycombinator.com/item?id=12822490

[Reg Tiangha]

On 05/21/2017 02:34 PM, xet7 wrote:
> Can anvil kernel module protections for rowhammer be added to Qubes?
>
> https://news.ycombinator.com/item?id=12822490
>

If it were to be incorporated into the Qubes kernel, I wonder what would
happen if it were tried to be run on an AMD machine?

Anyway, from what I can tell, there's nothing stopping you from
compiling this kernel module yourself and loading it into your VMs and
dom0 manually using modprobe, insmod, or adding it to /etc/modutils in
Debian or creating a specific anvil.conf file in /etc/modules-load.d on
Fedora. So if you really want to add the anvil module to your system,
you can do that right away on your own.

[Vít Šesták]

It probably can, but I am not sure if it is so great in context of Qubes. I guess it can handle exploitation of rowhammer within one VM, but not across all the VMs. IIUC, every VM would try to read the usage data, but it would have no idea of usage of adjacent memory by another VM.

Also, HVMs obscure the memory layout and it will be the only way for AppVMs in Qubes 4.

Regards,
Vít Šesták 'v6ak'

[Reg Tiangha]

On 05/21/2017 02:34 PM, xet7 wrote:

> Can anvil kernel module protections for rowhammer be added to Qubes?
>
> https://news.ycombinator.com/item?id=12822490
>

So I've skimmed through the whitepaper (https://iss.oy.ne.ro/ANVIL.pdf)
and because it says that it uses hardware performance counters to detect
rowhammer attacks, and while it probably works properly on a bare-metal
system, I can't tell if this is something that can only be used in dom0
or if it can actually be used in VMs as well (or if it'll work properly
at all in a Xen-based system).

But let's assume for a moment that it would work properly in Qubes. With
the Qubes security model, what are the most likely places to get hit by
a rowhammer attack that might benefit from something like Anvil? I would
assume sys-net is a prime candidate. Maybe even sys-firewall? But what
is the likelihood or in what circumstances would an AppVM behind a
sys-firewall be subjected to a rowhammer assault? Is this the sort of
thing an attacker would need remote access to in order to exploit? Or is
it something that can run on its own autonomously?

Clearly, I don't know much about how rowhammer would be used in a
real-world attack scenario, LOL.

[Tai...@gmx.com]

On 05/21/2017 04:34 PM, xet7 wrote:

> Can anvil kernel module protections for rowhammer be added to Qubes?
>
> https://news.ycombinator.com/item?id=12822490
>

Ideally to fix this issue you would buy quality RAM that is made
properly to avoid rowhammer (ask OEMs for test results, the large ones
have them) there are modules out there that can't be exploited even if
you half the refresh rate.

Mitigating this can only be done in libre firmware, you would have to
buy an open source init coreboot system (most "coreboot" systems such as
purism are not actually open source for hw init) and lower the DRAM
refresh rate (within spec) until you don't see any flips. Lowering the
refresh rate is how the major laptop OEM's do it at least.

I don't consider a kernel module a real mitigation as you have to apply
it each and every time you upgrade your kernel on most systems unless
you use ex: gentoo where you can auto-patch.

[xet7]

Is there list of available hardware that has libre firmware? So it could be used with quality RAM ?