Does Xorg server pin composition buffer's MFNs?
Rafal Wojtczuk
A bit of background: apparently under Linux, the mfn (say, physical address)
of the frame, that holds an usermode page, can change, unless precautions
are made. See http://lists.xensource.com/archives/html/xen-devel/2010-04/msg00514.html
why it might be an issue in certain virtualized environments.
Question 1: does X server (running under Linux), allocate the composition
buffers in an usual way (using malloc() or similar), or does it do anything
fancier in order to make sure the physical address of composition buffers is
stable in time ?
Question 2: Does ever X server schedule a DMA transaction (to graphics card
presumably) from a composition buffer ? (in such case it would make sense to
pin its physical pages).
Regards,
Rafal Wojtczuk
The Qubes OS Project
http://qubes-os.org
Adam Jackson
I'm assuming by "composition buffer" you mean the thing you're actually
scanning out on the display. I'll use the word "framebuffer" for that,
since that's the usual X name for it.
In the absence of an accelerated driver, X doesn't care about the
physical address of the framebuffer at runtime. Older versions of X
would mmap /dev/mem starting at the physical address of the framebuffer
(as gleaned from the PCI config registers), but after that point we just
write into it by virtual address. Newer ones do basically the same
thing but on the resource file in sysfs for the PCI device, with offset
0.
If you have an accelerated driver, then it depends on the particular
driver. Userspace-only drivers may need to know the physical address of
memory to initiate DMA. Kernel-based drivers typically do DMA from the
kernel.
It's not really clear what you're asking though, or what you're trying
to accomplish.
- ajax
Rafal Wojtczuk
> On Wed, 2010-04-14 at 20:07 +0200, Rafal Wojtczuk wrote:
> > Hello,
> > A bit of background: apparently under Linux, the mfn (say, physical address)
> > of the frame, that holds an usermode page, can change, unless precautions
> > are made. See http://lists.xensource.com/archives/html/xen-devel/2010-04/msg00514.html
> > why it might be an issue in certain virtualized environments.
> >
> > Question 1: does X server (running under Linux), allocate the composition
> > buffers in an usual way (using malloc() or similar), or does it do anything
> > fancier in order to make sure the physical address of composition buffers is
> > stable in time ?
> > Question 2: Does ever X server schedule a DMA transaction (to graphics card
> > presumably) from a composition buffer ? (in such case it would make sense to
> > pin its physical pages).
>
> I'm assuming by "composition buffer" you mean the thing you're actually
> scanning out on the display.
XCompositeRedirectSubwindows() function. Referred to as "windows backing
pixmap" in http://ktown.kde.org/~fredrik/composite_howto.html. Apologies if
I did not make it clear enough.
> I'll use the word "framebuffer" for that, since that's the usual X name for it.
> In the absence of an accelerated driver, X doesn't care about the
Unfortunately, that is not the piece of information I would like to know. In
fact if CompositeRedirectManual flag is specified, the actual framebuffer is
not touched by xlib calls at all.
> It's not really clear what you're asking though, or what you're trying
> to accomplish.
X server running in virtual machine A, and map it in the address space of
virtual machine B. Such mapping has to be defined in terms of physical
addresses; consequently, it is crucial to make sure that the frames backing a
composition buffer do not change in time.
So, once again: is it true that X server allocates a composition buffer (a
window backing pixmap) by a normal malloc call, and does no additional
effort to pin its physical pages so that they do not change in time ?
--
Subscription settings: http://groups.google.com/group/qubes-devel/subscribe?hl=en
Dave Airlie
instance EXA, can migrate the pixmap to from the driver from malloced memory
and the driver can migrate the pixmap to/from VRAM.
X has not access to the physical pages as its a userspace process.
So its a driver question really not a generic X server question.
Dave.
Adam Jackson
> On Wed, Apr 14, 2010 at 03:59:39PM -0400, Adam Jackson wrote:
> > I'm assuming by "composition buffer" you mean the thing you're actually
> > scanning out on the display.
>
> No. I mean the per-window offscreen storage mechanism, activated by
> XCompositeRedirectSubwindows() function. Referred to as "windows backing
> pixmap" in http://ktown.kde.org/~fredrik/composite_howto.html. Apologies if
> I did not make it clear enough.
Window pixmaps are like any other pixmap. Where they live is entirely
up to the driver. Unaccelerated drivers keep them in host memory with
malloc(). Accelerated drivers do that sometimes, and then sometimes put
them in video memory. Remember that you can have more video memory than
you can see through a PCI BAR, so you might not be able to address the
pixmap from the CPU at all.
> Briefly, the goal is to get the location of a composition buffer created by
> X server running in virtual machine A, and map it in the address space of
> virtual machine B. Such mapping has to be defined in terms of physical
> addresses; consequently, it is crucial to make sure that the frames backing a
> composition buffer do not change in time.
That's not going to do anything useful without some synchronization
work. Window pixmaps aren't created afresh for each frame. They're
long-lived. If you manage to get a pixmap shared between VMs A and B,
there's nothing to stop A from rendering into it while B is reading from
it.
The way compositing managers handle this is by taking an X server grab
while reading out of the window pixmap, which does prevent other client
rendering from happening. And as soon as you're doing _that_, just
XGetImage the pixels out instead of playing funny MMU games, it'll
probably be faster.
- ajax
Rafal Wojtczuk
> On Fri, 2010-04-16 at 10:47 +0200, Rafal Wojtczuk wrote:
> > On Wed, Apr 14, 2010 at 03:59:39PM -0400, Adam Jackson wrote:
> > > I'm assuming by "composition buffer" you mean the thing you're actually
> > > scanning out on the display.
> >
> > No. I mean the per-window offscreen storage mechanism, activated by
> > XCompositeRedirectSubwindows() function. Referred to as "windows backing
> > pixmap" in http://ktown.kde.org/~fredrik/composite_howto.html. Apologies if
> > I did not make it clear enough.
>
> Window pixmaps are like any other pixmap. Where they live is entirely
> up to the driver. Unaccelerated drivers keep them in host memory with
> malloc().
setup, dummy_drv.so is used), is the pixmap->devPrivate.ptr field guaranteed
to hold the pointer to actual pixels, or is it purely implementation (X server
version) dependent and may change in future ?
You can see the actual code at
http://gitweb.qubes-os.org/gitweb/?p=mainstream/gui.git;a=blob;f=xf86-input-mfndev/src/qubes.c;h=6b898c25f2bc4d7c68da45764c565157b96ddd4a;hb=aca457004e731da0b642486d8b6f01a9f2b76c4d#l404
> Accelerated drivers do that sometimes, and then sometimes put
> them in video memory. Remember that you can have more video memory than
> you can see through a PCI BAR, so you might not be able to address the
> pixmap from the CPU at all.
>
> > Briefly, the goal is to get the location of a composition buffer created by
> > X server running in virtual machine A, and map it in the address space of
> > virtual machine B. Such mapping has to be defined in terms of physical
> > addresses; consequently, it is crucial to make sure that the frames backing a
> > composition buffer do not change in time.
>
> That's not going to do anything useful without some synchronization
> work. Window pixmaps aren't created afresh for each frame. They're
> long-lived. If you manage to get a pixmap shared between VMs A and B,
> there's nothing to stop A from rendering into it while B is reading from
> it.
work: a video player running in A in full screen is correctly displayed in
B, all other apps work fine as well.
> The way compositing managers handle this is by taking an X server grab
> while reading out of the window pixmap, which does prevent other client
> rendering from happening. And as soon as you're doing _that_, just
> XGetImage the pixels out instead of playing funny MMU games, it'll
> probably be faster.
over the connection to Xserver, so over the unix socket ? So, you would need
XGetImage+copy XImage to B+XPutImage, which is 3 memory copies, 2 of them
involving additional unix socket overhead. With MIT SHM extensions you can get
rid of unix socket transfers, but still it is 3 memcpys.
In the current qubes code, X server running in B maps the composition
buffers from A, and then uses (slightly extended version of) XShmPutImage to
display them. No memory copies, besides ones needed by XShmPutImage to push
content to VGA.
BTW, it is possible that in case of accelerated driver, XShmPutImage uses
DMA from the MIT SHM region directly to VGA memory ?
Adam Jackson
> On Fri, Apr 16, 2010 at 11:41:51AM -0400, Adam Jackson wrote:
> > Window pixmaps are like any other pixmap. Where they live is entirely
> > up to the driver. Unaccelerated drivers keep them in host memory with
> > malloc().
>
> While we are at it: assuming nonaccelerated driver (in fact, in the actual
> setup, dummy_drv.so is used), is the pixmap->devPrivate.ptr field guaranteed
> to hold the pointer to actual pixels, or is it purely implementation (X server
> version) dependent and may change in future ?
Strictly, it's just a convention. In practice, it points to the pixels,
except when it doesn't. See xf86EnableDisableFBAccess() for the
details.
> > That's not going to do anything useful without some synchronization
> > work. Window pixmaps aren't created afresh for each frame. They're
> > long-lived. If you manage to get a pixmap shared between VMs A and B,
> > there's nothing to stop A from rendering into it while B is reading from
> > it.
>
> Currently synchronization is done by damage extension events. It seems to
> work: a video player running in A in full screen is correctly displayed in
> B, all other apps work fine as well.
You appear to be using DamageReportRawRectangles, so you'll eventually
be consistent. It looks like, due to how the X server implements it,
you'll end up getting a report for the whole pixmap all the time, as we
never empty the internal damage tracking. Which isn't the most
efficient thing ever, but might not actually matter given how most apps
end up double-buffering updates.
> BTW, it is possible that in case of accelerated driver, XShmPutImage uses
> DMA from the MIT SHM region directly to VGA memory ?
Yes, very possible. The intel and radeon drivers do this, probably
others too.
- ajax