shared memory confusion
Dirk Farin
I am really confused about the correct usage of
XShmQueryExtension(). In the documentation about
MIT-SHM it says:
Any code which uses the shared memory extension
should first check to see that the server provides
the extension. You could always be running over
the net, or in some other environment where the
extension will not work. To perform this check, call either
Status XShmQueryExtension (display)
or
Status XShmQueryVersion (display, major, minor, pixmaps)
Whereas a section I in the X11-FAQ says the opposite:
But be aware that XShmQueryExtension(dpy)
returns only information on whether or not the
server to which your program is connected is capable
of supporting the shared-memory extension -- it
doesn't confirm that your application is running
on the same machine on which you are running that server.
I really would like to know which one is correct. The
MIT-SHM documentation seems more sensible to me as I can
get the information about the pure availability of SHM
XQueryExtension().
Checking if server and client are running on the same machine
is not that easy and perhaps not even sufficient (?)
What checks do I have to do before I can use the extension?
Thanks,
Dirk Farin
----------------------------------------------------------------
Dirk Farin
University Mannheim / Dept. Circuitry and Simulation
B 6,26 EG, room 0.10 / D-68131 Mannheim / Germany
email: fa...@ti.uni-mannheim.de
----------------------------------------------------------------
Mattias Engdegård
>What checks do I have to do before I can use the extension?
It is, alas, somewhat tricky since even connections that *look* local to your
Xlib at a first glance (namely, those that it talks to via a unix domain
socket or other form of local IPC) may not be. Connections encrypted by
Kerberos or SSH cause this to happen, since they insert a local server
that takes care of the networky bits.
Thus, call XShmQueryVersion() or XQueryExtension() or both, to make
sure the extension is at least in the server. However, their success is
no guarantee for a local connection. The ultimate test is whether
XShmAttach() succeeds; you will almost certainly want to install an error
handler (XSetErrorHandler()) that gracefully takes care of a failure here.
It takes a couple of lines to get it right, but it is usually no big deal.
Dirk Farin
>
> >What checks do I have to do before I can use the extension?
>
> sure the extension is at least in the server. However, their success is
> no guarantee for a local connection. The ultimate test is whether
> XShmAttach() succeeds; you will almost certainly want to install an error
> handler (XSetErrorHandler()) that gracefully takes care of a failure here.
> It takes a couple of lines to get it right, but it is usually no big deal.
Mattias, thank you for the hint. I've changed my program and it now
works
fine. I'm still wondering why this has to be done this hard way.
Especially:
- Why doesn't the XShmAttach() call simply return False but rathers
requires
to catch the X11 error?
- Why has the function XShmQueryVersion() been introduced although it
does not
return more information than XQueryExtension()?
- Why does the documentation tell me wrong things?
Greetings,
Dirk Farin
Mikko Rauhala
>It is, alas, somewhat tricky since even connections that *look* local to your
>Xlib at a first glance (namely, those that it talks to via a unix domain
>socket or other form of local IPC) may not be. Connections encrypted by
>Kerberos or SSH cause this to happen, since they insert a local server
>that takes care of the networky bits.
Actually, at least SSH uses network sockets (setting the DISPLAY to
localhost:[port]), not UNIX sockets. Of course, this doesn't really
affect your conclusions, just thought to clarify this matter a bit.
--
Mikko Rauhala - m...@iki.fi - http://www.iki.fi/mjr/
Mattias Engdegård
>Actually, at least SSH uses network sockets (setting the DISPLAY to
>localhost:[port]), not UNIX sockets. Of course, this doesn't really
>affect your conclusions, just thought to clarify this matter a bit.
You are right, of course; thank you for the correction. Kerberos
(actually kx from kth-krb) does use Unix domain sockets, as do some
other X metaservers that provide various services such as multiplexing
or session persistance.
Using a loopback TCP socket instead of a Unix domain socket should
impose a slight performance penalty, but perhaps that is drowned in
the general SSH overhead. It does confuse clients a lot less, though.
Mattias Engdegård
>Especially:
>- Why doesn't the XShmAttach() call simply return False but rathers
>requires
> to catch the X11 error?
>- Why has the function XShmQueryVersion() been introduced although it
>does not
> return more information than XQueryExtension()?
>- Why does the documentation tell me wrong things?
It is a misdesign in retrospective. I suppose that it was originally
thought that XShmQueryVersion alone should detect the ability to use
MIT-SHM, so XShmAttach() was designed to use the standard X11 error
mechanism for speed. The X11 error handler allows errors to be detected and
handled asynchronously, and this gives a tremendous performance boost since
the client doesn't have to wait for a reply to each request.
As previously mentioned, the X server may have trouble detecting whether a
client is local or not, so this doesn't work. I suppose that meta-servers
(such as those enabling X over SSH and kerberos) could intercept
XShmQueryVersion and give a negative reply, but this would force them to do
more protocol parsing than at present and perhaps make things slower.
I agree that the MIT-SHM documentation should be updated.
Christian Sunesson
> Hello,
>
> I am really confused about the correct usage of
> XShmQueryExtension(). In the documentation about
> MIT-SHM it says:
That is perfectly normal. It confused me too. :) You have to deal with this
in an icky way, if the display supports the extension, go ahead, create
the shared segment and try to attach it. Catch failures with a handler.
If you got an error, you cant use mitshm, retreit to normal Images.
Even if the server might think it is intelligent and say that the client
is/isn't running on localhost, there are situations when it is fooled.
SSH for example, tunnel the trafic via an encrypted/compressed channel
and the server sees a local client, but yet the program is remote.
This code might guide you, http://www.mds.mdh.se/~cel98csn/source/xlib-4.c
I belive it did just these checks after i wrote it a few months back.
> I really would like to know which one is correct. The
> MIT-SHM documentation seems more sensible to me as I can
> get the information about the pure availability of SHM
> XQueryExtension().
> Checking if server and client are running on the same machine
> is not that easy and perhaps not even sufficient (?)
>
> What checks do I have to do before I can use the extension?
Use the source, luke :)
--
* email: cel9...@mds.mdh.se
Dirk Farin
> >requires
> > to catch the X11 error?
> It is a misdesign in retrospective. I suppose that it was originally
> thought that XShmQueryVersion alone should detect the ability to use
> MIT-SHM, so XShmAttach() was designed to use the standard X11 error
> mechanism for speed. The X11 error handler allows errors to be detected and
> handled asynchronously, and this gives a tremendous performance boost since
> the client doesn't have to wait for a reply to each request.
Theoretically it's true that asynchronous error handling is faster, but
if shared memory allocation fails, you certainly want to use standard
XLib calls instead. From a practical programming point of view you will
have to insert an XSync() call right after the XShmAttach() to catch the
error and fall back to standard XImages like this:
static bool shmfailed;
static int shmhandler(Display* display,XErrorEvent* err)
{
if (err->request_code == shmmajor &&
err->minor_code == X_ShmAttach)
shmfailed=true;
return 0;
}
main()
{
.......
bool d_UseShmExt=true;
tryagain:
if (d_UseShmExt)
{
shmfailed=false;
XSetErrorHandler(shmhandler);
d_ximg = XShmCreateImage(...);
.......
XShmAttach(d_display,&d_ShmSegInfo);
XSync(d_display,False);
XSetErrorHandler(NULL);
if (shmfailed)
{
.... clean up ...
d_UseShmExt = false;
goto tryagain;
}
}
else
{
... use std ximage ...
}
}
A simple False-return would be much easier. And think about how
often you create new XImages in your program. Probably only a few
that you will then use throughout the program, so error handling
speed is not the main concern. Especially as you can assume:
it failed once -> it will fail again next time
Greetings
Dirk Farin
Dirk Farin
>
> That is perfectly normal. It confused me too. :) You have to deal with this
> in an icky way, if the display supports the extension, go ahead, create
> the shared segment and try to attach it. Catch failures with a handler.
> If you got an error, you cant use mitshm, retreit to normal Images.
>
> I belive it did just these checks after i wrote it a few months back.
Unfortunately it does not. It has exactly the same problems as the first
version of mine and some more (eg. "shmctl(shminfo.shmid, IPC_RMID, 0);"
has
to be removed or it doesn't even run locally).
If anyone is interested in code that does all the checks, I can send it
to you.
Greetings
Dirk Farin
Christian Sunesson
> > This code might guide you, http://www.mds.mdh.se/~cel98csn/source/xlib-4.c
> > I belive it did just these checks after i wrote it a few months back.
>
> Unfortunately it does not. It has exactly the same problems as the first
> version of mine and some more (eg. "shmctl(shminfo.shmid, IPC_RMID, 0);"
> has
> to be removed or it doesn't even run locally).
Oh, but my unix systems keep a removed shared segment until the last
process detaches from it. So it works for me.
--
* email: cel9...@mds.mdh.se
Mattias Engdegård
>Unfortunately it does not. It has exactly the same problems as the first
>version of mine and some more (eg. "shmctl(shminfo.shmid, IPC_RMID, 0);"
>has
>to be removed or it doesn't even run locally).
What X server are you using, on what hardware? I have heard rumours of some
servers that detach and re-attach shared memory segments, so that the old
remove-segment-identifier-after-attaching trick doesn't work. Is that causing
you trouble? (For the record, I have had no trouble with either XFree86
or Solaris 2.5.1/2.6 XSun for ffb and cgsix.)
It would be reassuring if this was possible in general, since removing
segments left by crashing programs is tedious and hazardous.
Mattias Engdegård
[ Correct error handling from XShmAttach deleted ]
>A simple False-return would be much easier. And think about how
>often you create new XImages in your program. Probably only a few
>that you will then use throughout the program, so error handling
>speed is not the main concern. Especially as you can assume:
>it failed once -> it will fail again next time
No. If XShmAttach didn't fail the first time, it probably won't fail next
time either, so synchronous error handling would be wasteful all the
remaining times.
And don't assume that everyone else's programs have the same usage patterns
as yours. A client might use MIT-SHM as a general acceleration mechanism
for transporting images to the server, and creating shared XImages would
then be a common operation.
Now, it is true that attaching a shared memory segment is probably uncommon
enough that a roundtrip wouldn't be a big deal, and that the operation is
expensive enough so that the overhead isn't noticeable. But still, you
can always handle an asynchronous error synchronously, but never the
opposite. Thus, when in doubt, use an asynch method. Protocols live forever,
and a basic design error can be hard to fix later.
Dirk Farin
>
> Dirk Farin <fa...@ti.uni-mannheim.de> writes:
>
> Now, it is true that attaching a shared memory segment is probably uncommon
> enough that a roundtrip wouldn't be a big deal, and that the operation is
> expensive enough so that the overhead isn't noticeable. But still, you
> can always handle an asynchronous error synchronously, but never the
> opposite. Thus, when in doubt, use an asynch method. Protocols live forever,
> and a basic design error can be hard to fix later.
Ok, probably you are right, even though I cannot think of a scenario
where
this error could be handled asynchonously. But perhaps some helper
functions
to set up shared XImages would be nice, especially as it seems that
nearly
no one is able to write 100% correct code for this at his first try.
Dirk
Dirk Farin
> >version of mine and some more (eg. "shmctl(shminfo.shmid, IPC_RMID, 0);"
> >has
> >to be removed or it doesn't even run locally).
>
> What X server are you using, on what hardware? I have heard rumours of some
> servers that detach and re-attach shared memory segments, so that the old
> remove-segment-identifier-after-attaching trick doesn't work. Is that causing
> you trouble? (For the record, I have had no trouble with either XFree86
> or Solaris 2.5.1/2.6 XSun for ffb and cgsix.)
I'm using a HP-UX 10.20 workstation.
The solution to the problem is quite simple. Once again it's the
asynchon execution of XShmAttach() that's causing the failure.
Modify it this way:
XShmAttach(d, &shminfo); /* Tell the server to attach */
INSERT>> XSync(d,False);
shmctl(shminfo.shmid, IPC_RMID, 0);
and it works.
Dirk