X server on 3.2.5 kills some Xt clients

John LoSecco

unread,

Dec 23, 1993, 10:25:42 AM12/23/93

to

The X server shipping with AIX 3.2.5 (X11R5) kills X clients
linked with Xt from some vendors X11R4 versions of the Xt library.
The problem is a segementation violation in XtKeysymToKeycodeList.
The crash occurs because the new X server apparently supports more than
256 KeyCodes. The X toolkit spec claims up to 2^16 are supported.

Unfortunately XtKeysymToKeycodeList as shipped from MIT and
many vendors, at least for X11R4 expects and declares a uchar which
permits only 256 KeyCodes. The response to a server that supports more
than this is a crash of the client. Clearly a poor default response...

IBM claims that the problem is MIT's and the other vendor's.
The trouble is that it is the IBM server that broke the defacto standard.

In particular we have encountered this problem on code linked with the
X toolkit on SGI's Irix 4 running X11R4. I would be curious if other
vendor's also have this problem. Can someone suggest a workaround?

So much for the client-server model...
--
John M. LoSecco Phone:(219)631-6044 Fax:(219)631-5952
Physics Department HEPnet/SPAN: UNDHEP::LOSECCO or 47344::LOSECCO
University of Notre Dame Internet: los...@undpdk.hep.nd.edu
Notre Dame, Indiana 46556 USA Bitnet: losecco@undhep

David L. Crow

unread,

Jan 3, 1994, 6:36:03 PM1/3/94

to

First you say:

>256 KeyCodes. The X toolkit spec claims up to 2^16 are supported.

Then you say:

>The trouble is that it is the IBM server that broke the defacto standard.

Which defacto standard did we break? The one that was very different
from the stated real standard?

Sounds to me like your library code is not up to par with the spec
and that the IBM X server is running just fine as defined by the
specs it was designed to.

John LoSecco

unread,

Jan 24, 1994, 1:00:43 PM1/24/94

to

In article <crow.75...@waterloo.austin.ibm.com> cr...@austin.ibm.com writes:
>
> First you say:
>
>>256 KeyCodes. The X toolkit spec claims up to 2^16 are supported.
>
> Then you say:
>
>>The trouble is that it is the IBM server that broke the defacto standard.
>
> Which defacto standard did we break? The one that was very different
> from the stated real standard?
>
> Sounds to me like your library code is not up to par with the spec
> and that the IBM X server is running just fine as defined by the
> specs it was designed to.

The problem with this response is that IBM is marketing its
products in the real world. In the real world the customer doesn't much
care who is to blame. All he knows is that after updating his RS/6000
to AIX 3.2.5 applications that used to run, from a non IBM server, no longer
run.
It is quite likely that the customer still wants to run the application
and does not want to be in the middle of finger pointing.

This is similar to a response I got from IBM software deffect support.
Yes indeed your software does not have a defect. But it still does
not work or satisfy my need.

I note that the server has flags to support some bugs in the old
R4 code. I also suspect that the same Xtoolkit bug was present in earlier
distributed AIX X11R4 code that may now have been fixed.

But the problem is that ever since upgrading to AIX 3.2.5 I can not run
a number of key applications from any of our IBM workstations. I could run
them under 3.2.4. And people at IBM seem to be more interested in being
right than in fixing the problem.

John LoSecco

unread,

Feb 3, 1994, 2:23:23 PM2/3/94

to

> But the problem is that ever since upgrading to AIX 3.2.5 I can not run
>a number of key applications from any of our IBM workstations. I could run
>them under 3.2.4.

Many thanks to Bill Woodward at IBM. Bill has recompiled the 3.2.5 X server
to specify 254 keycodes, not the 255 in the default 3.2.5 server. IBM had
upped the number (from 141 in 3.2.4) to satisfy some European customers.
Dropping back to 254 does indeed fix the problem.

The APAR for the fix is IX41969. Bill has recommended that the fix be
released as a PTF.

Many thanks IBM for helping with the problem.
Shame on SGI for trying to duck the issue. The Xtoolkit library for X11R4
on both the IRIX 4 and IRIX 5 operating systems still has the client bug.

wpw...@darkwing.austin.ibm.com

unread,

Feb 14, 1994, 10:21:19 AM2/14/94

to

In article <2irivb$a...@news.nd.edu> los...@undpdk.hep.nd.edu (John LoSecco) writes:
>> But the problem is that ever since upgrading to AIX 3.2.5 I can not run
>>a number of key applications from any of our IBM workstations. I could run
>>them under 3.2.4.
>
>Many thanks to Bill Woodward at IBM. Bill has recompiled the 3.2.5 X server
>to specify 254 keycodes, not the 255 in the default 3.2.5 server. IBM had
>upped the number (from 141 in 3.2.4) to satisfy some European customers.
>Dropping back to 254 does indeed fix the problem.
>
>The APAR for the fix is IX41969. Bill has recommended that the fix be
>released as a PTF.
>

There have been a few calls coming into the support center asking
about this fix, so I wanted to post a few more details.
Under AIX 3.2.4 and before, with X11R5, the max keycode
value obtained from the X server (returned by the Xlib function
call, XDisplayKeycodes) was 141. The problem with this (relatively
small) value was that we had some European customers in some kind of
trading floor environment who were using custom keyboards and needed
this value to be raised. Since the X specification will permit a max
keycode value of up to 255, we just raised it to the maximum value
(which was a legal value).
The problem occurs because there is a bug in the X11R4 X
Toolkit Intrinsics library (libXt.a). This bug may have been fixed by
some X11R4 patchlevel from the X Consortium. I'm not sure, but it was
fixed in the X11R5 libXt.a.
The bug was in the XtKeysymToKeycodeList function in the X11R4
Xt library. Basically, there was an internal variable in
XtKeysymToKeycodeList called 'keycode' of type 'KeyCode'. The problem
was that 'KeyCode' resolved back to an unsigned char with a maximum
value of 255 (other places in the Xt library, KeyCode resolved back to
an unsigned int, I believe). This caused a problem because there was
a loop in XtKeysymToKeycodeList which basically did:

for (keycode = 0; keycode <= pd->max_keycode; keycode++)
<Write into the keycode'th element of an array>

the pd->max_keycode value is that max keycode retirned from the X
server, and as you can see, if that value was 255, then the loop
becomes infinite, and the code walks off the end of the array,
trashing memory and eventually core dumping.
The real solution to the problem is to fix the libXt bug. To
help John work around his problem (since SGI refused to fix it), I
added code to the X server to check if bug compatibility mode was on
(which it is by default on our X server) and, if so, to lower the max
keycode value to 254.
This solves John's problem when he displays to an IBM X
server, but the core dump in XtKeysymToKeycodeList will still occur
when displaying to any other X server which has a max keycode value of
255. Customer's should really try to get the libXt fix from their
other vendor and only order this fix from IBM as a last resort.

>
>Many thanks IBM for helping with the problem.
>Shame on SGI for trying to duck the issue. The Xtoolkit library for X11R4
>on both the IRIX 4 and IRIX 5 operating systems still has the client bug.

Yeah.

Mark Kilgard

unread,

Feb 16, 1994, 7:28:51 AM2/16/94

to

In article <WPWOOD.94F...@darkwing.austin.ibm.com>, wpw...@darkwing.austin.ibm.com writes:
|> Since the X specification will permit a max
|> keycode value of up to 255, we just raised it to the maximum value
|> (which was a legal value).

Sigh. I am *really hoping* that IBM didn't know about the bug in
Xt before changing max keycode value to 255 when making the value
254 would have avoided a lot of hassles. If IBM made the change
with full knowledge of the inter-operability problems it would
create, I would be quite irked. :-(

|> The real solution to the problem is to fix the libXt bug. To
|> help John work around his problem (since SGI refused to fix it), I

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Sigh!

Clearly there was some confusion as is usually the case when
inter-operability problems arise between vendors. The bug will be
fixed though I haven't talked about the release details with
anyone here yet.

The IBM support engineer's advice that customers should go demanding
a fixed libXt from all their other vendors is a bit presumptive. If
the Xt bug was some isolated bug in a single vendor's sources, he'd be
totally right. But when it is software that has been around for
some time and *lots* of X11R4 products are going to have the bug,
telling customers to demand a fix from all their other vendors
is a bit rich. What are you going to tell all the 3rd party
software vendors that have programs compiled against a static X11R4
Xt library? The kicker is that is sounds like IBM didn't even
need the number to be 255.

For IRIX 5 users, recompiling your application as an IRIX 5 binary
will fix the problem since the IRIX 5 Xt library is based on X11R5.
The IRIX 4 compatibility library is based on X11R4 for compatibility
reasons.

I do thank Bill Woodward for the details about this inter-operability
problem so SGI can fix its side of the problem; and I also thank
IBM for their willingness to supply a patch/workaround for the
problem on their side.

- Mark