X11, Xt, Xm and purify
Frode Tenneboe
What I found was quite disturbing. Apart from the tons of
UMRs (Uninitialised Memory Reads), which I understand comes from
dynamic memory allocations, there are also quite a few MLKs
(memory LeaKs) in various parts of X. TMstate.c and TMparse.c
are high on the list. So are functions like XmStringGetNextSegment,
XGetVisualInfo, XmStringInitContext and also possibly XmTextGetString.
And those are the only ones I have found in the short time I have
looked into this. I run Sun Solaris 2.5.1 with BlueStone motif 1.2.4.
Are there logical reasons for this?
Does a leak-free implementation of X/Motif exist for Solaris?
-Frode
--
^ Frode Tennebø | email: f...@edh.ericsson.se ^
| Ericsson Radar AS. N-1788 Halden | |
| Phone: +47 69 21 41 47 | Frode@IRC |
| with Standard.Disclaimer; use Standard.Disclaimer; |
Clive Harris
wrote:
<snip>
>
>Does a leak-free implementation of X/Motif exist for Solaris?
>
> -Frode
>
Frode
This is a very common concern of people who are trying to build
resilient applications based upon X.
I don't know if there are any leak-free implementations, but I work
for a company called Geodesic Systems (www.geodesic.com) whose product
- Great Circle - can absolutely eliminate these leaks in X, and in any
other third party library you may be using - even if you don't have
the source code available. In deployment mode (which is what you'll
need to use to eliminate X leaks) we are a link-time add-in so that by
default, you don't even have to recompile your code, just relink.
Email me (clive....@btinternet.com) if you would like me to send
you some more detailed information.
Regards
Clive Harris
Geodesic Systems.
Chris Nappin
I speak from personal experience of trying to reduce memory leaks in a large X application. Once down to a "small amount" of memory leakage, I had to give up and attribute the fault on our X/Motif libraries.
You will find that newer versions of Motif (e.g. newer than version 1.2) have much less memory leakage. There are several versions of X/Motif that are "memory leak free" but expect to pay for them.
You can of course, using purify, fix the leaks yourself ;-)
ta,
Chris Nappin.
David B. Lewis
|> Are there logical reasons for this?
In the more than 4 years since the release of 1.2.4, lots of these problems
have been fixed, as have similar problems with X11 (but note that Purify
gives off UMRs for code that is not problematic). I suggest that you upgrade.
The latest versions are Motif 2.1.10 and X11R6.4pl3. Those releases also
contain information on how to report problems.
--
David B. Lewis Editor, The Motif Developer: http://www.motifzone.com/tmd/
d...@motifzone.com WebWrangler, The Motif Zone: http://www.motifzone.com/
ken...@nojunk.rahul.net-
Chris Nappin <CNa...@inri.co.uk> wrote:
>Various parts of X, especially Motif, are well known to be very "leaky" in the memory dept!
I disagree. I think recent versions of X and Motif have few leaks and
those leaks are mostly in unusual cases (e.g., custom translation
tables aren't freed if the widget is destroyed).
Most leaks are due to application bugs and can be avoided by careful
programming. My web site has a couple of tutorials on avoiding memory
leaks and other memory corruption problems in X/Motif applications.
--
Ken Lee, http://www.rahul.net/kenton/
T.E.Dickey
> In article <360A59C3...@inri.co.uk>,
> Chris Nappin <CNa...@inri.co.uk> wrote:
>>Various parts of X, especially Motif, are well known to be very "leaky" in the memory dept!
> I disagree. I think recent versions of X and Motif have few leaks and
> those leaks are mostly in unusual cases (e.g., custom translation
> tables aren't freed if the widget is destroyed).
by recent, you must mean (and should specify) Motif 2.x and X11R6.x
(though both have multitudes remaining of references to uninitialized
data, as well as "permanent" leaks). Anyone using Purify learns to
filter that out (unfortunately).
But they're improved somewhat over X11R5 and Motif 1.x (which came out
shortly after Purify hit the street).
> Most leaks are due to application bugs and can be avoided by careful
> programming. My web site has a couple of tutorials on avoiding memory
> leaks and other memory corruption problems in X/Motif applications.
> --
> Ken Lee, http://www.rahul.net/kenton/
--
Thomas E. Dickey
dic...@clark.net
http://www.clark.net/pub/dickey
ken...@nojunk.rahul.net-
T.E.Dickey <dic...@shell.clark.net> wrote:
>> I disagree. I think recent versions of X and Motif have few leaks and
>> those leaks are mostly in unusual cases (e.g., custom translation
>> tables aren't freed if the widget is destroyed).
>by recent, you must mean (and should specify) Motif 2.x and X11R6.x
>(though both have multitudes remaining of references to uninitialized
>data, as well as "permanent" leaks). Anyone using Purify learns to
>filter that out (unfortunately).
I mean Motif 1.2.4 and X11R6.0, both of which have been available for at
least a few years.
UMRs are not bugs and can be safely ignored. Many popular libraries,
including X, use this optimization.
Can you be more specific on the multitudes of serious memory leaks that
you've found? I consider a memory leak to be serious if it happens
more than once in the lifetime of a widget.
T.E.Dickey
> I mean Motif 1.2.4 and X11R6.0, both of which have been available for at
> least a few years.
> UMRs are not bugs and can be safely ignored. Many popular libraries,
> including X, use this optimization.
UMRs are potential sources of core dumps. It says that the program is
copying unintialized data to a new location (unspecified).
> Can you be more specific on the multitudes of serious memory leaks that
> you've found? I consider a memory leak to be serious if it happens
> more than once in the lifetime of a widget.
not at the moment - but the next time I go that way, I'll collect a few.
ken...@nojunk.rahul.net-
T.E.Dickey <dic...@shell.clark.net> wrote:
>In comp.windows.x ken...@nojunk.rahul.net- wrote:
>> UMRs are not bugs and can be safely ignored. Many popular libraries,
>> including X, use this optimization.
>
>UMRs are potential sources of core dumps. It says that the program is
>copying unintialized data to a new location (unspecified).
That is false. The new location is specified properly. Copying
uninitialized data as fine as long as the destination is properly
allocated and not later dereferenced. All UMRs in X and Motif do this
properly. X (and other libraries) does this because a single copy of
an array or struct is faster than multiple copies or unnecessarily
initializing unused fields.
Pedro Proenca
> properly. X (and other libraries) does this because a single copy of
> an array or struct is faster than multiple copies or unnecessarily
> initializing unused fields.
That is non-portable code! The ANSI/ISO C standard does not specify the memory
layout of any data type. Padding and struct aligning can cause problems if you
build your app with different settings. This happens all the time in the
Microsoft Windows world.
Pedro Proenca
-----------------------
pro...@hks.com.NO_SPAM
T.E.Dickey
> In article <6uesm8$ic$1...@clarknet.clark.net>,
> T.E.Dickey <dic...@shell.clark.net> wrote:
>>In comp.windows.x ken...@nojunk.rahul.net- wrote:
>>> UMRs are not bugs and can be safely ignored. Many popular libraries,
>>> including X, use this optimization.
>>
>>UMRs are potential sources of core dumps. It says that the program is
>>copying unintialized data to a new location (unspecified).
> That is false. The new location is specified properly. Copying
> uninitialized data as fine as long as the destination is properly
> allocated and not later dereferenced. All UMRs in X and Motif do this
> properly. X (and other libraries) does this because a single copy of
> an array or struct is faster than multiple copies or unnecessarily
> initializing unused fields.
This will generate a UMR:
#include <stdio.h>
int main()
{
char *x;
char *y = x;
printf("%p\n", y);
return 0;
}
now: what happens if I do this (dumps core for me - after reporting a UMR):
#include <stdio.h>
int main()
{
char *x;
char *y = x;
printf("%s\n", y);
return 0;
}
> --
> Ken Lee, http://www.rahul.net/kenton/
--
Frode Tenneboe
: In article <6uesm8$ic$1...@clarknet.clark.net>,
: T.E.Dickey <dic...@shell.clark.net> wrote:
: >In comp.windows.x ken...@nojunk.rahul.net- wrote:
: >> UMRs are not bugs and can be safely ignored. Many popular libraries,
: >> including X, use this optimization.
: >
: >UMRs are potential sources of core dumps. It says that the program is
: >copying unintialized data to a new location (unspecified).
: That is false. The new location is specified properly. Copying
: uninitialized data as fine as long as the destination is properly
: allocated and not later dereferenced. All UMRs in X and Motif do this
: properly. X (and other libraries) does this because a single copy of
: an array or struct is faster than multiple copies or unnecessarily
: initializing unused fields.
As you can see there is some disagreement. However, I have recived
several replies. The product Great Circle looks promising and I'll
probably look into it. Anyway, Kens reply is favourable for the
latest releases and that is probably a fair assumption.
However, are there particular distributions which should be
avoided? Prevously I have only used BlueStone, IXI and
the one shiped from Sun. My impression is that the BS version
has been better in the memory leak department. Though, I don't know
if this is because it has used a different source base. Any
comments?
David B. Lewis
Or, even better, report the bugs. If you are capable of fixing the leaks,
then you must have source, and if you have source, you purchased it from
The Open Group, and you have a copy of the BUGREPORT file including the
address to send bugs and test-cases to. Other reports can go to the vendor
who provided you with your binary versions of the Motif libraries.
David B. Lewis
|> copying unintialized data to a new location (unspecified).
X calls often use the combination of a structure and a flag mask telling what
fields in the structure to pay attention to. The structure is copied en
masse. I believe Purify throws a UMR here, although the code is written such
that uninitialized fields are not dereferenced unless the flag in the mask
says that there's real data there.
T.E.Dickey
> |> You can of course, using purify, fix the leaks yourself ;-)
> Or, even better, report the bugs. If you are capable of fixing the leaks,
> then you must have source, and if you have source, you purchased it from
> The Open Group, and you have a copy of the BUGREPORT file including the
for Motif, of course.
(iirc, though, the Xt bundled with Motif was made incompatible with
the MIT sources to solve a shared-library quibble)
T.E.Dickey
> |> UMRs are potential sources of core dumps. It says that the program is
> |> copying unintialized data to a new location (unspecified).
> X calls often use the combination of a structure and a flag mask telling what
> fields in the structure to pay attention to. The structure is copied en
> masse. I believe Purify throws a UMR here, although the code is written such
> that uninitialized fields are not dereferenced unless the flag in the mask
> says that there's real data there.
maybe - but I'll have to assume that unless someone can automatically
verify that all of the UMRs are harmless, there's some bugs.
(Purify's more likely correct than whoever signed off on this ;-)
Nhi Vanye
Purify is just a tool, you need to understand both the code its analysing
and how it works.
I've seen too many bogus errors from it to take what it says as gospel.
Its good, but its no subsitute for knowing the code.
>Thomas E. Dickey
richard.
--
Richard M. Offer Widget FAQ --> http://reality.sgi.com/widgetFAQ
MTS-Core Design (Motif)
___________________________________________http://reality.sgi.com/offer
Michael Morrell
> |> UMRs are potential sources of core dumps. It says that the program is
> |> copying unintialized data to a new location (unspecified).
> X calls often use the combination of a structure and a flag mask telling what
> fields in the structure to pay attention to. The structure is copied en
> masse. I believe Purify throws a UMR here, although the code is written such
> that uninitialized fields are not dereferenced unless the flag in the mask
> says that there's real data there.
I believe that the above is considered a UMC (uninitialized memory copy),
not a UMR. UMCs are not bugs, but UMRs are. Purify attempts to distinguish
the two. One area where it fails is passing a struct pointer to a function.
If any of the fields in the structure are uninitialized, this is treated as
an UMR instead of the UMC it should be treated as.
Michael
ken...@nojunk.rahul.net-
Michael Morrell <mor...@cup.hp.com> wrote:
>> X calls often use the combination of a structure and a flag mask telling what
>> fields in the structure to pay attention to. The structure is copied en
>> masse. I believe Purify throws a UMR here, although the code is written such
>> that uninitialized fields are not dereferenced unless the flag in the mask
>> says that there's real data there.
>
>I believe that the above is considered a UMC (uninitialized memory copy),
>not a UMR.
Maybe I was using an older version of Purify, but last time I used it
the above was a UMR. I agree that this is safe to do.
Cameron Hayne
<dic...@shell.clark.net> wrote:
>maybe - but I'll have to assume that unless someone can automatically
>verify that all of the UMRs are harmless, there's some bugs.
>(Purify's more likely correct than whoever signed off on this ;-)
It's not an issue of which piece of software is more correct. As others
have tried to explain, it is merely a fact that Purify does not (cannot)
know that the memory copying is okay since Purify does not (cannot) understand
the code.
There may be bugs - that is always true - but a Purify message is not
necessarily evidence of a bug. Instead it is an indication that the
corresponding code should be examined to check if it might be a bug.
--
Cameron Hayne (ha...@crim.ca)
Centre de recherche informatique de Montreal
Glenn Pinkerton
: It's not an issue of which piece of software is more correct. As others
: have tried to explain, it is merely a fact that Purify does not (cannot)
: know that the memory copying is okay since Purify does not (cannot) understand
: the code.
: There may be bugs - that is always true - but a Purify message is not
: necessarily evidence of a bug. Instead it is an indication that the
: corresponding code should be examined to check if it might be a bug.
I think the problem is one of how we mere mortal humans can effectively
filter out the small number of valid messages from the plethora of unimportant
messages reported from any third party library. It's very much like turning
on the higest warning level in a compiler. Only a very few of the warnings
are meaningful, but you pretty much have to fix all of them to be able to
find the meaningful messages in the mess. I personally think that the
optimizations spoken of which cause the plethora of UMR's are not very important
from an overall throughput of the application perspective. I will almost always
opt for code that compiles and runs through a memory checker cleanly over any
optimizations that add significant numbers of warning messages.
All software will eventually break. The best we can do is give ourselves a
sporting chance to fix it when it breaks. I think eliminating large numbers of
UMR's in a library give people a more sporting chance to fix their software.
Glenn Pinkerton
g...@csn.org
Kevin W. Ramer
> In article <6uef57$f5e$2...@clarknet.clark.net>,
> T.E.Dickey <dic...@shell.clark.net> wrote:
>
> UMRs are not bugs and can be safely ignored. Many popular libraries,
> including X, use this optimization.
I entered this thread late and am looking through this discussion.
I strongly disagree with the above statement. I have been consulting at
a large telecommunications firm for the past three years. About a year
ago a problem came about where that company's customers were receiving
whopping credit (hundreds of thousands $) on their bills when adjustments were
being applied to them. I traced this to a UMR on a local double variable
that was
being used in a calculation.
>
> Can you be more specific on the multitudes of serious memory leaks that
> you've found? I consider a memory leak to be serious if it happens
> more than once in the lifetime of a widget.
Second example. Just these last two weeks our new Sun machines were
upgraded to 2.6 of Solaris. I was running purify and found that it was raining.
This and other subsequent problems caused quite a stir in Sun (yesterday a guy
appeared with source for all their current patch levels) as we are
attempting to put a pilot Sun installation 10-8. Fortunately, we discovered
that the installation of the upgrade introduced the problems.
> --
--
Kevin W. Ramer ram...@earthlink.net
Programmer-at-Large
If pro is the opposite of con then Congress must be the opposite
of Progress -- Gallagher
T.E.Dickey
> I think the problem is one of how we mere mortal humans can effectively
> filter out the small number of valid messages from the plethora of unimportant
> messages reported from any third party library. It's very much like turning
> on the higest warning level in a compiler. Only a very few of the warnings
> are meaningful, but you pretty much have to fix all of them to be able to
> find the meaningful messages in the mess. I personally think that the
> optimizations spoken of which cause the plethora of UMR's are not very important
> from an overall throughput of the application perspective. I will almost always
> opt for code that compiles and runs through a memory checker cleanly over any
> optimizations that add significant numbers of warning messages.
exactly (there's two ways to respond to it: one fixes the problem and the
other says the messages don't mean anything). We used to say that about lint.
(Even if only 10% of the messages mean something, that's worth doing).
-- I still use lint too.
ken...@nojunk.rahul.net-
> I think the problem is one of how we mere mortal humans can effectively
> filter out the small number of valid messages from the plethora of unimportant
> messages reported from any third party library.
Purify has a suppress command. You can use it to suppress UMRs in the
X (and other) libraries without affecting UMR reports from your application
code.
Glenn Pinkerton
Of course you can suppress various messages for various libraries. However,
one can never be certain if any of the messages suppressed in the library are
actually caused by improper usage of the library from your own code.
It really isn't that much effort to fix these problems. When I first started
using memory checking tools on my software, which has several hundred thousand
lines of code, I probably had over 1000 "unimportant" messages reported. I
fixed them all on a piece by piece basis when not being particularly active
with other things. I did find several real problems during this process, which
took about 2 months.
As far as the performance is concerned, I stick by my philosophy that the
performance gained by avoiding a memset (e.g. using calloc instead of malloc),
which is usually the fix for a UMR, is insignificant relative to good algorithm
development and relative to the speed of modern computers.
Glenn Pinkerton
g...@csn.org
David B. Lewis
|> necessarily evidence of a bug. Instead it is an indication that the
|> corresponding code should be examined to check if it might be a bug.
Exactly.
It is somewhat ironic that Purify is so good a tool that its output is
automatically trusted, as are the results from AcquaProva, a similar (and
much cheaper!) tool by CenterLine. These tools are a great help, but what
they report shouldn't be taken as Gospel. Rational distributes suppression
files for use with various versions of Xlib, Xt, and Motif.
David B. Lewis
|> that was being used in a calculation.
Ah, but that's a UMR that shouldn't be ignored. Purify is a win here.
T.E.Dickey
> |> There may be bugs - that is always true - but a Purify message is not
> |> necessarily evidence of a bug. Instead it is an indication that the
> |> corresponding code should be examined to check if it might be a bug.
> It is somewhat ironic that Purify is so good a tool that its output is
> automatically trusted, as are the results from AcquaProva, a similar (and
> much cheaper!) tool by CenterLine. These tools are a great help, but what
> they report shouldn't be taken as Gospel. Rational distributes suppression
> files for use with various versions of Xlib, Xt, and Motif.
Even better would be to fix the problems in Xlib, etc., rather than to assert
that Purify is incorrect (or should be taken with a grain of salt).
David B. Lewis
|> the MIT sources to solve a shared-library quibble)
You mean way back in Motif 1.0, which became obsolete in 1991?
Motif is now at 2.1.10, using X11R6.
David B. Lewis
He is referring to a structure copy.
Incidentally, I just tried using Purify to generate some UMRs for code that
does this. The tool appears not to do so, at least in version 4.2, for code
compiled with -g.
--
David B. Lewis At but not speaking for ICS (Cambridge, MA)
dbl AT ics.com
bob schulman
> Even better would be to fix the problems in Xlib, etc., rather than to assert
> that Purify is incorrect (or should be taken with a grain of salt).
Not necessarily. I've seen plenty of reports from Purify of what appear to
be egregious errors. Gee whiz...change -O to -g in the compile line, recompile,
and the problems disappear. So, um, is the problem in the code or the
code generator?
The problem's obviously in the code generator. One could go complain
to Cygnus or the folks at FSF (the code generator twas gcc), or one
could just get on with one's job.
In all my time Purify'ing Motif 2.0 and X11R6 the vast vast vast majority
of problems fall into the lap of aggresive optimizations of gcc rather
than the fault of the X Consortium or The Open Software Foundation.
bob
T.E.Dickey
> In comp.windows.x T.E.Dickey <dic...@shell.clark.net> wrote:
>> Even better would be to fix the problems in Xlib, etc., rather than to assert
>> that Purify is incorrect (or should be taken with a grain of salt).
> In all my time Purify'ing Motif 2.0 and X11R6 the vast vast vast majority
> of problems fall into the lap of aggresive optimizations of gcc rather
> than the fault of the X Consortium or The Open Software Foundation.
That doesn't seem to be how you viewed it before. Here's a quote courtesy
of DejaNews:
re: vxworks 5.2 and X memory leak. more options
Author: bob schulman
Email: b...@seaweed.com
Date: 1997/03/13
Forums: comp.os.vxworks
more headers
_________________________________________________________________
> I have no experience with a similar VxWorks configuration but having worked
> with X for a while, I know that it has a reputation for having a significant
> number of latent memory leaks. Virtually of the X windows systems I've used
> force you to kill X after a period of time to prevent you from exhausting you
r
> virtual memory. I don't think there is a fix for the MIT distribution; there
> may be some private distributions which have fixed them but if so, I am unawa
re
> of them.
Whoa! Not true at all. Seaweed's SeaWind product (X11R6+Motif 2.0 for
VxWorks) has been compiled up and run under the Purify environment
and Purify has almost no memory leak complaints (well, there are a couple
in the Drag 'n' Drop and a badly engineered free routine for a special
data structure in Motif). But the Xlib/Xt/(grand majority of)Motif 2.0
code is clean clean clean. So if there is a "reputation for having a
significant number of latent memory leaks", I'd have to say that such
a reputation is ill-deserved and probably based on old releases of X
or Motif.
By the way, Purify was implemented by a gentlemen who used to be a
Motif consultant who was so sick and tired of memory leaks,
array over/underruns, and references to uninitialized memory in
the code from OSF that he went and invented Purify. Pure Software turned out
to be much more lucrative than Motif consulting. So, Motif used to
be a pretty sick puppy, but those days are long gone.
bob
> bob