PB Application Memory Leak

[johnowens]

I am running a very small PB9 application continuously in
the background that is just running a few minor processes.
It checks a directory for flat files and imports them
(database table imports) exports pre-selected tables into
flat files, and then FTPs them. It's a very small program.
However, under windows 2000, this program keeps increasing
its memory footprint every times it goes through these three
event cycles. (Increases about 120k in memory every cycle).
There are dynamic arrays and cursors in this application,
but they are instance based variables so I dont see how they
would make it get larger like this. If I have a cycle run
once per minute, the application crashes after a day or two.
I have checked to make sure I destroy all of my database
transactions, and close all of my cursors. I have tried
commenting out almost everything in this code but I can't
pinpoint what the problem is. Does anyone know of common
things to look for, or know of a PB9 memory leak bug? This
seems to clean itself up automatically in windows xp. The
application usually starts at about 8000k, grows to about
16,000k, then goes back to 8000k in XP. Keeps growing in
windows 2000.

John Owens

[johnowens]

One interesting thing is that for each table that is
exported to a flat file, we create a dynamic datawindow.
(uses a CREATE). However we cannot destroy it. After a
destroy it, the next time we try to access this datawindow
object the system crashes. Even if we re-create it. If we
had the destroy at the end of the window object, the the
window closes, and on the next cycle is re-opened, the
system STILL crashes the next time we need to use that
object, even though that object has gone out of scope.

[dstone]

I also ran into memory leak problems while running some
conversion scripts. A few well placed GarbageCollect ( )
function calls did the trick for me after destroying any
unused objects.

Hope that helps

[johnowens]

What OS are you using? Where do you place your garbage
collect functions? I know we placed one of them, but it only
worked for us under windows XP.

[Jerry Siegel]

Why not use a datastore instead of a DW? As a non-visual object, it should
be safe to destroy when it is no longer needed. And yes, DS can be created
dynamically just like a DW.

<John Owens> wrote in message news:4117c467.15...@sybase.com...

[dstone]

I know it was pre XP, might even have been NT 4.0 (was a
while back.) We had a gui interface instantialing several
NVOs that were in turn creating several datastores. This
probelm hounding me for weeks, and I placed garbage collects
everywhere I could think of with varying results. The one
place it finally seemed to work was after destroying the
NVOs (though I still have all of the other previously
mentioned garbage collects still in place). If I were you I
would place them anywhere you can. There does not seem to
be a performance hinderance by doing this. Or you can place
some pauses in your code and run it while keeping an eye on
your task manager. If your app's memory usage does not go
down after destroying or closing an object, then put a
garbage collect there.

[Terry Voth]

Can you clarify what you mean? If you mean dw.Create(), that's OK, but
if you mean:

datawindow ldw
ldw = CREATE datawindow

then you've got problems. CREATE isn't meant for visual objects, which
a datawindow is. You have to decide if you want a visual object (and
use OpenUserObject()) or a non-visual object (and use a datastore).

As for the memory leak, if the above isn't the issue, I'd start with
CDMatch (http://members.cox.net/bdick/cdmatch.htm) if you've got
Enterprise.

Good luck,

Terry [TeamSybase] and Sequel the techno-kitten

Sequel's Sandbox: http://www.techno-kitten.com
Home of PBL Peeper, a free PowerBuilder Developer's Toolkit.
Version 2.2.06 now available at the Sandbox
See the PB Troubleshooting Guide at the Sandbox
^ ^
o o
=*=

[johnowens]

yes we used dw.create()

[johnowens]

Well we modified our program to use no datawindows, but to
use datastores instead. We matched up the create and
destroy statements. However the memory leak still exists.

I tried to use the cdmatch program, but it will not open up
our debug file. I dont know if it is too big or what. It
gives the error msg, "Error accessing external object
property : line 147 - wf_reduce_actions of object w_main".

I checked all of my transactions, and we disconnect and
destroy all of them.

The only other things we create besides normal local
variables are dynamic arrays and cursors. However they are
in a window that is opened and closed each cycle that this
program runs, so they should go out of scope after the
window closes shouldn't they? Plus I dont think you need to
destroy either of those objects.

Other than that I don't see anything that would cause a
memory leak. We tried commenting out all of the external
functions we had declared (for zipping and funky32 calls)
and the problem still occurs. Still having problems
pinpointing it since the problem isn't visible until the
entire cycle is over and the memory has almost all been
released (except for the memory leaked of course)

John

[Dave Fish [Team Sybase]]

What build number are you using. FWIW, garbage collection should take
care of unreferenced objects. I know there are a lot of grizzled old
veterans out there who don't trust it, but if it isn't working then
its a bug.

Regards,
Dave Fish
Sybase

[philipsalgannik]

Sounds like it's GDI handles leakage which won't be remedied
by garbage collection...

[johnowens]

Version 9.0.1 7066. is the build we are using.

I have put the garbage collection function in several places
but to no avail. So far everything we have tried besides
commenting out all of the code outright for our export
module doesnt seem to fix the problem.

In short, there is a small interface window that is visible
to the user where he can set the time interval, directories
to export files to, etc. The user presses start. Every
time the time interval hits, this window does a search on a
database table, and for every group of tables entered in the
database (traversed through a cursor) it opens a window
which is basically just a progress bar. This window has
several database transactions where it searches through
tables and exports them and any tables that have foreign key
references to those tabels into flat files. There are
several cursors used, dynamic arrays, and dynamic
datawindows. At the end the collection of flat files is
zipped up, the transactions are disconnected and destroyed,
and the window is closed. Garbage Collection functions are
both inside of this export window and after it is run.

We have tried adding more garbage collection functions, made
sure all of our "CREATE"s have "DESTROY"s, made sure all of
our cursors are closed. The window closes when finished, so
anything else in memory should be out of scope. There are a
few external function calls like feraseall (which is funky i
believe) and a run statement which does a sweep of your
export folder to turn all files to non-read-only before the
feraseall. There is a filehandle check for the finished
attrib window and a Send statement to close that handle.
That's about it. It's a pretty basic program.

We have tried updating powerbuilder to a more recent build
for other issues we thought were powerbuilder related a
couple of weeks ago but that caused other problems in our
system.

John Owens

[Terry Voth]

When you say "will not open up our debug file", you're referring to a
profiling file, not a PBDEBUG file, right? FWIW, most common reason
that a profiling file fails to open is that it wasn't closed properly
(particularly if the program crashes).

Just to be sure, to monitor memory consumption, you're using something
task-specific like PerfMon and not something system-generic like Task
Manager, right? Task Manager can indicate an increase in system memory
usage, but given the dozens to hundreds of things that can be going on
in Windows at any given time, it's a notoriously bad tool to identify
a problem with a specific app. Even with a task-specific monitor,
identifying a PB problem as opposed to a problem with a DBMS client
software or an external DLL call is pretty tough.

One place to go through a checklist is the Troubleshooting Guide
mentioned below. It may ring some bells.

Let's back up and go to basics. If you're app is truly consuming more
and more memory, one of two possibilities exist: your program *is*
actually consuming more memory, or it's not.

If you are, then you've got some serious leg work to do. Sometimes
just "matching" CREATEs and DESTROYs isn't sufficient; you have to
consider program flow. I've had to slap people around for putting a
RETURN between a CREATE and DESTROY (and, trust me, you get some
pretty serious looks when you're caught slapping yourself around in
your cubicle <BG>). You should monitor how much data is being
retrieved in any and all datawindows and datastores (a good place to
do this is the ancestor of each of these objects). Also, having some
kind of monitoring code going in your ancestor NVO (another good
reason to follow the PFC model, if you don't have one) might identify
something. I've never used it much, but maybe the Objects in Memory
tab in the debugger might help identify something. Frankly, CDMatch
will give you an incredible short cut on most of these tasks,
identifying which objects are garbage collected, particularly at the
end of the app, when everything not DESTROYed is garbage collected.
Getting this going will help your life immensely.

If you aren't, then it's one of two possibilities: either PB isn't
releasing memory it's using, as designed, or it isn't garbage
collecting.

Since <?>8.0.2</?>, PB has been designed to not release the memory it
consumes. This enhances performance by not having to go beg the OS for
memory allocation every time PB memory usage goes up and down. If you
are retrieving a data set that gets larger by 120K every cycle, then
your observation makes perfect sense, even if the data set is
destroyed. PB is going to keep that memory around for the next time it
needs it.

The other possibility is a lack of garbage collection. While most
people peg this as a time to clean up objects that are created but not
destroyed, a lot of other things get cleaned up that you don't have
control over, like DWOs (a lot are generated when you use dot notation
to access the datawindow contents) and OLEObjects (ironically, again,
the more "dots" you use, the more temporary OLEObjects get generated
for internal purposes). One possibility there is that you are not
letting PB garbage collect when it is designed to collect, when the
app is at "rest", and you are not forcing a garbage collection. The
other possibility is, like Dave says, a bug that PB isn't garbage
collecting something that it should. Until you've covered everything
above, that's going to be a pretty tough call to make. Unfortunately,
the profiling functionality is the only way to identify what has been
garbage collected. Branching off the approach CDMatch uses to identify
what has been garbage collected and where it was created, you could
probably also write something that identified what objects were in
memory at any given point.

This isn't exactly a solution, but maybe it will give you some ideas
to work on.

Good luck,

Terry [TeamSybase] and Sequel the techno-kitten

Sequel's Sandbox: http://www.techno-kitten.com

[johnowens]

I am using task manager actually. Since I am running a
stand alone DB client, that runs as a seperate APP. My app
that I made is very small and the memory usage for that app
is what continuously is increasing.

I have just re-coded the entire thing to be contained in a
user object instead of in a window, and instead of dynamic
datawindows I am using dynamic datastores. I make sure to
create and destroy every datastore, and I create and destroy
the user object every cycle. It seems that the problem
still exists.

I will check for returns between creates and destroys. Also
will check objects in memory in the debugger.

We checked the latest build of powerbuilder and still have
the same problems.

We are running garbage collection often.

To answer your question about datasets, in this test, the
dataset is identical each cycle, so that's not the problem.

John Owens

[johnowens]

Checked the objects in memory before the function call and
after. They are the same, yet the application is
continuously taking up more and more system memory.

[Mike Kruchten]

What database are you connecting to, and with what DB interface? I vaguely
remember something about a memory leak when using a specific version of the
MSSQL interface with some specific connection option. Sorry I don't remember
the details.

Mike Kruchten

<John Owens> wrote in message news:4117be1d.7f5...@sybase.com...

[johnowens]

We are using Sybase SQL Anywhere 8.

[Alex Whitney]

Just for grins, see what happens if you do not actually retrieve any rows.
Change your arguments so that you never retrieve any data and see if the
leak exists. This may give you some more information. I'd also add several
message boxes at various points in your code so you can watch task monitor
and get a better idea as to WHERE the leak occurs during the process.

As a last resort, you could make the app restart itself once an hour.

Alex
<John Owens> wrote in message news:4119136e.2f1...@sybase.com...

[Andy Neagu]

Are you using some external functions into an dll? Or all is pure PB?
Andy

<John Owens> wrote in message news:4117be1d.7f5...@sybase.com...

[Jeff Hill]

I encountered a similar problem a while back (several versions), where a
client's (Oracle) app had a mess of embedded SQL. We built dataobjects to
replace all the SQL and the problem went away. Could you get rid of the
cursors without a major effort?
<John Owens> wrote in message news:41191219.2ee...@sybase.com...

[johnowens]

We do have external functions, but we have commented them
out and observed the problem conitnue.

We are connecting to a Sybase SQL Anywhere 8 database.

What we are trying now is to split the application into 2
smaller apps, one will just open the other every cycle.
This way the 2nd program gets closed after every cycle and
the memory should get cleaned up.