Random crash and random disappearance PB11.5 application
CyberE
In the past we used PB6.5 and migrated successfully to PB11.5. The
following situation occurres in our current version but also in PB6.5.
The application is randomly crashing and sometimes disappears without
a warning. The following exception analyses logfile was generated:
**********************************************************************************************
FAULTING_IP:
+16d952f00d0df74
00000000 ?? ???
EXCEPTION_RECORD: ffffffff -- (.exr 0xffffffffffffffff)
ExceptionAddress: 00000000
ExceptionCode: 80000007 (Wake debugger)
ExceptionFlags: 00000000
NumberParameters: 0
BUGCHECK_STR: 80000007
PROCESS_NAME: ctc.exe
ERROR_CODE: (NTSTATUS) 0x80000007 - {Kernel Debugger Awakened} the
system debugger was awakened by an interrupt.
EXCEPTION_CODE: (HRESULT) 0x80000007 (2147483655) - Operation aborted
NTGLOBALFLAG: 0
APPLICATION_VERIFIER_FLAGS: 0
LOADERLOCK_OWNER_API:
_LdrpInitialize:LdrpInitializeThread:LdrpCallInitRoutine:
DERIVED_WAIT_CHAIN:
Dl Eid Cid WaitType
-- --- ------- --------------------------
x 0 c8c.f44 Speculated (Triage) -->
x 4 c8c.d84 Critical Section --^
WAIT_CHAIN_COMMAND: ~0s;k;;~4s;k;;
BLOCKING_THREAD: 00000f44
DEFAULT_BUCKET_ID: APPLICATION_HANG_DEADLOCK_HeapCorruption
PRIMARY_PROBLEM_CLASS: APPLICATION_HANG_DEADLOCK_HeapCorruption
LAST_CONTROL_TRANSFER: from 7c90df5a to 7c90e514
FAULTING_THREAD: 00000000
STACK_TEXT:
0012e2b4 7c90df4a 7c864e82 00000002 0012e3f0 ntdll!KiFastSystemCallRet
0012e2b8 7c864e82 00000002 0012e3f0 00000001 ntdll!
ZwWaitForMultipleObjects+0xc
0012e5c0 10001771 0012e600 00000000 00000000 kernel32!
UnhandledExceptionFilter+0x8b9
WARNING: Stack unwind information not available. Following frames may
be wrong.
0012e5d4 10001290 c0000005 0012e600 10001f1d ctc+0x1771
0012ffc0 7c817077 80000001 0173d9e4 7ffdd000 ctc+0x1290
0012fff0 00000000 100010dd 00000000 78746341 kernel32!BaseProcessStart
+0x23
FOLLOWUP_IP:
ctc+1771
10001771 5e pop esi
SYMBOL_STACK_INDEX: 3
SYMBOL_NAME: ctc+1771
FOLLOWUP_NAME: MachineOwner
MODULE_NAME: ctc
IMAGE_NAME: ctc.exe
DEBUG_FLR_IMAGE_TIMESTAMP: 49ade0fe
STACK_COMMAND: ~0s ; kb
BUCKET_ID: 80000007_ctc+1771
FAILURE_BUCKET_ID:
APPLICATION_HANG_DEADLOCK_HeapCorruption_80000007_ctc.exe!Unknown
WATSON_STAGEONE_URL:
http://watson.microsoft.com/StageOne/ctc_exe/8_3_0_0/49ade0fe/unknown/0_0_0_0/bbbbbbb4/80000007/00000000.htm?Retriage=1
Followup: MachineOwner
**********************************************************************************************
Currently we are testing the use of PB_HEAP environment variables, but
are waiting for the results. In the meanwhile...is there somebody that
has a cleu on what we are up against?
Kind regards,
![Jerry Siegel [TeamSybase]'s profile photo](https://lh3.googleusercontent.com/a/default-user=s40-c)
Jerry Siegel [TeamSybase]
of PB used to generate the deployables and the runtime DLLs. Process
Explorer http://technet.microsoft.com/en-us/sysinternals/bb896653.aspx is a
great free utility to check on that.
I would also look for a runaway loop extending an unbounded array or
creating objects, or maybe an external function corrupting storage.
--
Report Bugs to Sybase: http://case-express.sybase.com/cx/welcome.do
Product Enhancement Requests:
http://my.isug.com/cgi-bin/1/c/submit_enhancement
"CyberE" <eren...@gmail.com> wrote in message
news:84e107c2-f14e-41a5...@j4g2000yqa.googlegroups.com...
CyberE
Jerry, thank you for your reply. I checked the (Sybase) DLL's on the
client application and the Build of PB11.5 which is 3136...
pbdwe115.dll - 11.5.0.3136
pb010115.dll - 11.5.0.3136
pbshr115.dll - 11.5.0.3136
pbvm115.dll - 11.5.0.3136
libjcc.dll - 5.3.0.16
libjutils.dll - 5.3.0.16
Nothing funny there, I guess. But I don't know what those libjcc and
libjutils DLL's are...
Awaiting more replies, thanks in advance.
On 15 okt, 18:52, "Jerry Siegel [TeamSybase]"
<jNOsSPAMsiegel@yahoo!.com> wrote:
> The most common cause of "random" crashes is a mismatch between the version
> of PB used to generate the deployables and the runtime DLLs. Process
> Explorerhttp://technet.microsoft.com/en-us/sysinternals/bb896653.aspxis a
> great free utility to check on that.
> I would also look for a runaway loop extending an unbounded array or
> creating objects, or maybe an external function corrupting storage.
>
> --
> Report Bugs to Sybase: http://case-express.sybase.com/cx/welcome.do
> Product Enhancement Requests:http://my.isug.com/cgi-bin/1/c/submit_enhancement
>
> "CyberE" <erens...@gmail.com> wrote in message
> >http://watson.microsoft.com/StageOne/ctc_exe/8_3_0_0/49ade0fe/unknown...
Jerry Siegel [TeamSybase]
build the deployables, right?
I think the libj dlls have to do with memory management. They ARE required
for every deployment.
Re-reading the original post, I notice that you were getting crashes with
6.5. I'm still suspecting a runaway loop and maybe somewhere that objects
are getting created but not destroyed, or a fairly tight loop that is not
allowing automatic garbage collection to grab a time slice. I'd also be
leary of dot notation - under the hood they require a DWObject to be created
and destroyed. (as well as a cast from Any to the actual data type - use
functions for single data items!)
--
Report Bugs to Sybase: http://case-express.sybase.com/cx/welcome.do
Product Enhancement Requests:
http://my.isug.com/cgi-bin/1/c/submit_enhancement
"CyberE" <eren...@gmail.com> wrote in message
news:8612bb45-f89d-4a3a...@v25g2000yqk.googlegroups.com...
CyberE
All versions match. The development PB, the runtime, the rebuild PB,
PowerGen, you name it...
There are so many objects in our app. It's a needle in a haystack to
find the ones who are not destroyed properly. Is there some nifty tool
to check the entire app for all kinds of objects, whether they are
properly created and destroyed?
We use a whole lot of dot notations. So there might be a problem with
that. I'm not sure.
We also use the PFC layer. Quite an old one, but is it possible that
it is causing the problem?
On 16 okt, 09:24, "Jerry Siegel [TeamSybase]"
<jNOsSPAMsiegel@yahoo!.com> wrote:
> And that build matches the ones on the PB used to do the full rebuild and
> build the deployables, right?
> I think the libj dlls have to do with memory management. They ARE required
> for every deployment.
>
> Re-reading the original post, I notice that you were getting crashes with
> 6.5. I'm still suspecting a runaway loop and maybe somewhere that objects
> are getting created but not destroyed, or a fairly tight loop that is not
> allowing automatic garbage collection to grab a time slice. I'd also be
> leary of dot notation - under the hood they require a DWObject to be created
> and destroyed. (as well as a cast from Any to the actual data type - use
> functions for single data items!)
>
> --
> Report Bugs to Sybase: http://case-express.sybase.com/cx/welcome.do
> Product Enhancement Requests:http://my.isug.com/cgi-bin/1/c/submit_enhancement
>
> "CyberE" <erens...@gmail.com> wrote in message
>
Jerry Siegel [TeamSybase]
again - somebody please jump in if you know where it is!
http://www.techno-kitten.com/PowerBuilder_Help/Troubleshooting/troubleshooting.html
offers a lot of useful ideas.
PFC has been beaten on for a long time, I would put it way down the list of
suspects.
Stab in the dark - how often do you optimize your PBLs?
--
Report Bugs to Sybase: http://case-express.sybase.com/cx/welcome.do
Product Enhancement Requests:
http://my.isug.com/cgi-bin/1/c/submit_enhancement
"CyberE" <eren...@gmail.com> wrote in message
news:a83e6827-01c2-4756...@g31g2000yqc.googlegroups.com...
CyberE
good idea to do so.
I went through a large part of the troubleshooting guide you
mentioned, but still can not pinpoint the problem.
It would be great if you remember the name of the tool. I will try and
optimize the PBL's today.
On 16 okt, 10:19, "Jerry Siegel [TeamSybase]"
<jNOsSPAMsiegel@yahoo!.com> wrote:
> I have seen a create/destroy utility mentioned, darned if I can find it
> again - somebody please jump in if you know where it is!
>
> http://www.techno-kitten.com/PowerBuilder_Help/Troubleshooting/troubl...
> offers a lot of useful ideas.
>
> PFC has been beaten on for a long time, I would put it way down the list of
> suspects.
>
> Stab in the dark - how often do you optimize your PBLs?
>
> --
> Report Bugs to Sybase: http://case-express.sybase.com/cx/welcome.do
> Product Enhancement Requests:http://my.isug.com/cgi-bin/1/c/submit_enhancement
>
Jay Williams
http://members.cox.net/bdick/cdmatch.htm
"CyberE" <eren...@gmail.com> wrote in message
news:ea3ec830-999a-41be...@j19g2000yqk.googlegroups.com...
CyberE
I've tried that program yesterday but an error occurs, everytime I
want to open the pbp file.
"Error accessing external object property classname at line 47 in
function wf_reduceactions of object w_main."
A couple of hours ago I optimized the PBL's, but PowerGen does the
same before every build so it was quite obsolete for me to do it
manually.
Monday will be the release of a new version of our program. We will
see after that, but I don't think the problem will be gone.
Any more tips (or questions) are more then welcome...
On 16 okt, 13:53, "Jay Williams" <jwil...@cvps.com> wrote:
> I think this might be the tool Jerry is referring to:http://members.cox.net/bdick/cdmatch.htm
>
![Terry Voth [TeamSybase]'s profile photo](http://lh3.googleusercontent.com/a-/ALV-UjVklmaD3P7c9mYLj1EUbYO3kvYUxaVrJ1_UaYHgfBFyJ9WH=s40-c)
Terry Voth [TeamSybase]
crashes), they can't be read. A PBDEBUG trace is the only trace that
will be useful in a crash situation.
Other than mismatched DLLs, the other prime suspect in "random"
crashes would be external function calls. Make sure they're all
declared properly, and that REF STRINGs are properly pre-allocated.
(PBL Peeper, see below, can get you a list of external function
declarations, then do a Find on all of them.) If either of these are
off, then memory may be corrupted and that may cause a problem in
apparently random spots. The other thing about external function calls
is to make sure they're well behaved (if your team has written their
own) and aren't doing anything that would corrupt memory. When I've
diagnosed stuff like this in the past, the only clue was that the
crashes would always happen "shortly after" we passed some point in
the code. (And, yes, a PB version change can change things around in
memory so that what may have been a benign memory corruption in 6.5
could become a key location that will cause you all sorts of grief
now.)
Good luck,
Terry and Sequel the techno-kitten
On Fri, 16 Oct 2009 05:38:24 -0700 (PDT), CyberE <eren...@gmail.com>
wrote:
*********************************
Build your vocabulary while feeding the hungry
http://www.freerice.com
*********************************
Newsgroup User Manual
=====================
TeamSybase <> Sybase employee
Forums = Peer-to-peer
Forums <> Communication with Sybase
IsNull (AnswerTo (Posting)) can return TRUE
Forums.Moderated = TRUE, so behave or be deleted
*********************************
Sequel's Sandbox: http://www.techno-kitten.com
Home of PBL Peeper, a free PowerBuilder Developer's Toolkit.
Version 4.0.4 now available at the Sandbox
PB Futures updated June 25/2008
See the PB Troubleshooting & Migration Guides at the Sandbox
^ ^
o o
=*=
Jerry Siegel [TeamSybase]
arguments is null - easily possible if the value is coming from a database
and "random" as you don't know which rows have landmines. You need guard
code to substitute a safe value or bypass the call.
--
Report Bugs to Sybase: http://case-express.sybase.com/cx/welcome.do
Product Enhancement Requests:
http://my.isug.com/cgi-bin/1/c/submit_enhancement
"Terry Voth [TeamSybase]" <seq...@techno-kitten.com> wrote in message
news:u51hd5tpl0eek9epf...@4ax.com...
CyberE
within the next couple of day's.
On 17 okt, 02:40, "Jerry Siegel [TeamSybase]"
<jNOsSPAMsiegel@yahoo!.com> wrote:
> And I have seen some external functions cause a crash if one of the
> arguments is null - easily possible if the value is coming from a database
> and "random" as you don't know which rows have landmines. You need guard
> code to substitute a safe value or bypass the call.
>
> --
> Report Bugs to Sybase: http://case-express.sybase.com/cx/welcome.do
> Product Enhancement Requests:http://my.isug.com/cgi-bin/1/c/submit_enhancement
>
> "Terry Voth [TeamSybase]" <seq...@techno-kitten.com> wrote in messagenews:u51hd5tpl0eek9epf...@4ax.com...
>
> > Yes, if profiling files aren't closed properly (e.g. when your app
> > crashes), they can't be read. A PBDEBUG trace is the only trace that
> > will be useful in a crash situation.
>
> > Other than mismatched DLLs, the other prime suspect in "random"
> > crashes would be external function calls. Make sure they're all
> > declared properly, and that REF STRINGs are properly pre-allocated.
> > (PBL Peeper, see below, can get you a list of external function
> > declarations, then do a Find on all of them.) If either of these are
> > off, then memory may be corrupted and that may cause a problem in
> > apparently random spots. The other thing about external function calls
> > is to make sure they're well behaved (if your team has written their
> > own) and aren't doing anything that would corrupt memory. When I've
> > diagnosed stuff like this in the past, the only clue was that the
> > crashes would always happen "shortly after" we passed some point in
> > the code. (And, yes, a PB version change can change things around in
> > memory so that what may have been a benign memory corruption in 6.5
> > could become a key location that will cause you all sorts of grief
> > now.)
>
> > Good luck,
>
> > Terry and Sequel the techno-kitten
>
> > On Fri, 16 Oct 2009 05:38:24 -0700 (PDT), CyberE <erens...@gmail.com>
> ...
>
> meer lezen »
CyberE
IESHIMS.DLL and WER.DLL are missing and Dependency Walker gave the
following error:
Warning: At least one delay-load dependency module was not found.
Warning: At least one module has an unresolved import due to a missing
export function in a delay-load dependent module.
Are these DLL's necessary for PB?
> ...
>
> meer lezen »
Terry Voth [TeamSybase]
I wouldn't worry. These things may never be called. If I were
checking, I'd use Process Monitor from SysInternals/Microsoft and look
for File Not Found messages.
Good luck,
Terry and Sequel the techno-kitten
On Mon, 19 Oct 2009 02:03:01 -0700 (PDT), CyberE <eren...@gmail.com>
wrote:
>I checked the DLL's again with Dependency Walker.
>> meer lezen �
Jerry Siegel [TeamSybase]
IESHAMS.DLL is used by IE8.
Looks like WER.DLL is Windows Error Reporting.
--
Report Bugs to Sybase: http://case-express.sybase.com/cx/welcome.do
Product Enhancement Requests:
http://my.isug.com/cgi-bin/1/c/submit_enhancement
"Terry Voth [TeamSybase]" <seq...@techno-kitten.com> wrote in message
news:6mlod556k4agdvqe9...@4ax.com...
CyberE
further to DLL's.
We don't use that many external functions. The few we use look very
correct to me. There are a bunch of external function calls within the
PFC layer, but they must work I guess.
Maybe the application is using to much memory for some parts and is
the HEAP_SIZE not right. (No environmental variables have been set for
this on our workstations). Would it be a possibility to set the
following variables and maybe solve the problem or is this BS?
SYB_HEAP_SIZE
SYB_POOL_THRESHOLD
SYB_HEAP_MP_MAX_SLOT
SYB_HEAP_MP_MAX_SIZE
PS: What would be good values for these variables?
On 19 okt, 19:00, "Jerry Siegel [TeamSybase]"
<jNOsSPAMsiegel@yahoo!.com> wrote:
> http://technet.microsoft.com/en-us/library/dd433079.aspxindicates that
> IESHAMS.DLL is used by IE8.
> Looks like WER.DLL is Windows Error Reporting.
>
> --
> Report Bugs to Sybase: http://case-express.sybase.com/cx/welcome.do
> Product Enhancement Requests:http://my.isug.com/cgi-bin/1/c/submit_enhancement
>
> "Terry Voth [TeamSybase]" <seq...@techno-kitten.com> wrote in messagenews:6mlod556k4agdvqe9...@4ax.com...
>
> > Never heard of them, but if they aren't needed for *your* application,
> > I wouldn't worry. These things may never be called. If I were
> > checking, I'd use Process Monitor from SysInternals/Microsoft and look
> > for File Not Found messages.
>
> > Good luck,
>
> > Terry and Sequel the techno-kitten
>
> > On Mon, 19 Oct 2009 02:03:01 -0700 (PDT), CyberE <erens...@gmail.com>
> ...
>
> meer lezen »
Terry Voth [TeamSybase]
a problem like this, but I suppose you never know.
Have you tried a PBDEBUG trace? Maybe it's not random, but a commonly
used piece of code.
Any indication that this only happens in long-running processes? You
mention dot notation, and I've come up with a test case recently where
if the dwobjects created implicitly by dot notation aren't
GarbageCollect()'d, the app crashes much sooner than it would
otherwise. (e.g. crashes at 350M of memory consumed as opposed to
1.25G) However, this test case is pretty extreme as a long-running
process.
Good luck,
Terry and Sequel the techno-kitten
On Mon, 19 Oct 2009 23:25:18 -0700 (PDT), CyberE <eren...@gmail.com>
wrote:
>Hmmm, we can rule out those DLL's. I don't think I need to look any
CyberE
environment. It slows the application down too much. We tried it on
some test pc's but the app would not crash.
It is really random. Some times by opening a simple screen and
sometimes after hours of working and long-running processes.
We will look into the dot notations and the GarbageCollect(). Is there
something to check if the app is collecting garbage regularly?
On 20 okt, 14:17, "Terry Voth [TeamSybase]" <seq...@techno-kitten.com>
wrote:
> Well, managing the heap is something that I've never heard of solving
> a problem like this, but I suppose you never know.
>
> Have you tried a PBDEBUG trace? Maybe it's not random, but a commonly
> used piece of code.
>
> Any indication that this only happens in long-running processes? You
> mention dot notation, and I've come up with a test case recently where
> if the dwobjects created implicitly by dot notation aren't
> GarbageCollect()'d, the app crashes much sooner than it would
> otherwise. (e.g. crashes at 350M of memory consumed as opposed to
> 1.25G) However, this test case is pretty extreme as a long-running
> process.
>
> Good luck,
>
> Terry and Sequel the techno-kitten
>
> On Mon, 19 Oct 2009 23:25:18 -0700 (PDT), CyberE <erens...@gmail.com>
> ...
>
> meer lezen »
Jay Williams
sharedobjects (threading) in your app? Or maybe a try catch block where the
catch block is not doing anything? I imagine not as I am thinking try/catch
is fairly new.
"CyberE" <eren...@gmail.com> wrote in message
news:84e107c2-f14e-41a5...@j4g2000yqa.googlegroups.com...
CyberE
What do you mean with sharedobjects? I'm not familiar with that term.
If you ask me if we use shared datawindows or shared datastores, then
I say yes...a lot.
On 20 okt, 15:21, "Jay Williams" <jwil...@cvps.com> wrote:
> Check this thread for a similar problem I have had in PB10. Do you have any
> sharedobjects (threading) in your app? Or maybe a try catch block where the
> catch block is not doing anything? I imagine not as I am thinking try/catch
> is fairly new.
>
> http://groups.google.com/group/sybase.public.powerbuilder.general/bro...
>
> "CyberE" <erens...@gmail.com> wrote in message
> >http://watson.microsoft.com/StageOne/ctc_exe/8_3_0_0/49ade0fe/unknown...
Jay Williams
the help file for more information on their use.
"CyberE" <eren...@gmail.com> wrote in message
news:287eab1b-17fe-430e...@g1g2000vbr.googlegroups.com...
CyberE
SharedObjectRegister or SharedObjectGet what so ever.
On 20 okt, 16:34, "Jay Williams" <jwil...@cvps.com> wrote:
> Search your source code for SharedObjectRegister or SharedObjectGet. See
> the help file for more information on their use.
>
Terry Voth [TeamSybase]
empty (i.e. whenever the app has nothing to do, like when it is
waiting for user input). This handles most cases, but it might be
insufficient for long running processes or if the coding is sloppy,
not cleaning up objects it has created. (The exception is dot
notation, which you can't explicitly clean up.)
Good luck,
Terry and Sequel the techno-kitten
On Tue, 20 Oct 2009 06:06:18 -0700 (PDT), CyberE <eren...@gmail.com>
wrote:
>We did try the PBDEBUG trace. We can not use that in our production
CyberE
more after recent windows Updates.
Thanks for al the help though.
On 20 okt, 18:42, "Terry Voth [TeamSybase]" <seq...@techno-kitten.com>
wrote:
> GarbageCollect() happens implicitly whenever the message queue is
> empty (i.e. whenever the app has nothing to do, like when it is
> waiting for user input). This handles most cases, but it might be
> insufficient for long running processes or if the coding is sloppy,
> not cleaning up objects it has created. (The exception is dot
> notation, which you can't explicitly clean up.)
>
> Good luck,
>
> Terry and Sequel the techno-kitten
>
> On Tue, 20 Oct 2009 06:06:18 -0700 (PDT), CyberE <erens...@gmail.com>
> ...
>
> meer lezen »