vxworks debugging tools
Sergi Casas
> hi,
> ...
> Also, has anyone use vxvmi as a debug tool? I would think it would be
> possible to use this to track down errant memory accesses.
>
> Also, are there any other debug type tools available - wind river or 3rd
> party?
> thanks for any info,
> dave monachello
I am also VERY interested on alternative ways to debug with vxWorks,
specially erroneous memory accesses and memory leaks. Could you, please,
forward me any info you may get on this subject? I think the idea of using
vxvmi seems quite interesting as long as there is an MMU available; hope
someone else may point out how to take advantage of it.
Is there anything like Purify from Pure Software on vxWorks?
(Purify is a memory control&reporting tool through object code insertion
techniques, but as far as I know, does not support multitasking/multithreaded
applications)
I can talk about RTI's Sthetoscope & RTILib, which contain the
HeapCheck modules to check the vxworks memory partitions against corruption
of heap data structures. I'm using an i960 target, and RTILib has many
limitations on it, but maybe in some other architectures can be pretty useful.
I'm sure Stan Schneider will be glad of hearing this free advertisement and
will be able to add more info. It's your turn, Stan! :-)
Greetings, vxcoWorkers
--
|============================================================================|
| Sergi Casas ##### / ##### |
| Hewlett-Packard Company ### /_ _ ### e-mail: se...@hp-bpo.bpo.hp.com |
| Barcelona Division (BCD) ## / / / / ## |
| Avda. Graells, 501 ## / / /_/ ## Phone : +34 3 582 14 75 |
| 08190 S. Cugat (Barcelona) ### / ### FAX : +34 3 582 25 15 |
| SPAIN ##### / ##### |
|============================================================================|
David Monachello
I'm looking into ways to improve our debug environment. We are
currently using vxgdb and vxmon. I would like to use RDB and windview
but I'm not sure if they would work with our system because we have
our own tcp/ip protocol stack. Is this possible?
Also, has anyone use vxvmi as a debug tool? I would think it would be
possible to use this to track down errant memory accesses.
Also, are there any other debug type tools available - wind river or 3rd
party?
thanks for any info,
dave monachello
--
**************************************************
* Dave Monachello @ Micom Communications Corp. *
* email: da...@micom.com *
**************************************************
Stan Schneider
>> Submitted-by se...@hpbpq5.bpo.hp.com Tue Oct 25 06:40:12 1994
>>
>> I am also VERY interested on alternative ways to debug with vxWorks,
>> specially erroneous memory accesses and memory leaks. Could you, please,
>> forward me any info you may get on this subject? I think the idea of using
>> vxvmi seems quite interesting as long as there is an MMU available; hope
>> someone else may point out how to take advantage of it.
>>
>> Is there anything like Purify from Pure Software on vxWorks?
>> (Purify is a memory control&reporting tool through object code insertion
>> techniques, but as far as I know, does not support multitasking/multithreaded
>> applications)
>>
>> I can talk about RTI's Sthetoscope & RTILib, which contain the
>> HeapCheck modules to check the vxworks memory partitions against corruption
>> of heap data structures. I'm using an i960 target, and RTILib has many
>> limitations on it, but maybe in some other architectures can be pretty useful.
>> I'm sure Stan Schneider will be glad of hearing this free advertisement and
>> will be able to add more info. It's your turn, Stan! :-)
>>
OK, I can take a hint...
BTW, all the memory utilities are now supported on the i960.
I'll answer the question with an exerpt from RTILib's FAQ:
---------------------------------------------------------------------------
Some answers to FAQ's (Frequently-Asked Questions) about RTILib:
Q: What does HeapCheck do? How does it relate to VxVMI? To Purify?
HeapCheck checks the heap (the memory managed by malloc/free) for
inconsistencies. It detects any memory write that damages the heap (such as
writing off the end of an array, using a bad pointer, etc.). While it
detects these errors "after the fact", calls to HeapCheck can be patched in
likely places in your code to help catch the problem before the damage
spreads (memory errors can be like cancer). It's by no means a perfect
solution, but it serves two crucial purposes: a) it tells you when memory
errors are a problem, and b) it helps you track them down in a reasonable
timeframe. In our experience, a) is the more important, many of our
customers spend days looking for some intermittent or nasty bug they think
is completely unrelated to memory usage before they try HeapCheck. We've
never run into a problem that couldn't be found in an hour or so "after the
fact"; most are found in minutes.
In its default configuration, VxVMI protects program text segments from
being clobbered. VxVMI can be set up to protect other regions of memory.
[With some effort] VxVMI can also be configured to change allowed memory
regions on a task switch, and thus catch inter-task bad memory references.
It uses the MMU hardware, & so is fast & reliable.
However, VxVMI won't catch the more common case of a task that damages its
own memory, i.e. the heap. Problems like writing off the end of an array,
writing to a "freed" pointer, etc. [almost always] affect the task's own
memory space, and thus can't be caught by an MMU. HeapCheck can be used to
detect and isolate these cases. [
To use the UNIX analogy, VxVMI works like the UNIX memory manager; it
protects code segments, task interactions, and things that trash the
system. HeapCheck is similar to Purify; it finds problems within a task's
address space.
Q: What does HeapTrace do? How is it different from HeapCheck?
RTILib also contains HeapTrace, a utility that records what routine
allocated each block of memory. HeapTrace is useful for detecting memory
leaks (something VxVMI doesn't do at all).
HeapTrace patches the VxWorks memory management routines with routines that
record every transaction. Then, it can go through the heap and display what
blocks were allocated by what routines, etc. It does a good job of
finding memory leaks, as well as showing how and why memory is being used.
For instance, there was a thread on memory leaks last year:
>> From: win...@xray.aps.anl.gov (John R. Winans)
>> Why does the following program continuously eat memory?
[...program deleted, calls fopen/fclose repeatedly...]
>> memShow() says that I consume 288 bytes each time I go around the loop.
>> And once in a while, it jumps by 1704.
A display from HeapTrace after 7 fopen - fclose cycles:
-> HeapTraceShow 10
Blocks Total Call stack (to depth 10)
7 1960 _funcCall _fopen _open _iosOpen _netOpen _netGet _ftpXfer
_ftpHookup _socket _socreate
1 1408 _ipintr _tcp_input _sonewconn _tcp_usrreq _tcp_attach
_tcp_newtcpcb _m_getclr _m_more _m_expand _m_clalloc
The 280-byte blocks (1960/7 = 288 with header) are allocated by the socket
create code. The 1408-byte block (that with a 288-byte block and header
produces a 1704-byte jump) is a network buffer (mbuf) allocated by
m_clalloc down in the tcp code.
Purify finds leaks by scanning all of memory for blocks that aren't
referenced. That's much less reliable, slower, and harder to understand
than the interactive display options used by HeapTrace. Having an on-line
interactive interface is a real advantage that's simply not available to
Purify. (HeapTrace actually has a Purify-style memory scan routine called
"HeapTraceLeaks", but it's undocumented because of these issues. Anyone
that wants to try it, drop me a line.)
Q: How does FastBuffer differ from malloc?
FastBuffer provides memory allocation that's 10 times faster than
malloc/free. It allocates (user-specified) fixed-size memory buffers.
Malloc allows any size blocks, but it's relatively slow and
non-deterministic. FastBuffer can also be called in interrupt code, etc.
Q: What does Trace do? How is it related to VxGDB? To UNIX/PC tools?
Trace lets you monitor routines called and their parameters. For each
execution of a traced routine, a record is output showing what was called,
which task called it, what parameters it was called with, etc. For example,
you can find out what files a program opens via:
-> Trace fopen,"%s, %s"
->
rshell3: 1: _fopen(setup, r)
rshell3: 2: _fopen(setup.observer, r)
rshell3: 3: _fopen(/local/export/rts/root/target1/setup.vis, r)
rshell3: 4: _fopen(../log, w+)
You can trace any function call in the system, including VxWorks system
functions and your own functions. This can be a really useful tool.
VxGDB is a debugger. You can stop the program at breakpoints and look at
the parameter list, but it really doesn't do anything much like this. I
suppose you might be able to write some sort of TCL script to print these
records and continue, but it would be quite an effort to say the least.
There's a (sort of) Unix equivalent called "trace" on SunOS4, "truss" on
Solaris. There are a number of DOS system-call trace programs on the
market. It's a really popular tool in the DOS world.
[This one's only available on 68k right now.]
Q: What does Patch do? What good is it?
Patch allows you to insert a "hook" anywhere in any code. It inserts a
routine to be called before or after any other function in the compiled
code.
Patch is useful for lots of things. For instance, it's how Trace gets
installed to display function calls. Some other examples:
To interject a 1-second delay before every call to ReadA2D:
PatchSet(ReadA2D, taskDelay, 0, 60)
To see if task interactions are causing problems in MyRoutine:
PatchSet(MyRoutine, taskLock, taskUnlock)
Another way to do the above:
mutex = semBCreate(1, 1)
PatchSet(SendPacket, semTake, semGive, mutex, -1, 0);
To count the invocations of MyRoutine:
mycount = 0;
PatchSet(MyRoutine, PatchCount, 0, &mycount)
[This one's only available on 68k right now.]
Q. What's the difference between rshell and the VxWorks shell?
Rshell is a simple remote terminal server for VxWorks. It is *not* intended
as a replacement for the VxWorks shell. It only parses a very limited
subset of the C language.
However, rshell opens a new connection and spawns a shell process for every
incoming request. So, you can have as many connections to the target as you
like. You can run interactive programs without tying up the VxWorks shell.
You can also send commands directly from the UNIX command line.
Most users will always want a VxWorks shell connection as their primary
interface. The rshell connections are useful for ancilliary functions, such
as profiling, running a user menu, running spy, etc. It's also very useful
when running commands that you wish to set breakpoints in (easier than
spawning a process), restarting or analyzing a dead shell, etc.
---------------------------------------------------------------------------
RTILib has a bunch of other stuff as well, such as a menu and user input
processing package, a shared-memory server, etc., but these aren't really
"tools", & I've burned enough bandwidth for today...
-- Stan
===============================================================================
= = =
= Stan Schneider = email: st...@rti.com =
= Real-Time Innovations, Inc. = Phone: (408) 720-8312 =
= 954 Aster, Sunnyvale, CA 94086 = Fax: (408) 720-8419 =
= = =
===============================================================================