Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Need help patching an image to be notraceback

40 views
Skip to first unread message

Michael W. Wheeler

unread,
Aug 15, 1991, 11:22:00 AM8/15/91
to
A few years ago there was a patch that floated around the net
that would clear the traceback bit in the image header (or
something to that effect).

Well, the short of it is that it doesn't work here and now under
VAX/VMS 5.4-2. So my question is... does anyone out there have a
procedure that works under the current version of VMS? If yes or
you can help me out in some other way then please reply directly
to me and I'll summarize for the net.

And for those of you that are asking why would I want to do such
a thing in the first place... It's because we run an application
called MATLAB (version 3.5g) which has a stupid security check in
it that needs NETMBX priv which we don't grant to our students
and we don't want to start now.

No problem we thought. We'll just install it with NETMBX. No dice.
The image has been linked /TRACEBACK and the install utility says
it wont install images with traceback.

The folks at The MathWorks (MATLAB) just think it would be too
much trouble for them to help us out (even though we keep our
maint. license current.) Real thoughtful of them right? Dealing
with them is another whole story in and of itself! We asked for
them to supply us the image /NOTRACEBACK and we've gotten a
number of responses from them... First we got a "Maybe" then a
"Sure no problem" and now a "No, but that might change in two
weeks when so and so gets back from vacation and we ask them
about it." And all of this in the responsive time of just two
weeks. ;-) Sheesh!

Please send any help since The MathWorks apparently doesn't feel
obligated to help us.

Michael W. Wheeler (Bitnet: mww@tntech)
VAX/VMS Systems Support, Tennessee Technological University
Box 5071, Cookeville, TN 38505, Phone: 615-372-3977, FAX: 615-372-6172

Norm Lastovica

unread,
Aug 15, 1991, 9:58:17 PM8/15/91
to

In article <01G9EKM0S...@TNTECH.BITNET>, M...@TNTECH.BITNET ("Michael W. Wheeler") writes:

>A few years ago there was a patch that floated around the net
>that would clear the traceback bit in the image header (or
>something to that effect).
>
>Well, the short of it is that it doesn't work here and now under
>VAX/VMS 5.4-2. So my question is... does anyone out there have a
>procedure that works under the current version of VMS? If yes or
>you can help me out in some other way then please reply directly
>to me and I'll summarize for the net.

I'm very surprised to find that something that hacked the
image header no longer works. very surprised indeed. that stuff
hasn't changed in a very long time. perhaps your procedure was
broken to begin with? anyhow, here's an unsupported way to do
what you want. All of the information about how it works can be
found in the VMS internals and data structures book (the black
book).

$if p1 .eqs. "" then inquire p1 "Enter the name of the file to patch"
$if p2 .eqs. "" then inquire p2 "Y for debug ON, N for debug OFF"
$if p2 then gosub debug_on
$if .not. p2 then gosub debug_off
$exit
$
$debug_on:
$patch/absolute/nonew 'p1'
examine/word 2
define base = \
!
examine/long base + 8
define contents = \
deposit/long base + 0c = contents
!
examine/long base + 4
define contents = \
deposit/long base + 8 = contents
!
examine/long base
define contents = \
deposit/long base + 4 = contents
!
deposit/long base = 7ffedf68
!
examine/long 20 ! get the link flags longword
deposit/long 20 = \+1 ! set the LNKDEBUG bit
$return
$
$debug_off:
$patch/absolute/nonew 'p1'
examine/word 2
define base = \
!
examine/long base + 4
define contents = \
deposit/long base = contents
!
examine/long base + 8
define contents = \
deposit/long base + 4 = contents
!
examine/long base + 0c
define contents = \
deposit/long base + 8 = contents
!
!
examine/long 20 ! get the link flags longword
deposit/long 20 = \@-1@1 ! clear the LNKDEBUG bit
$return

----
the opinions expressed here are my own and hardly ever
reflect those of Digital Equipment Corporation.

Chris_F...@cup.portal.com

unread,
Aug 16, 1991, 8:01:46 AM8/16/91
to
Norm Lastovica writes:
>
> In article <01G9EKM0S...@TNTECH.BITNET>, M...@TNTECH.BITNET ("Michael W.

> Wheeler") writes:
>
> >A few years ago there was a patch that floated around the net
> >that would clear the traceback bit in the image header (or
> >something to that effect).
> >
> >Well, the short of it is that it doesn't work here and now under
> >VAX/VMS 5.4-2. So my question is... does anyone out there have a
> >procedure that works under the current version of VMS?
>
> I'm very surprised to find that something that hacked the
> image header no longer works. very surprised indeed. that stuff
> hasn't changed in a very long time. perhaps your procedure was
> broken to begin with?
>

First off, I, too, am very surprised to hear that the prior patch doesn't
work "now under VAX/VMS 5.4-2." Mine does. ;-) I can think of several
possible explanations... If your "fix" consisted of running an executable
image (George Carette's SETDEBUG.C, or my own SETDEBUG.MAR), perhaps the
VMS version under which the executable image was LINKed, was far enough
back in releases that you need to RELINK the executable. You would prob-
ably have said this, though, if that were the case. The second possibility,
almost equally unlikely, is that someone or something corrupted the data
or instructions in your patch, during or after the upgrade. If you did a
system backup-and-restore of user files from your old VMS version to the
new, but didn't use those Backup qualifiers just right, files marked /NOBACKUP
could have been copied as FILENAMES and allocations ONLY -- no DATA. I had
several files which appeared to have been moved correctly, but turned out
to contain only the pre-existing contents of their component disk blocks,
due to this problem across a VMS upgrade. Finally, maybe the problem with
the MATLAB image isn't exactly, entirely the same situation that your patch
was designed to address.

Norm L. goes on to provide a clever DCL procedure that invokes the PATCH
utility to tweak image transfer addresses and the LNKDEBUG flag bit to
"enable" and "disable" traceback. I'm concerned because it seems possible
for Norm's procedure to wipe out the target image's transfer addresses if
either of its options ("ON" or "OFF") is invoked several times in succession on
the same image. Norm, correct me if I'm wrong, but it _looks_ like you're
moving the Transfer address vectors "up" or "down" in the first-second-third
list, in addition to setting/clearing the LNK$_DEBUG flag bit. I don't see
any check (or, in DCL/Patch, any way TO check-and-branch) for whether, say,
this procedure has ALREADY been performed, i.e. if the first transfer address
is already SYS$IMGSTA (%X7ffedf68), and NOT moving things again and again til
ALL you have is SYS$IMGSTA in all three transfer-address fields. (On the
other hand, you don't look as though you're doing it the way I would; I'd
hack at offsets %X30 and %X34 in the image file VBN 1, but you look like
you're hacking offsets %X4, %X8, and %X0C. What'cha doin'? Write me,
my brain is buzzing...)

One ought to be aware, too, that if you use SETDEBUG.xxx AND Norm's pro-
cedure interchangeably, you'll be playing with the LNKDEBUG flag bit inde-
pendently of the image transfer addresses; the best thing I can say about
THAT is that it's better than the other way around...

As a last resort you could always use SETDEBUG to enable just the LNKDEBUG
bit, run MATLAB under the debugger to find out where, exactly, that check-
for-NETMBX-priv is taking place, and then use PATCH to eliminate/bypass it...
Sure the authors/sellers might get annoyed, but turnabout is fair play...

Good luck getting MATLAB to work; it SHOULD be possible...

Chris Chiesa
Chris_F...@cup.portal.com

Norm Lastovica

unread,
Aug 16, 1991, 9:51:58 AM8/16/91
to

In article <45...@cup.portal.com>, Chris_F...@cup.portal.com writes:
>Norm L. goes on to provide a clever DCL procedure that invokes the PATCH
>utility to tweak image transfer addresses and the LNKDEBUG flag bit to
>"enable" and "disable" traceback. I'm concerned because it seems possible
>for Norm's procedure to wipe out the target image's transfer addresses if
>either of its options ("ON" or "OFF") is invoked several times in succession on
>the same image. Norm, correct me if I'm wrong, but it _looks_ like you're
>moving the Transfer address vectors "up" or "down" in the first-second-third
>list, in addition to setting/clearing the LNK$_DEBUG flag bit. I don't see
>any check (or, in DCL/Patch, any way TO check-and-branch) for whether, say,
>this procedure has ALREADY been performed, i.e. if the first transfer address
>is already SYS$IMGSTA (%X7ffedf68), and NOT moving things again and again til
>ALL you have is SYS$IMGSTA in all three transfer-address fields.

Ok, if you turn it off several times, or on several times, it'll
hurt you. but heck, this is a DCL hack to solve the problem of an image
that was linked they way you don't want it to be. I don't think that is
such a bad restriction. besides, you'll have made a backup copy! ;-)

Anyhow, the problem being discussed in the base note was of
traceback, not debug. I believe that this will fix it for you (or will
give enough of a hint to do so). The reference to the IDSM was to try
and prod one into seeing exactly what the procedure does! Though

>(On the
>other hand, you don't look as though you're doing it the way I would; I'd
>hack at offsets %X30 and %X34 in the image file VBN 1, but you look like
>you're hacking offsets %X4, %X8, and %X0C. What'cha doin'? Write me,
>my brain is buzzing...)

the first examine (exam/word 2) gets the offset to the transfer
address array. from there, the offsets within the array are used (rather
than hardcoding in the addresses, I used the pointer to help ward off
any future changes to the fixed size part. the flags stay at a fixed
offset within the header, so there was no need to play any special games
there!

George J. Carrette

unread,
Aug 16, 1991, 5:49:20 AM8/16/91
to
In article <01G9EKM0S...@TNTECH.BITNET>, M...@TNTECH.BITNET ("Michael W. Wheeler") writes:
> No problem we thought. We'll just install it with NETMBX. No dice.
> The image has been linked /TRACEBACK and the install utility says
> it wont install images with traceback.

This procedure can be used to set traceback ON and OFF, and to also
turn ON and OFF the debug bit.

One use, you can do LINK/DEBUG, and the SETDEBUG the DEBUG bit off.
This makes the the result of a SET PROCESS/DUMP created dump a heck of
a lot more useful.

> The folks at The MathWorks (MATLAB) just think it would be too
> much trouble for them to help us out (even though we keep our
> maint. license current.) Real thoughtful of them right? Dealing
> with them is another whole story in and of itself! We asked for
> them to supply us the image /NOTRACEBACK and we've gotten a
> number of responses from them... First we got a "Maybe" then a
> "Sure no problem" and now a "No, but that might change in two
> weeks when so and so gets back from vacation and we ask them
> about it." And all of this in the responsive time of just two
> weeks. ;-) Sheesh!

Well, the real problem at Mathworks is that they less than ZERO knowledge
of VMS, and are unable to ask for help from the obvious places.
Oh. well.

$! SETDEBUG
$! Manipulate the flag and transfer vector fields in a VMS image header
$! so as to enable or disable the use of the debugger and the
$! traceback handler.
$!
$ IF P1 .EQS. "" THEN GOTO DOC
$ IF F$SEARCH(P1) .EQS. "" THEN GOTO NOFILE
$ SF = "SYS$SCRATCH:PATCH.TMP"
$ if p2 .eqs. "" then goto show
$ if p2 .eqs. "0" then goto clear_debug
$ if p2 .eqs. "1" then goto set_debug
$ if p2 .eqs. "T" then goto set_traceback
$ if p2 .eqs. "NT" then goto clear_traceback
$DOC:
$ WRITE SYS$OUTPUT "SETDEBUG <IMAGE-NAME> <P2>"
$ write sys$output "P2 must be 0 or 1, or T or NT"
$exit 44
$NOFILE:
$ WRITE SYS$OUTPUT "%SETDEBUG-E-NOFILE file not found: ''P1'
$EXIT 44
$!
$show:
$ patch/absolute 'p1'
e 20
e 30
e 34
$exit
$!
$clear_debug:
$ patch/absolute/nonew_version 'p1'
deposit 20
01000028
exit
exit
$exit
$!
$set_debug:
$ patch/absolute/nonew_version 'p1'
deposit 20
01000029
exit
exit
$exit
$!
$set_traceback:
$ gosub SET_TA_VALS
$ IF TRANSFER_ADDRESS_1 .EQ. TRACEBACK_TA THEN GOTO ALREADY_TB
$ IF TRANSFER_ADDRESS_2 .NE. 0 THEN GOTO ALREADY_TB
$ IF F$TRN("OUT") .NES. "" THEN CLOSE OUT
$ OPEN/WRITE OUT 'SF'
$ WRITE OUT "$ patch/absolute/nonew_version ''p1'"
$ WRITE OUT "deposit 30"
$ WRITE OUT F$FAO("!XL",TRACEBACK_TA)
$ WRITE OUT F$FAO("!XL",TRANSFER_ADDRESS_1)
$ WRITE OUT "EXIT"
$ WRITE OUT "EXIT"
$ CLOSE OUT
$ @'SF'
$EXIT
$ALREADY_TB:
$ WRITE SYS$OUTPUT "%SETDEBUG-E-ALREADY traceback already set"
$ EXIT
$!
$clear_traceback:
$ gosub SET_TA_VALS
$ IF TRANSFER_ADDRESS_1 .NE. TRACEBACK_TA THEN GOTO NOT_TB
$ IF TRANSFER_ADDRESS_2 .EQ. 0 THEN GOTO NOT_TB
$ IF F$TRN("OUT") .NES. "" THEN CLOSE OUT
$ OPEN/WRITE OUT 'SF'
$ WRITE OUT "$ patch/absolute/nonew_version ''p1'"
$ WRITE OUT "deposit 30"
$ WRITE OUT F$FAO("!XL",TRANSFER_ADDRESS_2)
$ WRITE OUT "0"
$ WRITE OUT "EXIT"
$ WRITE OUT "EXIT"
$ CLOSE OUT
$ @'SF'
$EXIT
$NOT_TB:
$ WRITE SYS$OUTPUT "%SETDEBUG-E-NOTTB traceback is not currently set"
$ EXIT
$!
$SET_TA_VALS:
$ TRACEBACK_TA = %X7FFEDF68
$ IF F$TRN("IN") .NES. "" THEN CLOSE IN
$ OPEN/READ IN 'P1'
$ READ IN REC
$ CLOSE IN
$ TRANSFER_ADDRESS_1 = F$CVUI(%x30 * 8,32,REC)
$ TRANSFER_ADDRESS_2 = F$CVUI(%x34 * 8,32,REC)
$ WRITE SYS$OUTPUT F$FAO("TRANSFER_ADDRESS_1 = !XL",TRANSFER_ADDRESS_1)
$ WRITE SYS$OUTPUT F$FAO("TRANSFER_ADDRESS_2 = !XL",TRANSFER_ADDRESS_2)
$RETURN

Chris_F...@cup.portal.com

unread,
Aug 18, 1991, 1:12:47 PM8/18/91
to
Thanks to Norm Lastovica for clearing up my mystification as to
the use of offsets in his DCL+PATCH procedure for enabling/disabling image
traceback! The missing link was that _I_ didn't know that offset 2 contained
a pointer to the transfer-address array. (Don't ask where I get my image-
header knowledge; it's pretty krufty...)

Chris_F...@cup.portal.com

0 new messages