There must be something different about the two ways you are using that file. I can't tell you what is different, since you did not describe both situations completely, but there must be some difference, otherwise the results would not be different.
I'll tell you some things about TACL that you might not be completely understanding, and maybe that will let you figure out why things are working differently in your two situations.
When you put TACL code into a file that you want to invoke by using that file's name, you have two ways to do it (or three ways, depending on whether you consider a macro different from a routine).
The simplest way is to create a file to be used as the IN file to TACL or as the file named in an OBEY command. The lines in such a file are interpreted by TACL just as if you had typed them at a TACL prompt.
The other way is to include "?TACL MACRO" or "?TACL ROUTINE" as the first line of the file and invoke the file with a RUN command. That can be either an explicit RUN command or an implicit RUN command, in which you just type the name of the file as the first word on the line. For an implicit RUN command, you must ensure that the #PMSEARCHLIST built-in variable includes the subvolume that contains the file, so the RUN command will find the file.
I assume you know the difference between a MACRO and a ROUTINE, so I won't explain that, but let me mention that a file is regarded by TACL as a MACRO or ROUTINE only when the file is invoked by a RUN command and contains a ?TACL line as the first line of the file. In particular, a ?TACL line that is encountered in TACL's IN file does not make the following lines be interpreted as a MACRO or a ROUTINE. In fact, there is a TACL command named "?", which displays a line from the command history that matches the specification following the "?". So a line like "?TACL ROUTINE", when read from TACL's IN file, is a command to display a line from the command history that contains the text "TACL ROUTINE".
That explains the error message you saw when you ran TACL with ABCTEST as its IN file. The error when a ? command does not find a matching line in the command history is not fatal. TACL continues to read lines and tries to execute them. If you did not see any other error messages in testout, I don't know why the other TACL commands in ABCTEST did not do their job of creating the file you expected. I suppose you would not have said the file was a ROUTINE if it does not use some of the features that are peculiar to ROUTINEs, such as #ARGUMENT, so I guess it is not very surprising that the TACL commands did not work as expected, since they were not being interpreted as in a ROUTINE. However, I would expect that you would have seen other error messages besides the one about the "?" command not finding a line to display. But since you did not show the commands in ABCTEST, I don't have enough information to analyze that point any further.
I don't know whether any of that information is new for you, or whether it helps you figure out why your file seems to work as you expect in one situation but not in another situation. That is all I can think to tell you right now. If that is not enough to let you figure out why your ?TACL ROUTINE file works as an IN file in one situation but not in another, and if you want to figure it out, post again with some more description of the two cases, and we might be able to help you figure out the difference.