Read/Write to TACL temporary file

[lyates]

I need to create a temp file and write to it and then read from it
within a TACL routine.
Any ideas?

[dimandja]

What ideas do you need? How to create a temp file? How to read/write
a temp file?

When you are specific, you can get a good answer in 1 or 2 posts,
intead of in 30 posts.

[Doug Miller]

In article <549ae02f-465c-443f...@w30g2000yqw.googlegroups.com>, lyates <lapto...@gmail.com> wrote:
>I need to create a temp file and write to it and then read from it
>within a TACL routine.

Why? Is there some reason you can't use a TACL variable for the same purpose?

>Any ideas?

Sure.

1) concoct a name for your temp file (optional)
2) create it
3) write to it
4) read from it

Seriously, though -- what have you tried so far, and what happened when you
tried it? If you're actually working on this and run into problems, I'll help
you solve them, but I'm not going to do your job for you. If you're completely
baffled and have no idea where to start, check out the descriptions of the
#CREATEFILE and #REQUESTER builtins in the TACL Reference Manual, and read
chapter 4 ("Accessing Files") in the TACL Programming Guide.

[Keith Dick]

He didn't actually say he wanted to create the file from TACL code. If the file is to be created from some other program, then read in a TACL routine, that is a possible reason for not using a TACL variable. Another possible reason could be that the amount of data is very large.

If the file is written in another program, and is not very large, then the FILETOVAR command followed by an #EXTRACT loop probably is easier than using #REQUESTER. If the file is large, then #REQUESTER probably is the best way.

This probably doesn't apply, but just in case: Guardian has a special sort of file called a temporary file, which is deleted immediately when it is closed. That kind of file is not appropriate for passing data from one program to another. It is useful only for temporarily holding data during the execution of a program when the data is too large to keep all in memory. I don't know whether one can create and use that kind of temporary file in TACL.

[dimandja]

Keith, I don't see how you can derive those conclusion from the blurry
original post;)

Anyway, temporary files can be created by any Tandem process and let
to die when the process expires However the process can choose to
make the temporary file permanent by formally renaming it.

Also a process can obtain (from Guardian) the name of the temporary
file and pass that name around without any restrictions.

The potential uses of a temporary file are up to the process that
creates it. The size of a temporary file does not depend solely on
the size of memory, it depends on the intentions of the creator
process.

TACL can create any type of file allowed by Guardian, Enscribe or
whatever database it is subscribing to.

[Doug Miller]

In article <w-GdnblSGbKKJKLR...@giganews.com>, kd...@acm.org wrote:
>Doug Miller wrote:
>> In article
> <549ae02f-465c-443f...@w30g2000yqw.googlegroups.com>, lyates
> <lapto...@gmail.com> wrote:
>>
>>>I need to create a temp file and write to it and then read from it
>>>within a TACL routine.
>>
>>
>> Why? Is there some reason you can't use a TACL variable for the same purpose?
>>
>>
>>>Any ideas?
>>
>>
>> Sure.
>>
>> 1) concoct a name for your temp file (optional)
>> 2) create it
>> 3) write to it
>> 4) read from it
>>
>> Seriously, though -- what have you tried so far, and what happened when you
>> tried it? If you're actually working on this and run into problems, I'll help
>> you solve them, but I'm not going to do your job for you. If you're completely
>> baffled and have no idea where to start, check out the descriptions of the
>> #CREATEFILE and #REQUESTER builtins in the TACL Reference Manual, and read
>> chapter 4 ("Accessing Files") in the TACL Programming Guide.
>
>He didn't actually say he wanted to create the file from TACL code.

No, but he did say that he wants to write to it, then read it, with a TACL
routine, which certainly implies that it *should* be created in the TACL code
as well.

> If the
> file is to be created from some other program, then read in a TACL routine,
> that is a possible reason for not using a TACL variable.

He's writing to it in the TACL routine.

>Another possible
> reason could be that the amount of data is very large.

Right, I understand that. I wanted to find out what his reasons were for
wanting a temp file. Those are valid reasons, but in the absence of such
reasons, he's probably better off using a TACL variable.

>
> If the file is written in another program, and is not very large, then the
> FILETOVAR command followed by an #EXTRACT loop probably is easier than using
> #REQUESTER. If the file is large, then #REQUESTER probably is the best way.

Again: he said he wants to write to it, then read it, in the TACL routine.

>
>This probably doesn't apply, but just in case: Guardian has a special sort of
> file called a temporary file, which is deleted immediately when it is closed.
> That kind of file is not appropriate for passing data from one program to
> another. It is useful only for temporarily holding data during the execution
> of a program when the data is too large to keep all in memory. I don't know
> whether one can create and use that kind of temporary file in TACL.

Yes, you can:
#CREATEFILE /FILENAMEV filevar/ $volume
creates a Guardian temp file $volume.#nnnnnnn and returns that name in
[filevar].

[Keith Dick]

This illustrates one reason that it is hard to answer questions via email or newsgroup posts. It really slows down things to ask for clarification before writing answers, so we sometimes make assumptions that turn out to be wrong.

There are (at least) two ways to parse his statement. The way you interpreted it, the "in TACL" applies to all three preceding actions. My point was that he did not make it clear that the "in TACL" applied to all three actions. It might be that he only meant that the file would be read in TACL. Maybe that is what he meant; maybe not.

>
>>If the
>>file is to be created from some other program, then read in a TACL routine,
>>that is a possible reason for not using a TACL variable.
>
>
> He's writing to it in the TACL routine.
>

Maybe.

>
>>Another possible
>>reason could be that the amount of data is very large.
>
>
> Right, I understand that. I wanted to find out what his reasons were for
> wanting a temp file. Those are valid reasons, but in the absence of such
> reasons, he's probably better off using a TACL variable.
>

Right. I tried not to say that using a variable was not right. It probably is better, assuming all the action is from a single TACL process and the amount of data is modest. But we don't know those conditions hold.

>
>>If the file is written in another program, and is not very large, then the
>>FILETOVAR command followed by an #EXTRACT loop probably is easier than using
>>#REQUESTER. If the file is large, then #REQUESTER probably is the best way.
>
>
> Again: he said he wants to write to it, then read it, in the TACL routine.
>

Maybe that is what he meant. We don't know for sure.

>
>>This probably doesn't apply, but just in case: Guardian has a special sort of
>>file called a temporary file, which is deleted immediately when it is closed.
>>That kind of file is not appropriate for passing data from one program to
>>another. It is useful only for temporarily holding data during the execution
>>of a program when the data is too large to keep all in memory. I don't know
>>whether one can create and use that kind of temporary file in TACL.
>
>
> Yes, you can:
> #CREATEFILE /FILENAMEV filevar/ $volume
> creates a Guardian temp file $volume.#nnnnnnn and returns that name in
> [filevar].

Thanks for the information. I had never tried to use a Guardian temporary file in TACL. I could imagine TACL checking the parameter and deciding $volume wasn't an acceptable form of filename for #CREATEFILE. From what you say, TACL does not restrict it.

The other point I was trying to make is, even if you can create a Guardian temporary file in TACL, I imagine you have to be careful how you use it to make sure that TACL opens it only once. For example, I expect you would not be able to use it in a VARTOFILE, then later in a FILETOVAR, since I imagine TACL would close it after the VARTOFILE, at which point it would disappear. I imagine using #REQUESTER would be safe, though I don't know that for sure.

My guess is that he did not actually mean Guardian temporary file when he wrote his question, but meant just an ordinary disk file that he would purge when he was done using it. But I wanted to point out the possible limitations if he really wanted to use a Guardian temporary file.

[Doug Miller]

In article <CJadnUCk5KAdKN3R...@giganews.com>, kd...@acm.org wrote:
[...]

>The other point I was trying to make is, even if you can create a Guardian
> temporary file in TACL, I imagine you have to be careful how you use it to
> make sure that TACL opens it only once.

Not exactly right. TACL can open it multiple times, as long as some of the
opens are simultaneous. For example, the sequence (open, open, close, open,
close, close) should work just fine, but (open, close, open, close, open,
close) won't because the file will go away after the first close.

>For example, I expect you would not
> be able to use it in a VARTOFILE, then later in a FILETOVAR, since I imagine
> TACL would close it after the VARTOFILE, at which point it would disappear.

Correct.

> I imagine using #REQUESTER would be safe, though I don't know that for sure.

Yes, it would, at least until a #REQUESTER /CLOSE/ command reduces the number
of opens on the file to zero.

[nols...@gmail.com]

My apologies if I accidentally replied to anyone directly, I'm just getting used to Google Groups and couldn't quite figure out what I was doing... (N00B!)

Regardless, after reading both the TACL Reference Manual and the TACL Programming Reference, I have a question about the buffer length on #REQUESTER WRITE. Both manuals indicate that a longer write buffer is possible via waited writes, which I am using. When I try to use /wait 320/ (to increase buffer length to 320) I get an error 520 (specified record length is too large or too small)

What my routine is trying to do is a brute-force splitting of a large file (175MB) using delimiters that are at the beginning of a line. It works, but the output records are truncated at 239 bytes, as indicated in the manuals.

Any ideas how to create a file that will eventually be a text file containing string data with a record length greater than 239 bytes?

My READ/WRITE code looks like so:

#SET open_err [#REQUESTER /wait/ READ [source] read_err
read_var prompt_var]
[#IF open_err = 0 |THEN|
#OUTPUT [source] opened successfully
|ELSE|
#OUTPUT [source] not open; error [read_err]
#RETURN
]

#SET open_err [#REQUESTER /WAIT 600/ WRITE [dest] write_err
write_var]
[#IF open_err = 0 |THEN|
#OUTPUT [dest] opened successfully
|ELSE|
#OUTPUT [dest] not open; error [open_err]
SINK [#REQUESTER CLOSE read_var] == close source
#RETURN
]

Thanks 1x10^6.

[Bill Honaker]

I don't have a manual handy, but based on the 239 byte limit, it
appears you're letting TACL create the [dest] file. It will create
that as an Edit file, and lines in Edit files (code 101) can never be
longer than 239 bytes.

You should create the file first using FUP CREATE. That way you can
create an Entry sequenced file if you want structred lines or an
unstrucured file, and have better control over the file creation.

Just my $.02
Bill

[Keith Dick]

Bill Honaker wrote:
> I don't have a manual handy, but based on the 239 byte limit, it
> appears you're letting TACL create the [dest] file. It will create
> that as an Edit file, and lines in Edit files (code 101) can never be
> longer than 239 bytes.
>
> You should create the file first using FUP CREATE. That way you can
> create an Entry sequenced file if you want structred lines or an
> unstrucured file, and have better control over the file creation.
>
> Just my $.02
> Bill

This is mostly correct.

One minor mistake is that it is not quite correct that lines in Edit files cannot be more than 239 characters. Due to the way the Edit file line compression works, the actual size limit for a line depends partly on the actual data in the line. Certain patterns of interleaving spaces with non-space characters in a line can get lines of length considerably greater than 239 into an Edit file. 239 is the upper limit for the length of a line that you are guaranteed to be able to put into an Edit file, no matter what data it contains, so people usually take the shortcut of saying the absolute limit is 239. It is even possible that some programs, maybe even TACL, impose that hard limit regardless of whether the line compression would accept a particular longer line. As I said, this is a minor point. Bill is correct that this is probably what is causing nolsen's problem.

If you choose to FUP CREATE the file as entry-sequenced, you'll be able to select a record size up to a little over 4000 characters, and the system will keep track of record boundaries. I don't know the upper limit on record size that TACL will let you write, but I imagine it is considerably more than the 320 that you tried to use.

When you create the file, remember to specify appropriate extent sizes if you expect to write a lot of data to the file. This is true for either entry-sequenced or unstructured files. The manual will tell you what the default sizes are. If you aren't sure how extent sizes relate to amount of data the file will hold, ask about it and we can explain that for you.

If you choose to FUP CREATE the file as unstructured (and not code 101), in general, the system will not keep track of record boundaries, so you must arrange to do that yourself. You could make the records be fixed length, if that suits your application. If you want variable-length records, you could put the record length into the file just before each record, or you could put some special character that does not appear in the data at the end of each record. If you want to read the data with a C program, you probably could put a character containing binary 10 at the end of each line (binary 10 is the newline character that C recognizes). If you want to make the file acceptable to C, it might help to specify code 180 when you FUP CREATE it, although I don't know whether that is necessary. I haven't tried to write a C-compatible file from TACL.

[ghr]

Since you are writing a large amount of data, remember to create the
output file 'buffered' to gain large performance improvement and
reduce load on the system.

[Randall]

While technically correct that the EDIT run-length encoding (RLE)
structure theoretically permits lines longer than 239, the TACL manual
is very clear that any lines longer than 239 are truncated. Search for
239 in that manual and you'll see the impacted commands. Lines longer
than 239 will not work in TEDIT - There may have been case about that
some 20 years ago where TEDIT couldn't open files with lines longer
than 239. I do know *someone* who was able to force lines larger that
239 into an EDIT file, but that was a bug in his code and it caused
all manner of chaos in various tools (you know who you are). With that
said, EDITREAD itself does not specify a limit, but we'd have to check
the guts inside the C libraries as to whether they could process lines
that long. My suspicion is not.

[Keith Dick]

You probably are correct about TACL and TEDIT. I haven't checked, and, as I said, many people took the shortcut of using 239 is the absolute limit.

Sometime in the past few years, I actually tested the situation and easily got EDIT to create and list lines in an Edit file that were much longer than 239 characters. I don't remember the theoretical limit right now, but I think it is between 400 and 450 characters, and I believe I was able to reach the theoretical limit, using the proper pattern of blanks and nonblanks (you have to alternate 15 blanks followed by 15 non-blank characters, if I remember correctly).

So up until quite recently, the original editor on Tandem still permitted such long lines. I doubt it has changed since then.

Others may disagree with me, but I'd claim that any software that misbehaves when faced with working with lines longer than 239 in an Edit file is defective software, given that the original code supports such longer lines. At times, programmers are terribly sloppy.

[nols...@gmail.com]

Thanks for all the help! I feel like I'm making some progress...but the destination lines are still truncated.

Is there a similar limit for variables? I'm wondering if I'm getting truncation when I read-in the data to my intermediate variable.

[Doug Miller]

On 11/16/2011 1:20 PM, nols...@gmail.com wrote:
> Thanks for all the help! I feel like I'm making some progress...but the destination lines are still truncated.
>
> Is there a similar limit for variables? I'm wondering if I'm getting truncation when I read-in the data to my intermediate variable.

There is a limit to what a TACL variable can hold, but you're not likely
anywhere near it. IIRC, it's 32767 bytes, so I doubt very much that
you're getting truncation on input.

Please post the relevant portions of your code.

[wbreidbach]

On 17 Nov., 00:23, Doug Miller <doug_at_milmacdot...@example.com>
wrote:

My idea would be to use FUP instead of TACL to copy the data. So you
would have to build a FUP command.

[Keith Dick]

If I understand his original post correctly, he has to examine the data in order to determine something about where to switch to a new output file, so he probably cannot have FUP do the copy.

Or do you mean to have the TACL code examine the data, counting the records, then build a series of FUP commands that include appropriate FIRST and COUNT parameters? As long as FUP can recognize the record boundaries in the input file, that could work, though it would require multiple passes over the data -- one in the TACL, and numerous partial passes for each FIRST.

Still, I don't see why he should be having truncation problems as long as he isn't trying to put long records into Edit files, unless, of course, the records exceed the maximum length allowed for a record in an Enscribe structured file.

[wbreidbach]

> Still, I don't see why he should be having truncation problems as long as he isn't trying to put long records into Edit files, unless, of course, the records exceed the maximum length allowed for a record in an Enscribe structured file.  - Zitierten Text ausblenden -
>
> - Zitierten Text anzeigen -

In one post I read "brute force split" and my impression was that he
was just spliiting the big file into smaller parts and that case FUP
would be an easy solution.

In case of using TACL the output files have to be created using FUP
anyway, specifying an appropiate recorde size if it is an Enscribe
file. The maximum record size is 4072. If the input file is an
unstructured file the handling would be different and depending on the
file itself.

[nols...@gmail.com]

brute-force using search...
what I'm parsing is an IBM EPSIDC (sic) file that contains multiple "locations" into a file that only contains a single location (or possibly multiple, sequential locations). Sections are delimited with unique 10-char string, which is my search1 input.

so what it's doing is reading an input line, seeing if the beginning matches the search string. if so, start pouring data into a new file, but still check the beginning of each line. when search2 is encountered, stop pouring lines into the new file and terminate the routine.

I'll post code tomorrow when I'm back at work. the full routine is probably 500 lines or so.

[Keith Dick]

Probably better than posting your full routine would be to pull out the code that opens and writes to the file where the lines are getting truncated and use them to make up a small test routine. You'll probably have to add a little code to that test routine to create a line or two of data to be written. If when you run that test routine, the data lines in the output file are truncated, then we have a nice small example to examine so we can easily see what your mistake is. If that test routine does not cause the truncation, then you'll just have to compare that test to what your full routine does and see what you are doing differently in the full routine.

Have you checked to be sure that the truncation is occurring on the output and not on the input?

If there is any question about whether the data lines inside the TACL routine are the proper length, use something like

#output [#charcount X]
or
[#if [[#charcount X] < 240 ] |then| [#output short line]]

to see how many characters are in the variable named X. If X contains multiple lines, #charcount will give you the sum of all the line lengths and line separators, so only do that test when the variable X contains just a single line.

[nols...@gmail.com]

[#IF [#FILEINFO/existence/ [dest]] | THEN |
[#LOOP | WHILE | [#EMPTYV do_i] | DO |
#INPUTV do_i "[dest] already exists! Would you like to purge it?"
[#CASE [do_i]
| Y | SINK [#PURGE [dest]]
| N |
| OTHERWISE | #SET do_i
]
]
#OUTPUT CREATING FILE.....
#SET create_err [FUP CREATE [dest], buffered, type E, rec 320, ext 40, maxextents 100] ==/ EXTENT 10, FILEFORMAT 1 /==ELIMINATE IF NOT NEEDED==
#SET do_i
[#IF create_err #RETURN] ==ELIMINATE IF NOT NEEDED==
]

== Open the source and destination files and
== associate variables with them.
#SET open_err [#REQUESTER /wait 320/ READ [source] read_err

read_var prompt_var]
[#IF open_err = 0 |THEN|
#OUTPUT [source] opened successfully
|ELSE|
#OUTPUT [source] not open; error [read_err]
#RETURN
]

#SET open_err [#REQUESTER /wait/ WRITE [dest] write_err

write_var]
[#IF open_err = 0 |THEN|
#OUTPUT [dest] opened successfully
|ELSE|
#OUTPUT [dest] not open; error [open_err]
SINK [#REQUESTER CLOSE read_var] == close source
#RETURN
]

== Initiate read and write operations.
#SET read_err 0 == initialize read_err
[#LOOP |DO|
== Start the read.
#APPEND prompt_var READIT
== Wait for read_var or read_err to change.
#SET ready [#WAIT read_var read_err]
== If read_var changed, the read was successful.
[#IF [#MATCH read_var.* [Ready]] |THEN|
== Set our COMPARE_STRING to the first 10 characters of the line
#SET cmp_str [#CHARGET read_var 1 for [max_l]]
== if the COMPARE_STRING is either the first or last text then we toggle
[#IF ("[#CHARGET cmp_str 1 for [f_length]]" '=' "[first_text]") | THEN |
toggle_out
]
[#IF ("[#CHARGET cmp_str 1 for [l_length]]" '=' "[last_text]") | THEN |
toggle_out
#SET read_err -1
]
== if Toggle is active, then we write out, otherwise keep going
[#IF [is_sending_out] |THEN|
== Wait for the last write to complete
SINK [#WAIT write_var]
== Move the record into write_var to initiate the
== write operation.
==#APPEND write_var [#EXTRACT read_var]
#EXTRACTV read_var write_var
#SET lines_copied [#COMPUTE [lines_copied] + 1]
| ELSE |
#EXTRACTV read_var dump_var
]
]
|UNTIL| ([read_err])
] == end of #loop
== Wait for the last write operation to finish.
SINK [#WAIT write_var]
== Close both files and terminate the #REQUESTER
== functions.
SINK [#REQUESTER CLOSE read_var]
SINK [#REQUESTER CLOSE write_var]

[Keith Dick]

nols...@gmail.com wrote:
> [#IF [#FILEINFO/existence/ [dest]] | THEN |
> [#LOOP | WHILE | [#EMPTYV do_i] | DO |
> #INPUTV do_i "[dest] already exists! Would you like to purge it?"
> [#CASE [do_i]
> | Y | SINK [#PURGE [dest]]
> | N |
> | OTHERWISE | #SET do_i
> ]
> ]
> #OUTPUT CREATING FILE.....

It appears to me that the FUP CREATE is only executed if the file already exists when the #FILEINFO/existence/ up above is executed. So when you run this routine when the file does not exist, you do not create an entry-sequenced file, and so when you write to the file farther down, TACL creates an Edit file for you, leading to the truncation you are seeing. You can check this very easily -- after a run in which truncation occurs, check the file code of the output file and see whether it is 0 or 101. If it is 101, I'm wrong. If I am misreading the code, please post again to tell me I'm wrong and I'll look more closely.

I haven't looked at the code farther down to see whether there are any other problems.

[Keith Dick]

Keith Dick wrote:
> nols...@gmail.com wrote:
>
>> [#IF [#FILEINFO/existence/ [dest]] | THEN |
>> [#LOOP | WHILE | [#EMPTYV do_i] | DO |
>> #INPUTV do_i "[dest] already exists! Would you like to purge it?"
>> [#CASE [do_i]
>> | Y | SINK [#PURGE [dest]]
>> | N |
>> | OTHERWISE | #SET do_i
>> ]
>> ]
>> #OUTPUT CREATING FILE.....
>
>
> It appears to me that the FUP CREATE is only executed if the file
> already exists when the #FILEINFO/existence/ up above is executed. So
> when you run this routine when the file does not exist, you do not
> create an entry-sequenced file, and so when you write to the file
> farther down, TACL creates an Edit file for you, leading to the
> truncation you are seeing. You can check this very easily -- after a
> run in which truncation occurs, check the file code of the output file
> and see whether it is 0 or 101. If it is 101, I'm wrong. If I am

Oops, I meant to say if the file code is 0 when you see truncation, I'm wrong.

[nols...@gmail.com]

You are correct.

However, it doesn't seem to matter as I'm still seeing truncation in the output file. Which, to me, indicates that I am probably getting truncation on the input rather than the output.

[Keith Dick]

nols...@gmail.com wrote:
> You are correct.
>
> However, it doesn't seem to matter as I'm still seeing truncation in the output file. Which, to me, indicates that I am probably getting truncation on the input rather than the output.

So, do you want further help? If you do, my first question would be: What kind of file is the input and how does the data get placed into that file?

[nols...@gmail.com]

Yes PLEASE! :)

The input file is an ENSCRIBE file which has records inserted to it by a COBOL Batch program:

$D10.ETCD.MICROFLX 30 Nov 2011, 11:22
ENSCRIBE
TYPE E
FORMAT 1
EXT ( 32766 PAGES, 5328 PAGES )
REC 320
BLOCK 4096
MAXEXTENTS 16

I'll try to be a bit better about checking back... this has been a huge help, even if I haven't made much progress yet. Only 1 co-worker knows anything about writing/troubleshooting TACL routines, and he's incredibly busy. I've had to learn all this myself by reading the manuals and working through things at the command line.

[Keith Dick]

Have you checked the contents of that file to be certain that there are records in that file that contain more than 239 characters? If not, start by checking that.

You can look at records in an entry-sequenced file with a command such as:

FUP COPY MICROFLX,,H,FIRST x,COUNT y

Note the two commas after the first argument. This will display y records in side-by-side hex and ascii format, starting x records into the file. The header of each record display shows its length, if I remember correctly, so you won't have to count characters.

If you find that the data has not been truncated before being put into that file, next would be to take the part of your TACL routine that reads the input and make a test routine that just reads the input file and checks the length of each line by using #charcount. Perhaps have the test count the number of records the TACL code finds that are greater than 250 characters long, or something like that. If there are long lines at the start of the input file, you can make the test program stop after reading just enough lines to be sure you have reached the long lines. That will make running test quicker.

Meanwhile, I will study the code you posted yesterday and see whether I can spot anything wrong with the part that does the copying.

[Keith Dick]

I think I spotted an error in your copying code. While you do specify WAIT 320 for the #requester read, you specify only WAIT for #requester write. I think the truncation is happening because the #requester write is using only the default 239-character buffer. Change the #requester write to specify WAIT 320, and I think that will cure your problem.

[N. Olsen]

AWESOME!
Thank you, Keith!

In the process of working through your suggestion that I check for a bug in TACL that might be causing the file not to be created properly, I moved the FUP CREATE statement outside of the IF.

I also added WAIT 320 to the WRITE.

Things appear to be working smoothly at this point! SUPER AWESOME!
THANKS EVERYONE! YOU GUYS ARE a **MIRACLE** for those young'ns working on a mainframe.

[Keith Dick]

You're welcome. I'm glad we could help you out.

[jakes...@gmail.com]

I am not sure if this post is dead or not but i am having a similar problem myself.

I want to create a file write data into it from a history command i am doing and then go through that file to try filter out certain messages and display them on screen.

Any pointers would be appreciated

[rkd...@gmail.com]

When you say "a history command", do you mean you want to put the list of previous commands that TACL's history command would show into a file and then go through that file to display just certain of those prior commands, or were you not referring to TACL's history command?

[Rich...@sbcglobal.net]

On Saturday, July 25, 2020 at 6:05:24 PM UTC-7, jakes...@gmail.com wrote:

Try something like this:

#PUSH #OUT
#SET #OUT tempfile
HISTORY 100
#POP #OUT
#PUSH myhistory command
filetovar tempfile myhistory
[#loop |while|not [#emptyv myhistory]
|do|
#extractv myhistory command
== process command
]

[jakes...@gmail.com]

Sorry if i was not clear enough.

I am currently doing an inline command that will output some history information.

[#DEF approval_rate ROUTINE
|BODY|

#set #inlineprefix **
histrpt /inline/
** window 00:01
** History all,,substandard rate
#inlineeof
]

I want to filter certain results from this history output and i'm sure i cannot do this through the inline code. So i thought i would try output the results into a temp file and maybe to a loop to read the contents only displaying the data I require on screen. as some data from this is spammed in the system and is not necessary to view so i want to make it easier to read.

I cannot find any examples to work off of so i am not sure where to start.

[Randall]

Have you considered doing this through osh? It might be a whole lot easier to write a shell script for what you are trying to accomplish.

[jakes...@gmail.com]

Can't say I have heard of that before but I'm still pretty new to the job so I have no idea if that would be possible let alone if it would work/be allowed with the system.

I am only doing this as a side thing at work to try to learn so the language as to move into a better department hopefully in the future so I don't think that would even work.

[rkd...@gmail.com]

On Monday, July 27, 2020 at 12:48:57 PM UTC-7, jakes...@gmail.com wrote:
> Can't say I have heard of that before but I'm still pretty new to the job so I have no idea if that would be possible let alone if it would work/be allowed with the system.
>
> I am only doing this as a side thing at work to try to learn so the language as to move into a better department hopefully in the future so I don't think that would even work.

You still are not being very clear about what you want to do.

In the TACL routine named approval_rate that you showed, is it the output of the histrpt command that you want to capture and display selected lines from it? If not that, what is it that you want to do?

[jakes...@gmail.com]

it's exactly that basically except but I want to filter out certain messages from the output not select certain messages.

There is way too many variations to be able to select only certain outputs I want.

I am trying to make a system check we do easier as 1-2 messages always get spammed. I want to filter those messages out of the histrpt history command so only the needed data is present to allow easier reading of said data.

[Keith]

I don't know whether you can change that approval_rate routine. If you can change it, a possible approach would be to make the histrpt command be:

histrpt/inline,outv raw_output/

Where raw_output is a TACL variable you #push earlier. The output of histrpt will be put into that variable. Then you could write a TACL loop after the #inlineeof command that does something like this:

#push raw_output
[#loop |while|not [#emptyv raw_output]
|do|
#extractv raw_output this_line
== here put some TACL code to look at the contents of raw_output
== and decide whether to display it or not

]

If you make such changes in approval_rate, it would be good to add #frame at the beginning of the routine, and #unframe just before the end, so it will automatically delete the variables you create within the routine.

If you cannot change approval_rate, perhaps you could make a new routine to do the same as the current approval_rate, but with the additions to make it filter out the lines you don't want to display. Of course, any command file that still uses approval_rate would still show all of the lines.

[Dave]

It would seem to me that Rich's idea would be the simplest if you just substitute your approval_rate ROUTINE for the HISTORY 100 line and stick a #FRAME and #UNFRAME around it clean up the variables.

[Randall]

Or, if running from OSS, you could do things like this:

gtacl -c 'histrpt' | \ # sends all output to the next process
grep 'pattern-to-match' | \ # removes all lines that do not match a pattern
head -n 100 | \ # Drop all but the first 100 matching lines
tail -n 10 | \ # Drop the last 10 lines from the 100
sed '1,$s/../.../' # Apply a translation to the content

I'm not trying to solve your specific requirement here, but this sort of filtering is radically easier when putting stuff through shell scripting.

[jakes...@gmail.com]

I have tried to put the output into a variable like you said but i keep getting an error. Here is what it looks like now.

#push chk_type raw_data



#ARGUMENT / VALUE chk_type/Text

[#DEF approval_rate ROUTINE
|BODY|

#set #inlineprefix **

histrpt /inline,outv raw_data/
** window 00:01
** History all,,approval rate
#inlineeof


]

But i keep getting this error


Running...Approval Rate...
** window 00:01
^
*ERROR* You have no INLINE process
Routine Trace:
approval_rate
^

Now that you gave me an idea on how it should look i managed to find some examples from other Routines on our system but i could not see much of a difference between mine and theirs.

So i am unsure why I am getting this error it's like the program is just no longer reading the inline code anymore.

[jakes...@gmail.com]

Thanks for the Idea Randall,

It does look way easier but I've only worked in my current job for maybe 6 months and have no idea if we use OSS I would have to ask and if we didn't then there would be no way some new operator could convince them to add it if i leaned it lol. I am just trying to make one of out jobs easier in my spare time not to much hassle if it turns out to be a bust but leaning comes from failure so yea :)

[rkd...@gmail.com]

Are you sure you are not overlooking an error prior to the error about no inline process? You are correct that the error you are seeing indicates that TACL has no process running that will accept the inline data you are trying to send it. But unless mixing inline and outv does not work as one would expect, I don't understand why the histrpt process is not accepting the inline input.

If you look again and do find an earlier error message, maybe you'll see how to fix things. Or ask for help about the earlier error.

I do not see any TACL commands that would display any of the output from histrpt in what you show as the current state of the TACL code. All the output would be put into the raw_data variable, so unless you do something with raw_data, you won't see any output. That's okay to debug the changes for getting the output of histrpt into raw_data, if that is all you intended right now. Just don't be surprised by no output. If you want to display all of raw_data, before going on to add code to filter out the lines you don't want, temporarily add the line:

[#output [raw_data]]

The outer [ and ] are necessary if raw_data contains more than one line. Actually, I suggest always including square brackets to enclose the entire TACL statement so you get in the habit of doing so, and so will never forget to do it when it actually is necessary.

As far as OSS goes, you should be able to find out very easily if you have access to it. Just enter the command osh at the TACL prompt. If that gives you a OSS command prompt, you have access to OSS. Try a few simple Unix commands, such as pwd or ls -a if you aren't sure that it is working. OSS commands are close to Linux commands, if you are familiar with Linux. There are some differences, though, so be a little cautious -- check the description of a command you want to use by checking it with man to be sure what the options are.

I am not sure whether the command sequence Randall suggested would work from an OSS prompt that was started via an osh command. The unusual stuff that goes on in order to switch from Guardian to OSS, then back to Guardian (via the gtacl command), might get things twisted up somehow. But I think it is not likely to cause any serious problem, so it would be worth trying, if you feel like trying that solution.

I think Dave's suggestion might have a problem if both TACL and the histrpt program try to write to the file used for #OUT. It can't hurt to try that. But if you get an error from histrpt that says it cannot open its OUT file, don't be surprised.

[jakes...@gmail.com]

There is no prior error to this one. The only thing i added in was the outv and the variable if it look that out again and ran it I would get the output i would normally expect

#push chk_type raw_data



#ARGUMENT / VALUE chk_type/Text

[#DEF approval_rate ROUTINE
|BODY|

#set #inlineprefix **

histrpt /inline/

** window 00:01
** History all,,approval rate
#inlineeof


]

But as soon as I add that in I get that error. I thought maybe it was my formatting or something but i'm sort of glad to hard that i was right about tacl just not reading the inline code.

As i was just playing around with it I knew i would get no output that us what i was hoping for as if i got no output it was working. Then i could go away update the other sections of my code with it and then play around to see if that loop is even possible.

[jakes...@gmail.com]

Also looks like osh in my terminal does something but it basically says unable to run so looks like we may have it but it's disabled? as it says root file set is not mounted

[Dave]

It appears that OSS is not installed. Maybe the problem is histrpt? Does it do standard startup processing (i.e. not some quick and dirty tal program skipping the startup message)? You might try ENFORM instead of histrpt just to rule that out. If your TACL wasn't named or it couldn't find histrpt you should receive additional errors before the no inline process message.

[rkd...@gmail.com]

Dave,

If histrpt was not reading its startup message, the inline wouldn't work, so I believe that is not the reason.

It might be that inline and outv interact badly, but don't produce an error, but I think that is kind of unlikely.

If histrpt does not recognize the form of the process name that outv sends as the OUT file name, and just exits without writing an error about that to the home terminal, that might explain it. That could be tested using a different program, such as ENFORM, as you suggest. ENFORM wouldn't recognize the commands this TACL script is sending it via the inline, but that doesn't matter for testing whether inline and outv work together and histrpt is the problem.

I think you are correct that the error message about the root not being mounted probably means OSS is not installed, or at least not initialized. So the approach Randall suggested probably is not usable there.

[Bill Honaker]

I tried a similar TACL routine using EDIT as the program instead of your histrpt. EDIT was run with INLINE and OUTV. It works as expected.
The most likely cause is that the program is abending trying to open the OUT file.
That OPEN must be occurring before the open of IN file, or the first Read (or writeread) from the IN file.
Note that the OUT file is a PROCESS with a qualifier, something like $Z023.#Z123.

Try turning saveabend on in the HISTRPT object. If no saveabend file shows up, try adding ,DEBUG to the run parameters. Step through and find the open.

If you didn't build the HISTRPT and have the source, try setting a breakpoint on Stop or Abend. When it tries to terminate, check the stack trace and see where it is in the code.

Good luck!

Bill

[rkd...@gmail.com]

Thanks for doing that test, Bill. What you found matches what I expected, but it always is good to get confirmation of these things.

Jake: From some of the things you said earlier, I get the impression that you don't have much experience using a NonStop system, so Bill's suggestions about how to use the debugger to find the problem might be more than you can follow easily at this time. If you want to dig in and try his suggestions, go for it!

However, if you want something a little easier to try, I can suggest the following.

Immediately after the command:

#push chk_type raw_data

add this command:

create tempfile

Then, change the command to run histrpt to be this:

histrpt/inline,out tempfile/

Note, this time, the run option is out, not outv.

If you try this and do not get the error about not having an inline process (and no other errors, either), enter the command:

fileinfo tempfile

and see whether tempfile has a size greater than 0. If it does, then the histrpt program is able to use a disk file name as its OUT file, and we probably can find a way to get histrpt to write its output to a disk file in a way that you can easily retrieve the output of histrpt from that disk file and display the lines with the lines you do not want filtered out.

I am assuming that histrpt will always display some lines of output, thus in this experiment, tempfile should always have a size greater than 0. If there are conditions in which histrpt might correctly exit without displaying anything, then see whether you can modify the inline commands to get histrpt to display at least one line.

The format of the data that this experiment puts into tempfile might not be something that can easily be read by TACL. Don't let that bother you. If it does put anything into tempfile, there are a couple of further easy experiments that we can do which probably will give us a way to get the output of histrpt into a form that you can process with some TACL code.

[Tone]

Note that TACL variables have a maximum size. If the fileinfo tempfile
shows an EOF greater than 5000 (or 32K, can't remember the limit) then
the reason histrpt fails when sending output to a variable is that it
exceeded the max size.

[wbreidbach]

My idea is a completely different approach.
Start an EMS-collector and write the messages to that collector. Afterwards you can use the EMS Event Viewer to access the collector and select all the messages you want to see.

[jakes...@gmail.com]

Thank you everyone for your help.

Yes, it is true I am very new to this I graduated last year getting my Degree and this is my first real IT job.

Coding itself Is not my job but I am trying to learn so I can make my way up the leader in a few years fingers crossed>

It's my weekend now so I can't try some of these new ideas but once I get back I'll give it a go and let you know how I get on.

I have found TACL/Tadem is harder to learn from the other languages I have had experience with just because of how old it is. There are only PDF manuals to work off of and the examples I can find in my current workplace. So I know it's going to be hard to learn but I'll keep going at it till I have an understanding on how everything works.

[rkd...@gmail.com]

It is okay to be just starting out. We who are trying to help you just need to be aware of that and be careful not to assume you know more than you do, and it often is hard to tell how much that is. So if we tell you to do something and you don't understand how to do it, ask us for a more clear explanation, or ask someone you are working with there, if you know someone there who can help.

I'm not sure you are finding TACL hard to learn because it is old. I imagine it is hard for you because you have not yet learned a language that has a lot of similarity to LISP. Or maybe it isn't that, but just that you have never learned a language that did not have a proper beginner's guide to introduce you to the language.

If you haven't already done so, try reading the first couple of chapters in the TACL Reference Manual and the TACL Programming Guide. They are not really written as a beginner's guide, but reading them can give you a lot of the basic rules about the language you need to know. At least on first reading, don't try to memorize every detail, but try to pick out what seems to be important and just sort of skim over what seems less important (though try to remember enough about the stuff you skim so that if you encounter a situation in which you do need to know those details, you'll remember where it was explained so you can go back and read it more carefully then). That isn't as good as having a tutorial that presents the basics in a logical order for beginners, it might help you.

In case you did not know already, those manuals for the NonStop system are available to anyone. Go to the web page at

www.hpe.com/info/nonstop-docs

then click the link in the table of links for customer documentation for the type of system you use. I imagine you use a J-series system or an L-series system. For many products, which type of system does not matter, because many of the products are exactly the same on the different types of systems, so if you are not sure which type of system you are using, just select the J-series link. Then on the next page, type some relevant keywords in the search box at the top of the page, and it will list the manuals that match the search terms. Click on the link to read it in your web browser. You can save a copy of the PDF file of the manual to your computer so you can access it more quickly if you are going to be using it often.

There is a lot to learn about TACL, but you can do a fair amount with it after learning only a modest amount of the whole. Good luck!

[Bill Honaker]

Tone, I think that limit may have gone away a while back. Variables are certainly limited by the available virtual memory, but I can't find any limits in either manual.

There are still limits for STRUCT variables (5000 bytes) but I don't know how that would be consistent with Larger Enscribe files...

I've certainly written simple routines that used /OUTV var/, and then vartofile and written a lot more than 32K. If someone is aware of a limit here, please chime in!

Bill

[jakes...@gmail.com]

Hi All Just thought i would update you on the solution.

I think the main problem I had was that I was told I had to run Histrpt from the correct location so the cnxobj is referencing it's location i guess.

I always try to figure this stuff out by myself first before emailing/asking others around work but i guess i was way off.

I am unsure what all the extra INLINE stuff does as I tried taking them out and it still worked and the tacl manual does not understand what plain English means so i will have to ask around for a more direct answer unless someone would like to explain but i'm not to worried about it.

Now to see if I can take this variable and work it into a loop wish me luck :), unless people want to discuss that as well. this as been way more helpful than stack overflow


#set #inlineprefix **
#set #inlineecho 0
#set #inlineout 0

[cnxobj].histrpt /inline, name [:pymk:prefix]jac/
#set #inlineto raw_data
** window 00:01
** History all,,substandard approval rate

#outputv raw_data

#set #inlineto
#set #inlineecho 1
#set #inlineecho 0

#inlineeof

[rkd...@gmail.com]

You definitely need the #inlineprefix line and the #inlineeof line.

If you used "outv raw_data" in the histrpt line as we suggested here earlier, you would not need the #inlineto line.

You definitely don't need the #inlineout or #inlineecho lines.

I imagine you are having problems with this because you have not yet learned enough basics of the NonStop system to understand what goes on when a program gets started, and how things work, like where its input comes from and where it output goes to. The descriptions in the manual are not for beginners, which is why you find them not to be understandable.

You might find explanations in the TACL Programming Guide a little more helpful, but I don't remember whether they start at a beginner level. I seem to remember a manual that taught the very basics of running programs and managing files, but I don't remember what it was called. If someone else recognizes what I'm referring to and remembers its title, please mention it, then we can look to see whether it is still available online.

From one of your statements, it sounds like you were asking questions about this on StackOverflow. What was the title of your post (or titles if you made more than one)? I'd like to read what the folks there told you.

[jakes...@gmail.com]

Yea I am basically going into this completely blind and trying to figure stuff out as i go and make some good notes/examples for any future people as the company (other than the tacl manuals) have no resources to learn the language.

The code was provided by my boss so when i get a chance to talk to him i will as he works from home and is only allowed in the office on Wednesdays so getting face to face with him can be hard.

As for stack overflow it's not worth it. only 1 person left a comment and it was instructions on how to get the history of previous commands you placed into TACL and when I explained that's not what i meant he just said oh ok changed the name of the posts to suit is answer and left lol. but here is the link if you want.

https://stackoverflow.com/questions/63094140/tacl-filtering-results-from-a-history-command/63106812?noredirect=1#comment111660180_63106812

It's going to be a slow process but in a few years i can hopefully move into a different department and focus more on coding as that is what i enjoy the most.

[rkd...@gmail.com]

So where did you get the suggestions for all those changes you showed today from what you showed last week? StackOverflow wasn't it. It seems you didn't get to talk with your boss. Did those changes come from some coworkers?

We can answer specific questions here on Google Groups. But there is an old saying that to ask the right question, you have to already know most of the answer. You know you have a lot of the very basic things to learn, and Google Groups is not a very practical way to teach or learn the very basic things.

I hope someone else here remembers the title of that manual that started with the very basic things about commands and files. If we could find you that, it might let you get going a lot faster.

[Dave]

Maybe the Guardian User's Guide would be helpful, particularly starting around Chapter 3.

https://support.hpe.com/hpesc/public/docDisplay?docLocale=en_US&docId=c02141946

[rkd...@gmail.com]

That Guardian User's Guide is not the manual I had in mind, but it has some similarity to what I was remembering. After looking at it, I think Chapter 3 and 4 could be helpful for Jake, and maybe the last part of Chapter 2 where it explains the command history, and especially the FC command, if he has not yet learned that. However, that manual does not do much to describe the nature of the NonStop system -- that it has a multiprocessor, message passing design, and how that affects how TACL and the processes you start from TACL interact, the concept of the IN, OUT, and TERM files, what a process started from TACL is supposed to do to use its IN and OUT files, and how TACL makes use of that to implement features such as INV, OUTV, and INLINE.

I'm wondering now whether what I'm remembering was from one of the classes the education folks taught. As I recall, those classes usually had very good student workbooks, and maybe what I am remembering is descriptions in a workbook for one of the introductory classes for internal developers (maybe there were equivalent courses for customers, too, that left out some of the deeper internal details). If such a class still exists, it would not be an option for Jake, unless one of his coworkers attended such a class and would let Jake read the workbook his coworker received when taking the class.

This situation is a little frustrating because I am pretty sure that if I were talking with Jake face to face, I could explain the things about the NonStop system that he needs to know to understand how to solve this problem in just a few short sessions, but doing the same via email or posts here is much more difficult because the face to face interaction is missing.

[MK]

On Tuesday, July 13, 2010 at 7:34:42 PM UTC-5, lyates wrote:
> I need to create a temp file and write to it and then read from it
> within a TACL routine.
> Any ideas?

Here is the example to create and write to a validation file "val_file". Hope this helps.

[#PUSH
#INLINEPREFIX
val_file
summary_var
]

======= Create the new validation file (file size 0) ========
[#if [#purge [val_file]]]
fup create [val_file], type u, code 101, buffered, ext 100, &
maxextents 100, oddunstr

#SETV summary_var "Adding some text to this file"
VARTOFILE summary_var [val_file]