ENFORM timestamps vs Tandem timestamps - I can offer advice, but I'm also asking for knowledge
Greg Shaw
I've recently been working through ENFORM and DDL, while trying to
make useful queries for work and for myself about various databases
at my workplace. One of them is the various and wide set of schemas
provided by ACI's BASE24 product. While investigating those, I have
come up against a serious brick wall about the ENFORM TIMESTAMP-DATE
and TIMESTAMP-TIME clauses, yet also worked out some useful oddities.
I have worked out that the starting point of the Julian timestamp in
ENFORM is half a day late from the Tandem's Julian timestamp's
beginning (or perhaps the other way around - work it out for
yourselves, but it's there..), back in 4713 BCE. So that's what I'm
offering to the community.
What I would be very grateful to learn is how ENFORM, when I use it,
appears to take Tandem timestamps, from 00:00 1974/12/31, and treat
them as I have seen. Let me explain, so you know that I haven't just
been playing....
In old-school BASE24, as I've seen in the DDL, there is a field called
LAST-FM.FM-TIMESTAMP, declared as three consecutive 16-bit binary
fields. Which should make one 48-bit (three-word) Tandem timestamp. So
far, so weird (why treat 48 bits as three 16-bit fields, eh?), and I
have proved that it is so - take each field, regard it as a two's
complement number (i.e., if negative when seen through an ENFORM
query, add 32768), then multiply the middle field by 2^16 and the top
field by 2^16*2^16 , add them all up - and you get a Tandem timestamp.
No question about it - if I give the resulting number to a TACL
#CONTIME, I get the exact time and date that I would if I were to view
the record via ACI's GOAFT PATHWAY method.
Now that's all very well and true, and I am proud of myself for having
worked that one out, blah blah, but I just don't get how ENFORM's
TIMESTAMP-DATE and TIMESTAMP-TIME functions treat these numbers.
ENFORM manuals claim that they provide one with human-friendly times
and dates from Tandem timestamps, but from my experience it is just
not true. ENFORM timestamps do start at 1974/12/31 00:00 - I have
tried asking ENFORM TIMESTAMP-DATE and TIMESTAMP-TIME to work on the
number 0 - but what clock do they work with after that? I substituted
Tandem timestamps from my previous calculations into ENFORM's
functions and they came up as some time early in 1974/12/31. I've
tried fudge-factors of various powers of two to multiply my timestamps
first, and it still makes no sense.
Perhaps my ENFORM query is wrongly-phrased. I don't know - but what
clock do ENFORM's TIMESTAMP-DATE and TIMESTAMP-TIME work on? Anyone
know?
Keith Dick
Well, I'm afraid I don't completely understand what you are trying to say in your post. It probably would have helped had you shown us your DDL and your ENFORM query.
The original system time for Guardian was a 48-bit value that counts the time from 00:00 12/31/1974 in your local civil time in units of .01 second. This is what the Guardian proc TIMESTAMP returns. Sometime later, a second system time was developed that is a 64-bit value that counts the time from noon 1/1/4713 BC GMT in microseconds. This is what the Guardian proc JULIANTIMESTAMP returns.
There is a simple linear formula that allows you to convert between the two forms of timestamp, except for taking Daylight Savings Time into account. It is possible to convert taking Daylight Savings Time into account, but it isn't a simple formula.
The simple formula has the form:
<64-bit timestamp> = <48-bit timestamp>*10000 + <big number> + <time-zone offset>
<time-zone offset> is the number of microseconds your local time differs from GMT. It usually is 3600000000*<hours from GMT>.
<big number> is the number of microseconds between noon 1/1/4713 BC and 00:00 12/31/1974. It is documented a number of places in the Tandem manuals and I always have to go look it up. I don't have it memorized. I believe it is the result of COMPUTETIMESTAMP for the date-time 1974/12/31 00:00:00.
A discussion a few months ago in this newsgroup or in the Yahoo mailing list Tandem_Computers (I don't remember which it was) had the formula when a discussion of these matters came up.
Enform's TIMESTAMP-DATE and TIMESTAMP-TIME work on the 48-bit timestamp. The Enform JULIAN-DATE function has nothing to do with the Guardian JULIANTIMESTAMP proc or the 64-bit timestamp.
I'll make a guess that you put the 48-bit timestamp value into a 64-bit integer field, right-justified. That isn't what TIMESTAMP-DATE and TIMESTAMP-TIME expect. The manual says you should put the TIMESTAMP value into a 6-character field, such as one declared as TYPE CHARACTER 6. In fact, I believe it also works if you put the TIMESTAMP value into a 64-bit integer field, but first shift the 48-bit value left by 16 bits. I think the important thing is that the address of the beginning of the field you pass to TIMESTAMP-DATE or TIMESTAMP-TIME be the first word of the 48-bit timestamp value.
If the above doesn't answer your questions, please post again and try to make your question more clear, including your code if you have some that isn't working as you think it should.
Greg Shaw
"I'll make a guess that you put the 48-bit timestamp value into a 64-
bit integer field, right-justified. That isn't what TIMESTAMP-DATE
and TIMESTAMP-TIME expect. The manual says you should put the
TIMESTAMP value into a 6-character field, such as one declared as TYPE
CHARACTER 6. In fact, I believe it also works if you put the
TIMESTAMP value into a 64-bit integer field, but first shift the 48-
bit value left by 16 bits. I think the important thing is that the
address of the beginning of the field you pass to TIMESTAMP-DATE or
TIMESTAMP-TIME be the first word of the 48-bit timestamp value."
You may be right, and I will check up on it. I thought it might have
had someting to do with the justification of the characters.....but
how? Now, the ENFORM Manuals are pretty awful when accessed from HP's
website, and most HP Manuals are useless unless you already understand
the subject matter. I used the Adobe searching tools for all examples
of "timestamp" and "date" and "time". Are you saying that if I'd tried
searching for "justif" then I would have found the answer? By then I
was getting unbelievably hacked off with crappy manuals. Why in
heaven's name would I put a 48-bit timestamp into a 6-character field?
How would I do that? Arbitrarily decide to chop each field in half and
assign it a value? Why, faced with a genuinely understandable
numerical 48-bit Tandem timestamp wrenched with much pain and thought
from these ridiculous three supposedly independent contiguous 16-bit
fields, would I stick it in a PIC X(6)? Where was that gem in the
manual?
This entire episode has taught me well about the origins of the Julian
microsecond timestamp, and although it's arbitrary, it's just as valid
as the Tandem centisecond one. I'll post my successful ENFORM query
based on ACI's Julian timestamp (poor, conflicted bastards to use
different timestamps for different files....) as soon as I get to
work. :-)
Tchah, you have given me something to think about, but you have not
reinforced my liking for Tandems. Mind you, that doesn't need to be
reinforced.
Doug Miller
>There is a simple linear formula that allows you to convert between the two
> forms of timestamp, except for taking Daylight Savings Time into account. It
> is possible to convert taking Daylight Savings Time into account, but it isn't
> a simple formula.
>
>The simple formula has the form:
>
> <64-bit timestamp> = <48-bit timestamp>*10000 + <big number> + <time-zone
> offset>
>
><time-zone offset> is the number of microseconds your local time differs from
> GMT. It usually is 3600000000*<hours from GMT>.
>
><big number> is the number of microseconds between noon 1/1/4713 BC and 00:00
> 12/31/1974. It is documented a number of places in the Tandem manuals and I
> always have to go look it up. I don't have it memorized. I believe it is the
> result of COMPUTETIMESTAMP for the date-time 1974/12/31 00:00:00.
It is. The value is 211024440000000000.
Below is some TACL code I wrote -- and documented -- lo these many years ago
to convert from 64-bit time to 48-bit time. Doing the reverse is a simple
exercise in basic algebra.
==============================================================================
==
?SECTION timestamp_offset TEXT
== Used in converting 64-bit timestamps to 48-bit timestamps, this number
== is the 64-bit timestamp of the base of the 48-bit timestamp system
== (00:00:00 31 Dec 1974).
==
== This number is the result of [#COMPUTETIMESTAMP 1974 12 31 00 00 00 000
000]
==
== To convert a 64-bit timestamp to a 48-bit timestamp, follow these steps:
== 1. If the timestamp is not in local civil time (LCT) already, convert
== it from GMT to LCT with [#CONVERTTIMESTAMP]
== 2. Subtract this number (giving microseconds since 00:00:00 31 Dec 1974)
== 3. Divide the result by 10000 (to convert to the 10-msec units used in
== 48-bit timestamps).
211024440000000000
==============================================================================
==
?SECTION convert_64bit_to_48bit ROUTINE
#FRAME
#PUSH GMT64, LCT64, LCT48
[SINK [#ARGUMENT /VALUE GMT64/ NUMBER
/MINIMUM 208657771200000000 == 1 Jan 1900 00:00:00
,MAXIMUM 214969204800000000/ == 1 Jan 2100 00:00:00
]
]
[#CASE [#ARGUMENT KEYWORD /WORDLIST GMT/ KEYWORD /WORDLIST LCT/]
| 1 | #SETMANY _ LCT64, [#CONVERTTIMESTAMP [GMT64] 0]
| 2 | #SETV LCT64 GMT64
] == end Case
#SET LCT48 [#COMPUTE ([LCT64] - [timestamp_offset]) / 10000]
#RESULT [LCT48]
#UNFRAME
==============================================================================
==
Doug Miller
>Now, the ENFORM Manuals are pretty awful when accessed from HP's
>website, and most HP Manuals are useless unless you already understand
>the subject matter. I used the Adobe searching tools for all examples
>of "timestamp" and "date" and "time".
Evidently not as carefully as you believed. :-)
> Are you saying that if I'd tried
>searching for "justif" then I would have found the answer? By then I
>was getting unbelievably hacked off with crappy manuals.
What manual were you searching?
>Why in
>heaven's name would I put a 48-bit timestamp into a 6-character field?
Because -- since there is no native 48-bit data type -- that's the *only way
possible* to reference a 48-bit field in Enform or DDL.
>How would I do that? Arbitrarily decide to chop each field in half and
>assign it a value?
Define thus in DDL:
01 FORTY-EIGHT-BIT-TIMESTAMP.
05 WORD-1 Type Binary 16.
05 WORD-2 Type Binary 16.
05 WORD-3 Type Binary 16.
>Why, faced with a genuinely understandable
>numerical 48-bit Tandem timestamp wrenched with much pain and thought
>from these ridiculous three supposedly independent contiguous 16-bit
>fields, would I stick it in a PIC X(6)?
How *else* do you propose referencing a 48-bit field? How would you define
this "genuinely understandable numerical 48-bit Tandem timestamp" in TAL or C?
> Where was that gem in the manual?
Enform Reference Manual, page 5-67, under the heading "TIMESTAMP-DATE Clause",
and again on the following page, under "TIMESTAMP-TIME Clause".
>
>This entire episode has taught me well about the origins of the Julian
>microsecond timestamp, and although it's arbitrary,
Actually, it's not entirely arbitrary. See
http://en.wikipedia.org/wiki/Julian_Period#History
for explanation of how it came to be defined thus.
> it's just as valid
>as the Tandem centisecond one.
More so, IMHO, since the basis of the centisecond timestamp system *is*
entirely arbitrary.
Keith Dick
> the subject matter. I used the Adobe searching tools for all examples
> of "timestamp" and "date" and "time". Are you saying that if I'd tried
> searching for "justif" then I would have found the answer?
> was getting unbelievably hacked off with crappy manuals. Why in
> heaven's name would I put a 48-bit timestamp into a 6-character field?
> How would I do that? Arbitrarily decide to chop each field in half and
> assign it a value? Why, faced with a genuinely understandable
> numerical 48-bit Tandem timestamp wrenched with much pain and thought
> from these ridiculous three supposedly independent contiguous 16-bit
> fields, would I stick it in a PIC X(6)? Where was that gem in the
> manual?
The reason it did not work when you put the 48-bit timestamp into the 64-bit binary field is that the TIMESTAMP-DATE and TIMESTAMP-TIME functions in ENFORM expect their parameter to point to the start of a 48-bit timestamp. When you put the 48-bit timestamp into a 64-bit binary field and pass the name of the 64-bit field to TIMESTAMP-DATE, the address passed to TIMESTAMP-DATE is not the start of a 48-bit timestamp -- it is two bytes to the left of the start of the 48-bit timestamp value (remember, the Tandem system is a big-endian system). So if you shift the 48-bit value left 16 bits before storing it in the 64-bit field, then the 48-bit timestamp will start at the beginning of the 64-bit field, and TIMESTAMP-DATE will get the result you expect. I'm not saying that that is the best way to store the 48-bit timestamp. It isn't -- it wastes two bytes and there is, as far as I know, no reason that you need to store the 48-bit timestamp in a numeric field. The most efficien
t way to store it in your database is to define a 6-character field in the database record and move the 6 bytes of the 48-bit timestamp value into that 6-character field, however your favorite programming language best lets you do so.
In your first post about this, you seemed to be rather befuddled about why the timestamp was defined as three 16-bit fields. It is an historical accident. The biggest standard binary field on the Tandem system was a 32-bit integer. The hardware to support 64-bit integers was optional, so could not be relied on for a standard system function. 32 bits wasn't big enough to hold a timestamp with the desired resolution. The next step up was three 16-bit fields, so that's what they chose to use for the timestamp. Since the main purpose was just representing a point in time, not doing arithmetic to compute on timestamps, it didn't matter that it wasn't easy to use a timestamp in an arithmetic expression. The reason that it was defined as three 16-bit values rather than six characters probably was that word addresses could be used to address all of the 64K word data segment in the early Tandem systems, while character addresses could be used only to address the lower half of
the 64K word data segment. So you might wonder why ENFORM says to put the timestamp in a six-character field rather than in an array of three 16-bit integers. I don't know why that was done, but my guess is that it was because when you put three 16-bit fields into a group item, that group item is considered a character field.
>
> This entire episode has taught me well about the origins of the Julian
> microsecond timestamp, and although it's arbitrary, it's just as valid
> as the Tandem centisecond one. I'll post my successful ENFORM query
> based on ACI's Julian timestamp (poor, conflicted bastards to use
> different timestamps for different files....) as soon as I get to
> work. :-)
Suit yourself. There isn't much point posting your code once you have it working. If you are having trouble getting something to work, show us the code you are having trouble with and tell us what you hoped you were doing with that code. Then we probably can quickly point out what point you are misunderstanding and how to fix the code to make it do what you want to do.
>
> Tchah, you have given me something to think about, but you have not
> reinforced my liking for Tandems. Mind you, that doesn't need to be
> reinforced.
>
The universe doesn't care much whether you like the Tandem system or not. But if you want to get useful work done using a Tandem system, either you have to learn how it works, the same way you learned whatever other systems you have mastered, or you should hire someone to do the work who does understand how the Tandem system works.
If you want to learn how the Tandem system works, ask here when you have trouble figuring out something and you will find several of us here who will be happy to explain the points you are having trouble with. For the quickest, most accurate answer, it is best to include as much detail as practical -- often including the relevant parts of the code that isn't working the way you think it ought to work will let us get to the core of the problem quickly. Sometimes when you don't understand the problem, you might not know how much detail is needed or what all of the relevant parts of the code are. In that case, we might have to ask for more information or to see other parts of code that we know are relevant, or the partial information might be enough to let us guess what the problem probably is.
Greg Shaw
information usually follows.
I apologise for my earlier antagonistic attitude. I'm sorry. I was in
a seriously bad mood, as I have a broken collar bone and I am not the
most friendly when in pain. But that doesn't excuse me getting all
bolshy.
Next time any of you fancy a fight, well just post here and I will try
to provide any knowledge I have to soothe. Thanks for all the
recommendations. I will read them.
Greg Shaw
I wrote it. ACI may have something to say about the file names. This
whole thread has been about how I can interpret LAST-AFM.FM.AFM-TS but
not LAST-FM, so this posting is late in the discussion. This ENFORM
script was written to be self-explanatory, so uses more memory than
might be needed. Any ways of interpreting LAST-FM are welcome.
?DICTIONARY <your dictionary>;
DECLARE MICROS-PER-DAY INTERNAL I11;
SET MICROS-PER-DAY TO 86400000000;
DECLARE JTIMESTAMP INTERNAL I18;
DECLARE JDAY INTERNAL I7;
DECLARE YEAR-MICROS INTERNAL I18;
DECLARE DAY-MICROS INTERNAL I18;
DECLARE DAY-SECS INTERNAL I5;
OPEN ATDD1;
LIST
TERM
AS A8
HEADING "TERM",
" ",
(IF ATDD1-CORE.TERM-TYP = "22"
THEN "NCR"
ELSE "unk"
)
HEADING "MAKE",
" ",
ATDD1-DEV-INFO-NCR-5XXX.ENCRYPT-PIN.ENCRYPT-KEYS.M-KEY
AS A8
HEADING "MKEY",
" ",
! Add half a day, because Tandem/BASE24 (and worldwide) Julian
timestamps !
! begin at 12:00, but ENFORM starts counting at
00:00 !
JTIMESTAMP := (LAST-AFM.FM.AFM-TS + (MICROS-PER-DAY/2) )
NOPRINT,
JDAY := (JTIMESTAMP / MICROS-PER-DAY)
AS DATE "Y4/M2/D2"
HEADING "DATE",
" ",
YEAR-MICROS := (JDAY * MICROS-PER-DAY)
NOPRINT,
DAY-MICROS := (JTIMESTAMP - YEAR-MICROS)
NOPRINT,
DAY-SECS := (DAY-MICROS / 1000000)
AS TIME "H2:M2:S2"
HEADING "TIME",
" ",
! Nested IF statement for update type - ENFORM has no CASE/switch
clause. !
! See ACI's DDL schemas for the derivation of the
values. !
(IF LAST-AFM.FM.UPDT-TYP = 2
THEN "CHG"
ELSE
(IF LAST-AFM.FM.UPDT-TYP = 0
THEN "ADD"
ELSE
(IF LAST-AFM.FM.UPDT-TYP = 1
THEN "OLD"
ELSE "DEL"
)
)
)
HEADING "UPD"
" ",
LAST-AFM.FM.USER-ID.GRP-NUM
AS I3
HEADING "GRP",
" ",
LAST-AFM.FM.USER-ID.USER-NUM
AS I3
HEADING "USR",
WHERE
TERM <> ATDD1-DEV-INFO-NCR-5XXX.ENCRYPT-PIN.ENCRYPT-KEYS.M-KEY
;
CLOSE ATDD1;