On Mar 15, 9:07 am, glen herrmannsfeldt <
g...@ugcs.caltech.edu> wrote:
> Thomas David Rivers <
riv...@dignus.com> wrote:
>
> > In this example:
> > SMALL: PROC;
> > DCL I FIXED BIN(31,127);
> > DCL D FIXED DECIMAL(15,100);
> > I = D;
> > END SMALL;
> > we have a conversion of a FIXED DECIMAL with a precision of 15 and a
> > scale of 100, to a FIXED BINARY. The FIXED BINARY has the maximum
> > precision & scale allowed..
> > FIrst question - the IBM Language Reference manual indicates
> > that result (I'm assuming it means the intermediate result) is
> > a FIXED BINARY(p2,q2) with these attributes:
> > The precision of the result is p2 = min(N,1+CEIL(p1*3.32)) and
> > q2=CEIL(ABS(q1*3.32))*SIGN(q1).
>
> I thought those were used when the rest of the attributes weren't
> known.
Look at "Source-to-target rules" for data conversions,
in which those ARE the quoted formulas.
The attributes ARE known (look at the declarations).
> > (I guess 'N' is defined somewhere - but I can't seem to find it..
> > so I take it to mean the maximum precision allowed by a
> > FIXED BINARY, which would be 63???
>
> Used to be 31, but is probably 63 by now.
Look in a current manual for the limits.
> > I'm not sure that matters as p2 would be 54 here... I sure would
> > like to know the definition of 'N'.)
>
> Yes it is the maximum allowed, but I don't know if that is,
> by now, 31 or 63 or even more.
Look in a current manual for the limits.
> > But, if you follow that rule - does it mean that the scale of the
> > intermediate result is 332? 332 is much larger than the
> > maximum scale (which is 127.) So - how
> > is that handled? (Just an internal matter.. or?)
>
> As well as I know it, when the destination precision and
> scale are known, then there is no need for the table.
David R. quoted the relevant formulas above.
You can find them in the manual.
> > And - our result scale is 127 - so presumably the intermediate value
> > would have to be left-shifted 205 bits... which, I imagine, could cause
> > an OVERFLOW condition (if the result had any 1 bits at all) and could
> > only be satisfied without the possibility of a condition if D were 0.
>
> Easy for the compiler, just test for zero, and either signal
> FIXEDOVERFLOW or assign zero. (Unless FIXEDOVERFLOW is disabled.)
SIZE is enabled, and is raised in situations when FOFL is raised.
> > Lastly - IBM provides a table of the values for calculating
> > CEIL(n*3.32), but the table only seems to go up to 33.
> > What about the other possible values?
>
> It should just be the actual value of CEIL(n*3.32).
??? That's what the table would give.
> I seem to remember at one time that when the value would have
> given extended precision,
??? There was no such thing as "extended precision" for fixed-point.
> but just barely, there was a table
> that rounded down just enough.
Why not look in a manual to find out what actually happens?
instead of guessing.
> CEIL(3.32*33) is CEIL(109.56) or 110. When extended precision
> has '1' for the leftmost hex digit it has 109 significant bits.
What? Floating-point now? That's irrelevant.
> CEIL(16*3.32) is CEIL(53.12), just barely more than double
> precision with a '1' leftmost hex digit. The table I remember
> gives 53 instead of 54 for that case.
That's irrelevant. The operands are fixed-point.