Unums "type 3" talk with live demonstration @ Stanford. February 1st 2017

[Thomas Sohmers]

https://www.youtube.com/watch?v=aP0Y1uAA-2Y&feature=youtu.be

Abstract on the Stanford website, as well as link to slides: http://web.stanford.edu/class/ee380/Abstracts/170201.html

First 50 minutes or so is an overview on unums and an introduction of "type 3 unums" that have two subtypes: Posits (discussed here), and Valids. In short, Posits have a greater mathematical accuracy AND dynamic range of floats, while in many cases also using fewer bits in the representation.

At around the 53 minute mark, the presentation switches over to Dr. Isaac Yonemoto who gives a live demonstration of a Julia implementation of Posits, showing it preforming favorably for both a FFT and neural network compared to low precision floats.

[William Tanksley, Jr]

I was just working on a fixed-width representation of the Stern-Brocot Tree -- and this pretty much solves how to do that (using a unary 'regime' counter for higher levels). Nice!

This also suggests a representation for arbitrary-width exact-precision rationals, with the obvious graceful degradation at a maximum width.

-Wm

--
You received this message because you are subscribed to the Google Groups "Unum Computing" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unum-computin...@googlegroups.com.
To post to this group, send email to unum-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/unum-computing/31a1220f-27b8-422f-a39c-c4532c399d69%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Job van der Zwan]

Even though I'm not in the line of work where I'll be able to use any of this until it has become mainstream computing, it's always fun to see progress on this topic! When can we expect a talk on valids?

Also, has the Posit Julia package been announced on https://discourse.julialang.org/ yet? It looks like it's much easier to just drop into any algorithm out there and compare them with traditional floats, compared to the previous unum packages, so that might make a lot of people more interested in playing with it and figuring out what the numerical benefits would be.

[Skip Cave]

John Gustafson's first example in his unums 3.0 video intended to show the failure of IEEE floating point is a matrix multiplication a x b. This was tested in the matrix language J, which uses standard hardware floating point. https://goo.gl/Ub548W It seems to provide the correct answer, contrary to what Gustafson predicts. ]a=.3.2e7 1 _1 8.0e7 NB. Assign a 32000000 1 _1 80000000 b=. 4.0e7 1 _1 _1.6e7 NB. Assign b a +/ . * b NB. Matrix multiply 2 JVERSION Engine: j804/j64/darwin Release: commercial/2015-12-21 18:06:25 Library: 8.04.15 Platform: Darwin 64 Also the same experiment was run using the array language APL https://goo.gl/txkkpP That experiment also got the correct answer. Why do these experiments seem to contradict John's claim that this test fails using standard floating point?

[johngustafson]

The language is not using standard hardware floating point. J has provisions for extended precision, especially with numbers that happen to be integers as in my example. Try writing out the dot product as multiplies and adds, from left to right, and I think you will get a very different answer from J. If you enter the problem in Mathematica, expressing the numbers as integers, it will also compute the correct answer 2; but if you put a decimal point at the end of every integer, then Mathematica uses "machine precision" (IEEE 754 64-bit binary floats) and makes the error.

3.2e7 * 4.0e7 = 1.28e15 (exact)

add 1*1, get 1.28e15 again, but round-to-nearest rules

add (–1)*(–1), get 1.28e15 again,

The product 8.0e7 * (–1.6e7) = –1.28e15 exactly, so adding that brings the sum to zero.

APL and J are very savvy about linear algebra accuracy, so I suspect they both may use the exact accumulator idea to prevent such problems. Exact accumulators are part of every unum-type environment.

[duda....@gmail.com]

Binary representation of 1.28e15 uses 51 bits, ieee754 double have 53 bit precision, so this example is calculated correctly.

[johngustafson]

Thank you for catching a transcription error. The example should be 32e7, not 3.2e7, and –16e7, no –1.6e7. Then the rounding behaves as described. I will correct my slide set!

[Skip Cave]

Actually, it looks like you need to increase all four constants by two orders of magnitude to cause a fail (at least in J):

   ]a=. 3.2e7 1 _1 8.0e7

32000000 1 _1 80000000

b=. 4.0e7 1 _1 _1.6e7

a +/ . * b

2

]a=. 3.2e8 1 _1 8.0e8

320000000 1 _1 800000000

b=. 4.0e8 1 _1 _1.6e8

a +/ . * b

2

   ]a=. 32e9 1 _1 8.0e9

32000000000 1 _1 8000000000

b=. 4.0e9 1 _1 _16e9

a +/ . * b

0

Fail.

Skip

[no_...@msn.com]

On Friday, February 10, 2017 at 7:59:18 PM UTC-5, johngustafson wrote:

Thank you for catching a transcription error. The example should be 32e7, not 3.2e7, and –16e7, no –1.6e7. Then the rounding behaves as described. I will correct my slide set!

For whatever it may be worth, I threw together a quick program in C:

#include<stdio.h>

int main()
{
    long double a[4] = {3.2e8, 1, -1,  8.0e8};
    long double b[4] = {4.0e8, 1, -1, -1.6e8};
    float fsum = 0;
    double dbsum = 0;
    long double ldbsum = 0;

    fsum = (float)a[0] * (float)b[0] +
          (float)a[1] * (float)b[1] +
          (float)a[2] * (float)b[2] +
          (float)a[3] * (float)b[3];

    dbsum = (double)a[0] * (double)b[0] +
           (double)a[1] * (double)b[1] +
           (double)a[2] * (double)b[2] +
           (double)a[3] * (double)b[3];

    ldbsum = a[0] * b[0] +
            a[1] * b[1] +
            a[2] * b[2] +
            a[3] * b[3];

    printf("fsum = %f\n", fsum);
    printf("dbsum = %f\n", (float)dbsum);
    printf("ldbsum = %f\n", (float)ldbsum);

    return 0;
}

Now full disclosure, this is version two. The first version used "e7" instead of "e8" for the arrays. Version one gave zero for fsum and two for the higher precisions. Version two gave zero for both the lower precisions and the correct answer (two) for the highest.

(I look forward to being told that all those casts were horribly excessive, and other comments on my coding. Also, the values are cast to floats for output because I didn't care to look-up the printf codes, and floats were enough for what the results were supposed to be.)

[no_...@msn.com]

OK, I've been trying to understand a few things on my own from the slides, but I just give up. Perhaps I just don't "get" it, or I just don't have the education, but I have two questions:

Firstly, I don't get how the regime bits work. It seems that they serve as a unary count of the exponent bits. Seems.

I understand floats, I just don't yet understand posits. And I find them interesting, so this is really starting to bug me.

Secondly-and this may be part of the first-is that it isn't clear if every posit has to have the exponent size declared explicitly, or if it varies naturally and only the overall size (8, 16, 32, etc.) needs to be declared. The chart on page/slide 40 ("Accuracy on a 32-Bit Budget") versus page/slide 19 ("nbits=5") makes this less than clear.

[johngustafson]

The regime bits and exponent bits are independent. If you look at any binary number, after the initial sign bit, it will have some number of identical bits at the start. Maybe just one bit. Or perhaps five zeros in a row, then a one. Those are the regime bits, terminated by the opposite bit. The next es bits are the exponent bits, and to match the dynamic range of floats, I'm recommending this table:

Size      es value

  16             1

  32             3

  64             4

128             7

256           10

I'll try to attach a table that has more detail, here.

[no_...@msn.com]

Firstly, thank you for the reply. Unfortunately, your table is above my comprehension level, but it looks like a except from something, so hopefully no work wasted there.

On Friday, February 17, 2017 at 10:57:39 PM UTC-5, johngustafson wrote:

The regime bits and exponent bits are independent.

I was afraid of that.
 

If you look at any binary number, after the initial sign bit, it will have some number of identical bits at the start. Maybe just one bit. Or perhaps five zeros in a row, then a one. Those are the regime bits, terminated by the opposite bit.

This was clear from the examples given in the slides. (Except for cases where the exponent and fractions disappeared and the regime took the whole size, but those cases were the saturation points and just before them.)
 

The next es bits are the exponent bits, and to match the dynamic range of floats, I'm recommending this table:

Size      es value

  16             1

  32             3

  64             4

128             7

256           10

So, you're saying that, unless the es values are standardized, they must be declared explicitly along with the number of bytes the variable takes up...

I am not the professor. I am not the educated one. I only have a HS Diploma and probably have no business even trying to speak on an equal plane with you...

However, unless standardized, this strikes me as a very, very big hurdle to what posits seem (to me) to be trying to be: a simple, no-thought-needed, "good enough" real number data type. (And also a hurdle to being a simple drop-in for floats, too.)

[John L. Gustafson]

My intention is to standardize on these es values for the reasons you say. The idea is to create a true standard, with bit-identical, reproducible results across systems. If you want to just do a drop-in replacement for IEEE floats, the table of es values should do the job.

Analogy: LEDs become bright enough to replace light bulbs. So companies create bulbs with compatible sockets that happen to contain those LEDs, even though LEDs can be packaged in a much wider variety of ways. Now people have a way to replace a 60-watt incandescent with an equally bright 10-watt LED bulb, or use the 60 watts to create a MUCH brighter light. And for those who want it, LEDs are also available in forms that look nothing like an old-fashioned light bulb. I think that's the way to view posits.

If someone builds, say, a custom chip for Deep Learning, they might want to use a 10-bit posit with an es value of zero. You need not be restricted to standard bit widths and dynamic ranges for such things.

--
You received this message because you are subscribed to the Google Groups "Unum Computing" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unum-computin...@googlegroups.com.
To post to this group, send email to unum-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/unum-computing/d031053a-af33-4347-815d-f08e403cbf00%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

---

Important: This email is confidential and may be privileged. If you are not the intended recipient, please delete it and notify us immediately; you should not copy or use it for any purpose, nor disclose its contents to any other person. Thank you.

[no_...@msn.com]

Firstly, thank you once again for the reply...

On Friday, February 17, 2017 at 11:31:29 PM UTC-5, John L. Gustafson wrote:

My intention is to standardize on these es values for the reasons you say. The idea is to create a true standard, with bit-identical, reproducible results across systems. If you want to just do a drop-in replacement for IEEE floats, the table of es values should do the job.

[...]

If someone builds, say, a custom chip for Deep Learning, they might want to use a 10-bit posit with an es value of zero. You need not be restricted to standard bit widths and dynamic ranges for such things.

...However, I originally had two questions, and while this more than satisfactorily answered the second, the first is left.

And that was how the regime bits are interpreted. (This was probably outside of the lecture's scope, so I doubt it was meant to be conveyed from just there.) If the regime bits are not a unary count of exponent bits, then I'm at a loss as to interpret them. All I can grasp is that values in (-1,1) are zeros terminated by one (except zero's all zeros all the time) and values in (-inf., -1] and [1, +inf.) are ones terminated by a zero (except +/-maxposit).

Also outside the lecture's scope, what is the sixteen-bit value 0x8000 ("negative zero") supposed to be? Quiet NaN and sign-less infinity are the first two things to come to mind.

[William Tanksley, Jr]

The regime is a unary count, yes; but it's assymetric so that zero is denoted by 10, but -1 is denoted by 01.

Consider a 4-bit unum with es=1. Then r=4 (that's the regime power).

Here's some of the positive unums; the negatives are simply the two's compliment of the positives. I always take take the two's complement when I see a negative number.

0000 = 0

001 = 4^-2

010 = 4^-1 * 2^0

011 = 4^-1 * 2^1

100 = 4^0 * 2^0 * (1)

101 = 4^0 * 2^0 * (1+ 1/2)

110 = 4^-1 * 2*1

111 = 4^-3

1000 = infinity

The unique part -- the part that's not like IEEE floats -- is the regime bits. They allow the unum to span much more (so that a C double can hold only a es=5 unum32). Understand those, and you can parse the rest out easily. The fraction part turns out to be dead simple in comparison.

-Wm

--

You received this message because you are subscribed to the Google Groups "Unum Computing" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unum-computin...@googlegroups.com.
To post to this group, send email to unum-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/unum-computing/58c1e705-3e7b-499c-86e0-bd443ab7b111%40googlegroups.com.

[no_...@msn.com]

First things first for context: Mr. Gustafson sent out a message that hasn't shown up on the Google Groups list yet. Hopefully it'll just have been taking a while.

Anyway, thank you once again for the reply. The attachment did indeed answer my questions to my satisfaction (and then some). And of course I have two more queries:

First, is it actually a good idea to implement negative posits in two's complement? I can only see that adding complication.

Secondly, I must admit that I've grown more... unhappy(?) with that existence of exceptions as time goes on and I've heard about people not using them well. As such, I'm not quite sure exceptions causing things to stop is a good idea. I'm not sure what it should be instead, however; perhaps a look at what Mill Computing has with their None values might be a good idea. (It's part of the metadata talk.)

Or perhaps I'm jumping at shadows in that last bit. Good luck.

[johngustafson]

Computers already support two's complement signed integers. By mapping the real numbers to such a representation and eliminating exception values like NaN, I actually make things much simpler than having an IEEE-style sign bit. Consider the need to test

x == y

with real values. With integers, it's trivial; either all bits are identical, or they aren't. But with IEEE floats, you have to make two exceptions and therefore have a different machine instruction for testing equality. First, if x is ±0 and y is ±0, then they are "equal" even though their bit patterns differ. Second, if x is a NaN or y is a NaN, the result is "not equal" even if the bit patterns are identical! What a mess. 

The testing of 

x < y, x ≤ y, x > y, x ≥ y

are similarly identical to the test for two's complement integers. No need to invoke an FPU adder by subtracting and checking the sign of the result, which is the way many processors do conditionals on floats.

When you say you are unhappy with exceptions, I believe you are expressing a favorable opinion of posits, correct? Because posits reduce the number of exceptions to just two: 0 and ±∞. No denormals (gradual underflow), no NaN values, no "negative zero," no invisible flags to set in the processor. I very deliberately cut the exception cases down to the minimum conceivable set.

[no_...@msn.com]

Once again, thank you for the reply. I'll cut the bits about two's complement and leave the bits to the experts and focus on the second part. (Also, the post with the 6MB attachment still hasn't shown up on Google Groups. If I had unsubscribed to the thread like I prefer to do, I wouldn't have even seen it.)

Unfortunately, my lack of education seems to be making this difficult for me, since I only have a gut feeling and neither the familiarity with the problem spaces nor numerical analysis to either quell the felling or express it to my satisfaction. Apologies in advance.

On Saturday, February 18, 2017 at 8:54:50 AM UTC-5, johngustafson wrote:

When you say you are unhappy with exceptions, I believe you are expressing a favorable opinion of posits, correct?

My concern is that, upon encountering a NaN situation, the computation should just stop. I would assume this would translate into a thrown exception, something I have not heard the greatest of things about.

The case I'm imagining is... a video game. It's raining and the player looks up for whatever reason. By chance, they look perfectly up, making something zero. Then, later in the calculations, that gets divided by zero and, since the authors decided to use these new-mangled posit thingies, an unexpected exception gets thrown and the program just crashes.

I'm trying to put a gut instinct issue into words and I still don't know if I'm just jumping at shadows or if this is something to genuinely consider.

[Job van der Zwan]

On Sunday, 19 February 2017 04:40:21 UTC+1, no_...@msn.com wrote:

The case I'm imagining is... a video game. It's raining and the player looks up for whatever reason. By chance, they look perfectly up, making something zero. Then, later in the calculations, that gets divided by zero and, since the authors decided to use these new-mangled posit thingies, an unexpected exception gets thrown and the program just crashes.

 

A divide by zero would result in a NaN right now, for which there would also be explicit code to capture it. If not, the NaN would propagate through the simulations and do all kinds of weird stuff too.

To reframe the question: can you think of any place where quietly receiving a NaN would not be a bug now, where a divide by zero-error would be? After all, you don't hear people complain about integer divide by zero either.

[Job van der Zwan]

Also, I was under the impression that divide by zero when using posits results in infinity, but that doesn't really answer your question of how to handle NaNs.

[johngustafson]

For posits, x/0 returns ±∞ unless x is zero, in which case it halts, that is, it signals a NaN. What happens next is up to the operator, but it makes no sense to continue computing. If this can occur in an application, it is an indication that the application is not yet debugged, since an exception like that is obviously preventable by putting an IF test in the code just before the divide, to protect against an unwanted interrupt.

Video games are VERY thoroughly tested, though I have had the experience of walking Turok between two rocks such that he could not get out, and I had to reset the game!

During development of an application, indeterminate forms might be produced like 0/0, ±∞ + ±∞, 0 * ±∞, and there also might be questions of accuracy; this is when you use valid representation, pairs of posits that indicate a start and end point on the projective reals. These work like Type 1 or Type 2 unums, with an uncertainty bit after the last significant bit of the fraction, and the width of the range allows detection of loss of information since valids keep track of their own accuracy. Valids can handle NaN situations without a hiccup, since they can represent the empty set (frequently a good substitute for having a quiet NaN) or the entire extended real number line (which is what 0/0, ±∞ + ±∞, 0 * ±∞ are, regarded as limits).

So if you find yourself needing to compute with NaN, it probably means you are still developing the app and should be using valids, Once there is no possibility of exception conditions, you can switch to posits and run at full speed.

[Asbjørn Heid]

On Saturday, February 18, 2017 at 3:15:44 AM UTC+1, no_...@msn.com wrote:

Firstly, I don't get how the regime bits work. It seems that they serve as a unary count of the exponent bits. Seems.

I tried to reverse engineer the regime bits from the talk. 

The way I understood it is that they provide a second, variable scale factor which is (2^es)^r, where r is the number of regime bits (possibly negated if inverse encoding is used and ignoring the stop bit), and es is the number of exponent bits.

Is this a correct interpretation?

Regards

- Asbjørn

[William Tanksley, Jr]

Almost correct - the bits '10' are encoded as 0, '110' as 1, and so on; but '01' is -1, '001' is -2, and so on. Zero introduces a necessary asymmetry.

--

You received this message because you are subscribed to the Google Groups "Unum Computing" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unum-computin...@googlegroups.com.
To post to this group, send email to unum-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/unum-computing/dbf3c88a-6a13-4a5b-a745-39afe46b910a%40googlegroups.com.

[Asbjørn Heid]

On Monday, February 20, 2017 at 5:30:51 PM UTC+1, William Tanksley, Jr wrote:

Almost correct - the bits '10' are encoded as 0, '110' as 1, and so on; but '01' is -1, '001' is -2, and so on. Zero introduces a necessary asymmetry. 

 

As soon as I posted it I thought about how zero was handled, and got a bit confused. Thank you for the clarification :)

Regards

- Asbjørn 

[John Gustafson]

Almost. It's (2^2^es)^r. The (2^2^es) is the "useed" and it always appears at the 45 degree point on the projective real ring. Te useed can be 2, 4, 16, 256, 65536,... Just keep squaring it to get the desired dynamic range. Or, to match the range provided by IEEE floats, which is what I recommend for a hardware implementation. I'm not suggesting that it be flexible at run time, though I wish it were; hardware engineers convince me that's too expensive, so instead we can follow in the footsteps of the IEEE floats but get the dynamic range (sometimes much more dynamic range) with MORE fraction bits left to express the value. With exceptions for very large and very small values where IEEE floats have more detail, posits can express any number that an IEEE float can, but also have fraction bits to spare.

To view this discussion on the web visit https://groups.google.com/d/msgid/unum-computing/CAFTBfO7V%3D%3DT2dpngSinLnxbtvYvM9Eu1AbcD4dyNEG%2B72Y0jLQ%40mail.gmail.com.

[William Tanksley, Jr]

"Just keep squaring it to get the desired dynamic range. Or, to match the range provided by IEEE floats, which is what I recommend for a hardware implementation."

As you add another exponent bit, you square useed.

The closest way to compare the range of a Posit to IEEE is to count the bits of exponent -- but that's not _just_ the 'es', but also the number of bits required in order to represent the regime. The IEEE exponent = (2^es*regime+exp), so you need to know how many bits represent the regime. For a 32-bit Posit, the regime can be between -32 and +31, so the regime needs 5 bits of the IEEE exponent. That leaves the IEEE binary32 roughly comparable to a Posit32(es=3), and IEEE binary64 comparable exactly to a Posit32(es=6) -- the latter should be exactly convertible, although I admit I haven't tried yet, and the former should be an inexact conversion because the binary32 doesn't have enough fraction bits.

That's all theory, though... I need to test my theories. It's based on the fact that the IEEE binary32 has 8 bits of exponent, and the IEEE binary64 has 11 bits of exponent. I'm pretty much ignoring the fraction; obviously binary64 always has more than we need, and binary32 almost always has less.

So it might work -- at least for a proof of concept -- to implement all the primitive operations of Posit32 as double, and just convert back and forth. This obviously wouldn't be as good as a native PPU (Posit Processing Unit) with its internal scratchpad, but it'd be fast and I suspect accurate (but again, I need to test that).

The bigger IEEE formats set aside less and less exponent relative to total size -- it goes from 8 bits, to 11, and then 15 (that using the utterly impractical and hardware-unsupported binary128 format). So in the higher bitsizes can absolutely drown them in fraction bits in the center AND beat them in range on the sides -- although of course without hardware support we're not much better than binary128, save for memory write speed.

To view this discussion on the web visit https://groups.google.com/d/msgid/unum-computing/F828CF6C-0FB7-4A1A-BCEC-C66DDBA8CD0D%40earthlink.net.

[Asbjørn Heid]

On Monday, February 20, 2017 at 7:35:29 PM UTC+1, johngustafson wrote:

Almost. It's (2^2^es)^r. 

Doh, that's what I meant to write.

I really enjoyed the presentation by the way. 

However one thing which I keep nagging me is losing signed infinities... I know some methods rely on them, for example this fast ray-AABB intersection test[1]. 

Though I guess keeping signed infinities will mess up the nice symmetrical bit-patterns?

As for raising an exception on NaN... a user of the ray-tracer I worked on complained that after rendering for a very long time, suddenly his image got all "messed up". The image got these weird "blooms" of bright colors spreading.

Long story short, a developer had tried to optimize for speed by rewriting "a/b * c/d * e/f" as "(a*c*e)/(b*d*f)", however in some cases the terms could be very tiny, causing both products a*c*e and b*d*f to become denormalized, and thus the single division returned NaN... the NaN then propagated into our Monte-Carlo integrator which messed up just about every calculation after that. 

Instead, I found that by how the math worked out, the terms a/b, c/d and e/f would always each be about two or three orders of magnitude at most, thus the original expression would always be stable.

Had the division instead raised an exception, instead of a silent NaN, the user would only had lost the data since the last checkpoint (~30 minutes), he could have rendered something else while we fixed the issue, and then he could have resumed his rendering... Instead he lost several days of rendering time.

[1]: https://tavianator.com/fast-branchless-raybounding-box-intersections/

Regards

- Asbjørn

[Isaac Yonemoto]

I've been working on some hardware implementations - and I have made some very important discoveries.  The biggest one is that having a two's complement negative means that your add/subtract bifurcates into only two conditions instead of four.  You can imagine how already being in two's complement simplifies addition/subtraction, but that it might muck up multiplication.  As it turns out, multiplication is rescued with a very slight tweak.  Still need to think about division a bit, however, that is not currently a priority, my intuition given my understanding of the division algorithms, is that it also will *not* be a problem.

[William Tanksley, Jr]

Asbjørn Heid <asbjor...@gmail.com> wrote:

However one thing which I keep nagging me is losing signed infinities... I know some methods rely on them, for example this fast ray-AABB intersection test[1]. 

That's a good point. One fundamental problem here is that the min/max operations cannot be defined on our +/- inf.

Another is that we treat 0 and infinity very differently; calculations can obviously round to 0, but they cannot round to infinity. This would probably be solved using a ubit.

There may be something more subtle, as well -- see the link to "part 2" on that same page, discussing dealing with rays that lie precisely on the bounding boxes.

I suspect one resolution to this problem in general would be to use Valids instead of Posits. That would seem natural for raytracing anyhow.... Although of course I don't know exactly what Valids actually are.

[no_...@msn.com]

I looked at my messages this mourning and saw this. Then I proceeded to not answer in favor of some time to think. (Good thing, too. I feel like I have a better understanding of where I'm coming from with this now.)

Naturally, I come back to eight more messages, one of which suggests that I'm completely wrong. My thought below, anyway:

On Monday, February 20, 2017 at 6:20:09 AM UTC-5, johngustafson wrote:

For posits, x/0 returns ±∞ unless x is zero, in which case it halts, that is, it signals a NaN. What happens next is up to the operator, but it makes no sense to continue computing. If this can occur in an application, it is an indication that the application is not yet debugged, since an exception like that is obviously preventable by putting an IF test in the code just before the divide, to protect against an unwanted interrupt.

Video games are VERY thoroughly tested, though I have had the experience of walking Turok between two rocks such that he could not get out, and I had to reset the game!

I have to ask how often you've seen speed-runs if you think games a so throughly tested. :P
 

During development of an application, indeterminate forms might be produced like 0/0, ±∞ + ±∞, 0 * ±∞, and there also might be questions of accuracy; this is when you use valid representation, pairs of posits that indicate a start and end point on the projective reals. These work like Type 1 or Type 2 unums, with an uncertainty bit after the last significant bit of the fraction, and the width of the range allows detection of loss of information since valids keep track of their own accuracy. Valids can handle NaN situations without a hiccup, since they can represent the empty set (frequently a good substitute for having a quiet NaN) or the entire extended real number line (which is what 0/0, ±∞ + ±∞, 0 * ±∞ are, regarded as limits).

So if you find yourself needing to compute with NaN, it probably means you are still developing the app and should be using valids, Once there is no possibility of exception conditions, you can switch to posits and run at full speed.

Moving on to seriousness, I've given this a good deal of thought. Especially that last bit.

My first impression was that I'm holding posits to be far more idiot-proof than you seem to. My example was video game graphics; why would you ever need Valids there!?

(Not quite joking: how much could get away with just 32-bit fixed-point and no one be the wiser?)

Thinking on that some more, I think I'm comparing Posits to how (IIRC-I don't feel like checking) the RISC-V ISA spec says to handle integer division exceptions: it always gives a default value and never throws exceptions. If you want exceptions, you need to check and throw them yourself. As such, I find simply stopping to be... what?

Thinking yet more, particularly on that last line, I wonder if we're looking at two different problems. The idea that you default to Valids and only switch to Posits after checking is baffling to me, but it would follow for... "high math" would probably suffice. I feel like I'm coming from far, far below: "Fixed-point won't cut it, what next on the complexity ladder?"

(A later post on lost ray tracing work suggests that I may be grossly mistaken. My issue was a gut feeling, anyway.)

On a more whimsical note, I started wondering if the exponent bits were actually needed. I don't quite know what that would do, but the 32-bit example makes me wonder. Also, I think it might give total reciprocal closure. (I look forward to being told that I'm completely wrong on that.) Still, I assume you considered that and discarded it for practical reasons.

(You did consider it, right? Because I will feel very silly if you hadn't.)

[no_...@msn.com]

On Monday, February 20, 2017 at 5:23:27 PM UTC-5, Isaac Yonemoto wrote:

I've been working on some hardware implementations - and I have made some very important discoveries.  The biggest one is that having a two's complement negative means that your add/subtract bifurcates into only two conditions instead of four.  You can imagine how already being in two's complement simplifies addition/subtraction, but that it might muck up multiplication.  As it turns out, multiplication is rescued with a very slight tweak.  Still need to think about division a bit, however, that is not currently a priority, my intuition given my understanding of the division algorithms, is that it also will *not* be a problem.

So, I feel like this is a good time to point out that both that slides form the lecture and the message that still hasn't shown up on Google Groups (the one with the 6MB attachment) show signed magnitude. That may be for audience benefit, but not noting that gives the wrong impression.

[John L. Gustafson]

I hope it makes sense to still answer a message this many weeks old. I have been focused on completing the posits paper, which again is too many megabytes to post here but with any luck will appear in Supercomputing Frontiers and Innovations (http://superfri.org/superfri) in a few months. Write to me if you would like to see the draft (dcs...@nus.edu.sg).

On a more whimsical note, I started wondering if the exponent bits were actually needed. I don't quite know what that would do, but the 32-bit example makes me wonder. Also, I think it might give total reciprocal closure. (I look forward to being told that I'm completely wrong on that.) Still, I assume you considered that and discarded it for practical reasons.

(You did consider it, right? Because I will feel very silly if you hadn't.)

Mathematically, the number of exponent bits can be zero, or any integer greater than zero. I recommend no exponent bits for 8-bit posits, 1 exponent bit for 16-bit posits (to exceed the dynamic range of IEEE half-precision floats), 3 exponent bits for 32-bit posits (same reason).

Total reciprocal closure fails unless the argument is 0, ±∞, or an integer power of 2, no matter what the exponent size is. Consider the point between 1 and 2; of course it is 3/2. With posits, the point between 1/2 and 1 is 3/4, not 2/3 as it would be for a Type II unum. The maximum product of x times its reflection (about the horizontal axis of the projective real circle) is 9/8, which is not too bad as an initial guess for the reciprocal, which can then be refined with Newton-Raphson or other algorithms.

[William Tanksley, Jr]

We're always waiting for your publications -- hope this one will be posted publicly accessible!

I'm playing around with a model in which 8-bit with 5-bit exponents are used to rough out a solution space, and then a higher-bitted version is used to polish the solutions. I suspect I'll have to wait for Valids to be explained before I can really get anywhere with this, though. I'm running into all kinds of complexities, and I think I'm missing something I need to understand Valids.

-Wm

--

You received this message because you are subscribed to the Google Groups "Unum Computing" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unum-computin...@googlegroups.com.
To post to this group, send email to unum-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/unum-computing/5999CAF5-55B5-4DEE-A24D-E87646E55BDA%40nus.edu.sg.

[Skip Cave]

John, 

To share large docs publicly, upload your paper as a PDF to Google Drive or Dropbox, depending which service you use. Then make the document public, and share the link on this forum. Then anyone with the link can download the doc.

Skip Cave

[no_...@msn.com]

On Thursday, April 13, 2017 at 6:49:50 AM UTC-4, John L. Gustafson wrote:

I hope it makes sense to still answer a message this many weeks old.

I have no issue. However I have another question, one befitting my level of proper education (i.e. an HS Diploma).

Why have projective infinity? Indeed, since values saturate to +/-maxposit, I don't see a need for an infinity of any type.

It isn't needed for intervals: simply exchanging the usual order of elements is all that is needed. (The only exceptions are the cases of the sets {+inf, -inf} and {projective_infinity}. I would hope those are rare enough cases not to matter, especially with saturation to non-infinities.)

All it seems (to me) to be doing is allowing you to have all bit patterns filled and making division by zero a defined operation.

The obvious other use for the bit pattern is a signaling NaN: it's never generated and there's a fault when it's an input. Is that special case a burden on a hardware implementation such that projective infinity is a better option? (There's not enough room for both positive and negative infinity, so there's only projective infinity or something else to fill the void.)

[William Tanksley, Jr]

It's symmetric to skip the infinity by also skipping zero. More likely to work with Valids than Posits, though.

--

You received this message because you are subscribed to the Google Groups "Unum Computing" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unum-computin...@googlegroups.com.
To post to this group, send email to unum-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/unum-computing/c98cc569-e4b7-4c93-a85c-d156c81595d5%40googlegroups.com.