small posit applications

[Theodore Omtzigt]

I had misread the posit spec and thought that the lower bound on posit configurations was a posit<3,0>. John pointed out that there are two more small posits, a posit<2,0> and a posit<3,1>. I just implemented and released those posits in the lib.

Does anyone have a good application/algorithmic use case of these small posits, beside as extreme valid bounds, that they can share?

Here are the value tables of the smallest posits:

Generate Posit Lookup table for a POSIT<2,0>

   #           Binary         Decoded       k    sign   scale          regime        exponent        fraction                         value

   0:               00              00      -1       1      -1               0               ~               ~                             0

   1:               01              01       0       1       0               1               ~               ~                             1

   2:               10              10       1      -1       1               0               ~               ~                           NaR

   3:               11              11       0      -1       0               1               ~               ~                            -1

Generate Posit Lookup table for a POSIT<3,0>

   #           Binary         Decoded       k    sign   scale          regime        exponent        fraction                         value

   0:              000             000      -2       1      -2               00               ~               -                             0

   1:              001             001      -1       1      -1               01               ~               -                           0.5

   2:              010             010       0       1       0               10               ~               -                             1

   3:              011             011       1       1       1               11               ~               -                             2

   4:              100             100       2      -1       2               00               ~               -                           NaR

   5:              101             111       1      -1       1               11               ~               -                            -2

   6:              110             110       0      -1       0               10               ~               -                            -1

   7:              111             101      -1      -1      -1               01               ~               -                          -0.5

Generate Posit Lookup table for a POSIT<3,1>

   #           Binary         Decoded       k    sign   scale          regime        exponent        fraction                         value

   0:              000             000      -2       1      -4               00               -               ~                             0

   1:              001             001      -1       1      -2               01               -               ~                          0.25

   2:              010             010       0       1       0               10               -               ~                             1

   3:              011             011       1       1       2               11               -               ~                             4

   4:              100             100       2      -1       4               00               -               ~                           NaR

   5:              101             111       1      -1       2               11               -               ~                            -4

   6:              110             110       0      -1       0               10               -               ~                            -1

   7:              111             101      -1      -1      -2               01               -               ~                         -0.25

[Shin Yee Chung]

Hi Theo,
I believe it is not just the two posit<2,0> and posit<3,1> mentioned. Last checked it is also not supporting large es values, such as when es + 2 >= nbits. I just tried posit<4,2> the compilation will hang for quite a bit.

I think a reference should try to match the design as much as possible, nbits >= 2, es >= 0. The practical use is up to the creativity of the users.

Otherwise, the document should also include the cases that are not supported for certain technical reasons such as efficiency or that C++ bitset template could not instantiate -ve size array.

Regards,
ShinYee

[John L. Gustafson]

There is no mathematical reason es cannot be any integer 0 or greater. I does not need to "fit" into the nbits. In the case of nbits = 3, while there are no es bits in the number, it changes the meaning of useed, which affects four of the eight possible posit meanings.

While common sense says you almost never should need an es value of 4 or greater, I can imagine a situation where you start an algorithm in posit<3,3>, get an initial guess or a crude convergence, then append five 0 bits to every value to get posit<8,3> representations (no conversion needed), then posit<16,3>, posit<32,3> and posit<64,3>, all done simply by padding on the right with 0 bits, a trivial operation. 

John

--
You received this message because you are subscribed to the Google Groups "Unum Computing" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unum-computin...@googlegroups.com.
To post to this group, send email to unum-co...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/unum-computing/34808fea-d2d6-4fa9-a52a-7554b0b47a95%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

---

Important: This email is confidential and may be privileged. If you are not the intended recipient, please delete it and notify us immediately; you should not copy or use it for any purpose, nor disclose its contents to any other person. Thank you.

[Theodore Omtzigt]

This is super-interesting for the generic programming interpretation of the posit library. Right now, the library implementation did make some design decisions that assume that 'es' bits are allocated. Nothing that can't be changed, so the following is a series of questions about the requirements that flow from this interpretation.

1- given the interpretation of es as the specifier of useed, it would be possible to articulate posits such as posit<3,3> ,posit<3,4>, posit<3,5> etc. Otherwise stated, nbits and es are independent, and the the state space of nbits > 0, and es >= 0. Is this independence between nbits and es helpful to the adoption of the posit number system?

2- posits with the same es value have a certain 'refinement' quality to them, as you point out, by padding 0's. This can be supported in software by conversion operators. These conversions would need to be explicit as I don't see yet how an implicit conversion of temporaries can be made consistent with 'computing' in a given precision. If somebody has insights how this algebra could work, I would love to explore. Mixing posit sizes will open a gnarly conversion rule debate, do we want to start this debate now?

3- if we are able to mix posit sizes during arithmetic operations, there will be a bifurcation in execution models for fixed function ASICs, and field programmable FPGAs. In the latter, we can support mixed size posits directly in the data path, but for ASICs (CPUs and GPUs) we would need some additional execution support which will negatively impact the efficiency of these compute engines. Is this efficiency impact beneficial or detrimental to the adoption of posits?

[Job van der Zwan]

On Saturday, 10 March 2018 11:19:28 UTC+1, Theodore Omtzigt wrote:

If we are able to mix posit sizes during arithmetic operations, there will be a bifurcation in execution models for fixed function ASICs, and field programmable FPGAs. In the latter, we can support mixed size posits directly in the data path, but for ASICs (CPUs and GPUs) we would need some additional execution support which will negatively impact the efficiency of these compute engines. Is this efficiency impact beneficial or detrimental to the adoption of posits?

Are you saying that the slowdown will come from runtime detection of posit size for every operation. I may slightly confused by what you mean, but aren't you then essentially asking whether to allow posit size to be a runtime configuarble thing? Because otherwise, assuming known size at compile time, shouldn't the code turn into the appropriate mixed-size or same-size version at compile time, implying no slow-down for same-sized posits? I mean, we don't need runtime detection to see whether we multiply a float with an integer either. I might be misunderstanding what you are saying, though.

[Theodore Omtzigt]

Take the problem of arbitrary precision arithmetic. You could build hardware for arbitrary precision arithmetic to accelerate important applications in geometry or number theory. That hw however would be less efficient on simple 32-bit float math than a dedicated 32-bit arithmetic data path with add/sub/mul/div/sqrt. 

A fixed posit configuration hardware data path can make optimizations, such as fixed size registers for exponent and fraction.

A variable posit configuration hardware data path on a baked technology ASIC would need to add some circuitry to keep track of the size of exponent and fraction registers. I use the term registers here in the way we hw folks use registers: that is, banks of flip-flops.

But a variable posit configuration hardware data path on a retargetable technology FPGA could still avoid the variable register control circuitry as the synthesis step transforms the computational graph into logic gates.

To use your terminology: posits will be runtime configurable in sw emulation and in retargetable hardware technologies. Posits can still be runtime configurable for baked ASIC technologies but there will be overhead to be paid in terms of design complexity and power efficiency.

So the question I posed is simply informed by the observation: if we want posits to replace floats, we don't need runtime configurability to provide parity.  Adding to the posit arithmetic specification that posit configurations can change size as in John's example of posit<3,3> expanding to bigger nbits is very cool, but has significant ramifications to hardware that need to support that arithmetic.

[Job van der Zwan]

Thank you for a very clear explanation, I understand what you mean now. Sadly I have nothing useful to contribute to the discusion, but I'm interested in following where it goes.

[Shin Yee Chung]

Hi Theo,

On Saturday, 10 March 2018 18:19:28 UTC+8, Theodore Omtzigt wrote:

This is super-interesting for the generic programming interpretation of the posit library. Right now, the library implementation did make some design decisions that assume that 'es' bits are allocated. Nothing that can't be changed, so the following is a series of questions about the requirements that flow from this interpretation.

1- given the interpretation of es as the specifier of useed, it would be possible to articulate posits such as posit<3,3> ,posit<3,4>, posit<3,5> etc. Otherwise stated, nbits and es are independent, and the the state space of nbits > 0, and es >= 0. Is this independence between nbits and es helpful to the adoption of the posit number system?

Firstly, it's nbits >= 2 & es >= 0. They are independent by design or specification.

Before one adopts, one will probably try out some reference posit implementations.

The correctness, robustness and completeness reflect the integrity of the reference implementation and the design,

especially if the implementation is endorsed by the designer.

2- posits with the same es value have a certain 'refinement' quality to them, as you point out, by padding 0's. This can be supported in software by conversion operators. These conversions would need to be explicit as I don't see yet how an implicit conversion of temporaries can be made consistent with 'computing' in a given precision. If somebody has insights how this algebra could work, I would love to explore. Mixing posit sizes will open a gnarly conversion rule debate, do we want to start this debate now?

Mixing posit sizes can be explicit, or by coercion rules (to be determined).

For posits of the same es value, a conversion between different nbits is either appending zeros or applying rounding.

Conversions of posits between different es values need not be difficult, since a larger useed value is always a multiple of a smaller useed value.

Applications do use 32-bit floats to compute, and accumulate into 64-bit floats.

What about 16-bit posits to compute, accumulate into quire, and finally store into 32-bit posits?

Or skipping the quire step?

Isn't that a requirement for posits to be drop-in replacement for floats for diversified applications?

 

3- if we are able to mix posit sizes during arithmetic operations, there will be a bifurcation in execution models for fixed function ASICs, and field programmable FPGAs. In the latter, we can support mixed size posits directly in the data path, but for ASICs (CPUs and GPUs) we would need some additional execution support which will negatively impact the efficiency of these compute engines. Is this efficiency impact beneficial or detrimental to the adoption of posits?

The baseline: mixed posits of different sizes can be pre-converted to a common size, similar to single-precision float converted to double-precision float before arithmetic operations.

Regards,

ShinYee

[John Gustafson]

Interesting discussing about supporting various es sizes in hardware for a given nbits size. I have some comments:

1) I'm in the process of building standard math library functions for various <nbits,es> combinations, starting with <16,1> Since posit math libraries must be correctly rounded for every input value, considerable care is needed. It will take a while to create optimized math libraries for a range of nbits-es combinations. I don't think it always works to compute at a higher precision and round down to the working precision, because that creates (rare) instances of the classic double-rounding error, making the result will be incorrect in the last bit for a few rare cases. That throws out the bitwise reproducibility of posit calculations, hence it's not compliant.

2) If you find yourself wishing you had a higher precision for just part of a program, consider using the quire instead. It is far more powerful than merely doubling the working precision for part of a calculation; it can refine calculations to 0.5 ULP and guarantee correctness, usually with no more than three times the number of operations of using the working precision. As many floating point users can attest, doubling or even quadrupling precision is no guarantee of improved accuracy... even when it seems to produce the same answer at all precisions. The quire provides a guarantee, a measure of numerical error without the need to resort to interval arithmetic.

3) In a deep learning application, it is clear that each layer of the network is best served by a different accuracy and dynamic range, and it's not that difficult to figure out what those accuracies and dynamic ranges are based on the input data. So in designing special-purpose AI chips, for both training and inference, the flexibility of nbits and es could be enormously helpful. Bill Dally (Chief Scientist at Nvidia) recently admitted that the blanket use of 16-bit IEEE floats for neural networks is not really what is needed, and that a more savvy approach would be to right-size the format for each layer in the network.

4) In software, if you have worked hard to create, say, a library hard-coded to <32,3>, then it is _very_ easy to modify it to support sizes <nbits,3> where 2 ≤ nbits ≤ 32. You simply need to change the location of the binary point where rounding occurs when normalizing the result, and also change the definitions of maxpos and minpos, but none of the data structures need to be changed in any way. It's a superset. It should not affect the number of clock cycles to do an operation. Of course, it would be highly inefficient to use it that way if all you ever needed was <8,3>, but it would work and give you something you can experiment with right away.

John

--
You received this message because you are subscribed to the Google Groups "Unum Computing" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unum-computin...@googlegroups.com.
To post to this group, send email to unum-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/unum-computing/d87e92ee-f34f-4808-9ff4-4a5367ff2712%40googlegroups.com.

[theo]

Shin Yee/John:

I am looking at how to implement the interpretation that nbits and es are independent. In the parameterizations of the arithmetic operators, the size of the data paths are a function of the size of the fraction. The largest fraction occurs at 1.0 and would be of size nbits - sign bit - minimum regime bits - exponent bits. Otherwise stated,  how nbits and es interrelate impacts the number of fraction bits. From this relationship I deduce that for the fraction interpretation nbits and es become coupled and the assertion that there are 'es' bits embedded in the representation is not inconsistent. 

I am trying to reconcile spec parameters with derived parameters that are necessary to define the data path. The number of fraction bits are derived quantity but a requirement nonetheless for the data path specification and this derived quantity becomes non-linear when you can't embed an es number of bits in the containing nbits at 1.0.

What is the right vocabulary to capture that constraint?

[Shin Yee Chung]

Hi Theo,

Max number of fraction bits = max(0, nbits - 1 - 2 - es), where -1 is for the sign bit, and -2 is for the minimum regime bits.

Regardless of es value, the max number of fraction bits is consistent.

Can't comment on how to uniformly handle fractions with 0 bit size in hardware.

Regards,

ShinYee

[John L. Gustafson]

Similarly, es represents the _maximum_ number of exponent bits. Some or all of the exponent bits can be crowded off the end of the posit by the regime bits, and there is nothing wrong with that, just like there is nothing wrong with crowding off the fraction bits (since the hidden bit is always 1 whether there are any fraction bits or not).

There may be practical applications for number systems with no fraction bits, by the way. You get back the perfect reciprocation of a Type II unum, and multiplication and division are exact so long as the result stays between minpos and maxpos in magnitude. If you want to capture, say, the intensity of sound or light, and you have a very limited number of bits, then it's much more important to have regime and exponent bits than fraction bits.

John

-- 
You received this message because you are subscribed to the Google Groups "Unum Computing" group.
To unsubscribe from this group and stop receiving emails from it, send an email to unum-computin...@googlegroups.com.
To post to this group, send email to unum-co...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/unum-computing/7e9348e1-e480-40a6-99fa-b75a01ed712c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.