Re: [PATCH] Add optional __Bfloat16 support

[Hongtao Liu]

On Fri, Jun 10, 2022 at 3:47 PM liuhongt via Libc-alpha
<libc-...@sourceware.org> wrote:
>
> Pass and return __Bfloat16 values in XMM registers.
>
> Background:
> __Bfloat16 (BF16) is a new floating-point format that can accelerate machine learning (deep learning training, in particular) algorithms.
> It's first introduced by Intel AVX-512 extension called AVX-512_BF16. __Bfloat16 has 8 bits of exponent and 7 bits of mantissa and it's different from _Float16.
>
> Movivation:
> Currently __bfloat16 is a typedef of short, which creates a problem where the compiler does not raise any alarms if it is used to add, subtract, multiply or divide, but the result of the calculation is actually meaningless.
> To solve this problem, a real scalar type __Bfloat16 needs to be introduced. It is mainly used for intrinsics, not available for C standard operators. __Bfloat16 will also be used for movement like passing parameter, load and store, vector initialization, vector shuffle, and .etc. It creates a need for a corresponding psABI.
>
> ---
> x86-64-ABI/low-level-sys-info.tex | 10 ++++++++--
> 1 file changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/x86-64-ABI/low-level-sys-info.tex b/x86-64-ABI/low-level-sys-info.tex
> index a8b69db..ba8db0d 100644
> --- a/x86-64-ABI/low-level-sys-info.tex
> +++ b/x86-64-ABI/low-level-sys-info.tex
> @@ -302,6 +302,12 @@ be used to represent the type, is a family of integer types.
> This permits the use of these types in allocated arrays using the common
> sizeof(Array)/sizeof(ElementType) pattern.
>
> +\subsubsection{Special Types}
> +
> +The \code{__Bfloat16} type uses a 8-bit exponent and 7-bit mantissa.
> +It is used for \code{BF16} related intrinsics, it cannot be
> +used with standard C operators.
> +
> \subsubsection{Aggregates and Unions}
>
> Structures and unions assume the alignment of their most strictly
> @@ -563,8 +569,8 @@ The basic types are assigned their natural classes:
> \item Arguments of types (signed and unsigned) \code{_Bool}, \code{char},
> \code{short}, \code{int}, \code{long}, \code{long long}, and
> pointers are in the INTEGER class.
> -\item Arguments of types \code{_Float16}, \code{float}, \code{double},
> - \code{_Decimal32},
> +\item Arguments of types \code{_Float16}, \code{__Bfloat16}, \code{float},
> + \code{double}, \code{_Decimal32},
> \code{_Decimal64} and \code{__m64} are in class SSE.
> \item Arguments of types \code{__float128}, \code{_Decimal128}
> and \code{__m128} are split into two halves. The least significant
> --
> 2.18.1
>


--
BR,
Hongtao

[Florian Weimer]

* liuhongt via Libc-alpha:

> +\subsubsection{Special Types}
> +
> +The \code{__Bfloat16} type uses a 8-bit exponent and 7-bit mantissa.
> +It is used for \code{BF16} related intrinsics, it cannot be
> +used with standard C operators.

I think it's not necessary to specify whether the type supports certain
C operators (surely assignment will work?). If they are added later,
the ABI won't need changing.

Thanks,
Florian

[H.J. Lu]

On Fri, Jun 10, 2022 at 2:38 AM Florian Weimer <fwe...@redhat.com> wrote:
>
> * liuhongt via Libc-alpha:
>
> > +\subsubsection{Special Types}
> > +
> > +The \code{__Bfloat16} type uses a 8-bit exponent and 7-bit mantissa.
> > +It is used for \code{BF16} related intrinsics, it cannot be

Please mention that this is an alternate encoding format for 16-bit floating
point. It has the same size and alignment as _Float16.

> > +used with standard C operators.
>
> I think it's not necessary to specify whether the type supports certain
> C operators (surely assignment will work?). If they are added later,
> the ABI won't need changing.
>

If _Bfloat16 becomes a fundamental type, the ABI should be changed to
move it together with other scalar types.

--
H.J.

[H.J. Lu]

On Fri, Jun 10, 2022 at 7:44 AM H.J. Lu <hjl....@gmail.com> wrote:
>
> On Fri, Jun 10, 2022 at 2:38 AM Florian Weimer <fwe...@redhat.com> wrote:
> >
> > * liuhongt via Libc-alpha:
> >
> > > +\subsubsection{Special Types}
> > > +
> > > +The \code{__Bfloat16} type uses a 8-bit exponent and 7-bit mantissa.
> > > +It is used for \code{BF16} related intrinsics, it cannot be
>
> Please mention that this is an alternate encoding format for 16-bit floating
> point. It has the same size and alignment as _Float16.

It also follows the same rules as _Float16 for parameter passing and function
return.

> > > +used with standard C operators.
> >
> > I think it's not necessary to specify whether the type supports certain
> > C operators (surely assignment will work?). If they are added later,
> > the ABI won't need changing.
> >
>
> If _Bfloat16 becomes a fundamental type, the ABI should be changed to
> move it together with other scalar types.
>
> --
> H.J.

--
H.J.

[Hongtao Liu]

On Sat, Jun 11, 2022 at 1:46 AM H.J. Lu <hjl....@gmail.com> wrote:
>
> On Fri, Jun 10, 2022 at 7:44 AM H.J. Lu <hjl....@gmail.com> wrote:
> >
> > On Fri, Jun 10, 2022 at 2:38 AM Florian Weimer <fwe...@redhat.com> wrote:
> > >
> > > * liuhongt via Libc-alpha:
> > >
> > > > +\subsubsection{Special Types}
> > > > +
> > > > +The \code{__Bfloat16} type uses a 8-bit exponent and 7-bit mantissa.
> > > > +It is used for \code{BF16} related intrinsics, it cannot be
> >
> > Please mention that this is an alternate encoding format for 16-bit floating
> > point. It has the same size and alignment as _Float16.
>
> It also follows the same rules as _Float16 for parameter passing and function
> return.

How about

+\subsubsection{Special Types}
+

+The \code{__Bfloat16} type has an alternate encoding format for 16-bit
+floating point with 8-bit exponent and 7-bit mantissa. It has the same size,
+alignment, parameter passing and return rules as _Float16.
+It is used for \code{BF16} related intrinsics, it cannot be used with standard
+C operators.
+

>
> > > > +used with standard C operators.
> > >
> > > I think it's not necessary to specify whether the type supports certain
> > > C operators (surely assignment will work?). If they are added later,
> > > the ABI won't need changing.
> > >
> >
> > If _Bfloat16 becomes a fundamental type, the ABI should be changed to
> > move it together with other scalar types.
> >
> > --
> > H.J.
>
>
>
> --
> H.J.
>

> --
> You received this message because you are subscribed to the Google Groups "X86-64 System V Application Binary Interface" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to x86-64-abi+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/x86-64-abi/CAMe9rOrvrZRbksjQ%2BY8caC%3DYo%3D%3DoFY5%2BmuOGf9UZRh6L7pUQjw%40mail.gmail.com.



--
BR,
Hongtao

[Phoebe Wang]

I think using the name __Bfloat16 is problematic. It’s too similar to _Float16, but the latter is an arithmetic type and already defined in C11 standard. I'm afraid if there will be a new _Bfloat16 type in the standard in future, it will confuse user with the double underscore one we defined here.

My top prefer is __bf16, given it’s a storage type and conceptually has the same naming conversion with __fp16 which is storage type too.

If __bf16 is not accepted, then a “__mbf16” or “__mbfloat” might be better which “__m” implies it’s used for intrinsics only just like “__m128”, “__mmask16” etc.

[Hongtao Liu]

On Mon, Jun 13, 2022 at 10:15 PM Phoebe Wang <phoeb...@gmail.com> wrote:
>
> I think using the name __Bfloat16 is problematic. It’s too similar to _Float16, but the latter is an arithmetic type and already defined in C11 standard. I'm afraid if there will be a new _Bfloat16 type in the standard in future, it will confuse user with the double underscore one we defined here.
> My top prefer is __bf16, given it’s a storage type and conceptually has the same naming conversion with __fp16 which is storage type too.

I don't have strong opinions for the name, __bf16 looks good to me.

> To view this discussion on the web visit https://groups.google.com/d/msgid/x86-64-abi/59ae5258-bdc6-4b69-9a49-3e189e08156an%40googlegroups.com.



--
BR,
Hongtao

[Hongtao Liu]

update v3:

diff --git a/x86-64-ABI/low-level-sys-info.tex
b/x86-64-ABI/low-level-sys-info.tex
index a8b69db..81d1618 100644
--- a/x86-64-ABI/low-level-sys-info.tex
+++ b/x86-64-ABI/low-level-sys-info.tex
@@ -302,6 +302,14 @@ be used to represent the type, is a family of

integer types.
This permits the use of these types in allocated arrays using the common
sizeof(Array)/sizeof(ElementType) pattern.

+\subsubsection{Special Types}
+

+The \code{__bf16} type is an alternate encoding format for 16-bit
+float with 8-bit exponent and 7-bit mantissa. It has the same size,

+alignment, parameter passing and return rules as _Float16.
+It is used for \code{BF16} related intrinsics, it cannot be used with standard
+C operators.
+

\subsubsection{Aggregates and Unions}

Structures and unions assume the alignment of their most strictly

@@ -563,8 +571,8 @@ The basic types are assigned their natural classes:

\item Arguments of types (signed and unsigned) \code{_Bool}, \code{char},
\code{short}, \code{int}, \code{long}, \code{long long}, and
pointers are in the INTEGER class.
-\item Arguments of types \code{_Float16}, \code{float}, \code{double},
- \code{_Decimal32},

+\item Arguments of types \code{_Float16}, \code{__bf16}, \code{float},

+ \code{double}, \code{_Decimal32},
\code{_Decimal64} and \code{__m64} are in class SSE.
\item Arguments of types \code{__float128}, \code{_Decimal128}
and \code{__m128} are split into two halves. The least significant

--
BR,
Hongtao

[Michael Matz]

Hello,

On Tue, 14 Jun 2022, Hongtao Liu wrote:

> update v3:
>
> diff --git a/x86-64-ABI/low-level-sys-info.tex
> b/x86-64-ABI/low-level-sys-info.tex
> index a8b69db..81d1618 100644
> --- a/x86-64-ABI/low-level-sys-info.tex
> +++ b/x86-64-ABI/low-level-sys-info.tex
> @@ -302,6 +302,14 @@ be used to represent the type, is a family of
> integer types.
> This permits the use of these types in allocated arrays using the common
> sizeof(Array)/sizeof(ElementType) pattern.
>
> +\subsubsection{Special Types}
> +
> +The \code{__bf16} type is an alternate encoding format for 16-bit
> +float with 8-bit exponent and 7-bit mantissa. It has the same size,
> +alignment, parameter passing and return rules as _Float16.
> +It is used for \code{BF16} related intrinsics, it cannot be used with standard
> +C operators.
> +

I agree with Florian here: this is cross-language psABI, it doesn't need
to say anything about C operators at all, and the mentioning of intrinsics
also doesn't seem usefully normative, but rather informative, so would
perhaps be appropriate as a footnote.

So, "\footnote{used in \code{BF16} intrinsics}" ?


Ciao,
Michael.

[Hongtao Liu]

Thanks for the suggestion, here's update patch v4

diff --git a/x86-64-ABI/low-level-sys-info.tex
b/x86-64-ABI/low-level-sys-info.tex
index a8b69db..871d1ed 100644
--- a/x86-64-ABI/low-level-sys-info.tex
+++ b/x86-64-ABI/low-level-sys-info.tex
@@ -302,6 +302,13 @@ be used to represent the type, is a family of

integer types.
This permits the use of these types in allocated arrays using the common
sizeof(Array)/sizeof(ElementType) pattern.

+\subsubsection{Special Types}
+

+The \code{__bf16}\footnote{This type is used in \code{BF16} intrinsics} type
+is an alternate encoding format for 16-bit float with 8-bit exponent and 7-bit
+mantissa. It has the same size, alignment, parameter passing and return rules
+as _Float16.

+
\subsubsection{Aggregates and Unions}

Structures and unions assume the alignment of their most strictly

@@ -563,8 +570,8 @@ The basic types are assigned their natural classes:

\item Arguments of types (signed and unsigned) \code{_Bool}, \code{char},
\code{short}, \code{int}, \code{long}, \code{long long}, and
pointers are in the INTEGER class.
-\item Arguments of types \code{_Float16}, \code{float}, \code{double},
- \code{_Decimal32},
+\item Arguments of types \code{_Float16}, \code{__bf16}, \code{float},
+ \code{double}, \code{_Decimal32},
\code{_Decimal64} and \code{__m64} are in class SSE.
\item Arguments of types \code{__float128}, \code{_Decimal128}
and \code{__m128} are split into two halves. The least significant

>
>

> Ciao,
> Michael.



--
BR,
Hongtao

[Florian Weimer]

* Phoebe Wang:

> My top prefer is __bf16, given it’s a storage type and conceptually
> has the same naming conversion with __fp16 which is storage type too.
> If __bf16 is not accepted, then a “__mbf16” or “__mbfloat” might be
> better which “__m” implies it’s used for intrinsics only just like
> “__m128”, “__mmask16” etc.

Does this __bf16 have the same layout as the __bf16 already in use by
the GCC 32-bit Arm front end? It's also not entirely internal to GCC,
GDB has this (apparently in generic code):

| const struct floatformat **
| default_floatformat_for_type (struct gdbarch *gdbarch,
| const char *name, int len)
| {
| const struct floatformat **format = NULL;
|
| /* Check if this is a bfloat16 type. It has the same size as the
| IEEE half float type, so we use the base type name to tell them
| apart. */
| if (name != nullptr && strcmp (name, "__bf16") == 0
| && len == gdbarch_bfloat16_bit (gdbarch))
| format = gdbarch_bfloat16_format (gdbarch);

So if it doesn't match existing usage, I think we should pick a
different name for x86-64.

Thanks,
Florian

[Jan Beulich]

In gas we've (mostly) unified Arm's and x86'es bfloat16 handling, for
being the same layout. I think this can be extended to gcc, gdb, and
whichever else component might care.

Jan

[Hongtao Liu]

On Wed, Jun 15, 2022 at 4:35 PM Florian Weimer <fwe...@redhat.com> wrote:
>
> * Phoebe Wang:
>
> > My top prefer is __bf16, given it’s a storage type and conceptually
> > has the same naming conversion with __fp16 which is storage type too.
> > If __bf16 is not accepted, then a “__mbf16” or “__mbfloat” might be
> > better which “__m” implies it’s used for intrinsics only just like
> > “__m128”, “__mmask16” etc.
>
> Does this __bf16 have the same layout as the __bf16 already in use by
> the GCC 32-bit Arm front end? It's also not entirely internal to GCC,

They're the same, they're all coming from google Brain Floating Point
Format which is a truncated (16-bit) version of the 32-bit IEEE 754
single-precision floating-point format.
Update V5 with some description for Brain Floating Point Format.
diff --git a/x86-64-ABI/low-level-sys-info.tex
b/x86-64-ABI/low-level-sys-info.tex
index a8b69db..f051be9 100644
--- a/x86-64-ABI/low-level-sys-info.tex
+++ b/x86-64-ABI/low-level-sys-info.tex
@@ -302,6 +302,16 @@ be used to represent the type, is a family of

integer types.
This permits the use of these types in allocated arrays using the common
sizeof(Array)/sizeof(ElementType) pattern.

+\subsubsection{Special Types}
+

+The \code{__bf16}\footnote{This type is used in \code{BF16} intrinsics.} type
+is an alternate encoding format for 16-bit float with 8-bit exponent and 7-bit
+mantissa. \code{__bf16} is short for Brain Floating Point Format\footnote{This
+ type is used to accelerate machine learning (deep learning training,
+ in particular) algorithms.} which is a truncated (16-bit) version of the
+32-bit IEEE 754 single-precision floating-point format. It has the same size,

+alignment, parameter passing and return rules as _Float16.

+
\subsubsection{Aggregates and Unions}

Structures and unions assume the alignment of their most strictly

@@ -563,8 +573,8 @@ The basic types are assigned their natural classes:

\item Arguments of types (signed and unsigned) \code{_Bool}, \code{char},
\code{short}, \code{int}, \code{long}, \code{long long}, and
pointers are in the INTEGER class.
-\item Arguments of types \code{_Float16}, \code{float}, \code{double},
- \code{_Decimal32},
+\item Arguments of types \code{_Float16}, \code{__bf16}, \code{float},
+ \code{double}, \code{_Decimal32},
\code{_Decimal64} and \code{__m64} are in class SSE.
\item Arguments of types \code{__float128}, \code{_Decimal128}
and \code{__m128} are split into two halves. The least significant

> GDB has this (apparently in generic code):
>
> | const struct floatformat **
> | default_floatformat_for_type (struct gdbarch *gdbarch,
> | const char *name, int len)
> | {
> | const struct floatformat **format = NULL;
> |
> | /* Check if this is a bfloat16 type. It has the same size as the
> | IEEE half float type, so we use the base type name to tell them
> | apart. */
> | if (name != nullptr && strcmp (name, "__bf16") == 0
> | && len == gdbarch_bfloat16_bit (gdbarch))
> | format = gdbarch_bfloat16_format (gdbarch);
>
> So if it doesn't match existing usage, I think we should pick a
> different name for x86-64.
>
> Thanks,
> Florian
>

> --
> You received this message because you are subscribed to the Google Groups "X86-64 System V Application Binary Interface" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to x86-64-abi+...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/x86-64-abi/87sfo6pbmr.fsf%40oldenburg.str.redhat.com.



--
BR,
Hongtao

[H.J. Lu]

\code{__bf16} represents Brain Floating Point Format

or

\code{__bf16} follows Brain Floating Point Format

> + type is used to accelerate machine learning (deep learning training,
> + in particular) algorithms.} which is a truncated (16-bit) version of the
> +32-bit IEEE 754 single-precision floating-point format. It has the same size,
> +alignment, parameter passing and return rules as _Float16.
> +
> \subsubsection{Aggregates and Unions}
>
> Structures and unions assume the alignment of their most strictly
> @@ -563,8 +573,8 @@ The basic types are assigned their natural classes:
> \item Arguments of types (signed and unsigned) \code{_Bool}, \code{char},
> \code{short}, \code{int}, \code{long}, \code{long long}, and
> pointers are in the INTEGER class.
> -\item Arguments of types \code{_Float16}, \code{float}, \code{double},
> - \code{_Decimal32},
> +\item Arguments of types \code{_Float16}, \code{__bf16}, \code{float},
> + \code{double}, \code{_Decimal32},
> \code{_Decimal64} and \code{__m64} are in class SSE.

I don't think this change is needed since __bf16 is passed the same
way as _Float16. We should limit the scope of __bf16.

> \item Arguments of types \code{__float128}, \code{_Decimal128}
> and \code{__m128} are split into two halves. The least significant
>
> > GDB has this (apparently in generic code):
> >
> > | const struct floatformat **
> > | default_floatformat_for_type (struct gdbarch *gdbarch,
> > | const char *name, int len)
> > | {
> > | const struct floatformat **format = NULL;
> > |
> > | /* Check if this is a bfloat16 type. It has the same size as the
> > | IEEE half float type, so we use the base type name to tell them
> > | apart. */
> > | if (name != nullptr && strcmp (name, "__bf16") == 0
> > | && len == gdbarch_bfloat16_bit (gdbarch))
> > | format = gdbarch_bfloat16_format (gdbarch);
> >
> > So if it doesn't match existing usage, I think we should pick a
> > different name for x86-64.
> >
> > Thanks,
> > Florian
> >


--

H.J.

[Hongtao Liu]

Changed.

> > + type is used to accelerate machine learning (deep learning training,
> > + in particular) algorithms.} which is a truncated (16-bit) version of the
> > +32-bit IEEE 754 single-precision floating-point format. It has the same size,
> > +alignment, parameter passing and return rules as _Float16.
> > +
> > \subsubsection{Aggregates and Unions}
> >
> > Structures and unions assume the alignment of their most strictly
> > @@ -563,8 +573,8 @@ The basic types are assigned their natural classes:
> > \item Arguments of types (signed and unsigned) \code{_Bool}, \code{char},
> > \code{short}, \code{int}, \code{long}, \code{long long}, and
> > pointers are in the INTEGER class.
> > -\item Arguments of types \code{_Float16}, \code{float}, \code{double},
> > - \code{_Decimal32},
> > +\item Arguments of types \code{_Float16}, \code{__bf16}, \code{float},
> > + \code{double}, \code{_Decimal32},
> > \code{_Decimal64} and \code{__m64} are in class SSE.
>
> I don't think this change is needed since __bf16 is passed the same
> way as _Float16. We should limit the scope of __bf16.

Deleted __bf16 from this part.

>
> > \item Arguments of types \code{__float128}, \code{_Decimal128}
> > and \code{__m128} are split into two halves. The least significant
> >
> > > GDB has this (apparently in generic code):
> > >
> > > | const struct floatformat **
> > > | default_floatformat_for_type (struct gdbarch *gdbarch,
> > > | const char *name, int len)
> > > | {
> > > | const struct floatformat **format = NULL;
> > > |
> > > | /* Check if this is a bfloat16 type. It has the same size as the
> > > | IEEE half float type, so we use the base type name to tell them
> > > | apart. */
> > > | if (name != nullptr && strcmp (name, "__bf16") == 0
> > > | && len == gdbarch_bfloat16_bit (gdbarch))
> > > | format = gdbarch_bfloat16_format (gdbarch);
> > >
> > > So if it doesn't match existing usage, I think we should pick a
> > > different name for x86-64.
> > >
> > > Thanks,
> > > Florian
> > >
>
>
> --
> H.J.

Update in V6

diff --git a/x86-64-ABI/low-level-sys-info.tex
b/x86-64-ABI/low-level-sys-info.tex
index a8b69db..72385dd 100644

--- a/x86-64-ABI/low-level-sys-info.tex
+++ b/x86-64-ABI/low-level-sys-info.tex
@@ -302,6 +302,16 @@ be used to represent the type, is a family of
integer types.
This permits the use of these types in allocated arrays using the common
sizeof(Array)/sizeof(ElementType) pattern.

+\subsubsection{Special Types}
+
+The \code{__bf16}\footnote{This type is used in \code{BF16} intrinsics.} type
+is an alternate encoding format for 16-bit float with 8-bit exponent and 7-bit

+mantissa. \code{__bf16} represents Brain Floating Point Format\footnote{This

+ type is used to accelerate machine learning (deep learning training,
+ in particular) algorithms.} which is a truncated (16-bit) version of the
+32-bit IEEE 754 single-precision floating-point format. It has the same size,
+alignment, parameter passing and return rules as _Float16.
+
\subsubsection{Aggregates and Unions}

--
BR,
Hongtao

[Hongtao Liu]

New merge request for it.

https://gitlab.com/x86-psABIs/x86-64-ABI/-/merge_requests/35

--
BR,
Hongtao