[Python-Dev] Require a C compiler supporting C99 to build Python 3.11

[Victor Stinner]

Hi,

I made a change to require C99 <math.h> "NAN" constant and I'm was
asked to update the PEP 7 to clarify the C subset is needed to build
Python 3.11.

Python 3.6 requires a subset of the C99 standard to build defined by the PEP 7:
https://www.python.org/dev/peps/pep-0007/

I modified Python 3.11 to require more C99 features of the <math.h> header:

* bpo-45440: copysign(), hypot(), isfinite(), isinf(), isnan(), round()
* bpo-46640: NAN

After my NAN change (bpo-46640), Petr Viktorin asked me to update the
PEP 7. I proposed a change to simply say that "Python 3.11 and newer
versions use C99":
https://github.com/python/peps/pull/2309

I would prefer to not have to give an exhaustive list of C99 features
used by CPython, since it's unclear to me what belongs to C99 or to
ISO C89. As I wrote before, Python already uses C99 features since
Python 3.6.

On my PEP PR, Guido van Rossum asked me to escalate the discussion to
python-dev, so here I am :-)

In "C99", the number "99" refers to the year 1999, the standard is now
23 years old:
https://en.wikipedia.org/wiki/C99

In 2022, C99 is now well supported by C compilers supported by Python:
GCC, clang, MSVC.

I don't know if AIX XLC supports C99. AIX provides a "c99" compiler
compatible with C99. It also seems like GCC is usable on AIX.

I don't know if ICC supports C99. Python doesn't officially the ICC
compiler, the ICC buildbots are gone a few years ago. But sometimes I
make some changes to enhance the ICC support, when the change is small
enough.

Note: Python also uses C11 <stdatomic.h>, but it's not required: there
are fallbacks for compilers which don't support it.

Victor
--
Night gathers, and now my watch begins. It shall not end until my death.
_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/J5FSP6J4EITPY5C2UJI7HSL2GQCTCUWN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Mark Dickinson]

On Mon, Feb 7, 2022 at 5:11 PM Victor Stinner <vsti...@python.org> wrote:

I made a change to require C99 <math.h> "NAN" constant [...]

There's a separate discussion topic lurking here. It's equally in need of discussion here (IMO), but it's orthogonal to the "should we require C99" discussion. I've changed the subject line accordingly to try to avoid derailing that discussion.

Unlike the other things Victor mentions ("copysign", "round", etc.), the NAN macro is not required to be present by C99. Instead, the standard says that "NAN is defined if and only if the implementation supports quiet NaNs for the float type" (C99 §7.12p5).

Victor is proposing in GH-31160 to require the presence of the NAN macro in order for CPython to build, which under C99 is equivalent to requiring that the C float type supports quiet NaNs. That's not the same as requiring IEEE 754 floating-point, but it's not far off - there aren't many non-IEEE 754 floating-point formats that support NaNs. (Historically, there are essentially none, but it seems quite likely that there will be at least some non-IEEE 754 formats in the future that support NaNs; Google's bfloat16 format is one example.)

So there (at least) three questions here:

- Should we require the presence of NaNs in order for CPython to build?

- Should we require IEEE 754 floating-point for CPython-the-implementation?

- Should we require IEEE 754 floating-point for Python-the-language?

For the first two, I'd much prefer either to not require NaNs, or to go the whole way and require IEEE 754 for CPython. Requiring NaNs but not IEEE 754 feels like an awkward halfway house: in practice, it would be just as restrictive as requiring IEEE 754, but without the benefits of making that requirement explicit (e.g., being able to get rid of non-IEEE 754 paths in existing code, and being able to tell users that they can reasonably expect IEEE 754-conformant behaviour).

Note that on the current main branch there's a Py_NO_NAN macro that builders can define to indicate that NaNs aren't supported, but the Python build is currently broken if Py_NO_NAN is defined (see https://bugs.python.org/issue46656). If the answer to the first question is "No", then we need to fix the build under Py_NO_NAN. That's not a big deal - perhaps a couple of hours of work.

-- 

Mark

[Victor Stinner]

Hi Mark,

Aha, good, you posted an email to python-dev, good :-) Last days, I
was trying to collect more data about this topic, especially find
platforms which *don't* support IEEE 754, before posting to
python-dev.

Nowadays, outside museums, it's hard to find computers which don't
implement IEEE 754.

* Since 1998, IBM S/390 supports IEEE 754.
* Compaq/DEC VAX didn't use IEEE 754, but are no longer built since
2000 (you cant still use Python 3.10 if it can be built on it!)
* Some Cray computers like Cray SV1 (1998) don't implement IEEE 754,
but other Cray computers like Cray T90 (1995) implement IEEE 754.

There are embedded devices with no hardware FPU: in this case, the FPU
is implemented in software. I expect it to implement IEEE 754. Is
CPython a good target for such small CPUs which have no hardware FPU?
MicroPython may better fit their needs.

On the other side, all moderns CPU architectures support IEEE 754:
Intel x86, ARM, IBM Power and PowerPC, Compaq/DEC Alpha, HP PA-RISC,
Motorola 68xxx and 88xxx, SGI R-xxxx, Sun SPARC.

Sources:

* https://en.wikipedia.org/wiki/Floating-point_arithmetic#IEEE_754:_floating_point_in_modern_computers
* https://stackoverflow.com/questions/2234468/do-any-real-world-cpus-not-use-ieee-754

On Mon, Feb 7, 2022 at 7:25 PM Mark Dickinson <dick...@gmail.com> wrote:
> - Should we require the presence of NaNs in order for CPython to build?
> - Should we require IEEE 754 floating-point for CPython-the-implementation?

In the past, when we deprecated the support for an old platform, we
didn't suddenly remove the code. We made sure that it's no longer
possible to build on it. So if anyone notices, it's easy to revert
(ex: remove the few lines in configure).

Would it make sense to trigger a build error on plaforms which don't
support IEEE 754 in Python 3.11, and later decide if it's time to
remove the code in in Python 3.12?

Well. If you ask *me*, I'm in favor of removing the code right now. If
someone needs to support a platform which doesn't support IEEE 754,
the support can be maintained *outside* the Python source code, as
external patches or as a Git fork, no?

Honestly, I never got access to any machine which doesn't support IEEE
754 (or nobody told me!).

> - Should we require IEEE 754 floating-point for Python-the-language?

Right now, I have no opinion on this question.

> Note that on the current main branch there's a Py_NO_NAN macro that builders can define to indicate that NaNs aren't supported, but the Python build is currently broken if Py_NO_NAN is defined (see https://bugs.python.org/issue46656). If the answer to the first question is "No", then we need to fix the build under Py_NO_NAN. That's not a big deal - perhaps a couple of hours of work.

My comment on bpo-46656: "Python uses Py_NAN without "#ifdef Py_NAN"
guard since 2008. Building Python with Py_NO_NAN never worked. Nobody
reported such build failure in the last 14 years..."

If anyone would try building/using Python on a platform without NAN, I
would expect that we would get a bug report or an email. I'm not aware
of anything like that, so it seems like nobody uses Python on such
platform.

Victor
--
Night gathers, and now my watch begins. It shall not end until my death.
_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/PPVBMCKL5E2MVTWVFGSI4VTT6IAJ5QBF/

[Gregory P. Smith]

It's 2022. We should just require both NaN and IEEE-754.

By the time a system is large enough to build and run CPython, it is exceedingly unlikely that it will not be able to do that. Even if it means software emulation of IEEE-754 floating point on something large yet odd enough not to have a FPU.

The places where non-IEEE-754 and non-NaN floating point are likely to exist are in specialized parallelized coprocessor hardware. Not the general purpose CPU running CPython.

 

-gps

[Brett Cannon]

On Mon, Feb 7, 2022 at 11:02 AM Victor Stinner <vsti...@python.org> wrote:

Hi Mark,

Aha, good, you posted an email to python-dev, good :-) Last days, I
was trying to collect more data about this topic, especially find
platforms which *don't* support IEEE 754, before posting to
python-dev.

Nowadays, outside museums, it's hard to find computers which don't
implement IEEE 754.

* Since 1998, IBM S/390 supports IEEE 754.
* Compaq/DEC VAX didn't use IEEE 754, but are no longer built since
2000 (you cant still use Python 3.10 if it can be built on it!)
* Some Cray computers like Cray SV1 (1998) don't implement IEEE 754,
but other Cray computers like Cray T90 (1995) implement IEEE 754.

There are embedded devices with no hardware FPU: in this case, the FPU
is implemented in software. I expect it to implement IEEE 754. Is
CPython a good target for such small CPUs which have no hardware FPU?
MicroPython may better fit their needs.

On the other side, all moderns CPU architectures support IEEE 754:
Intel x86, ARM, IBM Power and PowerPC, Compaq/DEC Alpha, HP PA-RISC,
Motorola 68xxx and 88xxx, SGI R-xxxx, Sun SPARC.

RISC-V also supports the 2008 IEEE-754 spec: https://riscv.org/technical/specifications/ .

-Brett

[Brett Cannon]

Do we have a buildbot that has a CPU or OS that can't handle IEEE-754? What are the chances we will get one? If the answers are "none" and "slim", then it seems reasonable to require NaN and IEEE-754.

[Jeremiah Vivian]

On Mon, Feb 7, 2022 at 5:11 PM Victor Stinner vsti...@python.org wrote:

> I made a change to require C99 <math.h> "NAN" constant and I'm was
> asked to update the PEP 7 to clarify the C subset is needed to build
> Python 3.11.

What if it was built with MSVC which is usually just C90?

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/JSQX3OY4XMTBLLCERP3JDJXPPT2BMQQ6/

[Brett Cannon]

I think if those compilers fully C99 at this point we should consider just moving completely over to C99.

-Brett

[Victor Stinner]

On Tue, Feb 8, 2022 at 12:19 AM Brett Cannon <br...@python.org> wrote:
> Do we have a buildbot that has a CPU or OS that can't handle IEEE-754? What are the chances we will get one? If the answers are "none" and "slim", then it seems reasonable to require NaN and IEEE-754.

All CPU architectures of current buildbot workers use IEEE 754 (little
or big endian).

I looked at the following CPU architectures:

* x86 (Intel 32-bit)
* x86-64 (Intel 64-bit)
* armv7l (ARM 32-bit)
* aarch64 (ARM 64-bit)
* ppc64, ppc64le (PowerPC 64-bit)
* s390x (IBM)

There is also a "SPARCv9 Oracle Solaris 11.4" worker, but sadly it
seems to be disconnected for 3 months. The SPARCv9 architecture uses
IEEE 754.

On buildbots, I checked float.__getformat__("double"). It is logged as
"builtins.float.double_format" in the "pythoninfo" step of buidbots.

Little-endian examples:

* x86-64 (AMD64 Arch Linux Asan Debug 3.x): IEEE, little-endian
* x86 (Gentoo Non-Debug with X 3.x): IEEE, little-endian
* aarch64 (CentOS9 LTO + PGO 3.10): IEEE, little-endian
* aarch64 (Fedora Rawhide Clang 3.x): IEEE, little-endian
* aarch64: MacOS M1 (ARM64 macOS 3.x): IEEE, little-endian
* aarch64 (ARM64 Windows 3.x): IEEE, little-endian
* PPC64LE (CentOS9 3.10): IEEE, little-endian
* armv7l (ARM Raspbian 3.x): IEEE, little-endian

Big-endian examples:

* PPC64 (AIX 3.x): IEEE, big-endian
* s390x (Debian 3.x): IEEE, big-endian

float.__getformat__("double") is initialized by _PyFloat_InitState()
at Python startup with this code:
---
#if SIZEOF_DOUBLE == 8
{
double x = 9006104071832581.0;
if (memcmp(&x, "\x43\x3f\xff\x01\x02\x03\x04\x05", 8) == 0)
detected_double_format = ieee_big_endian_format;
else if (memcmp(&x, "\x05\x04\x03\x02\x01\xff\x3f\x43", 8) == 0)
detected_double_format = ieee_little_endian_format;
else
detected_double_format = unknown_format;
}
#else
detected_double_format = unknown_format;
#endif
---

Victor
--
Night gathers, and now my watch begins. It shall not end until my death.
_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/4PJP7CB7UXQ6KRRWBBQUIQCLO2LL5JNO/

[Christopher Barker]

From the perspective of some that writes a lot of computational code:

On Mon, Feb 7, 2022 at 10:25 AM Mark Dickinson <dick...@gmail.com> wrote:

- Should we require the presence of NaNs in order for CPython to build?

- Should we require IEEE 754 floating-point for CPython-the-implementation?

Yes, and yes, together, as Mark suggests

- Should we require IEEE 754 floating-point for Python-the-language?

I would say no, but it should be recommended -- see Victor's example of Micro Python -- though does anyone have an authority over that? IIUC, Micro Python already has a few differences -- but is anyone saying they shouldn't call it Python? Though, with a quick perusal of teh docs, micro Python does seem to support NaN and Inf, at least -- not sure about the rest of 754.

While we're at it are  64 bit floats required for either cPython or Python the language?

IIUC, the main platform that this was an issue for, for computationally heavy code, was Cray -- but it seems they're 754 compatible now, yes?

Thanks for doing this,

-CHB

--

Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython

[Inada Naoki]

https://www.ibm.com/docs/en/SSGH3R_16.1.0/pdf/getstart.pdf

As far as reading page 3,
xlclang fully supports C89/C99/C11.
xlc fully supports C89/C99, and partially supports C11.

> Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/ZLDOBJUVMTIRETVRNHPWWO5MBHTXYEW3/

> Code of Conduct: http://python.org/psf/codeofconduct/

--
Inada Naoki <songof...@gmail.com>

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/XIMQWFVP7RJW7CTV74YPWI74L7ZUU2PX/

[Jim J. Jewett]

- Should we require the presence of NaNs in order for CPython to build?
- Should we require IEEE 754 floating-point for CPython-the-implementation?
- Should we require IEEE 754 floating-point for Python-the-language?

I don't have strong opinions on the first two, but for the language definition, I think the most we should say is "if an implementation does not support IEEE 754 floating-point, this must be mentioned in the documentation as an implementation limit."

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/YEXX363XX6DS7ZC653RBLIPNQIHBVYTK/

[Gregory P. Smith]

On Mon, Feb 7, 2022 at 4:52 PM Christopher Barker <pyth...@gmail.com> wrote:

From the perspective of some that writes a lot of computational code:

On Mon, Feb 7, 2022 at 10:25 AM Mark Dickinson <dick...@gmail.com> wrote:

- Should we require the presence of NaNs in order for CPython to build?

- Should we require IEEE 754 floating-point for CPython-the-implementation?

Yes, and yes, together, as Mark suggests

- Should we require IEEE 754 floating-point for Python-the-language?

I would say no, but it should be recommended -- see Victor's example of Micro Python -- though does anyone have an authority over that? IIUC, Micro Python already has a few differences -- but is anyone saying they shouldn't call it Python? Though, with a quick perusal of teh docs, micro Python does seem to support NaN and Inf, at least -- not sure about the rest of 754.

PEP 7 and PEP 11 are only about CPython. Other Python VMs full or partial are free to do what they want with how they implement floats if at all. (they're build time optional in MicroPython as are many Python language syntax features)

While we're at it are  64 bit floats required for either cPython or Python the language?

CPython: yes.  we use a double.

Python the language: no.  (float is single precision on many micropython platforms as it saves precious ram and performance, plus microcontroller fpu hardware like an M4 is usually single precision 32bit)

[Serhiy Storchaka]

07.02.22 19:09, Victor Stinner пише:

> After my NAN change (bpo-46640), Petr Viktorin asked me to update the
> PEP 7. I proposed a change to simply say that "Python 3.11 and newer
> versions use C99":

But public headers should be compatible with C++, and not all C99
features are compatible with C++. It is not a matter of style, it is a
matter of compatibility.

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/ZDGZN46UQAPJY5EF5A3GYA5DT4XX6AQO/

[Petr Viktorin]

Does MSVC support all of C99? I haven't found any documentation claiming
that... but I'm also not familiar with MSVC.

Do we need to support a subset like "C99 except the features that were
removed in C11"?

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/XUNLZB5Q5BCWLX7C77WBD34ZU33RPVNM/

[Steve Dower]

On 2/8/2022 1:35 AM, Gregory P. Smith wrote:
> While we're at it are  64 bit floats required for either cPython or
> Python the language?
>
>
> CPython: yes.  we use a double.
> Python the language: no.  (float is single precision on many micropython
> platforms as it saves precious ram and performance, plus microcontroller
> fpu hardware like an M4 is usually single precision 32bit)

Agreed. CPython should be specific, Python should be as vague as
possible. Otherwise, we would prevent _by specification_ using Python as
a scripting language for things where floats may not even be relevant.

It's not even about hardware capabilities (unless you believe Python is
only for running in a terminal, in which case we'll just have to
disagree). It could be about an app using short Python snippets as a
customisation language for some domain-specific task rather than
inventing a brand new language for it. The more
"it's-only-Python-if-it-has-X" restrictions we have, the less appealing
we become.

Cheers,
Steve

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/W4GVXJZJ6EKNIEQVFIKRGU7PNN3OTNUP/

[Steve Dower]

On 2/8/2022 9:52 AM, Petr Viktorin wrote:
> On 07. 02. 22 18:09, Victor Stinner wrote:
>> In 2022, C99 is now well supported by C compilers supported by Python:
>> GCC, clang, MSVC.
>
>
> Does MSVC support all of C99? I haven't found any documentation claiming
> that... but I'm also not familiar with MSVC.
>
> Do we need to support a subset like "C99 except the features that were
> removed in C11"?

I couldn't find any information either (from public sources, at least,
haven't asked the team directly yet).

All the C99 library is supposedly supported, but there are (big?) gaps
in the compiler support. Possibly these are features that were removed
in C11? I don't know what is on that list. It seems C11 and C17 are
supported [1], though I know for sure we have contributors who aren't
using a recent enough compiler version to have it :)

Personally, I see no reason to "require" C99 as a whole. We have a style
guide already, and can list the additional compiler features that we
allow along with guidance on updating existing code and ensuring
compatibility.

I don't see much risk requiring the C99 standard library, though. It's
the compiler features that seem to have less coverage.

Cheers,
Steve

[1]:
https://devblogs.microsoft.com/cppblog/c11-and-c17-standard-support-arriving-in-msvc/

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/UZACKMZGN2PUGNQ74FQX2V4UTQAA6DWB/

[Erlend Aasland]

According to the docs[^1], MSVC does _not_ support all of C99. OTOH they claim
conformance with both C11 and C17, using the /std:c11 and /std:c17 compiler
flags.


[^1]: https://docs.microsoft.com/en-us/cpp/build/reference/std-specify-language-standard-version?view=msvc-170

> On 8 Feb 2022, at 10:52, Petr Viktorin <enc...@gmail.com> wrote:
>
> Does MSVC support all of C99? I haven't found any documentation claiming that... but I'm also not familiar with MSVC.

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/AYZC3F73THZXNMSYWSFPXFKAUQ5NBTDO/

[Victor Stinner]

On Tue, Feb 8, 2022 at 2:02 PM Steve Dower <steve...@python.org> wrote:
> All the C99 library is supposedly supported, but there are (big?) gaps
> in the compiler support.

Some Visual Studio 2019 updates on the Microsoft blog.

March 2020: C99 _Pragma
https://devblogs.microsoft.com/cppblog/announcing-full-support-for-a-c-c-conformant-preprocessor-in-msvc/
> The _Pragma operator has been one of the long-standing deficiencies of the preprocessor, and a blocker in being standard conformant in C++ and C99.

September 2020: C11 and C17
https://devblogs.microsoft.com/cppblog/c11-and-c17-standard-support-arriving-in-msvc/

Maybe a more practical approach would be to use C99 "except of
features not supported by MSVC of Visual Studio 2019"? In practice, we
can try to support VS 2017, the version currently recommended by the
devguide:
https://devguide.python.org/setup/#windows-compiling

We already have fallbacks for compilers which don't support C99. We
don't have to remove these fallbacks if they are still needed by
popular C compilers. For example, I'm fine with keeping basic
compatbility support with ICC if it remains reasonable, even if we
don't "officially" support ICC (ex: no buildbot).

Victor
--
Night gathers, and now my watch begins. It shall not end until my death.

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/L5TIKLYYAHGCSUUWFHXW4ZBHH2TDIHUB/

[Steven D'Aprano]

On Tue, Feb 08, 2022 at 12:44:46PM +0000, Steve Dower wrote:

> Agreed. CPython should be specific, Python should be as vague as
> possible. Otherwise, we would prevent _by specification_ using Python as
> a scripting language for things where floats may not even be relevant.

I don't think that being *vague* is desirable. Being *permissive* on the
other hand may be.

> It's not even about hardware capabilities (unless you believe Python is
> only for running in a terminal, in which case we'll just have to
> disagree). It could be about an app using short Python snippets as a
> customisation language for some domain-specific task rather than
> inventing a brand new language for it.

I'm sorry, I don't understand what point you are trying to make here.

Regarding "running in a terminal", I don't see how the environment
(terminal, GUI, something else) is relevent.

Regarding your second point, about domain-specific tasks, I don't
understand. If your domain-specific task doesn't need to use floats,
just don't use floats. There's surely no need to invent a whole new
language for some task because your app only needs to do string
processing, say. Have I misunderstood what you are trying to say?


--

Steve
_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/CFLJJN64M4WW6DTRLSGZZTQLU7OHW5OH/

[Steven D'Aprano]

On Mon, Feb 07, 2022 at 05:35:17PM -0800, Gregory P. Smith wrote:

> CPython: yes. we use a double.
> Python the language: no. (float is single precision on many micropython
> platforms as it saves precious ram and performance, plus microcontroller
> fpu hardware like an M4 is usually single precision 32bit)

If we are to *officially* support non-double floats, it would be nice if
sys.float_info were to tell us explicitly how wide the floats are rather
than having to try to reverse engineer it from the other information
there.

A floating point expert can probably look at this:

sys.float_info(max=1.7976931348623157e+308, max_exp=1024,
max_10_exp=308, min=2.2250738585072014e-308, min_exp=-1021,
min_10_exp=-307, dig=15, mant_dig=53, epsilon=2.220446049250313e-16,
radix=2, rounds=1)

and immediately recognise that those values imply a 64-bit float, but I
expect most people will not. If Python the language is going to support
single, double, quad precision floats, and maybe even minifloats with
just 16 or even fewer bits, then can we please add a field to float_info
telling us how many bits the floats have?


--
Steve

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/EUWN4TDI62FK7L6SOG6IAV3AOINDBH6O/

[Steven D'Aprano]

On Mon, Feb 07, 2022 at 06:23:52PM +0000, Mark Dickinson wrote:

> - Should we require IEEE 754 floating-point for CPython-the-implementation?
> - Should we require IEEE 754 floating-point for Python-the-language?

If the answer to those questions are Yes, that rules out using Unums,
posits, sigmoid numbers etc as the builtin float. (The terminology is a
bit vague, sorry.) Do we want that?

https://ieeexplore.ieee.org/document/8080000

https://en.wikipedia.org/wiki/Unum_%28number_format%29

https://github.com/interplanetary-robot/SigmoidNumbers

Posits are hyped as "better than IEEE-754", I have no idea if it is all
hype or if they actually are better or just different.


--
Steve

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/7LUAQ32ZAHKWBJHLHUYEB7I5BZNDXGB7/

[Antoine Pitrou]

On Tue, 08 Feb 2022 01:12:22 -0000
"Jim J. Jewett" <jimjj...@gmail.com> wrote:
> - Should we require the presence of NaNs in order for CPython to build?
> - Should we require IEEE 754 floating-point for CPython-the-implementation?
> - Should we require IEEE 754 floating-point for Python-the-language?
>
> I don't have strong opinions on the first two, but for the language definition, I think the most we should say is "if an implementation does not support IEEE 754 floating-point, this must be mentioned in the documentation as an implementation limit."

This is more about platform support than about implementation support.
An implementation using a C "float" or "double" (or whatever
the equivalent in another implementation language) will de facto support
IEEE 754 on platforms that support it.

Sure, a Python implementation may also choose to rewrite its own
floating-point type from scratch, but that sounds unlikely.

Regards

Antoine.

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/JO6BB3JZBPELKVT5HGNFK2RTQBZ3C2Y7/

[Gregory P. Smith]

On Tue, Feb 8, 2022 at 2:25 PM Steven D'Aprano <st...@pearwood.info> wrote:

On Mon, Feb 07, 2022 at 05:35:17PM -0800, Gregory P. Smith wrote:

> CPython: yes.  we use a double.
> Python the language: no.  (float is single precision on many micropython
> platforms as it saves precious ram and performance, plus microcontroller
> fpu hardware like an M4 is usually single precision 32bit)

If we are to *officially* support non-double floats, it would be nice if
sys.float_info were to tell us explicitly how wide the floats are rather
than having to try to reverse engineer it from the other information
there.

A floating point expert can probably look at this:

sys.float_info(max=1.7976931348623157e+308, max_exp=1024,
max_10_exp=308, min=2.2250738585072014e-308, min_exp=-1021,
min_10_exp=-307, dig=15, mant_dig=53, epsilon=2.220446049250313e-16,
radix=2, rounds=1)

and immediately recognise that those values imply a 64-bit float, but I
expect most people will not. If Python the language is going to support
single, double, quad precision floats, and maybe even minifloats with
just 16 or even fewer bits, then can we please add a field to float_info
telling us how many bits the floats have?

There is no need to know how many bits it is. The meaningful information about precision and accuracy from a math point of view is already expressed in float_info.  the size in bits isn't relevant.  You can derive the size from that if you'd like and are willing to assume a binary format.  binary_bits = ~mant_dig+log2(max_exp)... But that tells you less than sys.float_info.

-gps

[Gregory P. Smith]

On Tue, Feb 8, 2022 at 2:41 PM Steven D'Aprano <st...@pearwood.info> wrote:

On Mon, Feb 07, 2022 at 06:23:52PM +0000, Mark Dickinson wrote:

> - Should we require IEEE 754 floating-point for CPython-the-implementation?
> - Should we require IEEE 754 floating-point for Python-the-language?

If the answer to those questions are Yes, that rules out using Unums,
posits, sigmoid numbers etc as the builtin float. (The terminology is a
bit vague, sorry.) Do we want that?

It does not rule anything else out should they become viable.  This is just a statement that to build cpython we require ieee754 support.  It does not say anything about how our Python float type is implemented internally.

Should a meaningful large-os platform come along that promotes the use of a different format available from C we could make use of that and loosen the policy as needed.

What updating our requirement for CPython would mean is that the likely unexercised outside our own __set_format__ using test suite "unknown_format" code in https://github.com/python/cpython/blob/main/Objects/floatobject.c could go away until such time as a platform with an actual different format springs into existence.

Driveby floatobject.c code inspection: It is odd that we do ieee_be/ieee_le/unknown conditionals as a runtime check rather than configure time check as that means we compile the code for three implementations into our float implementation on all platforms despite them each using only one - I guess that was done for testing purposes presumably in the 1.x era when viable platforms were weirder as standards traction grew - today I'd call that dead code bloat.

-gps

[h.vet...@gmx.com]

> Maybe a more practical approach would be to use C99 "except of
features not supported by MSVC of Visual Studio 2019"?

This could be formulated in a more neutral way by saying

"C99 without the things that became optional in C11", or perhaps
"C11 without optional features" (at least from the POV of MSVC,
haven't checked the other compilers/platforms for C11-compliance).

> In practice, we can try to support VS 2017, the version currently
recommended by the devguide:

That is becoming dated quickly, as Microsoft has deprecated, and is removing,
that version quite rapidly from their CI services (azure/GHA), i.e. mid March, see:
https://github.com/actions/virtual-environments/issues/4312.

It's understandable in the sense that they don't want to support a third version
in addition to vs2022 and vs2019, but the net effect is that very few (open source)
projects will keep using vs2017 going forward.

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/IIZ6LXK2MANUHZAMYSXDF5KPF3VIRKDJ/

[Steven D'Aprano]

On Tue, Feb 08, 2022 at 05:48:46PM -0800, Gregory P. Smith wrote:
> On Tue, Feb 8, 2022 at 2:41 PM Steven D'Aprano <st...@pearwood.info> wrote:

> > If the answer to those questions are Yes, that rules out using Unums,
> > posits, sigmoid numbers etc as the builtin float. (The terminology is a
> > bit vague, sorry.) Do we want that?
>
> It does not rule anything else out should they become viable. This is just
> a statement that to build cpython we require ieee754 support. It does not
> say anything about how our Python float type is implemented internally.

Posits do not implement IEEE-754. They aren't merely a different
internal representation of the IEEE-754 floating point standard, they
implement a different numeric system altogether.

Things that IEEE-754 require, such as signed zero and signed infinity,
are not supported by posits. Posits include a single unsigned zero and a
single unsigned infinity.

Earlier, I made a mistake: I misremembered that posits support a single
NAN, and so I removed Mark's question about requiring NANs from my
quoting. I was wrong to do so: posits do not have any NANs.

So if we require NANs, or IEEE-754, that rules out posits as the builtin
float.

--
Steve
_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/IKJSL3CEL7YZX3XPCJM4SA6BXVFN27Q3/

[Petr Viktorin]

On 09. 02. 22 4:39, h.vet...@gmx.com wrote:
>> Maybe a more practical approach would be to use C99 "except of
> features not supported by MSVC of Visual Studio 2019"?
>
> This could be formulated in a more neutral way by saying
>
> "C99 without the things that became optional in C11", or perhaps

That sounds like a better wording!

> "C11 without optional features" (at least from the POV of MSVC,
> haven't checked the other compilers/platforms for C11-compliance).

That's an interesting idea -- what's keeping us from C11?
In other words: the main thing keeping us from C99 is MSVC support, and
since that compiler apparently skipped C99, should we skip it as well?

>> In practice, we can try to support VS 2017, the version currently
> recommended by the devguide:
>
> That is becoming dated quickly, as Microsoft has deprecated, and is removing,
> that version quite rapidly from their CI services (azure/GHA), i.e. mid March, see:
> https://github.com/actions/virtual-environments/issues/4312.
>
> It's understandable in the sense that they don't want to support a third version
> in addition to vs2022 and vs2019, but the net effect is that very few (open source)
> projects will keep using vs2017 going forward.
> _______________________________________________
> Python-Dev mailing list -- pytho...@python.org
> To unsubscribe send an email to python-d...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/IIZ6LXK2MANUHZAMYSXDF5KPF3VIRKDJ/
> Code of Conduct: http://python.org/psf/codeofconduct/
_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/ERQQFXNAGANR2ND3SRDVCIQZKYTT6OBM/

[Steve Dower]

On 2/8/2022 9:54 PM, Steven D'Aprano wrote:
> Regarding your second point, about domain-specific tasks, I don't
> understand. If your domain-specific task doesn't need to use floats,
> just don't use floats. There's surely no need to invent a whole new
> language for some task because your app only needs to do string
> processing, say. Have I misunderstood what you are trying to say?

Only that some people get very concerned about following "standards",
and we can help them by making our standard easier to follow ("more
permissive," as you suggest, is fine wording by me).

Some people see things that are written down as rules to be followed,
while others see them as guidelines. When we write something down wrong
or loosely, we can cause a surprising amount of angst within the first
group (for example, I fairly frequently get worried questions about the
VS version for building CPython due to how the wiki and devguide mention
it).

I'm just trying to make sure we don't add more of these, by ensuring
that we properly scope things. In this case *CPython requires IEEE 754
floats*, but the Python language does not.

Cheers,

Steve

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/KANDB5WROSPKE7TANO4A75IOD3IDBQZI/

[Jim J. Jewett]

I think you skimmed over "A floating point expert can probably look at this ... "

I remember a time when I just assumed more bits was better, and a later time when I figured smaller was better, and a still later time when I wanted to match the published requirements for bitsize. So that was several years when I didn't really understand the tradeoffs, but could benefit from (or at least write better documentation) knowing the size. During those years, I would have recognized the importance of 1024, but would probably not have bothered interpreting 2.220446049250313.

A method (or docstring) with a more friendly interface would be good.

-jJ

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/R4BW2ZL46Y23UYYQCOSWJ2B3KTSRO5LK/

[Brett Cannon]

On Wed, Feb 9, 2022 at 4:19 AM Petr Viktorin <enc...@gmail.com> wrote:

On 09. 02. 22 4:39, h.vet...@gmx.com wrote:
>> Maybe a more practical approach would be to use C99 "except of
> features not supported by MSVC of Visual Studio 2019"?
>
> This could be formulated in a more neutral way by saying
>
> "C99 without the things that became optional in C11", or perhaps

That sounds like a better wording!

> "C11 without optional features" (at least from the POV of MSVC,
> haven't checked the other compilers/platforms for C11-compliance).

That's an interesting idea -- what's keeping us from C11?

No one asking before, probably because we have been trying to get to C99 for so long. 😄

 

In other words: the main thing keeping us from C99 is MSVC support, and
since that compiler apparently skipped C99, should we skip it as well?

If we think "C11 without optional features" is widely supported then I think that's a fine target to have.

For anyone not sure what's optional in C11, I found https://en.wikipedia.org/wiki/C11_%28C_standard_revision%29#Optional_features . Other than atomics being discussed on Discord for mimalloc, leaving those things out seem reasonable to me.

-Brett

[Brett Cannon]

On Tue, Feb 8, 2022 at 10:20 PM Steven D'Aprano <st...@pearwood.info> wrote:

On Tue, Feb 08, 2022 at 05:48:46PM -0800, Gregory P. Smith wrote:
> On Tue, Feb 8, 2022 at 2:41 PM Steven D'Aprano <st...@pearwood.info> wrote:

> > If the answer to those questions are Yes, that rules out using Unums,
> > posits, sigmoid numbers etc as the builtin float. (The terminology is a
> > bit vague, sorry.) Do we want that?
>
> It does not rule anything else out should they become viable.  This is just
> a statement that to build cpython we require ieee754 support.  It does not
> say anything about how our Python float type is implemented internally.

Posits do not implement IEEE-754. They aren't merely a different
internal representation of the IEEE-754 floating point standard, they
implement a different numeric system altogether.

Things that IEEE-754 require, such as signed zero and signed infinity,
are not supported by posits. Posits include a single unsigned zero and a
single unsigned infinity.

Earlier, I made a mistake: I misremembered that posits support a single
NAN, and so I removed Mark's question about requiring NANs from my
quoting. I was wrong to do so: posits do not have any NANs.

So if we require NANs, or IEEE-754, that rules out posits as the builtin
float.

As Greg said, all we are discussing is requiring NaN and IEEE-754 in order to compile CPython. There's nothing saying we can't change how things are implemented in CPython in the future if something else made sense. And if someone wants to fork CPython to implement something else they can as well. And none of this has any bearing on the language based on the trend of this conversation's answer to Mark's third question. So I am not seeing how any decision we make from this thread about CPython and IEEE-754 rules anything out in the future?

[Inada Naoki]

On Thu, Feb 10, 2022 at 3:49 AM Brett Cannon <br...@python.org> wrote:
> On Wed, Feb 9, 2022 at 4:19 AM Petr Viktorin <enc...@gmail.com> wrote:
>> On 09. 02. 22 4:39, h.vet...@gmx.com wrote:
>>
>> That's an interesting idea -- what's keeping us from C11?
>
> No one asking before, probably because we have been trying to get to C99 for so long. 😄
>
>> In other words: the main thing keeping us from C99 is MSVC support, and
>> since that compiler apparently skipped C99, should we skip it as well?
>
> If we think "C11 without optional features" is widely supported then I think that's a fine target to have.
>
> For anyone not sure what's optional in C11, I found https://en.wikipedia.org/wiki/C11_%28C_standard_revision%29#Optional_features . Other than atomics being discussed on Discord for mimalloc, leaving those things out seem reasonable to me.
>

I like it. I want to use anonymous union. It makes complex structure
like PyDictKeysObject simple a little.

I confirmed that XLC supports it.
https://www.ibm.com/docs/en/xl-c-and-cpp-aix/13.1.3?topic=types-structures-unions#strct__anonstruct

Regards,
--
Inada Naoki <songof...@gmail.com>

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/MMTUDLFIB7ET6T3Q73HDLODDGZY74X54/

[Petr Viktorin]

On 10. 02. 22 0:30, Inada Naoki wrote:
> On Thu, Feb 10, 2022 at 3:49 AM Brett Cannon <br...@python.org> wrote:
>> On Wed, Feb 9, 2022 at 4:19 AM Petr Viktorin <enc...@gmail.com> wrote:
>>> On 09. 02. 22 4:39, h.vet...@gmx.com wrote:
>>>
>>> That's an interesting idea -- what's keeping us from C11?
>>
>> No one asking before, probably because we have been trying to get to C99 for so long. 😄
>>
>>> In other words: the main thing keeping us from C99 is MSVC support, and
>>> since that compiler apparently skipped C99, should we skip it as well?
>>
>> If we think "C11 without optional features" is widely supported then I think that's a fine target to have.
>>
>> For anyone not sure what's optional in C11, I found https://en.wikipedia.org/wiki/C11_%28C_standard_revision%29#Optional_features . Other than atomics being discussed on Discord for mimalloc, leaving those things out seem reasonable to me.
>>
>
> I like it. I want to use anonymous union. It makes complex structure
> like PyDictKeysObject simple a little.
>
> I confirmed that XLC supports it.
> https://www.ibm.com/docs/en/xl-c-and-cpp-aix/13.1.3?topic=types-structures-unions#strct__anonstruct

Ah, I've also wanted anonymous unions in the past!
There's a little problem in that they're not valid in C++, so we can't
have them in public headers.

We'll need to mention C++ if we update the standard.

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/XLRK7P3Q4ANNC4ZEIPIHWJHFKFCWWMRD/

[Victor Stinner]

On Thu, Feb 10, 2022 at 10:28 AM Petr Viktorin <enc...@gmail.com> wrote:
> Ah, I've also wanted anonymous unions in the past!
> There's a little problem in that they're not valid in C++, so we can't
> have them in public headers.
>
> We'll need to mention C++ if we update the standard.

IMO we only have to care about C++ in Include/ header files: make sure
that "Python.h" is usable in C++.

Outside this directory, we don't have to care about C++ compatibility.

Victor
--
Night gathers, and now my watch begins. It shall not end until my death.

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/AYRNS2PGB6MOUPEY5IJXFMGEVW4T5L6E/

[Inada Naoki]

On Thu, Feb 10, 2022 at 6:31 PM Petr Viktorin <enc...@gmail.com> wrote:
>
> >
> > I like it. I want to use anonymous union. It makes complex structure
> > like PyDictKeysObject simple a little.
> >
> > I confirmed that XLC supports it.
> > https://www.ibm.com/docs/en/xl-c-and-cpp-aix/13.1.3?topic=types-structures-unions#strct__anonstruct
>
> Ah, I've also wanted anonymous unions in the past!
> There's a little problem in that they're not valid in C++, so we can't
> have them in public headers.
>

C++ 11 supports anonymous union with some reasonable limitations.
https://en.cppreference.com/w/cpp/language/union

XL C/C++ also support it. So we can use it if we decided to use it.

Regards,

--
Inada Naoki <songof...@gmail.com>

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/GE3RFUXUDFY3GZQHFEVZAIQW3CCMLFK7/

[Victor Stinner]

IMO we need to distinguish the public C API which should be as much
compatible as possible, target the oldest C standard, and the Python
internals can require a more recent C standard.

For example, today maybe it's reasonable to requires C99 for Include/
headers (public .h files) and support C11 for Python internals
(private .c files).

Victor

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/LG5WBBU5JK44IO6UCKSQEUVCJ55X6ZBE/

[Antoine Pitrou]

On Wed, 09 Feb 2022 03:39:42 -0000
h.vet...@gmx.com wrote:
>
> That is becoming dated quickly, as Microsoft has deprecated, and is removing,
> that version quite rapidly from their CI services (azure/GHA), i.e. mid March, see:
> https://github.com/actions/virtual-environments/issues/4312.
>
> It's understandable in the sense that they don't want to support a third version
> in addition to vs2022 and vs2019, but the net effect is that very few (open source)
> projects will keep using vs2017 going forward.

Not every project using CPython is an open source project, though.
And CPython probably supports more platforms than GHA or Azure do.

Regards

Antoine.

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/FRC6L4QON4TPBPSJCOUSJNRWHLBS5LCB/

[Victor Stinner]

Hi,

I updated my PEP 7 PR to use C99 in the public C API and "a subset of"
C11 in Python internals:

"Python 3.11 and newer versions use C99 in the public C API and use a
subset of C11 in Python internals. The public C API should be
compatible with C++. The C11 subset are features supported by GCC 8.5,
clang 8.0, and MSVC of Visual Studio 2017."

https://github.com/python/peps/pull/2309/files

GCC 8.5 is the version chosen by RHEL 8. It should provide C11
features that we care about.

I pickled clang 8.0 because it's had been released in 2019 and so
should be available on most operating systems. FreeBSD uses clang by
default. FreeBSD 13 uses clang 11.

And it seems like we still care about support Visual Studio 2017, even
if Visual Studio 2019 and 2022 are available.

I chose to not require supporting AIX XLC. Inada-san wrote: "xlclang
fully supports C89/C99/C11. xlc fully supports C89/C99, and partially
supports C11." I guess that in practice, we can test a PR on buildbots
when trying "new shiny" C11 feature.

Moreover, if a C11 feature is missing, it's usually not too
complicated to use a workaround for C99 and older.

Victor

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/5IUV6IEB2ARZ4R6MCVK7FZFKRLH7M5NV/

[h.vet...@gmx.com]

> And it seems like we still care about support Visual Studio 2017, even
> if Visual Studio 2019 and 2022 are available.

Can we make this more concrete? Do we know of affected parties? Numbers
of affected users? Or are we just conservatively guesstimating the thickness
of the long tail of support?

On top of that, I think the formulation you chose does not map correctly to
actual compiler capabilities. C99 (minus C11 optionals) only arrived in VS2019,
there were still gaps in VS2017.

I would advocate bumping the requirement to VS2019. Yes, there is a built-in
reluctance to update toolchains, but MSFT has been _extremely_ conservative
w.r.t. ABI-compatibility (e.g. the whole story with [[msvc::no_unique_address]]),
and I just don't see the argument why a non-negligible portion of users cannot
upgrade to the fully-compatible-with-everything-previously VS2019.

> "Python 3.11 and newer versions use C99 in the public C API and use a
> subset of C11 in Python internals. The public C API should be
> compatible with C++. The C11 subset are features supported by GCC 8.5,
> clang 8.0, and MSVC of Visual Studio 2017."

If we were to require VS2019, then the formulation could be much nicer (IMO):
"""
Python 3.11 and newer versions use C99 in the public C API (without those
features that became optional in C11), while Python internals may use C11
(again without optional features). The public C API should be compatible with C++.

"""
_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/PAIVGORIDRIGLPCSQI5KN6U6CKFTBHPX/

[Victor Stinner]

Ok, let me try something simpler:

"Python 3.11 and newer versions use C11 without optional features. The

public C API should be compatible with C++."

https://github.com/python/peps/pull/2309/files

Victor

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/C32XCK5WHMLJNV6RGLMN2XNBDWR4DI3V/

[Barry]

> On 24 Feb 2022, at 11:45, Victor Stinner <vsti...@python.org> wrote:
>
> Ok, let me try something simpler:

>
> "Python 3.11 and newer versions use C11 without optional features. The
> public C API should be compatible with C++."
> https://github.com/python/peps/pull/2309/files

Should is often read as meaning optional when writing specs.
Can you say “must be compatible with C++”.

Barry

>
> Victor
> _______________________________________________
> Python-Dev mailing list -- pytho...@python.org
> To unsubscribe send an email to python-d...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/C32XCK5WHMLJNV6RGLMN2XNBDWR4DI3V/
> Code of Conduct: http://python.org/psf/codeofconduct/
>

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/AKW6Y5TIHWIZKCSQD4I6GD4Q7GZQYUZ7/

[Victor Stinner]

On Thu, Feb 24, 2022 at 11:10 PM Barry <ba...@barrys-emacs.org> wrote:
> > "Python 3.11 and newer versions use C11 without optional features. The
> > public C API should be compatible with C++."
> > https://github.com/python/peps/pull/2309/files
>
> Should is often read as meaning optional when writing specs.
> Can you say “must be compatible with C++”.

I plan to attempt to write an actual test for that, rather than a
vague sentence in a PEP. For now, "should" is a deliberate choice: I
don't know exactly which C++ version should be targeted and if it's
really an issue or not.

For example, C++20 reserves the "module" keyword, whereas Python uses
it in its C API. Example:

PyAPI_FUNC(int) PyModule_AddType(PyObject *module, PyTypeObject *type);

See:

* https://bugs.python.org/issue39355
* https://github.com/pythoncapi/pythoncapi_compat/issues/21

--

I made a change in the datatable project to add Python 3.11 support
using the pythoncapi_compat.h header file. Problem: this *C* header
file produced new warnings in datatable extension module which with
built with a C++ compiler:
https://github.com/h2oai/datatable/pull/3231#issuecomment-1032864790

Examples:

| src/core/lib/pythoncapi_compat.h:272:52: warning: zero as null
pointer constant [-Wzero-as-null-pointer-constant]
| || tstate->c_profilefunc != NULL);
| ^~~~
| nullptr

and

| src/core/lib/pythoncapi_compat.h:170:12: warning: use of old-style
cast [-Wold-style-cast]
| return (PyCodeObject *)_Py_StealRef(PyFrame_GetCode(frame));
| ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

I made pythoncapi_compat.h compatible with C++ (fix C++ compiler
warnings) by using nullptr and reinterpret_cast<TYPE>(EXPR) cast if
the __cplusplus macro is defined, or NULL and ((TYPE)(EXPR)) cast
otherwise.

datatable also uses #include "Python.h". I don't know there were only
C++ compiler warnings on "pythoncapi_compat.h". Maybe because
datatable only uses static inline functions from
"pythoncapi_compat.h", but it may also emit the same warnings if
tomorrow some static inline functions of "Python.h" are used.

For now, I prefer to put a reminder in PEP 7 that the "Python.h" C API
is consumed by C++ projects.

Victor
--
Night gathers, and now my watch begins. It shall not end until my death.

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/RGNBM5CSUPBQSTZND4PHEV3WUEKS36TP/

[Victor Stinner]

The consensus is to require IEEE 754 to build CPython, but not require
it in the Python language specification.

Updates (changed merged in bpo-46656):

* Building Python 3.11 now requires a C11 compiler without optional
C11 features. I wrote it in What's New in Python 3.11 and the PEP 7.
* Building Python 3.11 now requires support for floating point
Not-a-Number (NaN): remove the Py_NO_NAN macro.

Victor

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/EP625OXOLAQ3DSGFWICYAGVKAPWDSF2V/

[Gregory P. Smith]

On Thu, Feb 24, 2022 at 3:27 PM Victor Stinner <vsti...@python.org> wrote:

On Thu, Feb 24, 2022 at 11:10 PM Barry <ba...@barrys-emacs.org> wrote:
> > "Python 3.11 and newer versions use C11 without optional features. The
> > public C API should be compatible with C++."
> > https://github.com/python/peps/pull/2309/files
>
> Should is often read as meaning optional when writing specs.
>  Can you say “must be compatible with C++”.

I plan to attempt to write an actual test for that, rather than a
vague sentence in a PEP. For now, "should" is a deliberate choice: I
don't know exactly which C++ version should be targeted and if it's
really an issue or not.

Agreed.  "should" is good because we're not even clear if we currently actually comply with C++ standards.  i.e. https://bugs.python.org/issue40120 suggests we technically may not for C++ (it is not strictly a superset of C as we all like to pretend), though for practical purposes regardless of standards compilers tend to allow that.

We're likely overspecifying in any document we create about what we require because the only definition any of us are actually capable of making for what we require is "does it compile with this compiler on this platform? If yes, then we appear to support it. can we guarantee that? only with buildbots or other CI" - We're generally not versed in specific language standards (aside from compiler folks, who is?), and compilers don't comply strictly with all the shapes of those anyways for either practical or hysterical reasons. So no matter what we claim to aspire to, reality is always murkier.  A document about requirements is primarily useful to give guidance to what we expect to be aligned with and what is or isn't allowed to be used in new code.  Our code itself always has the final say.

-gps

[Steve Dower]

On 23Feb2022 2240, h.vet...@gmx.com wrote:
> On top of that, I think the formulation you chose does not map correctly to
> actual compiler capabilities. C99 (minus C11 optionals) only arrived in VS2019,
> there were still gaps in VS2017.

Additionally, Visual Studio version doesn't directly map to the compiler
version. For example, VS2022 contains multiple older versions of the
compiler.

Currently we use the v142 toolset for releases (selected by default in
desktop development workloads in VS 2019), and can't migrate to v143
(selected by default in VS 2022) until some compiler bugs are fixed. If
we're going to specify anything about MSVC, I'd prefer it was the v142
label rather than the Visual Studio version (though we can include the
latter as a convenience for the reader, and hopefully clearly enough
that people stop filing bug reports about it...)

Cheers,
Steve

_______________________________________________
Python-Dev mailing list -- pytho...@python.org
To unsubscribe send an email to python-d...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/

Message archived at https://mail.python.org/archives/list/pytho...@python.org/message/KJ5XCX7IOV6GR2MWPXV6TNJ3P5G5NLSW/