Minimum Python version to 3.8
113 views
Skip to first unread message
Travis Scrimshaw
May 10, 2021, 7:12:48 PM5/10/21
to sage-devel
On #30423, which is getting close to completion, we will be using multiprocessing.shared_memory, which is only available on Python 3.8+. However, right now we are at least allowing Python 3.7 (as per the patchbot). So my main proposal would be to bump the minimum required Python version to 3.8, which was released Oct. 14, 2019.
On that ticket, we can make it so that the main entry point runs things in serial if the Python version is sufficiently small and that the doc builds, but the doctests for the parallel code will fail. So the first alternative option would be to mark certain doctests (or the file) as needing a large Python version.
A second alternative is that this can be split off as a separate package (either an optional Sage package or pip installable). Yet, it is somewhat tightly coupled with the FusionRing code (and root systems) in Sage, so this is not so desirable. A less invasive way would be to just split the parallel part off, but this would take some work to do I think.
What do people think?
Best,
Travis
Travis
Dima Pasechnik
May 10, 2021, 7:20:00 PM5/10/21
to sage-devel
On Tue, May 11, 2021 at 12:12 AM 'Travis Scrimshaw' via sage-devel
<sage-...@googlegroups.com> wrote:
>
> On #30423, which is getting close to completion, we will be using multiprocessing.shared_memory, which is only available on Python 3.8+. However, right now we are at least allowing Python 3.7 (as per the patchbot). So my main proposal would be to bump the minimum required Python version to 3.8, which was released Oct. 14, 2019.
>
> On that ticket, we can make it so that the main entry point runs things in serial if the Python version is sufficiently small and that the doc builds, but the doctests for the parallel code will fail. So the first alternative option would be to mark certain doctests (or the file) as needing a large Python version.
As 3.7 still has 2 years to go before EOL, we'd rather keep it - I'd
<sage-...@googlegroups.com> wrote:
>
> On #30423, which is getting close to completion, we will be using multiprocessing.shared_memory, which is only available on Python 3.8+. However, right now we are at least allowing Python 3.7 (as per the patchbot). So my main proposal would be to bump the minimum required Python version to 3.8, which was released Oct. 14, 2019.
>
> On that ticket, we can make it so that the main entry point runs things in serial if the Python version is sufficiently small and that the doc builds, but the doctests for the parallel code will fail. So the first alternative option would be to mark certain doctests (or the file) as needing a large Python version.
just say tag the doctests with bigger versions of Python if needed.
>
> A second alternative is that this can be split off as a separate package (either an optional Sage package or pip installable). Yet, it is somewhat tightly coupled with the FusionRing code (and root systems) in Sage, so this is not so desirable. A less invasive way would be to just split the parallel part off, but this would take some work to do I think.
>
> What do people think?
>
> Best,
> Travis
>
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/e9bd34ce-3fac-4737-a8f8-a5203f2fd87en%40googlegroups.com.
Travis Scrimshaw
May 10, 2021, 7:51:54 PM5/10/21
to sage-devel
Do we have a doctest tag for a minimum python version?
Best,
Travis
Dima Pasechnik
May 10, 2021, 7:55:05 PM5/10/21
to sage-devel
On Tue, 11 May 2021, 00:51 'Travis Scrimshaw' via sage-devel, <sage-...@googlegroups.com> wrote:
Do we have a doctest tag for a minimum python version?
make your own, say:
# optional: py38
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/a353d199-5bb3-4fcf-92af-b301f48f9ed1n%40googlegroups.com.
Matthias Koeppe
May 10, 2021, 8:40:04 PM5/10/21
to sage-devel
-1. Even NEP 29 (https://numpy.org/neps/nep-0029-deprecation_policy.html) does not drop Python 3.7 support before end of the year.
Volker Braun
May 11, 2021, 5:10:53 PM5/11/21
to sage-devel
Yet another possibility is to look for a backport that implements sufficient functionality for your needs for now.
Travis Scrimshaw
May 12, 2021, 8:34:50 PM5/12/21
to sage-devel
Own tag might be a goood way forward as the code itself can run on Python 3.7 by avoiding the multiprocessing. It would be purely to state that we don't want to test the mp parts of the code. How do I make an optional tag? I am worried that since this is not simply an optional package to test but against a Python version, it would require a slightly invasive change to the doctesting framework. There also is an option of testing for a more specific import from Python too.
Best,
Travis
Travis
David Roe
May 13, 2021, 2:16:54 AM5/13/21
to sage-devel
One way to do it is to modify the auto_optional_tags variable in sage.doctest.control lines 47-54 to add a more specific python version tag. This doesn't get you inequalities in versions though.
You could also just add an if statement around your test if there are few enough of them (e.g. int(sys.version.split('.')[1]) >= 8).
David
--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/ae100fbf-8b77-4c9c-b92a-cde41e6d17f8n%40googlegroups.com.
jonatha...@googlemail.com
May 13, 2021, 4:27:53 AM5/13/21
to sage-devel
It looks to me as if all of this ticket can be "fixed" to work with python 3.7 by just trying to import shared_memory.
`start_worker_pool` should probably raise a proper error message, if the import of shared_memory failed previosly.
The few doctests that explicitly call `start_worker_pool` can do something as:
sage: if sys.version_info[:2] != (3, 7): f.start_worker_pool() # requires python 3.8+
Also `use_mp=True` could be ignored on python 3.7 (and this should be documented).
Jonathan
Dima Pasechnik
May 13, 2021, 6:09:25 AM5/13/21
to sage-devel
On Thu, May 13, 2021 at 1:34 AM 'Travis Scrimshaw' via sage-devel
<sage-...@googlegroups.com> wrote:
>
> Own tag might be a goood way forward as the code itself can run on Python 3.7 by avoiding the multiprocessing. It would be purely to state that we don't want to test the mp parts of the code. How do I make an optional tag? I am worried that since this is not simply an optional package to test but against a Python version, it would require a slightly invasive change to the doctesting framework.
I always thought that it's trivial:
<sage-...@googlegroups.com> wrote:
>
> Own tag might be a goood way forward as the code itself can run on Python 3.7 by avoiding the multiprocessing. It would be purely to state that we don't want to test the mp parts of the code. How do I make an optional tag? I am worried that since this is not simply an optional package to test but against a Python version, it would require a slightly invasive change to the doctesting framework.
sage: bar() # optional:foo
would stop bar() being tested, unless "--optional=...,foo,..."
is given to ./sage -t
Am I wrong?
Dima
> There also is an option of testing for a more specific import from Python too.
>
> Best,
> Travis
>
> On Wednesday, May 12, 2021 at 7:10:53 AM UTC+10 Volker Braun wrote:
>>
>> Yet another possibility is to look for a backport that implements sufficient functionality for your needs for now.
>>
>> On Tuesday, May 11, 2021 at 2:40:04 AM UTC+2 Matthias Koeppe wrote:
>>>
>>> -1. Even NEP 29 (https://numpy.org/neps/nep-0029-deprecation_policy.html) does not drop Python 3.7 support before end of the year.
>>>
>>>
>>> On Monday, May 10, 2021 at 4:12:48 PM UTC-7 Travis Scrimshaw wrote:
>>>>
>>>> On #30423, which is getting close to completion, we will be using multiprocessing.shared_memory, which is only available on Python 3.8+. However, right now we are at least allowing Python 3.7 (as per the patchbot). So my main proposal would be to bump the minimum required Python version to 3.8, which was released Oct. 14, 2019.
>>>>
>>>> On that ticket, we can make it so that the main entry point runs things in serial if the Python version is sufficiently small and that the doc builds, but the doctests for the parallel code will fail. So the first alternative option would be to mark certain doctests (or the file) as needing a large Python version.
>>>>
>>>> A second alternative is that this can be split off as a separate package (either an optional Sage package or pip installable). Yet, it is somewhat tightly coupled with the FusionRing code (and root systems) in Sage, so this is not so desirable. A less invasive way would be to just split the parallel part off, but this would take some work to do I think.
>>>>
>>>> What do people think?
>>>>
>>>> Best,
>>>> Travis
>>>>
David Roe
May 13, 2021, 11:52:53 AM5/13/21
to sage-devel
On Thu, May 13, 2021 at 6:09 AM Dima Pasechnik <dim...@gmail.com> wrote:
On Thu, May 13, 2021 at 1:34 AM 'Travis Scrimshaw' via sage-devel
<sage-...@googlegroups.com> wrote:
>
> Own tag might be a goood way forward as the code itself can run on Python 3.7 by avoiding the multiprocessing. It would be purely to state that we don't want to test the mp parts of the code. How do I make an optional tag? I am worried that since this is not simply an optional package to test but against a Python version, it would require a slightly invasive change to the doctesting framework.
I always thought that it's trivial:
sage: bar() # optional:foo
would stop bar() being tested, unless "--optional=...,foo,..."
is given to ./sage -t
Am I wrong?
You're right. I was just suggesting a mechanism where the python version could be detected in the doctest code and you wouldn't need to include "--optional=py38".
David
Dima
> There also is an option of testing for a more specific import from Python too.
>
> Best,
> Travis
>
> On Wednesday, May 12, 2021 at 7:10:53 AM UTC+10 Volker Braun wrote:
>>
>> Yet another possibility is to look for a backport that implements sufficient functionality for your needs for now.
>>
>> On Tuesday, May 11, 2021 at 2:40:04 AM UTC+2 Matthias Koeppe wrote:
>>>
>>> -1. Even NEP 29 (https://numpy.org/neps/nep-0029-deprecation_policy.html) does not drop Python 3.7 support before end of the year.
>>>
>>>
>>> On Monday, May 10, 2021 at 4:12:48 PM UTC-7 Travis Scrimshaw wrote:
>>>>
>>>> On #30423, which is getting close to completion, we will be using multiprocessing.shared_memory, which is only available on Python 3.8+. However, right now we are at least allowing Python 3.7 (as per the patchbot). So my main proposal would be to bump the minimum required Python version to 3.8, which was released Oct. 14, 2019.
>>>>
>>>> On that ticket, we can make it so that the main entry point runs things in serial if the Python version is sufficiently small and that the doc builds, but the doctests for the parallel code will fail. So the first alternative option would be to mark certain doctests (or the file) as needing a large Python version.
>>>>
>>>> A second alternative is that this can be split off as a separate package (either an optional Sage package or pip installable). Yet, it is somewhat tightly coupled with the FusionRing code (and root systems) in Sage, so this is not so desirable. A less invasive way would be to just split the parallel part off, but this would take some work to do I think.
>>>>
>>>> What do people think?
>>>>
>>>> Best,
>>>> Travis
>>>>
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/ae100fbf-8b77-4c9c-b92a-cde41e6d17f8n%40googlegroups.com.
--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAAWYfq1z28PNZeR%3Dn5n4cNX4crBqYcNEnV9EvThH_Oa-cSiaeA%40mail.gmail.com.
Dima Pasechnik
May 13, 2021, 12:06:55 PM5/13/21
to sage-devel
On Thu, May 13, 2021 at 4:52 PM David Roe <roe...@gmail.com> wrote:
>
>
>
> On Thu, May 13, 2021 at 6:09 AM Dima Pasechnik <dim...@gmail.com> wrote:
>>
>> On Thu, May 13, 2021 at 1:34 AM 'Travis Scrimshaw' via sage-devel
>> <sage-...@googlegroups.com> wrote:
>> >
>> > Own tag might be a goood way forward as the code itself can run on Python 3.7 by avoiding the multiprocessing. It would be purely to state that we don't want to test the mp parts of the code. How do I make an optional tag? I am worried that since this is not simply an optional package to test but against a Python version, it would require a slightly invasive change to the doctesting framework.
>>
>> I always thought that it's trivial:
>>
>> sage: bar() # optional:foo
>>
>> would stop bar() being tested, unless "--optional=...,foo,..."
>> is given to ./sage -t
>>
>> Am I wrong?
>
>
> You're right. I was just suggesting a mechanism where the python version could be detected in the doctest code and you wouldn't need to include "--optional=py38".
As a hack we can have an optional empty package "py38", which is
>
>
>
> On Thu, May 13, 2021 at 6:09 AM Dima Pasechnik <dim...@gmail.com> wrote:
>>
>> On Thu, May 13, 2021 at 1:34 AM 'Travis Scrimshaw' via sage-devel
>> <sage-...@googlegroups.com> wrote:
>> >
>> > Own tag might be a goood way forward as the code itself can run on Python 3.7 by avoiding the multiprocessing. It would be purely to state that we don't want to test the mp parts of the code. How do I make an optional tag? I am worried that since this is not simply an optional package to test but against a Python version, it would require a slightly invasive change to the doctesting framework.
>>
>> I always thought that it's trivial:
>>
>> sage: bar() # optional:foo
>>
>> would stop bar() being tested, unless "--optional=...,foo,..."
>> is given to ./sage -t
>>
>> Am I wrong?
>
>
> You're right. I was just suggesting a mechanism where the python version could be detected in the doctest code and you wouldn't need to include "--optional=py38".
installed whenever Sage's Python is newer than python 3.7.* :-)
Travis Scrimshaw
May 14, 2021, 7:18:58 PM5/14/21
to sage-devel
Thank you for the pointer for how to add the optional marker. Adding a dummy spkg for when the Python version is sufficiently large feels like a much more invasive hack around by going into the build system. ;)
Jonathan, yes, that is how we can fix the code. The problem as I recall is the doctests for the code that explicitly tests the parallel code (e.g. the shm_managers). Perhaps those can all be avoided in the same way as a workaround without modifying the actual doctesting framework. This could be a good solution as well. I will think about it and discuss it with the authors.
Best,
Travis
Dima Pasechnik
May 15, 2021, 6:35:53 AM5/15/21
to sage-devel
On Sat, May 15, 2021 at 12:19 AM 'Travis Scrimshaw' via sage-devel
<sage-...@googlegroups.com> wrote:
>
> Thank you for the pointer for how to add the optional marker. Adding a dummy spkg for when the Python version is sufficiently large feels like a much more invasive hack around by going into the build system. ;)
no, why, creating a dummy package like this is (almost) declarative
<sage-...@googlegroups.com> wrote:
>
> Thank you for the pointer for how to add the optional marker. Adding a dummy spkg for when the Python version is sufficiently large feels like a much more invasive hack around by going into the build system. ;)
programming, as opposed to actually hacking doctest code.
I forgot, there were script packages at some point, are they gone now?
Reply all
Reply to author
Forward
0 new messages