Can we restore `mean`?

[Samuel Lelievre]

Dear sage-devel,

Taking averages is a common operation, and a `mean` function
such that `mean(xx)` returns `sum(xx) / len(xx)` regardless of
the type of objects in the iterable `xx` is extremely convenient.

For instance, for a polygon whose vertices `uu` have coordinates
in a number field and are represented as vectors over that field,
`mean(uu)` finds the centre of that polygon. To center the polygon
at the origin, use `c = mean(uu)` and `vv = [u - c for u in uu]`.

In Sage 9.5, using `mean` displays a warning:
```
DeprecationWarning: sage.stats.basic_stats.mean is deprecated;
use numpy.mean or numpy.nanmean instead
```
This is after #29662 was merged in SageMath 9.5.beta1.

Alas, `numpy.mean` cannot find the mean of a list of vectors
over a number field.

Of course, as a workaround, I can define

def mean(xx):
    r"""
    Return the mean of this iterable.
    """
    return sum(xx) / len(xx)

and place that in an `init.sage` file in my `~/.sage` folder.

Having that built into Sage is so much more convenient though.
Can we have it back?

Kind polygonal regards,  --Samuel

[William Stein]

This happened in https://trac.sagemath.org/ticket/29662, as requested
by kcrisman. I just looked at that
ticket and added a comment about several additional examples where
deprecating mean breaks things
in subtle ways...

> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/60fe4b1c-db88-4caa-8d13-670b9b3edb79n%40googlegroups.com.



--
William (http://wstein.org)

[kcrisman]

My original ticket title was "Clarify stats module role", not deprecate.   "This ticket is to split the more technical stuff (which presumably may still be used for researchers, but not for the sort of things basic R or pandas data frames would be) into a separate module where it can be taken care of."  The whole notion of removing something as basic as "mean" from global namespace is not something I would support.

However, that does bring up a broader issue of who the end users of Sage are supposed to be.  I understand the idea of Sage solely as a Python library like Sympy or whatever.  However, not just historically have we behaved otherwise, but it seems rather opposed to the mission statement of Sage.  Our goal isn't to be "best standard Python library for math" but "open-source competitor to ..." and they are stand-alone programs, by design and rightly so. (This could include having a pip-installable version, obviously.) This question is naturally not directly part of this thread, but it seems to come up fairly often.  As for me, too many memories of needing warnings in Maple like "Did you remember to load the plots package via the with(plots): command?"   Nils has a good quote on the ticket, "A python library is just not a very good match for what people expect from an interactive CAS, so I think it's good we have a shim layer that makes sagemath behave a little more like a traditional CAS."

[Matthias Koeppe]

IMO the best long-term solution is to make sure that the built-in stats module (https://docs.python.org/3/library/statistics.html) can be used. However, as explained in the ticket https://trac.sagemath.org/ticket/29662 (and https://trac.sagemath.org/ticket/28234), there are bugs that prevent users from using it with some Sage types. 

[David Roe]

The documentation of Python's statistics module notes that "Unless explicitly noted, these functions support int, float, Decimal and Fraction. Behaviour with other types (whether in the numeric tower or not) is currently unsupported. Collections with a mix of types are also undefined and implementation-dependent."

https://trac.sagemath.org/ticket/28234 is not easy to fix.  Even if a long term goal is being able to use the built in stats module (which may or may not be a good idea given the disclaimer above), I think we should add mean back in.

David

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/72970d04-877d-41bb-8af5-7dce382f887bn%40googlegroups.com.

[Matthias Koeppe]

On Sunday, February 13, 2022 at 12:58:41 PM UTC-8 David Roe wrote:

The documentation of Python's statistics module notes that "Unless explicitly noted, these functions support int, float, Decimal and Fraction. Behaviour with other types (whether in the numeric tower or not) is currently unsupported. Collections with a mix of types are also undefined and implementation-dependent."

Yes, they need the Sage community's input to develop this further.

 

 

[Vincent Delecroix]

Please review #33453 that does restore a mean and median into the global
namespace.

[David Roe]

I went to take a look at it, and clicking on the branch yields

"Failure to create temporary git repository clone for merge preview of 0a44a4fb8c05d379fc472fcccc86bb6e099b22d0: Cloning into '/tmp/tmpZRzotQ'... error: copy-fd: write returned No space left on device fatal: failed to copy file to '/tmp/tmpZRzotQ/.git/objects/pack/pack-16e470024f6802c73d1da2cea1ce07a05048f419.pack': No space left on device fatal: The remote end hung up unexpectedly."

Is the trac virtual machine full of logs?

David

--
You received this message because you are subscribed to the Google Groups "sage-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/ccaf3ecf-a116-f533-4d79-18f344719c4c%40gmail.com.

[Dima Pasechnik]

On Thu, Mar 3, 2022 at 10:00 AM David Roe <roed...@gmail.com> wrote:
>
> I went to take a look at it, and clicking on the branch yields
>
> "Failure to create temporary git repository clone for merge preview of 0a44a4fb8c05d379fc472fcccc86bb6e099b22d0: Cloning into '/tmp/tmpZRzotQ'... error: copy-fd: write returned No space left on device fatal: failed to copy file to '/tmp/tmpZRzotQ/.git/objects/pack/pack-16e470024f6802c73d1da2cea1ce07a05048f419.pack': No space left on device fatal: The remote end hung up unexpectedly."
>
> Is the trac virtual machine full of logs?

It is - 14Gb of apache logs. Trying to clean carefully now.

> David
>
> On Thu, Mar 3, 2022 at 4:57 AM Vincent Delecroix <20100.d...@gmail.com> wrote:
>>
>> Le 13/02/2022 à 22:21, Matthias Koeppe a écrit :
>> > On Sunday, February 13, 2022 at 12:58:41 PM UTC-8 David Roe wrote:
>> >
>> >> The documentation of Python's statistics module notes that "Unless
>> >> explicitly noted, these functions support int, float, Decimal and Fraction.
>> >> Behaviour with other types (whether in the numeric tower or not) is
>> >> currently unsupported. Collections with a mix of types are also undefined
>> >> and implementation-dependent."
>> >>
>> >
>> > Yes, they need the Sage community's input to develop this further.
>>
>> Please review #33453 that does restore a mean and median into the global
>> namespace.
>>
>> --
>> You received this message because you are subscribed to the Google Groups "sage-devel" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.
>> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/ccaf3ecf-a116-f533-4d79-18f344719c4c%40gmail.com.
>
> --
> You received this message because you are subscribed to the Google Groups "sage-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sage-devel/CAChs6_nhZwxSKZjcOY9hg3A5DBo0xh2-iTXQbuLZ9pdsGJ1GRQ%40mail.gmail.com.

[Dima Pasechnik]

On Thu, Mar 3, 2022 at 10:19 AM Dima Pasechnik <dim...@gmail.com> wrote:
>
> On Thu, Mar 3, 2022 at 10:00 AM David Roe <roed...@gmail.com> wrote:
> >
> > I went to take a look at it, and clicking on the branch yields
> >
> > "Failure to create temporary git repository clone for merge preview of 0a44a4fb8c05d379fc472fcccc86bb6e099b22d0: Cloning into '/tmp/tmpZRzotQ'... error: copy-fd: write returned No space left on device fatal: failed to copy file to '/tmp/tmpZRzotQ/.git/objects/pack/pack-16e470024f6802c73d1da2cea1ce07a05048f419.pack': No space left on device fatal: The remote end hung up unexpectedly."
> >
> > Is the trac virtual machine full of logs?
>
> It is - 14Gb of apache logs. Trying to clean carefully now.

should be OK now.