Proposal: A New Statistics Module for SageMath

[Dubey Vivek santoshkumar]

Dear Sage Community,

I hope this message finds you well. I am writing to initiate a discussion about a project I am deeply interested in: creating a new, comprehensive statistics module for SageMath.

I have researched this topic in detail. After going through the relevant tickets and discussions—particularly #29662 (deprecation of sage.stats.basic_stats), #33432 (restoration of basic stats commands), and #33453 (the stalled statistics restoration effort)—it is clear that the community has been aware of this gap for a long time. The module was deprecated because it was underdeveloped and inconsistent (for example, mean worked with vectors but variance did not). The restoration efforts have not progressed, and the official recommendation is now to use numpy, scipy.stats, or pandas instead.

This brings me to the question: can we do better?

Why This Matters Now

We are in the middle of an Artificial Intelligence, Machine Learning, Deep Learning, NLP, and related fields revolution. These fields are fundamentally built on statistics and probability. As students, educators, and researchers use SageMath to understand the mathematics behind algorithms, they often have to leave its core environment for statistical work. A native module would allow users to combine SageMath's symbolic power with modern statistical methods, all within a single, integrated environment.

SageMath is widely used as a calculator, a research tool, and a teaching platform. Having a robust statistics module would make it even more valuable for the AI, ML, DL, and NLP community.

The Proposal: A New Module

I propose building a new, well-designed statistics module for SageMath. This would not attempt to revive or patch sage.stats.basic_stats. Instead, it would be a clean, modern library built from the ground up, designed to work seamlessly with SageMath's existing infrastructure.

The module would aim to cover:

- Descriptive statistics (mean, median, variance, standard deviation, etc.)
- Probability distributions (Normal, Beta, Binomial, Poisson, etc.) with symbolic support
- Statistical tests (t-test, chi-square, ANOVA, etc.)
- Regression models (linear, logistic)
- Time series analysis

This would leverage SageMath's symbolic engine and the existing RealDistribution class in sage.probability, which is still maintained and provides a solid foundation.

Links for Reference

- Deprecation of sage.stats.basic_stats: #29662
- Restoration of basic stats commands (discussion): #33432
- Stalled statistics restoration effort: #33453
- Python statistics module incompatibility with Sage numbers: #28234

Request for Feedback

Before I invest time in a detailed design document, I would like to hear the community's thoughts:

- Is there interest in such a module?
- Are there specific features that should be prioritized?
- Would the community be willing to review a detailed proposal?

Thank you for your time and consideration.

Vivek Dubey