HyperLogLog Over RSA

[Shahriar Iravanian]

I came across this proposal (https://karpinski.org/HyperLogLogOverRSA.jl/dev/01-executive-summary/) from one of the creators of the Julia programming language, which may have some relevance to Sympy (or more broadly, the scientific Python community).  

The basic idea is how to solve the problem of tracking the installation and usage of our packages. For example, I would love to know how many copies of Symjit are installed and what the distributions of Python versions, Sympy versions, CPU types, and OS types are. I think the standard way to do so is to use telemetry. However, background telemetry is ethically dubious, susceptible to various security and privacy challenges, and probably illegal, at least in some jurisdictions. Opt-in telemetry is somewhat better, but then the data is less useful. The proposal attempts to solve this problem using cryptographic protocols to ensure users' privacy and security while providing useful statistics (but not individual users' identities) to the package writers. However, this would work only if the user base is reasonably large. I suspect the Sympy user base is probably large enough to support such protocols. Of course, if such a protocol is deployed at a higher level, say numpy, it would be even more helpful. 

I would like to hear your opinion about this protocol. 

Regards,

Shahriar

[Oscar Benjamin]

Hi Shahriar,

Isn't this something that would have to be implemented in e.g. PyPI?

It doesn't sound like something that SymPy could do.

Oscar

> --
> You received this message because you are subscribed to the Google Groups "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/sympy/ec43dbce-cc65-4e8e-8748-810abe1c5dden%40googlegroups.com.

[Shahriar Iravanian]

Hi Oscar,

The question is where the right level/place for such a thing is. Unfortunately, PyPi or Conda are not the right place. They provide download statistics, but these are nearly useless because most downloads are by bots, crawlers, and such. Especially in the case of the Python ecosystem, there are tons of ways to install a package (pypi, conda, uv, from source, apt-get, brew, ...). In the end, the statistics should come from the installed systems themself; however, it should be done in a secure and ethical way.  I agree that SymPy is not the right place to implement this. Ideally, it should be implemented as a separate Python package, which is then ported and used as a dependency by packages like SymPy. 

Shahriar