GSoC Project: Rebuilding SymPy Gamma using Pyodide

[Aman Sharma]

Hello SymPy community, 

I am Aman Sharma (mostlyaman on github). I plan to apply for GSoC 2022 for porting SymPy Gamma to Pyodide. I have used SymPy Gamma countless times during my first year calculus and electrical course. In my opinion, SymPy Gamma is an incredibly useful tool and has great potential to be expanded upon.

If you want to suggest some features / share opinions to the project, please do so at https://github.com/sympy/sympy_gamma/issues/188

SymPy Gamma currently runs on the Google App Engine, with Sympy 1.6 which is quite old. Because of the high server costs, Gamma was going to be shut down. So, it is important to modify SymPy Gamma to run in the browser without any backend computation. This eliminates the need to maintain a server and the application could be just hosted on Github pages with zero cost. This is achieved using Pyodide.

I have already been working on this and you can see the current demo here. The GSoC Project would consist of adding remaining functionalities and improving the capabilities of SymPy Gamma.

I think it is also worthwhile to have some developer documentation for Sympy Gamma, about how it works and how a new contributor can start looking at the code. New contributors are essential to keeping this awesome project alive and growing. I look forward to your opinions on this suggestion.

There are some good suggestions in SymPy Gamma Issue Tracker for proposed features. I will be adding some of them in the project timeline.

I have the first draft ready for my proposal here. I will finalise it very soon. I hope potential mentors may take a look at my proposal and suggest improvements.

Aman Sharma

[Oscar Benjamin]

On Mon, 18 Apr 2022 at 18:43, Aman Sharma
<b21...@students.iitmandi.ac.in> wrote:
>
> Hello SymPy community,
> I am Aman Sharma (mostlyaman on github). I plan to apply for GSoC 2022 for porting SymPy Gamma to Pyodide. I have used SymPy Gamma countless times during my first year calculus and electrical course. In my opinion, SymPy Gamma is an incredibly useful tool and has great potential to be expanded upon.
> If you want to suggest some features / share opinions to the project, please do so at https://github.com/sympy/sympy_gamma/issues/188
>
> SymPy Gamma currently runs on the Google App Engine, with Sympy 1.6 which is quite old. Because of the high server costs, Gamma was going to be shut down. So, it is important to modify SymPy Gamma to run in the browser without any backend computation. This eliminates the need to maintain a server and the application could be just hosted on Github pages with zero cost. This is achieved using Pyodide.

There are a number of projects/PRs attempting to do something similar
to this right now. You mention SymPy Beta in your proposal but I
thought there was at least one other. How is this project idea
different from those?

--
Oscar

[Aaron Meurer]

There are proposals to replace SymPy Live with jupyterlite, but I'm
not aware of other SymPy Gamma proposals. SymPy Gamma works quite
differently from SymPy Live, so I'm not sure if there will be an
off-the-shelf replacement like jupyterlite, although if there is, we
should look into it.

Aaron Meurer

>
> --
> Oscar
>
> --
> You received this message because you are subscribed to the Google Groups "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAHVvXxSgJFw%2BXZ4sLDH%3DwfhkxxVG_7mLawN9ukKWZc33uje_%2Bg%40mail.gmail.com.

[Oscar Benjamin]

On Mon, 18 Apr 2022 at 20:37, Aaron Meurer <asme...@gmail.com> wrote:
>
> On Mon, Apr 18, 2022 at 1:16 PM Oscar Benjamin
> <oscar.j....@gmail.com> wrote:
> >
> > On Mon, 18 Apr 2022 at 18:43, Aman Sharma
> > <b21...@students.iitmandi.ac.in> wrote:
> > >
> > > Hello SymPy community,
> > > I am Aman Sharma (mostlyaman on github). I plan to apply for GSoC 2022 for porting SymPy Gamma to Pyodide. I have used SymPy Gamma countless times during my first year calculus and electrical course. In my opinion, SymPy Gamma is an incredibly useful tool and has great potential to be expanded upon.
> > > If you want to suggest some features / share opinions to the project, please do so at https://github.com/sympy/sympy_gamma/issues/188
> > >
> > > SymPy Gamma currently runs on the Google App Engine, with Sympy 1.6 which is quite old. Because of the high server costs, Gamma was going to be shut down. So, it is important to modify SymPy Gamma to run in the browser without any backend computation. This eliminates the need to maintain a server and the application could be just hosted on Github pages with zero cost. This is achieved using Pyodide.
> >
> > There are a number of projects/PRs attempting to do something similar
> > to this right now. You mention SymPy Beta in your proposal but I
> > thought there was at least one other. How is this project idea
> > different from those?
>
> There are proposals to replace SymPy Live with jupyterlite, but I'm
> not aware of other SymPy Gamma proposals. SymPy Gamma works quite
> differently from SymPy Live, so I'm not sure if there will be an
> off-the-shelf replacement like jupyterlite, although if there is, we
> should look into it.

There is SymPy Beta which is apparently also based on pyodide:

https://github.com/eagleoflqj/sympy_beta
https://sympy-beta.vercel.app/

--
Oscar

[Aaron Meurer]

I would go into more technical details, if you have them, about how
things will be implemented. Particularly how the pyodide will work and
any challenges there.

Something I would like to know more about is what adding a new feature
would look like on the technical side. How hard would it be for
someone not familiar with the code? Just how much of that will require
writing Javascript vs. writing something in Python? Ideally as much
will be in Python as possible, as that is the easiest for SymPy
developers to work with, and it also makes it easiest to interface
directly with SymPy. Also any features that make sense to be in SymPy
itself should go in SymPy itself. Getting the design for modularity
right is more important than actually adding lots of features. If we
have that, then adding new features will be straightforward.

Aaron Meurer

> --
> You received this message because you are subscribed to the Google Groups "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/9eb8656f-086c-4e16-af68-f30e9d4a5615n%40googlegroups.com.

[Aaron Meurer]

On Mon, Apr 18, 2022 at 1:44 PM Oscar Benjamin

<oscar.j....@gmail.com> wrote:
>
> On Mon, 18 Apr 2022 at 20:37, Aaron Meurer <asme...@gmail.com> wrote:
> >
> > On Mon, Apr 18, 2022 at 1:16 PM Oscar Benjamin
> > <oscar.j....@gmail.com> wrote:
> > >
> > > On Mon, 18 Apr 2022 at 18:43, Aman Sharma
> > > <b21...@students.iitmandi.ac.in> wrote:
> > > >
> > > > Hello SymPy community,
> > > > I am Aman Sharma (mostlyaman on github). I plan to apply for GSoC 2022 for porting SymPy Gamma to Pyodide. I have used SymPy Gamma countless times during my first year calculus and electrical course. In my opinion, SymPy Gamma is an incredibly useful tool and has great potential to be expanded upon.
> > > > If you want to suggest some features / share opinions to the project, please do so at https://github.com/sympy/sympy_gamma/issues/188
> > > >
> > > > SymPy Gamma currently runs on the Google App Engine, with Sympy 1.6 which is quite old. Because of the high server costs, Gamma was going to be shut down. So, it is important to modify SymPy Gamma to run in the browser without any backend computation. This eliminates the need to maintain a server and the application could be just hosted on Github pages with zero cost. This is achieved using Pyodide.
> > >
> > > There are a number of projects/PRs attempting to do something similar
> > > to this right now. You mention SymPy Beta in your proposal but I
> > > thought there was at least one other. How is this project idea
> > > different from those?
> >
> > There are proposals to replace SymPy Live with jupyterlite, but I'm
> > not aware of other SymPy Gamma proposals. SymPy Gamma works quite
> > differently from SymPy Live, so I'm not sure if there will be an
> > off-the-shelf replacement like jupyterlite, although if there is, we
> > should look into it.
>
> There is SymPy Beta which is apparently also based on pyodide:

Right, I meant aside from SymPy Beta. The author of SymPy Beta has
declined to upstream the changes to SymPy Gamma, so if we want to
continue to maintain SymPy Gamma, we will need to do the work
separately.

Aaron Meurer

>
> https://github.com/eagleoflqj/sympy_beta
> https://sympy-beta.vercel.app/
>
> --
> Oscar
>

> --
> You received this message because you are subscribed to the Google Groups "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.

> To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAHVvXxS4X0xKpNHKW%3DkgLZjXWisnsF3eKXu1jb6-avcC%2BUY%2Bcg%40mail.gmail.com.

[Oscar Benjamin]

To me it seems like a good thing that someone else is maintaining
SymPy Beta. SymPy Gamma and SymPy Live have both languished because
the SymPy maintainers don't have the right skillset to maintain them.

Also I might be wrong but as I understood it the reason for not
upstreaming was just the difference in license but I didn't really see
a change of license as a big deal. SymPy Gamma is not a library in the
way that SymPy is so it doesn't need the same license.

--
Oscar

[Qijia Liu]

Hi all,

Thanks for mentioning SymPy Beta.

If I understand correctly, SymPy Gamma's goal is to make SymPy end-user-friendly in a Wolfram|Alpha way, and some of the users (like Aman) may contribute back to SymPy.

If I'm allowed to be as ambitious as I choose, SymPy Beta's goal is to be an open source alternative to Wolfram|Alpha, as close as possible.

That is to say, although currently more than 90% of its kernel is powered by SymPy, in the future it may involve functionalities from other libraries, like NumPy and SciPy, or even outside Python ecosystem. At that time, it should be renamed, and maintained by a different group of people rather than only SymPy community.

So, do you need a pure-SymPy web service, or equivalently continue SymPy Gamma? Likely. That's why Aman's GSoC makes sense. And I'd be glad to borrow code and ideas from it.

SymPy will never be hurt by my forking. I'm not a selfish guy who only cares his own project. Instead I upstream every possible improvements. #23379, #23349, #23239 are examples. If you find any AGPLed code should belong to SymPy, please let me know.

For SymPy Gamma, I do want its users to use SymPy Beta instead, because it is modern and (at least currently) better maintained, has more functionalities, bug fixes and less confusing behavior. And it doesn't add a cent to my bill :-)

But as the license granted, you may fork SymPy Beta back at any time you want.

Sorry for being off topic. If you have more questions or thoughts, please discuss at https://github.com/sympy/sympy_gamma/issues/186 or https://github.com/eagleoflqj/sympy_beta

Best,

Qijia

[Aman Sharma]

Thank you Qijia for your reply. SymPy Beta is surely a great project . SymPy Beta was a proof of concept to me, and motivated me to spend time learning Web Development and Pyodide.
SymPy Gamma brings the power of SymPy to users without learning the correct Python syntax or the SymPy API. I would like to see it grow and getting used by students like me in the future utilising the capabilities of complete SymPy library without a coding background. (I have been talking about SymPy Gamma to my friends and they have already become daily users : )

If we go ahead and decide to continue SymPy Gamma, I completely agree with Aaron here

"""

Something I would like to know more about is what adding a new feature
would look like on the technical side. How hard would it be for
someone not familiar with the code? Just how much of that will require
writing Javascript vs. writing something in Python?

"""

Adding a feature should be done majority in Python. I will elaborate  on this in a few hours.

Aman

[Aman Sharma]

"""

Something I would like to know more about is what adding a new feature
would look like on the technical side.

"""

Adding a new feature like a new card could be done entirely in python. The Pyodide coderunner handles the input expression provided by javascript and returns the cards in the form of a JS Map (equivalent to a Python dict). The javascipt loops over all the items in the Map, and evaluates any card if needed. Currently, it is hard-coded in Javascript which cards have to be evaluated. We can update the python logic to include a Boolean value in the dict whether the card needs any evaluation.

Eg: Currently, when JS recieves a card like this, 

>>> s.eval('x**2')[2]
{'card': 'roots', 'var': 'x', 'title': 'Roots', 'input': 'solve(x**2 -1 ,  x)', 'pre_output': 'x', 'parameters': []}

It then sends the following code to  run in Python

>>> s.eval_card('roots', 'x**2-x', 'x', {})
{'value': '[0, 1]', 'output': '<ul>\n<li><script type="math/tex; mode=display">0</script></li>\n<li><script type="math/tex; mode=display">1</script></li>\n</ul>'}

Here, the required html is provided to the JS and it is inserted into the proper div for the card.

The JS here follows the same routine for all cards. Hence, adding a new card should be possible to do entirely in python.

"""

Particularly how the pyodide will work and any challenges there.

"""

Pyodide runs on a browser on a service worker, and their documentation have provided a basic API to run Python code in JS.

In my source code, there is a "main" function which deals with the Pyodide API and returns the result. It can evaluate mathematical expressions in different flavours by changing the type parameter in the "main" function.

like for running a pure python command, we can set type = "input"

for generating cards using SymPyGamma object, we can set type = "eval"

and for evaluating a card we can set type = "eval-card"

The major challenge about using Pyodide is transferring Python objects to Javascript. Pyodide usually converts the basic types very well, like Python dicts to JS Maps. I have not experimented much and have currently stuck with strings to share information between JS and Python.

If some change to the UI is to be implemented, it will most probably have to be done in JS. Thankfully, Vue enables us to create components which can be reused. like the same component can be used for the search bar and the context menu (see https://github.com/sympy/sympy_gamma/issues/188 ) and the expression recieved from each can be executed differently using the type parameter in main function. 

In my opinion, we can certainly bring in new contributors for Gamma if we have some guides on how to navigate the code. Learning how to interface with Python codebase was much more work for me than to implement it using JS. Thats why, I have given Phase 2 of GSoC to majorly create documentation for new contributors.

I will update my proposal with some tasks to make it easier to implement features using Python. 

Nevertheless, I totally understand if SymPy Gamma is discontinued in the future. I enjoyed working on this project.