GSOC 2019 Idea: Python/C Parser for sympy

188 views
Skip to first unread message

Nikhil Maan

unread,
Mar 27, 2019, 7:39:49 PM3/27/19
to sympy
Hello everyone,

My name is Nikhil Maan and I am a sophomore at Amity University, India as a CSE major.
I have a few years of experience with C/C++; that makes me relatively new to python. What I love the most about Python is that it's dynamically typed. Coming from C and C++, it saves a lot of work and in general a very cool feature.
I have also been using sympy for some time and I  really love the idea of code generation for other languages. Also, lambdify is my favorite function. I can just solve my mathematics in sympy and generate the corresponding code to use in other languages in my program. Being a mathematics enthusiast, it is really satisfying to be able to solve maths using proper symbols instead of a mixture of some random functions and operators which is not very pretty or convenient.

I was looking at sympy's idea page for GSoC and I saw the idea for creating parsers to convert syntax from other languages to create sympy expression. It would be very convenient for programmers in these languages like me who like to use sympy if we can just convert our code to sympy's syntax, solve it and generate code for results back to the language we are using instead of creating new expressions every time we need to use sympy. It would save a lot of time for a large number of people. And I would like to use my summer break to help in making that possible so that it can be convenient for everyone later.

I would like to ask: Is anyone working on a python or C parser for sympy? I would like to take up parsing python/C code to sympy as a GSoC project if the community likes that. Hoping there will be someone willing to mentor for this project.


Ondřej Čertík

unread,
Mar 27, 2019, 8:16:14 PM3/27/19
to sympy
Hi Nikhil,
Depending on the application of this, it probably makes sense to use Clang to parse the C or C++ code and then go over the Clang AST and extract expressions from it and convert them to SymPy where it makes sense.

Would you be interested in doing this (also) for Fortran? See these ideas:

https://github.com/sympy/sympy/wiki/GSoC-2019-Ideas#lfortran-sympy-project-ideas

especially the "Parsing Fortran code to SymPy". That would be something I would be very interested in, and yes, I would be happy to mentor it.

Perhaps we could do both C and Fortran in your proposal. For C that would be relatively easy due to Clang. For Fortran you would use my LFortran compiler. There the parser actually requires some development. C already has a modern production open source compiler/parser (Clang), but Fortran doesn't yet, and that's one thing I am trying to fix with LFortran (see https://docs.lfortran.org/ for other motivation).

On the SymPy side, we definitely want to do both C and Fortran, so this would naturally lead you to design a good interface that works for both. Similarly as our code generation interface works for both C and Fortran.

If you are interested in developing a parser and semantic representation of the language so that this can be done, then Fortran actually would be a great choice because this is exactly where we need help with LFortran. While for C this has already been developed in Clang, so I would suggest to just use it.

Let me know your interests, and we can discuss further details.

Ondrej
Message has been deleted

Nikhil Maan

unread,
Mar 27, 2019, 8:51:52 PM3/27/19
to sympy
Hello Ondrej,

On Thursday, March 28, 2019 at 5:46:14 AM UTC+5:30, Ondřej Čertík wrote:
Hi Nikhil,



Depending on the application of this, it probably makes sense to use Clang to parse the C or C++ code and then go over the Clang AST and extract expressions from it and convert them to SymPy where it makes sense.

Would you be interested in doing this (also) for Fortran? See these ideas:

https://github.com/sympy/sympy/wiki/GSoC-2019-Ideas#lfortran-sympy-project-ideas

especially the "Parsing Fortran code to SymPy". That would be something I would be very interested in, and yes, I would be happy to mentor it.

Perhaps we could do both C and Fortran in your proposal. For C that would be relatively easy due to Clang. For Fortran you would use my LFortran compiler. There the parser actually requires some development. C already has a modern production open source compiler/parser (Clang), but Fortran doesn't yet, and that's one thing I am trying to fix with LFortran (see https://docs.lfortran.org/ for other motivation).

On the SymPy side, we definitely want to do both C and Fortran, so this would naturally lead you to design a good interface that works for both. Similarly as our code generation interface works for both C and Fortran.

If you are interested in developing a parser and semantic representation of the language so that this can be done, then Fortran actually would be a great choice because this is exactly where we need help with LFortran. While for C this has already been developed in Clang, so I would suggest to just use it.

Let me know your interests, and we can discuss further details.

Ondrej


Hello Ondrej,

Thank you for showing interest in my project. It will be an honor to work with you on this project.

I would love to include parsing Fortran in the proposal. I would like to help as much as I can. Sympy has made doing mathematics so much convenient, whatever I can do to make it more convenient for others, I will try my best.

There was a dicussion i saw in a thread about being able to understand simple instructions like 'integrate sin x over x' and convert them to sympy syntax. That will be a long and complex task. But this project can be extended further after fortran and C to understand natural language.
But, for now first i would like to focus on Fortran and C first. I'll read the clang and Lfortran documentation asap and start working on a plan.

I would also like to thank you for creating sympy. it made doing complex mathematics so much better. Instead of just looking at some mix of keywords and operators which is very hard to understand, we can actually see and understand what we're doing.

I'll get to work now

Sincerely,
Nikhil Maan
 
 

Ondřej Čertík

unread,
Mar 27, 2019, 11:09:51 PM3/27/19
to sympy
Hi Nikhil,

> Hello Ondrej,
>
> Thank you for showing interest in my project. It will be an honor to
> work with you on this project.
>
> I would love to include parsing Fortran in the proposal. I would like
> to help as much as I can. Sympy has made doing mathematics so much
> convenient, whatever I can do to make it more convenient for others, I
> will try my best.
>
> There was a dicussion i saw in a thread about being able to understand
> simple instructions like 'integrate sin x over x' and convert them to
> sympy syntax. That will be a long and complex task. But this project
> can be extended further after fortran and C to understand natural
> language.
> But, for now first i would like to focus on Fortran and C first. I'll
> read the clang and Lfortran documentation asap and start working on a
> plan.
>
> I would also like to thank you for creating sympy. it made doing
> complex mathematics so much better. Instead of just looking at some mix
> of keywords and operators which is very hard to understand, we can
> actually see and understand what we're doing.
>
> I'll get to work now

Perfect. I am glad you found SymPy useful. For the GSoC, also don't forget the patch requirement and send one or more patches to SymPy.

To clarify my motivations regarding LFortran: one or two students working on it during GSoC will allow me to pull it off. And I talked with Aaron and as long as the projects help both LFortran and SymPy, it's ok to to do it as part of SymPy's GSoC.

What I am trying to achieve with LFortran is to have an interactive environment for numerical scientific computing like Python, Julia or MATLAB. Of these projects, it's closest to Julia, also in terms of the motivation (fixing the so-called two language problem of coding in Python for interactivity, and then rewriting to C++: instead just do everything in Julia... or Fortran) as well as the technology of interactive compiling. But instead of inventing a new language, I am simply using Fortran, just making it interactive. Fortran as a language is actually really nice for array oriented scientific computing --- if it had nice modern tooling supporting it, which is what I am trying to do.

While doing it, I am also designing the Fortran compiler to be nicely designed as a library with a nice Python API. SymPy does this to symbolic mathematics --- instead of using some symbolic manipulation software as a black box, it nicely gives you access to the symbolic tree as a Python library and you can do things with it. With LFortran I am trying to do the same with Fortran, instead of using the compiler as a black box, it nicely exposes Fortran as an AST and ASR trees/representations, allowing people to write tools on top.

I open sourced it just a few days ago, and I am getting very positive feedback on it, e.g.:

https://twitter.com/DegenerateConic/status/1109930553069973516
http://degenerateconic.com/fortran-and-llvm/

and the GitLab repo has 14 stars, the GitHub repo has 26 stars, that's the largest rate of increase of any of my open source projects that I open sourced.

So I have no doubts a lot of people want this and will use it, we just need to develop it enough.

Ondrej

Aaron Meurer

unread,
Mar 28, 2019, 2:07:00 PM3/28/19
to sy...@googlegroups.com
On Wed, Mar 27, 2019 at 6:51 PM Nikhil Maan <nikhil...@gmail.com> wrote:
>
> Hello Ondrej,
>
> On Thursday, March 28, 2019 at 5:46:14 AM UTC+5:30, Ondřej Čertík wrote:
>>
>> Hi Nikhil,
>>
>>
>>
>> Depending on the application of this, it probably makes sense to use Clang to parse the C or C++ code and then go over the Clang AST and extract expressions from it and convert them to SymPy where it makes sense.
>>
>> Would you be interested in doing this (also) for Fortran? See these ideas:
>>
>> https://github.com/sympy/sympy/wiki/GSoC-2019-Ideas#lfortran-sympy-project-ideas
>>
>> especially the "Parsing Fortran code to SymPy". That would be something I would be very interested in, and yes, I would be happy to mentor it.
>>
>> Perhaps we could do both C and Fortran in your proposal. For C that would be relatively easy due to Clang. For Fortran you would use my LFortran compiler. There the parser actually requires some development. C already has a modern production open source compiler/parser (Clang), but Fortran doesn't yet, and that's one thing I am trying to fix with LFortran (see https://docs.lfortran.org/ for other motivation).
>>
>> On the SymPy side, we definitely want to do both C and Fortran, so this would naturally lead you to design a good interface that works for both. Similarly as our code generation interface works for both C and Fortran.
>>
>> If you are interested in developing a parser and semantic representation of the language so that this can be done, then Fortran actually would be a great choice because this is exactly where we need help with LFortran. While for C this has already been developed in Clang, so I would suggest to just use it.
>>
>> Let me know your interests, and we can discuss further details.
>>
>> Ondrej
>
>
>
> Hello Ondrej,
>
> Thank you for showing interest in my project. It will be an honor to work with you on this project.
>
> I would love to include parsing Fortran in the proposal. I would like to help as much as I can. Sympy has made doing mathematics so much convenient, whatever I can do to make it more convenient for others, I will try my best.
>
> There was a dicussion i saw in a thread about being able to understand simple instructions like 'integrate sin x over x' and convert them to sympy syntax. That will be a long and complex task. But this project can be extended further after fortran and C to understand natural language.
> But, for now first i would like to focus on Fortran and C first. I'll read the clang and Lfortran documentation asap and start working on a plan.

Natural language parsing is a whole other beast. I would suggest
sticking to C and Fortran. A project to handle natural language
queries could easily fill a whole summer on its own.

Aaron Meurer

>
> I would also like to thank you for creating sympy. it made doing complex mathematics so much better. Instead of just looking at some mix of keywords and operators which is very hard to understand, we can actually see and understand what we're doing.
>
> I'll get to work now
>
> Sincerely,
> Nikhil Maan
>
>
>
> --
> You received this message because you are subscribed to the Google Groups "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.
> To post to this group, send email to sy...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sympy.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/36c4c6db-3f9c-4c38-a38c-227b9e183b2b%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

Nikhil Maan

unread,
Mar 29, 2019, 5:41:30 PM3/29/19
to sympy


On Thursday, March 28, 2019 at 11:37:00 PM UTC+5:30, Aaron Meurer wrote:
On Wed, Mar 27, 2019 at 6:51 PM Nikhil Maan <nikhil...@gmail.com> wrote:

> Hello Ondrej,
>
> Thank you for showing interest in my project. It will be an honor to work with you on this project.
>
> I would love to include parsing Fortran in the proposal. I would like to help as much as I can. Sympy has made doing mathematics so much convenient, whatever I can do to make it more convenient for others, I will try my best.
>
> There was a dicussion i saw in a thread about being able to understand simple instructions like 'integrate sin x over x' and convert them to sympy syntax. That will be a long and complex task. But this project can be extended further after fortran and C to understand natural language.
> But, for now first i would like to focus on Fortran and C first. I'll read the clang and Lfortran documentation asap and start working on a plan.

Natural language parsing is a whole other beast. I would suggest
sticking to C and Fortran. A project to handle natural language
queries could easily fill a whole summer on its own.

Aaron Meurer




Dear  Ondrej and Aaron,

I am currently only focusing on Fortran and C. I was just mentioning that I read that somewhere. We can expand in that direction once this project is complete.

i want to say that the semester is about to end; this is the last week of classes and i don't get as much free time as I would like. So, I'm sorry if I'm not active enough. But I will prepare the first draft of the proposal/plan before Monday. Whenever you are free, you can review it and tell me if I missed something or if something more can be done than what I'm proposing.

Also, Can I send the draft in a personal message. i don't think making a draft public is a good idea. But if you want me to post it publicly, I will happily do it,.

Regards,
Nikhil Maan

Ondřej Čertík

unread,
Mar 29, 2019, 7:03:19 PM3/29/19
to sympy
Hi Nikhil,
Also please do not forget about the patches for SymPy.

Ondrej

Nikhil Maan

unread,
Apr 2, 2019, 12:10:19 AM4/2/19
to sympy


On Saturday, March 30, 2019 at 4:33:19 AM UTC+5:30, Ondřej Čertík wrote:
Hi Nikhil,

Also please do not forget about the patches for SymPy.

Ondrej

Dear Ondrej,

I have shared a draft of my proposal with sympy. Whenever you have time, you can review it and tell me if I should add
something else and if you want something done before something else, the timeline is flexible.

I also found some of the Lfortran documentation to be minimalist. So, I've also included improving Lfortran documentation in the proposal.

For the patches, I am currently looking at test coverage. Some modules have a little low test coverage.

@Aaron Meurer
           What are your thoughts about the proposal?

Regards.
Nikhil Maan


Aaron Meurer

unread,
Apr 2, 2019, 1:08:18 AM4/2/19
to sympy
I would suggest focusing on the patch requirement at this point.

Aaron Meurer
> --
> You received this message because you are subscribed to the Google Groups "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.
> To post to this group, send email to sy...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sympy.
> To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/241b9ce5-cce2-4ee4-8737-ebfb758bca17%40googlegroups.com.

Ondřej Čertík

unread,
Apr 2, 2019, 5:21:30 PM4/2/19
to sympy


On Mon, Apr 1, 2019, at 10:10 PM, Nikhil Maan wrote:
>
>
> On Saturday, March 30, 2019 at 4:33:19 AM UTC+5:30, Ondřej Čertík
> wrote:Hi Nikhil,
> >
> > Also please do not forget about the patches for SymPy.
> >
> > Ondrej
>
> Dear Ondrej,
>
> I have shared a draft of my proposal with sympy. Whenever you have
> time, you can review it and tell me if I should add
> something else and if you want something done before something else,
> the timeline is flexible.

Can you please share the link? I can't find it. I'll have a look.

As Aaron said, concentrate on the patch requirement now.

Ondrej

Nikhil Maan

unread,
Apr 3, 2019, 11:38:44 AM4/3/19
to sympy


On Wednesday, April 3, 2019 at 2:51:30 AM UTC+5:30, Ondřej Čertík wrote:


Can you please share the link? I can't find it. I'll have a look.

As Aaron said, concentrate on the patch requirement now.

Ondrej


Here's the link. It is the first draft, more work needs to be done, but I wanted a review of the plan before making improvememnts.

I am focusing on patches now. Will have at least one PR by morning.
P.S. : It's 9 PM here.

Regards,
Nikhil Maan

Ondřej Čertík

unread,
Apr 3, 2019, 1:59:06 PM4/3/19
to sympy
Hi Nikhil,

Thanks. I left there some comments. I will give more feedback soon. Ping me once you submit a patch, that is now the highest priority.

Ondrej
> --
> You received this message because you are subscribed to the Google
> Groups "sympy" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to sympy+un...@googlegroups.com.
> To post to this group, send email to sy...@googlegroups.com.
> Visit this group at https://groups.google.com/group/sympy.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/sympy/8db59684-6ed6-4c7d-9d4f-a95be9d8e9ea%40googlegroups.com <https://groups.google.com/d/msgid/sympy/8db59684-6ed6-4c7d-9d4f-a95be9d8e9ea%40googlegroups.com?utm_medium=email&utm_source=footer>.
Message has been deleted

Nikhil Maan

unread,
Apr 5, 2019, 1:22:37 AM4/5/19
to sympy


On Wednesday, April 3, 2019 at 11:29:06 PM UTC+5:30, Ondřej Čertík wrote:
Hi Nikhil,

Thanks. I left there some comments. I will give more feedback soon. Ping me once you submit a patch, that is now the highest priority.

Ondrej

 
Dear Ondrej,

i have updated the proposal according to the comments. i am also working on the next draft.

As for the patch requirements, I have submitted a patch regarding increasing test coverage.

[#16557] (https://github.com/sympy/sympy/pull/16557)


i'm sorry for the delay, the module was a little larger than expected.

Regards,
Nikhil Maan

Ondřej Čertík

unread,
Apr 8, 2019, 3:05:58 PM4/8/19
to sympy
Perfect. We'll discuss the PR at github. Regarding your application, it's time to submit it now.
-- 
You received this message because you are subscribed to the Google Groups "sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.
To post to this group, send email to sy...@googlegroups.com.

Nikhil Maan

unread,
Apr 8, 2019, 3:25:58 PM4/8/19
to sympy


On Tuesday, April 9, 2019 at 12:35:58 AM UTC+5:30, Ondřej Čertík wrote:
Perfect. We'll discuss the PR at github. Regarding your application, it's time to submit it now.
 
You received this message because you are subscribed to the Google Groups "sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sy...@googlegroups.com.

Dear Ondrej,

i will be submitting the final PDF of the proosal now. But, if you get time to see it and want some changes, I might have time to edit and reupload.
The draft will still be there on the google docs link in sync with the proposal pdf.

Regarding the patch requirements, I have one merged PR and few other open ones, few of which are to be merged.

And thank you for helping me with the project even before it started.

Regards,
Nikhil

Reply all
Reply to author
Forward
0 new messages