symbol with double colon

[Giorgino R]

Hi all,

I am using `parse_exr` to parse some strings as sympy symbols and then generate cxx code with `cxxcode`. When I use an expression with double colon, parse_expr is trying to split it. In contrast, if I define this expression with Sympy Symbol, I can create cxx code (see below). 

Any idea how I could overwrite this behavior in `parse_expr`?

Many thanks in advance,

Giorgino

[Oscar Gustafsson]

This behavior is from the Python tokenizer.

In [13]: input_code = StringIO('l_::p+3')

In [14]: list(generate_tokens(input_code.readline))

Out[14]:

[TokenInfo(type=1 (NAME), string='l_', start=(1, 0), end=(1, 2), line='l_::p+3'),

TokenInfo(type=54 (OP), string=':', start=(1, 2), end=(1, 3), line='l_::p+3'),

TokenInfo(type=54 (OP), string=':', start=(1, 3), end=(1, 4), line='l_::p+3'),

TokenInfo(type=1 (NAME), string='p', start=(1, 4), end=(1, 5), line='l_::p+3'),

TokenInfo(type=54 (OP), string='+', start=(1, 5), end=(1, 6), line='l_::p+3'),

TokenInfo(type=2 (NUMBER), string='3', start=(1, 6), end=(1, 7), line='l_::p+3'),

TokenInfo(type=4 (NEWLINE), string='', start=(1, 7), end=(1, 8), line=''),

TokenInfo(type=0 (ENDMARKER), string='', start=(2, 0), end=(2, 0), line='')]

I guess you can add a transformation that merges NAME, OP=':', OP=':', NAME sequences into a single name token.

BR Oscar

--
You received this message because you are subscribed to the Google Groups "sympy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to sympy+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CA%2Bc1-%2B-iMn8BiYWU3T23yb2_Yj3ASTW3LDiy6DWr149TJRGJkg%40mail.gmail.com.

[Giorgino R]

Hi Oscar,

Many thanks for this, I see. I just went through the documentation and I found the transformations. I guess I will have to add this to the standard transformations (e.g. (merge_double_colon, ) + standard_transformations) so as to work.

BW

Giorgino

To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAFjzj-%2Bh7bXZMpp0n4OKFK%2B9Jegj9J%2BX9xhC7Lo7YomrDZ1mDg%40mail.gmail.com.

[Aaron Meurer]

Yes, you will need to write a custom parser transformer that merges colon tokens into name tokens. This should be straightforward because colon tokens wouldn't appear in any other context in an expression (only in a larger block of code, which parse_expr() cannot handle). So you can unconditionally merge colons in your transformer.

This is actually something that would be nice to document better. For now, Oscar's suggestion gives the basic idea of what you need to do. If you need help with the Python tokenizer, I have written a guide here that goes over it https://www.asmeurer.com/brown-water-python/

Aaron Meurer

To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CA%2Bc1-%2B_4nJb%3D3POZ5L5X0LEndj0YHyndJTCmc0FkPBLH29h6ig%40mail.gmail.com.

[Giorgino R]

Hi Aaron, 

Many thanks for your response. I think it is clear how to create the parser transformer. I just followed the transformations already written inside sympy's parser module.

Thanks for sharing the documentation for the python tokenizer, as well :).

BW

Giorgino 

To view this discussion on the web visit https://groups.google.com/d/msgid/sympy/CAKgW%3D6JoAcMDO2JeLNqv-m0XWNODvz%3DWTUH%3Dg8sEVfRv4brEqA%40mail.gmail.com.