Question about parsing Python files
Kent Tenney
Files which mix statements with class and function declarations
parse in a way I find confusing.
Given the following:
parsetest.py = """\
s1 = "s1"
def f1():
pass
s2 = "s2"
def f2():
pass
s3 = "s3"
"""
The node
@auto parsetest.py
contains the following
@language python
@tabwidth -4
@others
s3 = "s3"
and has 3 children:
<parsetest declarations>
s1 = "s1"
<f1>
def f1():
pass
<f2>
s2 = "s2"
def f2():
pass
I would prefer if nodes were created for each statement block.
As it is, the string definitions are hard to find.
Thanks,
Kent
Terry Brown
Kent Tenney <kte...@gmail.com> wrote:
> Files which mix statements with class and function declarations
> parse in a way I find confusing.
marked as duplicate of
http://groups.google.com/group/leo-editor/browse_thread/thread/54afe963abdca1a
:-)
I think the unhelpful bottom line was that it's impossible to tell with which def a same level statement should be grouped.
But then...
> I would prefer if nodes were created for each statement block.
> As it is, the string definitions are hard to find.
I think that was the conclusion zpcspm and I came to too :)
Post a bug I guess.
Cheers -Terry
Edward K. Ream
I agree, I think. A typical pattern is:
def abc():
...
synonymForAbc = abc
OTOH, comments probably belong to the following def.
Edward
Terry Brown
"Edward K. Ream" <edre...@gmail.com> wrote:
> I agree, I think. A typical pattern is:
>
> def abc():
> ...
>
> synonymForAbc = abc
>
> OTOH, comments probably belong to the following def.
thingyClass = Foo
def abc(self):
self.thingy = thingyClass()
your example is probably more common, but it can go either way. I think everything in its own node and if people don't like it they can use @shadow or @thin (unless @thin is @file now, in which case @file :-)
Cheers -Terry
Kent Tenney
are there more than ``comment`` and ``declaration``?
How about a hint about node contents in the headline, like:
declaration <s1 = "s1">
\
s1 = "s1"
inifile = "/my/ini/file"
comment <# Houston, we have ... >
\
# Houston, we have a problem, the following doesn't work
# FIXME
> --
> You received this message because you are subscribed to the Google Groups "leo-editor" group.
> To post to this group, send email to leo-e...@googlegroups.com.
> To unsubscribe from this group, send email to leo-editor+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/leo-editor?hl=en.
>
>
Edward K. Ream
>I think everything in its own node and if people don't like it they can use @shadow or @thin (unless @thin is @file now, in which case @file :-)
No. Too many nodes would be the worst possible "solution". @shadow or
@thin are irrelevant to this discussion. We can not remedy defects in
@auto by resorting to @thin!
Edward
Terry Brown
"Edward K. Ream" <edre...@gmail.com> wrote:
True, but I don't think there's any way @auto can know which def node a declaration belongs to, and I really don't think it should guess and potentially hide critical information. Like this:
def doSomething(self):
...
def doSomethingCachedVersion(self):
...
doSomething = doSomethingCachedVersion
Ok, not great code, but you see this kind of thing often enough. If the declaration is hidden in the def doSomethingCachedVersion node, you could spend years, perhaps decades :-), trying to debug the first definition of doSomething, seeing that would seem to be the active code (assuming calls to self.doSomething elsewhere).
I don't think auto can put declarations in def nodes without risking concealment of critical info.
Cheers -Terry
Edward K. Ream
> True, but I don't think there's any way @auto can know which def node a declaration belongs to, and I really don't think it should guess and potentially hide critical information.
Imo, the cure is worse than the disease. Too many nodes would be a
really bad thing. One can contrive situations in which the "active"
code will be pretty hidden no matter what we do. I think we have to
live with it.
Having said that, I do think that synonyms that follow a def belong in
the node containing the def. This has annoyed be for a very long
time. A bug should be filed. And fixed.
Edward
Kent Tenney
After starting this thread, I think someone else should file the bug,
since the example I started with seems to be considered correct,
the only behaviour to be addressed is the which def node to put
synonyms in, a use case I haven't encountered.
It's fairly rare for me to sprinkle comments and statements between
defs, so my concern was with their disappearance. I can see however
that a file can be contrived which would be confusing in the other direction.
I think that for my files, I would prefer nodes indicating the
location of sprinklings to trying to remember if they exist and which
def node I should look in.
A hard one to resolve it seems.
It seems that most Python code I look at doesn't do a great deal of
sprinkling, but I suppose it's out there.
Is it considered good / bad style to do this?
What about properties?
foo = property(f1)
just stick them with a nearby def?
>
> Edward
Terry Brown
Kent Tenney <kte...@gmail.com> wrote:
> It's fairly rare for me to sprinkle comments and statements between
> defs, so my concern was with their disappearance. I can see however
> that a file can be contrived which would be confusing in the other direction.
Not really sure what the next step is. I think you, zpcspm, and I are of the same opinion, but Edward differs. Does this mean... options? Arghh, the horror! :)
So, behaviors would be...
All non-def blocks get their own node... except stuff immediately following the class line, which goes in the same node as the class line. (Kent, zpcspm, me)
or, controlled by an option,
interspersed code is automagically grouped with the def to which it's most relevant.
I guess I'm not 100% clear on the current behavior.
Cheers -Terry
zpcspm
> Not really sure what the next step is. I think you, zpcspm, and I are of the same opinion, but Edward
> differs. Does this mean... options? Arghh, the horror! :)
things.
Yes, I have complained earlier about a similar pattern.
But I mostly encounter it when dealing with legacy code (which is not
my own).
I import this code into leo using @shadow.
I short: I think it would be nice if users would have a custom option
to control the logic of code splitting, but this has implications (at
least more complexity of the importing code). I understand this is a
clear situation when the best can become an enemy of the good.
Kent Tenney
> On Jun 15, 9:49 pm, Terry Brown <terry_n_br...@yahoo.com> wrote:
>> Not really sure what the next step is. I think you, zpcspm, and I are of the same opinion, but Edward
>> differs. Does this mean... options? Arghh, the horror! :)
>
> My requirement is lighter and I can live with the current state of
> things.
> Yes, I have complained earlier about a similar pattern.
> But I mostly encounter it when dealing with legacy code (which is not
> my own).
Right. My impression is good style dictates putting declarations at the top,
followed by defs, as Leo expects currently.
The use case which prompted my post involved experiments with triple quoted
blocks inline for testing and documentation. This doesn't look like a good idea,
so I'm not wed to the splitting issue, but interested enough for a bit
more discussion.
> I import this code into leo using @shadow.
Then what?
Do you make nodes for the comments, assignments etc
which are between defs?
ie: do you manually do what we are considering asking Leo to do?
>
> I short: I think it would be nice if users would have a custom option
> to control the logic of code splitting, but this has implications (at
> least more complexity of the importing code). I understand this is a
> clear situation when the best can become an enemy of the good.
In Edward's contrived file which would be cluttered with too many nodes,
under the current scheme, lots of code would be hidden in def nodes.
I tend to prefer the surfeit of nodes, which says to me "please organize your
code a bit better" instead of the current approach which says "I'm playing
hide the code with you"
:-]
Regarding options, I'm curious about how file splitting is defined.
If splitting were defined by some kind of template or rule set which
wasn't too complex, the user could choose between.
I've wondered about this when I encounter structured files which
I wish Leo could split for me, or I could easily specify splitting requirements.
Thanks,
Kent
Edward K. Ream
>> I short: I think it would be nice if users would have a custom option
>> to control the logic of code splitting
This is an interesting topic. More flexibility would be good, imo.
The question is, how to get it?
Because we are talking about @auto, we can't make many assumptions
about coding style: that's up to the original authors of the code.
We could consider per-file options that say, in effect, I want *this*
file formatted like *this*. These options can be cached if we
continue to assume that can't alter the files themselves.
Alternatively, a leo-editor line, similar to a vim line, could specify
options. Such lines aren't odious.
I continue to believe that code "fragments" belong with either the
preceding or following node, but if we have per-file options that's
only a preference that can be over-ridden
These are just preliminary thoughts...
Edward
zpcspm
> >On Wed, Jun 16, 2010 at 3:27 AM, zpcspm <zpc...@gmail.com> wrote:
> > I import this code into leo using @shadow.
>
> Then what?
> Do you make nodes for the comments, assignments etc
> which are between defs?
> ie: do you manually do what we are considering asking Leo to do?
disk and noticed that my changes weren't persistent. At that point I
brought the issue to this group. Here's the thread:
http://groups.google.com/group/leo-editor/browse_thread/thread/054afe963abdca1a#
It pretty much intersects with what is being discussed here.
For now I don't touch the outline structure that leo makes when
importing python source files. I sigh somethimes but I'm getting used
to it.
Terry Brown
"Edward K. Ream" <edre...@gmail.com> wrote:
> On Wed, Jun 16, 2010 at 6:44 AM, Kent Tenney <kte...@gmail.com> wrote:
>
> >> I short: I think it would be nice if users would have a custom option
> >> to control the logic of code splitting
>
> This is an interesting topic. More flexibility would be good, imo.
> The question is, how to get it?
>
> Because we are talking about @auto, we can't make many assumptions
> about coding style: that's up to the original authors of the code.
>
> We could consider per-file options that say, in effect, I want *this*
> file formatted like *this*. These options can be cached if we
> continue to assume that can't alter the files themselves.
> Alternatively, a leo-editor line, similar to a vim line, could specify
> options. Such lines aren't odious.
I can't see why it's so complicated.
Firstly, @decorators - mentioning these first to get them out of the way, they belong in the same node as the following def. Cases like this:
# add inspection
@inspector(log_file,
debug)
# also enable tracing
@trace
# now define foo
def foo(self):
...
might take a little care, but basically once you start seeing decorators you start storing lines in the node of the next def. In the obnoxious example above, if the comment about the first decorator got separated, I don't think that would be the end of the world, until python implements `import understand_human_language` there are going to be edge cases.
So, @decorators aside, Leo has to decide where to put non-blank lines which occur between defs. Leo's already making that decision, so to me it's just a matter of a @setting to let the user decide whether it's `preceding_node`, `following_node`, or `own_node`. I'd want the same approach applied to all files I load with @auto.
Maybe I'm missing something.
Cheers -Terry