Need help with opening a Leo file

[rengel]

I have a leo file stored in 'D:/dev/.leo/Dev.leo'.

When I open this file with Leo (on Windows 10), I get

'@edit D:/dev/.leo/Dev.leo'

in the outline pane while the complete leo file is shown in the body pane. 

This is the only node in outline pane.

I have no idea how this node '@edit D:/dev/.leo/Dev.leo' came about. 

I didn't type it in. Maybe I hit some key combination that I wasn't aware of.

I have no idea how I can I get rid of this node and restore the normal function of this leo file.

Any ideas what I could do?

Thanks in advance!

PS: Other leo files in the same location open and work correctly.

[rengel]

Maybe I should add a screenshot of the situation:

[Edward K. Ream]

On Mon, Sep 16, 2024 at 12:56 AM 'rengel' via leo-editor <leo-e...@googlegroups.com> wrote:

I have a leo file stored in 'D:/dev/.leo/Dev.leo'.

When I open this file with Leo (on Windows 10), I get

'@edit D:/dev/.leo/Dev.leo'

Strange things happen. I would just delete the node.

Edward

[rengel]

Thanks for your suggestion!

But this is the only node. In its body text it contains hundreds of othe nodes wrapped in an xml file. So deleting that node would delete all the contents of this Leo file.

In the meantime I was able to repair the Leo file using an external editor.

It contained some Erlang source files. The Erlang source code uses some data structures that look like Leo section references '<<...>>'. That caused some hiccups so that in the end Leo wasn't able to parse the Leo file correctly and put everything under one node. Basically, the repair was just removing the t nodes that contained Erlang code.

I was just about to report this solution, when I saw your answer. Thanks again!

[rengel]

BTW: Is there a safe way to include "<<...>>" structs in source code in the body pane? How can Leo be prevented from looking for such structs in the body pane?

[Thomas Passin]

I haven't tried it but Leo has an alternate JSON file format.  That ought to work for the erlang files.

[Thomas Passin]

Leo *ought* to be escaping key XML characters like '<' when it imports a file.  If it isn't handling them right, I would consider that to be a bug.

[Edward K. Ream]

On Mon, Sep 16, 2024 at 5:38 AM 'rengel' via leo-editor <leo-e...@googlegroups.com> wrote:

Thanks for your suggestion!

But this is the only node. In its body text it contains hundreds of othe nodes wrapped in an xml file. So deleting that node would delete all the contents of this Leo file.

Leo will create a .leo file containing just an @edit node in some circumstances. I think that may have been what happened.

To see all the places where that happens, do a cff on "@edit" (body only).

The most likely possibilities:

- Importing a .txt file: LeoQTreeWidget.createAtFileNode

- Creating a file from a url as part of a drag and drop: LeoQTreeWidget.doFileUrlHelper & helper

- Trying to open a bad .leo file: LM.openBadLeoFile.

- Opening an external file: LM.openExternalFile.

- Opening a file from a script: g/LM.openWithFileName.

- Opening a file from a recursive import script: ric.import_one_file.

Leo reads .xml (.leo) files correctly (with all necessary xml escapes), but if Leo won't handle xml escapes if Leo thinks it is importing a .txt file.

In short, something strange happened :-)  There is little or no chance that Leo's code that reads/writes .leo files is buggy.

Edward

[Edward K. Ream]

On Mon, Sep 16, 2024 at 5:41 AM 'rengel' via leo-editor <leo-e...@googlegroups.com> wrote:

BTW: Is there a safe way to include "<<...>>" structs in source code in the body pane? How can Leo be prevented from looking for such structs in the body pane?

<<...>> always means a section reference for @file nodes. There are no exceptions. This constraint is the only way of reading @file nodes reliably, so this constraint is a fundamental part of Leo.

<<...>> may be just text in other kinds of @<file> trees.  See Leo's reference for details.

In the meantime I was able to repair the Leo file using an external editor.

It contained some Erlang source files. The Erlang source code uses some data structures that look like Leo section references '<<...>>'. That caused some hiccups so that in the end Leo wasn't able to parse the Leo file correctly and put everything under one node. Basically, the repair was just removing the t nodes that contained Erlang code.

Glad to hear it. It's possible that the Erlang importer may be the culprit.

Edward

[rengel]

Let's see if I can demonstrate this:

Given is the Erlang file 'erlang_test.erl' that I import in a fresh Leo outline (first image, erlang_test1.png).

Then I change the '@auto' directive to '@file' and try to save the outline.

That results in an error (second image, erlang_test2.png).

[rengel]

This is the Erlang test file:

[Edward K. Ream]

On Mon, Sep 16, 2024 at 7:42 AM 'rengel' via leo-editor <leo-e...@googlegroups.com> wrote:

This is the Erlang test file:

Thanks. The .erl file contains several lines that look like section references. Leo should report problems when writing the enclosing .leo file.

Neither of us seems sure about how the one-node .leo file got created, but the general problem is coming into focus.

I wonder whether the erlang importer should be retired.

Edward

[Edward K. Ream]

On Mon, Sep 16, 2024 at 7:40 AM 'rengel' via leo-editor <leo-e...@googlegroups.com> wrote:

Given is the Erlang file 'erlang_test.erl' that I import in a fresh Leo outline (first image, erlang_test1.png).

Then I change the '@auto' directive to '@file' and try to save the outline.

That results in an error (second image, erlang_test2.png).

I think we can agree that this error is expected.

Edward

[Edward K. Ream]

On Monday, September 16, 2024 at 7:38:18 AM UTC-5 Edward K. Ream wrote:

On Mon, Sep 16, 2024 at 5:41 AM 'rengel' via leo-editor wrote:

BTW: Is there a safe way to include "<<...>>" structs in source code in the body pane? How can Leo be prevented from looking for such structs in the body pane?

<<...>> always means a section reference for @file nodes. There are no exceptions. This constraint is the only way of reading @file nodes reliably, so this constraint is a fundamental part of Leo.

<<...>> may be just text in other kinds of @<file> trees.  See Leo's reference for details.

This page tells what kinds of @<file> trees may contain section references.

Edward

[Thomas Passin]

An @edit file is always considered to be a text file, isn't it? So we're thinking that an @edit node was created when it should have been something else, right?

[Thomas Passin]

Looking at the test erlang file, I see that the "<< ... >> " syntax seems to be an essential part. I had at first thought they were symbols that had not been escaped by the XML ingesting code, but I see that was wrong. I wonder if the  section delineator characters "<< .... >> " could be changed using a directive, much like the comment character can be changed. Perhaps that would not be too much of a code change?

[Edward K. Ream]

On Mon, Sep 16, 2024 at 8:56 AM Thomas Passin <tbp1...@gmail.com> wrote:

An @edit file is always considered to be a text file, isn't it? So we're thinking that an @edit node was created when it should have been something else, right?

It looks that way, but I can't be sure. Let's see if Viktor has any other comments.

Edward

[Edward K. Ream]

On Mon, Sep 16, 2024 at 9:05 AM Thomas Passin <tbp1...@gmail.com> wrote:

Looking at the test erlang file, I see that the "<< ... >> " syntax seems to be an essential part. I had at first thought they were symbols that had not been escaped by the XML ingesting code, but I see that was wrong. I wonder if the  section delineator characters "<< .... >> " could be changed using a directive, much like the comment character can be changed. Perhaps that would not be too much of a code change?

Good question. Maybe 20+ years ago Leo had some escapes that roughly did the same thing.

One of the best design decisions I ever made was to eliminate those hacks/conventions. There is no way to read @<file> nodes with any kind of exception.  I'm not going to change my mind about this while I live.

Edward

[rengel]

So basically that means you can't reliably use Leo's @file nodes for Elixir adn Erlang source code?

[Edward K. Ream]

On Mon, Sep 16, 2024 at 12:34 PM 'rengel' via leo-editor <leo-e...@googlegroups.com> wrote:

So basically that means you can't reliably use Leo's @file nodes for Elixir adn Erlang source code?

Yes, that's what I'm saying. @edit will usually be better than @asis.

Edward

[Jacob Peck]

Doesn't that kind of defeat the whole purpose of having an outliner?

Strings of the pattern `<<some_token>>` occur with some frequency in bash scripts as well, for example:

```

log=/path/to/logfile.log

sftp user...@example.com<<EOF>>$log

  mget *.txt

  bye

EOF
```

I admit, I did test that above snippet earlier, and it worked without issue both as a @file and an @edit, so I'm not certain what exactly is triggering this issue.  But I do think it's worthwhile to figure it out.

Jake

--
You received this message because you are subscribed to the Google Groups "leo-editor" group.
To unsubscribe from this group and stop receiving emails from it, send an email to leo-editor+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/leo-editor/CAMF8tS24H1Vei-AgxJXNj23qV_ODYtPGGvxw1T9es32-k7wNWA%40mail.gmail.com.

[Thomas Passin]

The section marker has to be at the very left of the current indentation level, doesn't it?

[Jacob Peck]

Interesting.  I’ve been using <<sections>> for 12 years now and have never previously stumbled onto that limitation.  I don’t see it documented, but at the same time I’ve never been good at searching Leo’s docs.

It does appear that a @section-delims directive exists, per issue 2276.  Perhaps the misbehaving importers could automatically add that directive with an appropriate default value known to be non-existent (or extremely uncommon) in the given language?

I’m probably wildly misunderstanding the issue, but just a quick thought.

Jake

On Sep 16, 2024, at 2:50 PM, Thomas Passin <tbp1...@gmail.com> wrote:

The section marker has to be at the very left of the current indentation level, doesn't it?

To view this discussion on the web visit https://groups.google.com/d/msgid/leo-editor/663c2b66-852c-4799-81b1-54faf73bb5cbn%40googlegroups.com.

[Thomas Passin]

I didn't phrase that right but it doesn't matter because I just tested and found that the section reference will be found anywhere in a line.  This agrees with the initial posting. And the sample erlang file has some "<<...>>" structures that are the first non-whitespace characters in their lines anyway.

[Thomas Passin]

I tried out the directive and it worked as desired for the sample erlang file.  I specified that the section delimiters would be "[[[...]]]":

@section-delims [[[ ]]]

Edward may think of this as a hack today (as opposed to 20  years ago) but there are good precedents: SQL statements that declare functions need to escape the delimiters if they are to emit a legal SQL statement. SQL has syntax to temporarily change the delimiters so this can be done. One common use is that a ";" by default terminates an SQL statement.  So if you want your function to create but not run this statement:

SELECT 1;

as soon as the processor came to the ";' it would execute the current statement, which is wrong because you are just defining it to be used later.  So:

DELIMITER // -- Parser now looks for "//" to end the current statement

SELECT 1;  -- The ";" is part of the statement that will be executed later.

DELIMITER ; -- Return parser behavior to process ";" as usual.

(I've left out other details needed to make this complete function definition, just to make the point).

Here is what the saved erlang @file looks like with the changed section delimiters:

%@+leo-ver=5-thin
%@+node:tom.20240916163919.1: * @file c:\temp\erlang_name_sections.er
%@@language erlang
%@@section-delims [[[ ]]]

%% ErlangTest

-module(erlang_test).

colourize_image(Alpha, {R,G,B}) ->
    << <<R:8,G:8,B:8,A:8>> || <<A:8,_:8,_:8>> <= Alpha >>.
%@+[[[ interleave_rgb ]]]
%@+node:tom.20240916164123.1: ** [[[ interleave_rgb ]]]
interleave_rgb_and_alpha(RGB, Alpha) ->
    list_to_binary(
      lists:zipwith(fun({R, G, B}, A) ->
    <<R, G, B, A>>
end,
    [{R,G,B} || <<R, G, B>> <= RGB],
    [A || <<A>> <= Alpha])).
%@-[[[ interleave_rgb ]]]
%@-leo

So Leo can happily create @file erlang files even when the contain what look like named sections, and you can use named sections within such a file.  Nice work, Edward!

[Edward K. Ream]

On Tue, Sep 17, 2024 at 1:35 AM 'rengel' via leo-editor <leo-e...@googlegroups.com> wrote:

Reproducing the error is a little tricky,

so this is to clarify how the error can be reproduced.

- My test file is at location "D:/Downloads/erlang_test.erl".

- From there I import this file with "File->Import Files->Import Any File...".

- In Leo I get a new node "Imported Files" with the subnode "@auto D:/Downloads/erlang_test.erl".

- In the subnode, I change "@auto" to "@file". 

- Then I press Ctrl+S to save the outline.

- This creates the "undefined section" error.

If I recreate an "@file D:/Downloads/erlang_test2.erl"" node manually and copy and paste the Erlang code from the first node into the body of this node and then save the outline, the ERROR DOES NOT OCCUR.

Interesting. Thanks for these notes.

So what is happening when "@auto" is changed to "@file"?

I'm not completely sure and I can't fully explain the behavior you describe.

The following table, from the directives reference, shows that Leo writes @auto and @file trees differently.

                         Sections &   File data in
@<file> kind  Sentinels?  @others?    .leo file?    Notes
------------  ---------- -----------  ------------  -----
@asis            no         no           yes
@auto            no         yes          no         1, 2
@auto-xx         no         yes          no         1, 2
@clean           no         yes          yes
@edit            no         no           no
@file            yes        yes          no
@nosent          no         yes          yes

It's hard to predict what the exact consequences of changing the kind of the @<file> tree will be!

Aha! There is no Erlang importer

This means that ic.createOutline will call ic.scanUnknownFileType, which adds @language erlang and copies the entire file to the root node.

ic.scanUnknownFileType already contains a few special cases. It would be possible to add an @delims directive for .erl files.

Summary

In Erlang, <<...>> may appear in various contexts, so Erlang's syntax clashes with Leo's default syntax for section references.

As Thomas points out, using the @section-delims directive is a workaround.

@section-delims [[[ ]]]

For the record, I see nothing wrong in using this directive for Erlang.  Just make sure that "[[[" doesn't appear anywhere in the code!

A cff on "delims" reveals 64 matches, including various unit tests. This code will stay exactly as it is.  In particular, Leo's @file read code, fast_at.scan_lines, handles @delims correctly.

Leo's unit tests are a bit light on covering @delims, this issue does not suggest that Leo's code is buggy.

Edward

[Edward K. Ream]

On Mon, Sep 16, 2024 at 3:09 PM Jacob Peck <gates...@gmail.com> wrote:

Interesting.  I’ve been using <<sections>> for 12 years now and have never previously stumbled onto that limitation.  I don’t see it documented, but at the same time I’ve never been good at searching Leo’s docs.

Many languages have << and >> operators, but few languages have an explicit <<...>> syntax.

For example, all C-related languages have bit-shift operators, but it's rare to see these two operators on the same line.

C doesn't much care about line breaks, so it's easy to put these operators on separate lines if Leo should complain about a missing section.

Edward

[rengel]

Confirmed! When using "@section-delims [[[ ]]]" the error disappears.

Thank you!

Is there a way and/or place to define this language-specific?

[Edward K. Ream]

On Tue, Sep 17, 2024 at 6:59 AM 'rengel' via leo-editor <leo-e...@googlegroups.com> wrote:

Confirmed! When using "@section-delims [[[ ]]]" the error disappears.

Thank you!

Is there a way and/or place to define this language-specific?

Not yet. But an Erlang-specific special case in ic.scanUnknownFileType would be reasonable.

More general solutions are possible, but I see no reason to encourage @section-delims.

Edward