show nodes w/ content stored in .leo, not @ <files>

[HansBKK]

I'd like to ensure that **all** content nodes are being stored in my derived-output dirstruc. I see that (most of) the @ <file> types enforce "no orphans" in subtrees, but I'm looking for notification on any "lost orphans" that I neglected to place under the "supervision" of an @ <file> caretaker. Of course the simple answer is "just be careful to make sure. . .", but I'm looking for a way to automate this, without having to actually parse the .leo XML data itself.

Any suggestions?

And while I'm at it, I'm also seeking more general advice, confirmation I'm on the right track in getting to know Leo - feel free to skip the below if my verbosity annoys.

I'm trying to establish a SOP for bringing pre-existing plaintext files into Leo, in this case from external sources where the original file contents must remain exactly as they are. Any and all comments welcome,but especially if you see something I'm doing wrong.

  - Keep external source file in the source-native VCS
  - Import with @edit
  - Change to @shadow
  - Start extracting to "chunks" via section item nodes, creating a "canonical tree"
  - Keep both the original file and the Leo derived-output open in a diff tool, regularly checking the latter matches the former
  - Create "custom trees" - topic-based, function-based, audience-based etc
  - Further split the content into sub-sections to an appropriate level of "granularity" to suit intended later re-use
  - Ensure that every content node is cloned into appropriate locations in the custom trees
  - Once initial-pass extraction is complete, "archive" the canonical tree off where it won't need to be touched,
    -  - unless further, finer-level chunking is needed later, in which case go back to using the diff tool to ensure original is preserved
  - Start to add more "organizing" and "view" nodes, as well as custom content to the custom trees
  - Start inserting @shadow "roll-up" nodes, creating new custom files in a completely separate dirstruc tree from the original source filesystem
  - Put those dirstrucs into own authoring VCS, separate repo from the external source-native one;
    -  - consider putting Leo file in VCS for continuous snapshotting/rollback as well.

If this is a software project's source-code, complete this process with the documentation first, then depending on the language used and my ability/interest, perhaps also start doing the same with the program code itself, inserting clones of snippets into the custom documentation trees, in effect reverse-engineering the source dirstruc into a Leo-based "literate programming" fork.

Even if I'm unable to contribute to the code itself, I should then be in a good position to be able to contribute enhancements to the project's documentation. Leo enables me to output completely customized "packages" of the source project's own content, keeping it in sync with future updates by continuing to update/pull into the "canonical" tree, checking for changes needed to my self-authored nodes as I go along.

OK, sorry this was so long and if you've actually read this far thank you. As I said any and all feedback welcome, but I particularly would like a heads-ups on possible problems from more Leo-knowledgeable people on the implementation details, before I get too much further along in my current project. Seems like a good way to get to know Leo, as well as perhaps taking baby steps down the road toward becoming a "real coder" one day 8-)

[Terry Brown]

On Sun, 11 Dec 2011 19:54:32 -0800 (PST)
HansBKK <han...@gmail.com> wrote:

> I'd like to ensure that **all** content nodes are being stored in my
> derived-output dirstruc.

You can run this script to list all non-@<file> nodes in the log pane.

def report(p):

if not p.v.anyAtFileNodeName():

g.es(p.get_UNL(False))

for i in p.children():
report(i)

for i in c.rootPosition().self_and_siblings():
report(i)

Just paste it into a node and click run-script.

Reading over your importing projects ideas quickly they seem like they
could work. For recognized languages (Python, C, Javascript, rst) you
can use the @auto import to get the initial structure into nodes, and
switch to @shadow after that, if you want.

Cheers -Terry

[HansBKK]

On Monday, December 12, 2011 8:28:11 PM UTC+7, Terry wrote:

You can run this script to list all non-@<file> nodes in the log pane.

Thanks Terry I'll give it a shot.
 

Reading over your importing projects ideas quickly they seem like they

could work.  For recognized languages (Python, C, Javascript, rst) you
can use the @auto import to get the initial structure into nodes, and
switch to @shadow after that, if you want.

No these are just text files, mostly "lightly" marked up with txt2tags, some asciidoc.

I'd love to automate the breaking up into sections based on header format, but when I had a quick look at the "create a new importer" docs I got a bit intimidated. The "extract" command is very useful though.

What I've found a bit frustrating is the fact that that one really does need to be careful about editing in Leo vs outside. When I kept getting "Recovered nodes" and "path changed" errors at one point I said OK, just delete the shadow folder and recreate it, but I ended up with a completely flat @shadow node - all its section children were deleted and the section << marks >> wiped! Fortunately I'd already clones all the section definition nodes over to my custom tree, so it was just a matter of reconstructing their order and copy/pasting the marks back in.

I figure from now on I'll have an "external files backup" tree without any @ <file> directive, just plain text copy and paste of the section marks, off in a hidden corner of the leo file.

And of course don't mess with the shadow folders 8-)

I would have thought on re-reading @shadow nodes, if the leo file contains a conflicting information the Leo would prompt the user, or do a "Recovery node" like when the conflict goes the other way - it's obviously very cautious about the external files, but less so about the internal data. . .

Of course it's likely the root cause is the nut behind the wheel, I've just started figuring Leo out. . .

So far **such** a kewl tool!

 

[HansBKK]

Terry,

The report isn't just listing nodes with content, it includes all the "org" and "view" nodes (of which I have a lot) as well.

Could the criteria exclude nodes with no body content?

And let me know how I can return the favor sometime - testing something from a noob user POV, documentation etc.

Or I'll buy you a beer next time you're in Bangkok 8-)

Thanks,

Hans

[Terry Brown]

On Mon, 12 Dec 2011 06:16:49 -0800 (PST)
HansBKK <han...@gmail.com> wrote:

> Could the criteria exclude nodes with no body content?

def report(p):

if p.b.strip() and not p.v.anyAtFileNodeName():

g.es(p.get_UNL(False))

for i in p.children():
report(i)

for i in c.rootPosition().self_and_siblings():
report(i)

would be nodes with body content only, I think.

Cheers -Terry

[HansBKK]

Thanks for your efforts on this, but I'm afraid the criteria matching varies based on the location of the nodes in the tree.

Also depending on where the script is located in the tree (NBD).

If nodes-with-content-not-being-derived-into-files are present right up to the root, no problem. Any empty nodes in between "block", they don't get reported. Unfortunately everything in my data is organized with empty nodes above. . .

[Terry Brown]

On Mon, 12 Dec 2011 16:48:07 -0800 (PST)
HansBKK <han...@gmail.com> wrote:

> If nodes-with-content-not-being-derived-into-files are present right up to
> the root, no problem. Any empty nodes in between "block", they don't get
> reported. Unfortunately everything in my data is organized with empty nodes
> above. . .

Didn't think of that, what about

def report(p):

if p.v.anyAtFileNodeName():
return

if p.b.strip():

g.es(p.get_UNL(False))

for i in p.children():
report(i)

for i in c.rootPosition().self_and_siblings():
report(i)

Cheers -Terry

[HansBKK]

Works great in order to show the nodes I've missed putting into files.

I had put all the @path directives in the body of my filesystem-mirroring nodes so the headlines just shows a neat
/path/to/directory/

because I'd rather not clutter up the outline

My workaround for the moment is to copy and paste the log into my editor and run a regex like this to strip out the path-only headers:
^\/.*\/\n

I haven't had a look yes at your "filesystem mirroring" plugin, plan to once I've gotten farther along learning Leo basics, but I reckon some functionality like this would fit in well there.

Thanks much for your time man, and I hope one day I'll be able to look at your code and actually have clue how it's doing its thing. . .