Syntax coloring cloned node

[lnoorde...@gmail.com]

Hi,

I'm in the process of writing a programming book.

The literate programming concept is attractive because it uses the code in two distinct ways:
a) as program source
b) as part of the book contents
Important here is that they are exactly the same lines being used in two different contexts.

The usual literate programming tools however leave room for improvement, and so I found Leo.

So I have a section of code annotated with
@language assembly_x86
which displays well in the code branch.

When I clone this section into the book branch it needs the sphinx annotation
.. code:: nasm
or else it isnt recognized for block layout (all lines of assembly are displayed on a single line in the browser) and syntax highlighting.

So far I have failed to get Leo to use the cloned section to display properly in both contexts.

I've tried anonymous sections with @others, named sections <<section>> with and without preceding the named sections with rst annotation but it just doesnt come out well.

I am really not very familiar with Leo yet, but I do understand the importance of cloning sections and would love to use Leo for my project.
So is there some way that I can properly display a section with assembly code in both contexts?
Or am I using Leo improperly and how should I organize my work to 'Leonize it'.

thanks in advance,
Leo

[Edward K. Ream]

On Sun, Sep 25, 2016 at 9:20 AM, <lnoorde...@gmail.com> wrote:

I'm in the process of writing a programming book.

​..​

The usual literate programming tools however leave room for improvement, and so I found Leo.

​Welcome aboard.​

 

So I have a section of code annotated with
@language assembly_x86
which displays well in the code branch.

When I clone this section into the book branch it needs the sphinx annotation

​​

.. code:: nasm
or else it

​ isn't recognized for block layout (all lines of assembly are displayed on a single line in the browser) and syntax highlighting.

So far I have failed to get Leo to use the cloned section to display properly in both contexts.

​​

 You have identified what could be called the Holy Grail of documentation

​, namely to use a single source that can generate different text depending on context.

In all such "challenging" situations, the most straightforward approach is to use a script to create "helper" nodes from you list of clones.  Here is a tested prototype:

use_tags = False
if use_tags:
    tc = c.theTagController
    assert tc, 'when use_tags is True this script requires nodetags.py plugin'
root1 = g.findNodeAnywhere(c, 'my book')
assert root1, 'my book not found'
root2 = g.findNodeAnywhere(c, 'my book (sphinx)')
if root2:
    while root2.hasChildren():
        root2.firstChild().doDelete()
else:
    last = c.lastTopLevel()
    root2 = last.insertAfter()
nasm = '​.. code:: nasm\n\n'
for p in root1.self_and_subtree():
    if not use_tags or use_tags and 'nasm' in tc.get_tags(p):
        print(p.h)
        p2 = root2.insertAsLastChild()
        p2.h = p.h
        p2.b = nasm + p.b
root2.expand()
c.redraw()

​This creates a copy (not clones) of the previously cloned nodes.  If use_tags is true, it adds '​.. code:: nasm\n\n' to all nodes tagged with 'nasm'.  Otherwise, it adds
'​.. code:: nasm\n\n' to all nodes.

Again, this is just a prototype, but it shows that relatively simple Leo scripts can massage text in any desired manner.  This script could be placed in an @button node.

Please study this script (running it would be best) and ask any further questions you like.

Edward

[Edward K. Ream]

On Mon, Sep 26, 2016 at 6:21 AM, Edward K. Ream <edre...@gmail.com> wrote:

Here is a tested prototype:

​Oops.  The script works properly only if there is already a 'my book (sphinx)'​ node.

To fix this, add the line:

    root2.h = 'my book (sphinx)'​

after the line:

    root2 = last.insertAfter()

​EKR​

[lnoorde...@gmail.com]

Hi Edward,

Thanks for helping out.
To be frank I find your solution of copying nodes isn't quite as elegant as cloning.
So I kept fooling around a bit and found out that when using the @clean node to create the rst files and then invoking make by hand I get very satisfactory results.

Doing so I lost the rst3 support, but I was using sphinx anyway.
It dit get me:
a) cloned nodes
b) syntax coloring in both source and documentation

Left to do:
* clean up a bit, using proper defaults would make things simpler
* figure out how to create a button that traverses the tree and will call make on any Makefile node it finds (your script could be instrumental to that).

Thanks for your support so far.
When I'm happy with the results I will post the resulting .leo file.

Leo

[vitalije]

One possible solution would be to use Leo-UNL instead of clones. For example in your @rst file you put somewhere following:

.. code:: nasm
    LEOUNL:: <here paste leo unl path to desired node>

Then run the following script (could be turned in to button) to generate your rst files:

@first # -*- coding:utf-8 -*-
import re
import os
LEOUNLpat = re.compile(u'LEOUNL::([^#]*)#(.*)$', re.M)
def process_rst_files():
    for p1 in c.allNodes_iter():
        if p1.h.startswith(u'@rst '):
            fname = p1.h[5:].strip()
            g.es(u'processing %s'%fname)
            do_one_file(p1, fname)

def do_one_file(p1, fname):
    c.rstCommands.processTree(p1, None, toString=True, justOneFile=True)[0]
    fname = os.path.join(c.getNodePath(p), fname + '.txt')
    s = c.rstCommands.source.decode('utf-8')
    s = putLeoCode(s).encode('utf-8')
    s1 = getfile(fname)
    if s1 != s:
        putfile(fname, s)
        g.es('changed')
    else:
        g.es('unchanged')

def getfile(fname):
    try:
        s = open(fname,'rb').read()
    except IOError:
        s = ""
    return s

def putfile(fname, s):
    out = open(fname, 'wb')
    out.write(s)
    out.close()

def putLeoCode(txt):
    i = 0
    res = []
    while i < len(txt):
        m = LEOUNLpat.search(txt, i)
        if not m:
            res.append(txt[i:])
            break
        else:
            res.append(txt[i:m.start()])
            ind = u'\n' + indentation(txt, m.start())
            leofile = m.group(1)
            unl = m.group(2)
            found, depth, p1 = g.recursiveUNLSearch(unl.split(u"-->"), c)
            if found:
                kod = c.p.b.splitlines(False)
                res.append(ind.join(kod))
            else:
                print (u"Leofile:"+leofile)
                print (u"unl:[%s]"%unl)
                print (u'not found')
                raise ValueError, unl
            i = m.end()
    return u''.join(res)

def indentation(txt, i):
    j = i
    while j > 0 and txt[j-1] == ' ':
        j -= 1
    return ' '*(i-j)
process_rst_files()

I am still using python2. If you use python3 I believe it would complain a little about syntax. In that case remove all .encode('utf-8') and .decode('utf-8') and all string literals that are prefixed with u should be without that prefix.
HTH VItalije

[Edward K. Ream]

On Mon, Sep 26, 2016 at 3:20 PM, <lnoorde...@gmail.com> wrote:

​...​your solution of copying nodes isn't quite as elegant as cloning.

​I agree.  However, munging text with a script is the most general solution.
​

 

So I kept fooling around a bit and found out that when using the @clean node to create the rst files and then invoking make by hand I get very satisfactory results.

​Excellent.​

 

EKR

[Edward K. Ream]

On Tue, Sep 27, 2016 at 3:44 AM, vitalije <vita...@gmail.com> wrote:

One possible solution would be to use Leo-UNL instead of clones. For example in your @rst file you put somewhere following:

.. code:: nasm
    LEOUNL:: <here paste leo unl path to desired node>

Then run the following script (could be turned in to button) to generate your rst files:

Good work.  Scripts are the way to get exactly what we want.  Once you go down this path, you will never be satisfied with flat text.

There are many possible ways to "mark" text for inclusion in documents. Which is best for you depends on your circumstances and preferences.

EKR