LaTex to PDF workflow using Leo

[Largo84]

In another post, Israel Hands asked about a straightforward Leo workflow to use LaTex to get PDF final output. So, here is mine for whatever it's worth. Readers will decide for themselves how 'straightforward' it is; it works for me and I use it a lot. Maybe others will get some benefit from it and maybe some will offer suggestions to improve it. If there's interest, I can attach a minimal .leo file that demonstrates some of the following techniques.

Typical Use case

I teach a variety of classes in a business environment and need handouts, teaching aids, worksheets and training manuals that are specifically customized for each client. These documents are easier to manage, print and protect using standard PDFs.

Workflow Overview

1. Document content comes from a primary resource directory arranged by topic (not client specific).
2. I have a Resources.leo file that helps me keep that directory organized.
3. All of the content files are written in LaTex (I use a .txi file extension of my own invention to indicate the file is an 'input' file only, not the main output file which uses .tex).
4. I have a Client.leo file for each client in their own directory to organize work specific to each client.
5. For each document needed for a client project, I create a Document.tex file from a standard template and change the document properties as needed for the specific client, project and document.
6. The Document.tex file acts as the presentation 'shell' for the document and I simply add \input{"\ResourcePath Content.txi"} after the \begin{document} statement (\ResourcePath is a shortcut command to the location of the content resource). This shell determines such things as the document title, document type, client name, header/footer information and revision date.
7. Since I work primarily in Windows, I use TeXNicCenter to process (typeset) the Document.tex file to create PDF output. (I do not use TeXNicCenter for editing, only file processing).

Workflow Notes and Shortcuts

1. Years ago, I discovered the incredible exam class for LaTex and now use it almost exclusively. It makes it much easier to create student and teacher versions of the same content (eg. handout for students and training manual with speaking notes for the teacher).
2. I use the new @outline-data tree-abbreviations in Leo to create each new Document.tex file from a template with variables (very cool!)
3. I created many @data abbreviations in Leo to speed up typing of standard LaTex structures (would be happy to share them if anyone is interested).
4. All document content stays in the Resources directory and only 'shell' documents are in the client directories.
5. These shell documents allow for client-specific information to be added to the headers, footers and in some cases as variables inside the content area itself (using \theClient variable that I define).

Software Needed

1. Leo (of course, and its dependencies).
2. MiKTex for the LaTex distribution and package management (I have it set to auto-update as needed).
3. TeXNicCenter for processing (typesetting) to PDF output.

Regards,

Rob........

[Largo84]

Just posted a Leo file with examples on GitHub here.

Regards,

Rob........

[Israel Hands]

Thanks Rob - will have a look at the example tomorrow. Can you control TexNicCenter from Leo - or do you process the file and preview process manually? Ta

IH

[Largo84]

No, I run processing (typesetting) in TeXNicCenter manually. Typically, I create a blank project file and open however many files required for that project. That way it's easier to clean up the extra files created during the process steps. HTH. There might be a way to invoke the LaTex commands through Leo, but that's way above my pay grade.

Rob...........

[Israel Hands]

Thanks Rob - I've had a quick look and there's a lot there to get my head round - thanks for sharing it. The manual processing is the same in Scrivener which will compile to a Latex file but then I load that into TexStudio for processing and previewing.  I haven't used @outline-data tree-abbreviations  - so I'm going to start there, I'm sure there will be questions to follow!  Ta IH

[Arjan]

Thanks, interesting to see what people are using for LaTeX. I'll describe my process, I'd be happy to hear about improvements. This is all just hardcoded and hacked together, as I currently work on just one LaTeX project.

I believe there are older threads which discuss more elaborate processing schemes, but I wasn't sure how to use those.

I'm using Leo to organize sections like \chapter, \section and \subsection. Since I'm just writing latex in Leo, I need to keep track of the right hierarchies, so I can't freely move nodes around in the hierarchy or I end up with \subsection at the same level as \section, etc. It would be great to be able to let Leo handle this.

MyProject

  - @clean myproject/myproject.tex

  - @clean myproject/references.bib

  - Compile myproject.tex

The compile node has something like this:

@language python

import os
import subprocess
import sys

repository_dir = os.path.abspath(os.curdir)

# The system commands should be run from the folder containing the tex/cls/clo/bib files.
working_dir = os.path.join(repository_dir, 'myproject')
os.chdir(working_dir)

# The commands to run.
run_xelatex = 'xelatex ' + working_dir + os.sep + 'myproject.tex'
run_bibtex =  'bibtex ' + working_dir + os.sep + 'myproject'

g.es('Running XeLaTeX and BibTeX')
# os.system starts a new subshell
# @todo: is it possible to run the below commands in one subshell consecutively?
os.system(run_xelatex)
os.system(run_bibtex)
os.system(run_xelatex)

# Platform-independent file opening
def open_file(filename):
    if sys.platform == "win32":
        os.startfile(filename)
    else:
        opener ="xdg-open"
        subprocess.call([opener, filename])

open_file('myproject.pdf')

Arjan

[Largo84]

Thanks, Arjan for the compile code, will have to try it out on my system.

I had a similar experience, sometimes finding that I had Leo's node structure out of sync with LaTex structure tags. I sort of solved that by putting all content in separate input files (I use .txi as an extension to avoid confusion) and only using LaTex structure elements in the final output shell. Since I mix and match content a lot, this makes it a lot easier for me. It's still possible to have the structure not match up, but it's easier to fix now. If you look at the example Leo file I posted yesterday you will see how I implement that strategy.

HTH

Rob.......

I'm using Leo to organize sections like \chapter, \section and \subsection. Since I'm just writing latex in Leo, I need to keep track of the right hierarchies, so I can't freely move nodes around in the hierarchy or I end up with \subsection at the same level as \section, etc. It would be great to be able to let Leo handle this.

Arjan

[Edward K. Ream]

On Tue, Apr 11, 2017 at 8:46 AM, Arjan <arjan...@gmail.com> wrote:

I'm using Leo to organize sections like \chapter, \section and \subsection. Since I'm just writing latex in Leo, I need to keep track of the right hierarchies, so I can't freely move nodes around in the hierarchy or I end up with \subsection at the same level as \section, etc. It would be great to be able to let Leo handle this.

​This would be an interesting enhancement.

Leo's rst3 command does exactly this, but the code in leo/core/leoRst.py might induce heart palpitations.  There are reasons for all that complexity, and probably none of them relate to latex.

In fact, it should be relatively simple to write a script or plugin to do something useful.  Define the latex level of a node p to be p.level() - root.level(), where root is the root node of your hierarchy or latex markup. Use this computed level to choose the latex markup.

HTH.  Feel free to ask questions or to create an enhancement request.

Edward

[Largo84]

Hmm, not sure how that would work. It might be relatively easy to invoke such a script when setting up the node at first, but what happens if/when you decide to move the node and its level changes? What would be really useful would be a way to traverse the hierarchy after it's written and ready for compiling to verify that each node is labeled correctly. I can't even imagine how that would be possible with a script (I do it manually now). But, I suppose just about anything is possible with enough ingenuity.

Rob.........

[Edward K. Ream]

On Wed, Apr 12, 2017 at 12:10 AM, Largo84 <lar...@gmail.com> wrote:

Hmm, not sure how that would work. It might be relatively easy to invoke such a script when setting up the node at first, but what happens if/when you decide to move the node and its level changes?

​The situation is much like the rst3 command. I envisage invoking the script manually whenever you change the source tree.  That would at least be a good prototype.

Edward

[Josef]

Hello Edward,

much of my work involves editing LaTeX files, but I do have to work together with others. I tried Leo's @shadow and @thin nodes, for minimal interference, with mixed success. The main problem is the way Latex uses multi-file input.

As you may remember, the way the rst plugin automatically translates the Leo hierarchy into headings came from a tiny piece of code I once contributed. So, naturally I also thought about creating a latex plugin with automatic hierarchy translation to headings. That would work well as long as the Latex file is one single, large file.

Most people writing large Latex documents break them down into different chunks and then glue them together with \input. That's where the problems start with using Leo: Leo would have to recreate the same file structure in order to work well together with others, but that would mean one @file statement per .tex file. These @file headings do not necessarily correspond to the document structure. The \input can appear anywhere in the text. An \input may correspond to a new (sub-)section but it also may not. Not every heading will have a corresponding \input either. The Leo tree therefore will not only contain section headings, but also @file nodes corresponding to \input statements and not necessarily corresponding to the document structure.

Also, one must find the master tex file in order to see the complete hierarchy. Since the files may be distributed over different directories, the importer would have to be told about the master file and would then have to look for all the \input statements recursively (and ignoring commented out \inputs).

I don't know how to solve this problem yet.

- Josef

[Largo84]

Josef, I work with large LaTex files also and have for several years; almost all of which contain multiple \input files. My system may not work for you, but it works well for me. A few things to note about my system:

1. My master document outline is a shell only (no actual content). It contains only the preamble, document class and anything else that helps to determine what the document will look like. It's usually auto-created from a set of standard templates.
2. *All* content resides in separate file outline nodes that might be anywhere and almost never is in the same place as the master document file. I generally use either @file (if sentinels are OK) or @clean (if not) for the content files.
3. The master document outline nodes will have a sectioning environment, maybe \section{Some Section} followed by \input{"\path\Some_Section.tex"}. This pulls in the content from Some_File.tex wherever it happens to be.
4. I prefer to *not* include sectioning environments in the content files because for one document, Some_Section content might be a \section{Some Section}, where another document might have the same content as a \subsection{Some Section}.

Good luck and maybe we can share some other ideas to make this work better. I'm not a Leo expert, but I depend on it a lot for my writing. HTH

Rob........

[Arjan]

Reviving this old topic. I made a crude attempt, hard-coded etc., to just rewrite the section definitions via regex search and replace. Works just on a single .tex file (I don't see a benefit to using multiple input files when managing things via Leo anyway), but it's still useful in my case.

How would I add Undo to this? Any other obvious improvements to make?

Cheers

@language python

"""

Changes LaTeX section definition levels in the subtree of an @clean file node to their subtree level.

Only one LaTeX section level can therefore be used within a single node body.

"""

import re

section_levels = {

    1: 'chapter',

    2: 'section',

    3: 'subsection',

    4: 'subsubsection',

    5: 'paragraph',

    6: 'subparagraph'

}

def latex_convert_section_levels(body, adjusted_level_name):

    """ Replaces LaTeX section definition levels found on a single line (re multiline mode).

    Returns the modified node body."""

    newbody = re.sub(r'\\(chapter|section|subsection|subsubsection|paragraph|subparagraph)(\[.*?\])?({.*})',

        r'\\'+adjusted_level_name+r'\g<2>\g<3>', body, re.M)

    return(newbody)

if re.match(r'@clean [A-Za-z\./]+\.tex', p.h):

    root_level = p.level()

    for node in p.subtree():

        level = node.level() - root_level

        if level < 7:

            level_name = section_levels[level]

        else:

            level_name = 'subparagraph'

        node.b = latex_convert_section_levels(node.b, level_name)

    g.es('Updated section levels based on outline structure.')

else:

    g.es('not a LaTeX file node?')

[Edward K. Ream]

On Thursday, March 8, 2018 at 10:48:28 AM UTC-6, Arjan wrote:

Reviving this old topic. I made a crude attempt, hard-coded etc., to just rewrite the section definitions via regex search and replace...

How would I add Undo to this? Any other obvious improvements to make?

Here is a version with undo:

@language python

"""
Changes LaTeX section definition levels in the subtree of an @clean file node to their subtree level.
Only one LaTeX section level can therefore be used within a single node body.
"""

import re

section_levels = {
    1: 'chapter',
    2: 'section',
    3: 'subsection',
    4: 'subsubsection',
    5: 'paragraph',
    6: 'subparagraph'
}

def latex_convert_section_levels(p, adjusted_level_name):

    """ Replaces LaTeX section definition levels found on a single line (re multiline mode).
    Returns the modified node body."""

    return re.sub(r'\\(chapter|section|subsection|subsubsection|paragraph|subparagraph)(\[.*?\])?({.*})',
        r'\\'+adjusted_level_name+r'\g<2>\g<3>', p.b, re.M)

u, undoType = c.undoer, 'change-latex'
h = p.h.strip()
if g.match_word(h, 0, '@clean') and h.endswith('.tex'):
    bunch = u.beforeChangeTree(c.p)
    changed, dirtyVnodeList = 0, []
    root_level = p.level()
    for p in p.subtree():
        level = p.level() - root_level
        if level < 7:

            level_name = section_levels[level]
        else:
            level_name = 'subparagraph'

        s = latex_convert_section_levels(p, level_name)
        if s != p.b:
            bunch2 = u.beforeChangeNodeContents(p)
            p.b = s
            u.afterChangeNodeContents(p, undoType, bunch2, dirtyVnodeList=dirtyVnodeList)
            p.v.setDirty()
            changed += 1
    if changed:
        u.afterChangeTree(c.p, undoType, bunch)
        g.es('Changed %s node%s.' % (changed, g.plural(changed)))
    else:
        g.es('No nodes changed')

else:
    g.es('not a LaTeX file node?')

The only important change besides the addition of undo code was to test for .tex files with the following instead of regex.

 if g.match_word(h, 0, '@clean') and h.endswith('.tex'):

The regex will fail in some cases, especially on Windows:

And that's all.  It's a good @button node.

Edward

[Arjan]

Thanks, Edward!

[Edward K. Ream]

On Mon, Mar 12, 2018 at 5:44 PM, Arjan <arjan...@gmail.com> wrote:

Thanks, Edward!

​You're welcome.

Edward

[Arjan]

The above script used to work, but recently (after not having used it for a long time), I noticed it errors:

exception executing script

TypeError: afterChangeNodeContents() got an unexpected keyword argument 'dirtyVnodeList'

only 6 lines

--------------------

  line 49:             p.b = s

* line 50:             u.afterChangeNodeContents(p, undoType, bunch2, dirtyVnodeList=dirtyVnodeList)

  line 51:             p.v.setDirty()

  line 52:             changed += 1

Has this method changed? What should it look like now to have undo?

Best, Arjan

[tbp1...@gmail.com]

Yes, the method signature has changed.  It's now:

def afterChangeNodeContents(self, p, command, bunch, inHead=False):
    """Create an undo node using d created by beforeChangeNode."""

The command argument seems to be the name of the Leo command that would cause the undoable changes.  Presumably this plays the role of the previous undoType.

[Edward K. Ream]

On Tue, Sep 29, 2020 at 3:01 PM Arjan <arjan...@gmail.com> wrote:

The above script used to work, but recently (after not having used it for a long time), I noticed it errors:

exception executing script

TypeError: afterChangeNodeContents() got an unexpected keyword argument 'dirtyVnodeList'

The way to answer all such questions is to search for the method in leoPy.leo, aka leoPyRef.leo.

Searching for "def afterChangeNodeContents" (without the quotes) yields:

def afterChangeNodeContents(self, p, command, bunch, inHead=False):

If you want more information, do as follows...

To find the commit that changed this line, do:

git blame leo\core\leoUndo.py >blame.txt

Load blame.txt and search for  "def afterChangeNodeContents". You will see:

245a439a34 leo/core/leoUndo.py (Edward K. Ream 2019-12-13 18:56:10 -0600  458)     def afterChangeNodeContents(self, p, command, bunch, inHead=False):

Now do "gitk leo\core\leoUndo.py" and search for 245a439a34

You will see:

QQQ

Author: Edward K. Ream <edre...@gmail.com>  2019-12-13 18:56:10
Committer: Edward K. Ream <edre...@gmail.com>  2019-12-13 18:56:10
Parent: b92dc485b36c60a62a4b8ac2655d9384a1117f89 (Removed dirtyVnodeList from u.afterInsertNode.)
Child:  c36f073f4dad99dea9fa4327af2265ccbfa882f5 (Remove disabled code)
Branches: layouts, master, remotes/origin/layouts and many more (70)
Follows: v6.1, v6.2-dev
Precedes: v6.2-b1

    A mass change, removing all dirtyVnodeList logic.
   
    It remains to be seen whether this really will work.

QQQ

HTH.

Edward