An example of a suggested (semi-) literate programming syntax for PreTeXt

[Jones, Bryan]

All,

Per my conversation with Rob, the code below provides an example of a (semi-) literate programming syntax [1] I'd like PreTeXt to support. I've applied it to a script used to build a proposal that I just wrote with some of the PreTeXt community. (It takes a programmer to convert a simple task into a complex program...)

The text and attached file is a bash script; hence, comments begin with a #. Here is my suggested approach:

1. Comments in lines which contain a mix of code and comment, such as:
   mv foo.txt bar.txt # The frobulator expects only bars!
   are not interpreted as PreTeXt.
2. Comments which lack a space after the comment character, such as:
   #!/usr/bin/env bash
   are not interpreted as PreTeXt.
3. All other comments are interpreted as PreTeXt. Leading spaces before the comment character must be preserved, so that comments line up with the surrounding code.
   

For an example of these principles in action, see SourceClassifier.py - a module to lex source code; in this instance, the underlying Python source code contained comments in ReStructuredText/Sphinx (Runestone's previous markup). I'd like to be able to do the same thing, but use PreTeXt as the underlying markup language.

My questions:

1. I would appreciate ideas/thoughts/feedback on this approach.
2. What's the next step -- shall I suggest detailed PreTeXt markup that this file should translate into? (That was Rob's guidance; I hesitate to do that here, for fear of lengthening an already long e-mail)

Thanks!

Bryan

[1] While PreTeXt already provides good support for literate programming (LP) (see https://pretextbook.org/doc/guide/html/topic-literate-programming.html), I advocate a different approach, particularly given the lack of widespread adoption of LP. In my flavor, source code must be usable as-is, with no translation needed to extract it; however, the same program can (hopefully, in the future) be transformed into a PreTeXt document.

#!/usr/bin/env bash

# <section>

#   <title>

#       <c>build-proposal-html</c>: Build the proposal as HTML

#   </title>

#

#   <introduction>

#       <p>

#           This script requires the <pretext/> repository and the

#           <c>xsltproc</c> command to run. It does not support

#           Windows (outside of WSL).

#       </p>

#   </introduction>

#

#   <subsection>

#       <title>Setup</title>

#

# <p>

#   Source a custom file with two path names, <c>PTX</c> and

#   <c>PRO</c>. See <xref ref="paths.sh.template">; you must copy

#   this file to <c>paths.sh</c> then edit it to set these paths

#   based on when they exist on your computer.

# </p>

#

# <p>

#   Get the path to the directory this script is located in; see

#   <url href="https://stackoverflow.com/a/59916">Stack

#   Overflow</url> (hereafter abbreviated SO).

# </p>

DIR="$(dirname "$0")"

# Note that <c>.</c> is bash shorthand for <c>source</c>. For

# alternatives, see <url

# href=http://stackoverflow.com/questions/192292">SO</url>.

. ${DIR}/paths.sh

# <p>

#   Compute a temporary output directory; per <xref

#   ref=".gitignore">, this directory isn't tracked by Git.

#   Therefore, you must manually copy desirable PDF versions into

#   the tracked <c>${PRO}/output/proposal</c>.

# </p>

#

# <p>

#   TODO: Why do we need to define <c>PRO</c> if we can compute

#   <c>DIR</c>?

#</p>

OUT="${PRO}/temp-output"

# Preparation: Use <url

# href="https://linux.die.net/man/1/install">install</url> to copy

# <pretext/> NSF proposal XSLT files into the <c>OUT</c> directory.

# TODO: Why is this needed?

install -d ${PTX}/user ${OUT}

# Overwrite the NSF XSLT files in the pretext repository with the

# custom version from this repository.

cp -a ${PRO}/xsl/nsf-proposal-html.xsl ${PTX}/user

# <p>

#   TODO: I don't understand these instructions.

#   copy images folder (include final dot!)

#   2022 not used yet, check managed directories

# </p>

#cp -a ${PRO}/src/images .

#

#   </subsection>

#

#   <subsection>

#       <title>Supporting functions</title>

#

# <p>

#   Print a banner to stderr, so it's clear where any errors for the

#   current build come from.

# </p>

#

# <p>

#   Parameters: any parameters (<c>$*</c>) are included in the

#   message.

# </p>

banner () {

    # Redirect echo's output to stderr. See <url

    # href="https://stackoverflow.com/a/23550347">SO</url>.

    >&2 echo "*******************************************************"

    >&2 echo "Building $*..."

    >&2 echo "*******************************************************"

}

# <p>Run <c>xsltproc</c> configured to build HTML output.</p>

#

# <p>Parameters:

#   <ul>

#       <li>$1: The <pretext/> source file to build.</li>

#       <li>

#           all other arguments: command-line options to pass to

#           <c>xsltproc</c>.

#       </li>

#   </ul>

# </p>

run_xsltproc () {

    # Per the <url href="http://xmlsoft.org/xslt/xsltproc.html"

    # >xsltproc docs</url>, note that all command-line parameters

    # must come <em>before</em> XSLT and XML filenames<mdash/>hence

    # the ordering in this command. Per <url

    # href="https://stackoverflow.com/a/9057392">SO</url>, the

    # expression <c>${@:2}</c> pass all arguments except for the

    # first one.

    xsltproc --xinclude ${@:2} ${PTX}/user/nsf-proposal-html.xsl ${PRO}/src/$1

}

#   </subsection>

#

#   <subsection>

#       <title>Build process</title>

#

# <p>Perform the build in the <c>OUT</c> directory.</p>

cd ${OUT}

banner "project proposal"

run_xsltproc proposal.xml

banner "project summary"

# The project summary must not have numbered heading, hence the

# <c>numbering.maximum.level</c>.

run_xsltproc summary.ptx --stringparam numbering.maximum.level 0

banner "data management plan"

run_xsltproc data-management-plan.ptx

# Return to the previous directory; see <url

# href="https://unix.stackexchange.com/a/330885">SO</url>.

cd -

#   </subsection>

# </section>

--

Bryan A. Jones, Ph.D.
Associate Professor
Department of Electrical and Computer Engineering
231 Simrall / PO Box 9571
Mississippi State University
Mississippi State, MS 39762
http://www.ece.msstate.edu/~bjones
bjones AT ece DOT msstate DOT edu
voice 662-325-3149
fax 662-325-2298

Our Master, Jesus Christ, is on his way. He'll show up right on
time, his arrival guaranteed by the Blessed and Undisputed Ruler,
High King, High God.
- 1 Tim. 6:14b-15 (The Message)

[Rob Beezer]

Dear Bryan,

Thanks for the extensive example, that is very helpful. I've been thinking
about this one since we last discussed it.

You've set a rather high bar. You explicitly want your file/program to *not* be
XML. But then you want lots of XML inside it. So you'll need to recover all
that XML, as XML, not text.

1. You can xi:include the file, but you *must* use parse="text" since your
file cannot have an overall XML root element. Your previous comments suggested
the include would indicate some structure. It can't. File structure is immaterial.

2. I suggest surrounding the include with

<program language="bash" literate="yes">

(a) There is no support for bash yet (I don't think) but we will want to add to
the languages table the character used for a comment. Just experiment with "#"
hard-coded in a variable for now.

(b) We already have a version of literate programming. So this needs some other
moniker to avoid confusion (sorry, early bird gets the worm). So do not use
@literate. Do use yes/no.

(c) This can be your playground - nobody else is using this attribute, so
nothing you do will impact other parts of PreTeXt once you do the following.

3. The pretext-assembly.xsl stylesheet is the pre-processor. It has several
passes. Add a new template in the "representations" pass that matches on

program[@literate='yes']

4. Your goal is to parse the text/program (a recursive template to go
line-by-line?) and create XML. Both XML to guide your later conversion to LaTeX
and HTML, and to recover/convert the PreTeXt XML into *actual* XML (i.e. nodes,
not characters). You can use unofficial ad-hoc XML here, perhaps namespaced
with the "pi" prefix, to make it harder for authors to abuse.

Almost certainly, you will want to use the exsl:node-set() function to convert
a Result Tree Fragment (text that *looks* like XML) into nodes you can output in
the XML-to-XML conversion that this stylesheet does. Do not use
"disable-output-escaping" - that is a red flag.

5. The pretext/pretext script (not the CLI) has a format "assembly-static"
which you can use for testing. Getting any PreTeXt XML out of the program's
comments and into this output is the first big step. Aligning indentation in
output formats feels like the other big step.

As always, we can chat during Drop-In (but not tomorrow).

Rob

On 7/20/22 20:44, Jones, Bryan wrote:
> All,
>
> Per my conversation with Rob, the code below provides an example of a (semi-)
> literate programming syntax [1] I'd like PreTeXt to support. I've applied it to
> a script used to build a proposal that I just wrote with some of the PreTeXt
> community. (It takes a programmer to convert a simple task into a complex
> program...)
>
> The text and attached file is a bash script; hence, comments begin with a #.
> Here is my suggested approach:
>

> 1. Comments in lines which contain a mix of code and comment, such as:

> mv foo.txt bar.txt # The frobulator expects only bars!
> are not interpreted as PreTeXt.

> 2. Comments which lack a space after the comment character, such as:

> #!/usr/bin/env bash
> are not interpreted as PreTeXt.

> 3. All other comments are interpreted as PreTeXt. Leading spaces before the

> comment character must be preserved, so that comments line up with the
> surrounding code.
>
> For an example of these principles in action, see SourceClassifier.py - a module
> to lex source code

> <https://codechat.readthedocs.io/en/master/CodeChat/SourceClassifier.py.html>;

> in this instance, the underlying Python source code

> <https://codechat.readthedocs.io/en/master/_sources/CodeChat/SourceClassifier.py> contained

> comments in ReStructuredText/Sphinx (Runestone's previous markup). I'd like to
> be able to do the same thing, but use PreTeXt as the underlying markup language.
>

> *My questions:*
>
> 1. I would appreciate ideas/thoughts/feedback on this approach.
> 2. What's the next step -- shall I suggest detailed PreTeXt markup that this

> file should translate into? (That was Rob's guidance; I hesitate to do that
> here, for fear of lengthening an already long e-mail)
>
> Thanks!
>
> Bryan
>
> [1] While PreTeXt already provides good support for literate programming (LP)
> (see https://pretextbook.org/doc/guide/html/topic-literate-programming.html

> <https://pretextbook.org/doc/guide/html/topic-literate-programming.html>), I

> http://www.ece.msstate.edu/~bjones <http://www.ece.msstate.edu/~bjones>

> bjones AT ece DOT msstate DOT edu
> voice 662-325-3149
> fax 662-325-2298
>
> Our Master, Jesus Christ, is on his way. He'll show up right on
> time, his arrival guaranteed by the Blessed and Undisputed Ruler,
> High King, High God.
> - 1 Tim. 6:14b-15 (The Message)
>

> --
> You received this message because you are subscribed to the Google Groups
> "PreTeXt development" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to pretext-dev...@googlegroups.com
> <mailto:pretext-dev...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/pretext-dev/CAHhYbD-Mq%3DQGyOw2OPf1SxKwK1UbFrL6EL2cyaZrmJz7vjt%3D1g%40mail.gmail.com
> <https://groups.google.com/d/msgid/pretext-dev/CAHhYbD-Mq%3DQGyOw2OPf1SxKwK1UbFrL6EL2cyaZrmJz7vjt%3D1g%40mail.gmail.com?utm_medium=email&utm_source=footer>.

[David Farmer]

I am trying to understand how literate programming will be
used in a document. I understand very little, and so I can be
thrown off by a minor understanding.

Here are two lines from your example:

# Note that <c>.</c> is bash shorthand for <c>source</c>. For

# Preparation: Use <url

My understanding is that those become PreTeXt. But, if I take away
the leading "# ", those are not in p tags.

My main question: Is the point that you have a file which is
a valid computer program in a particular language. If you
delete all lines that are not of the form

(whitespace)#(space)(something)

Then on the lines that remain, if you delete
#(space)
then you have a valid PreTeXt document?

I know I have heard you talk about this, but it didn't
make it into my head the previous times.

David

> email to pretext-dev...@googlegroups.com.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/pretext-dev/cc9caaa9-93f2-e848-cd17-85e85f48907d%40ups.edu.
>

[Jones, Bryan]

David,

Thanks for the careful look!

To support the most common case -- of a single-paragraph comment -- I propose that we wrap any text that doesn't begin and end with the same tag in a <p> tag.

The goals are:

1. To define a format that allows programs to run with no pre-processing or modification; the input I envision is source code with PreText-formatted comments.
2. To convert this to a PreTeXt document, we can separate code from comments. The code gets wrapped in <code> blocks, while the comments become standard PreTeXt (mostly). That way, the result is a document-like rendering of the program, where the source code is accompanied by explanatory, well-formatted documentation.

Does that make sense?

Bryan

To view this discussion on the web visit https://groups.google.com/d/msgid/pretext-dev/alpine.LRH.2.21.2207210750400.27280%40li375-150.members.linode.com.

--

Bryan A. Jones, Ph.D.
Associate Professor
Department of Electrical and Computer Engineering
231 Simrall / PO Box 9571
Mississippi State University
Mississippi State, MS 39762
http://www.ece.msstate.edu/~bjones

[David Farmer]

Let's not think only about the most common case.

Suppose this is what you wanted (please adjust if this is not what
you actually want) (All I am trying to do is repeat what I think
has already been said, so I know what is being proposed.)

1. There is some text which is a valid program in some language
(which I will refer to as code + comments)
(Note: The # character might not mark a comment in that language.)

2a. That text will be inserted in a way that the result is a valid
XML document.

2b. And even better, a valid PreTeXt document.

3. When converted to PDF, the comments become visible text, and
the code is formatted appropriately as code.

Step 2 seems to require wrapping the document in some tag.
If that tag is already in PreTeXt, then some new attribute will
indicate that the contents are a literate program. Maybe an
additional new tag will indicate how comments are done in that
language. (Maybe "code" is a suitable tag? Using "p" does not
seem right to me.)

If Step 2 requires a new tag, then please reconsider!

Step 3 involves extracting the contents and converting to a
different form which also is valid PreTeXt. A question is whether
the XSL used by PreTeXt is sufficient to do that conversion.

If 2a happened but 2b did not, then a preprocessor can convert the
non-PreTeXt source to PreTeXt source. But that is a less desirable
approach.

On Thu, 21 Jul 2022, Jones, Bryan wrote:

> David,
>
> Thanks for the careful look!
>
> To support the most common case -- of a single-paragraph comment -- I propose that we wrap any text that doesn't begin
> and end with the same tag in a <p> tag.
>
> The goals are:

> 1. To define a format that allows programs to run with no pre-processing or modification; the input I envision is source
> code with PreText-formatted comments.
> 2. To convert this to a PreTeXt document, we can separate code from comments. The code gets wrapped in <code> blocks,

[Jones, Bryan]

Rob,

Thanks. I've intermingled my responses with your e-mail below.

Bryan

On Thu, Jul 21, 2022 at 6:10 AM Rob Beezer <bee...@ups.edu> wrote:

Dear Bryan,

Thanks for the extensive example, that is very helpful.  I've been thinking
about this one since we last discussed it.

You've set a rather high bar.  You explicitly want your file/program to *not* be
XML.  But then you want lots of XML inside it.  So you'll need to recover all
that XML, as XML, not text.

1.  You can xi:include the file, but you *must* use  parse="text"  since your
file cannot have an overall XML root element.  Your previous comments suggested
the include would indicate some structure.  It can't.  File structure is immaterial.

2.  I suggest surrounding the include with

<program language="bash" literate="yes">

(a) There is no support for bash yet (I don't think) but we will want to add to
the languages table the character used for a comment.  Just experiment with "#"
hard-coded in a variable for now.

(b) We already have a version of literate programming.  So this needs some other
moniker to avoid confusion (sorry, early bird gets the worm).  So do not use
@literate.  Do use yes/no.

(c) This can be your playground - nobody else is using this attribute, so
nothing you do will impact other parts of PreTeXt once you do the following.

So, how about something like:

<program language="bash" codechat="yes">

  <xi:include href="build-proposal-html" parse="text"/>

</program>

3.  The  pretext-assembly.xsl  stylesheet is the pre-processor.  It has several
passes.  Add a new template in the "representations" pass that matches on

program[@literate='yes']

The code I have to transform source code to PreTeXt is written in Python, so I'll need a way to get access to the XML tree, then run the converter on any nodes matching program[@codechat='yes']. It looks like I should modify the code in the xsltproc function in pretext.py -- does this seem reasonable?

 

4.  Your goal is to parse the text/program (a recursive template to go
line-by-line?) and create XML.  Both XML to guide your later conversion to LaTeX
and HTML, and to recover/convert the PreTeXt XML into *actual* XML (i.e. nodes,
not characters).  You can use unofficial ad-hoc XML here, perhaps namespaced
with the "pi" prefix, to make it harder for authors to abuse.

Almost certainly, you will want to use the  exsl:node-set() function to convert
a Result Tree Fragment (text that *looks* like XML) into nodes you can output in
the XML-to-XML conversion that this stylesheet does.  Do not use
"disable-output-escaping" - that is a red flag.

5.  The pretext/pretext script (not the CLI) has a format "assembly-static"
which you can use for testing.  Getting any PreTeXt XML out of the program's
comments and into this output is the first big step.  Aligning indentation in
output formats feels like the other big step.

OK, I can test with that -- thanks!

 

To unsubscribe from this group and stop receiving emails from it, send an email to pretext-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pretext-dev/cc9caaa9-93f2-e848-cd17-85e85f48907d%40ups.edu.

--

Bryan A. Jones, Ph.D.
Associate Professor
Department of Electrical and Computer Engineering
231 Simrall / PO Box 9571
Mississippi State University
Mississippi State, MS 39762
http://www.ece.msstate.edu/~bjones

[Jones, Bryan]

David,

Thanks for thinking about this! Yes to your steps. For step 3, I use Python code to do the source code to PreTeXt transformation -- doing this in XSL would be very challenging.

My current thought it to uses a tag Rob suggested:

<program language="bash" codechat="yes">

Hopefully, my code will look for this tag, then translate anything inside it (which must be source code) to valid PreTeXt.

Bryan

--
You received this message because you are subscribed to the Google Groups "PreTeXt development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pretext-dev...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/pretext-dev/alpine.LRH.2.21.2207241209040.5935%40li375-150.members.linode.com.

[Jones, Bryan]

Rob,

Based on our conversation in today's hangout, here's some C code whose lines may be difficult to classify as either code or commentary (comments preceded only by whitespace). Note that the current approach I use employs >1000 lines of Python and relies on Pygments to do the lexical analysis of the underlying language. I'm concerned that translating this to XSLT would be difficult or impossible. In addition, recognizing Python docstrings is challenging -- my code uses Python's AST module to parse and extract docstrings, again difficult / impossible in XSLT. 

#include <stdio.h>

#include <stddef.h>

/* This is a comment. It should be treated as commentary.

// This comment should include the double slashes.

This comment includes an open quote " but not a closing quote, which does

not make the rest of the file part of a string.

*/

#define A_MACRO "This string spans\

// NOT A COMMENT!\

multiple lines.\n"

int main(void) {

    printf(A_MACRO);

    // See https://www.open-std.org/jtc1/sc22/wg14/www/docs/n1256.pdf#page=74

    // for the C99 standard on string literals.

    printf("This string also\

    // NOT A COMMENT!\

    spans multiple lines.\n");

    printf("Strings may contain escaped double-quotes: \""

    // This comment is commentary!

    " and escaped backslashes: \\"

    // This comment is commentary!

    "\n"

    );

    // Because the following open quote is inside a character literal, the rest

    // of this file is not inside a string.

    printf("A C character literal: %c\n", '"');

    printf("Bye!\n"); /* Note that

    // This comment is part of a multi-line comment, and should not be treated

    // as "commentary".

    */

    /* In contrast, this comment IS commentary. */

    // Commentary: another case to consider (wide character strings). Note

    // that C++ adds a several additional prefixes (u8, u, U, plus all these

    // with R for raw strings). See

    // https://en.cppreference.com/w/cpp/language/string_literal.

    wchar_t* wstr = L"This is a wide character string.";

}

Here's the output I get using gcc v9:

bjones@gv41y33:/mnt/c/Users/bjones/Documents/git/CodeChat/CodeChat$ gcc tricky.c; ./a.out
This string spans// NOT A COMMENT!multiple lines.
This string also    // NOT A COMMENT!    spans multiple lines.
Strings may contain escaped double-quotes: " and escaped backslashes: \
A C character literal: "
Bye!

Bryan

[Rob Beezer]

Dear Bryan,

Thanks very much for the detailed example. That helps.

It strikes me that the characters for markup which indicate comments are being
asked to indicate comments in some cases, but in other cases maybe they do not
need to be escaped to retain other meanings. Yes, I just said markup, and I
know this is really a part of the language grammar/definition. But of course,
for PreTeXt source I think entirely about sensible markup.

An idea I had after yesterday's discussion. Author provides markup within a
"program" element that is alternately commentary and code (with element names to
be decided). Then to create an executable program file a really simple XSL
stylesheet produces legitimate program files (such as the one you just
provided). This would be a single command-line with our pretext/pretext
script, which could possibly be included in the CLI.

Rob

On 8/9/22 19:12, Jones, Bryan wrote:
> Rob,
>
> Based on our conversation in today's hangout, here's some C code whose lines may
> be difficult to classify as either code or commentary (comments preceded only by
> whitespace). Note that the current approach I use employs >1000 lines of Python

> <https://codechat.readthedocs.io/en/master/CodeChat/SourceClassifier.py.html>
> and relies on Pygments <https://pygments.org/> to do the lexical analysis of the

> underlying language. I'm concerned that translating this to XSLT would be
> difficult or impossible. In addition, recognizing Python docstrings

> <https://peps.python.org/pep-0257/> is challenging -- my code uses Python's AST

> <https://en.cppreference.com/w/cpp/language/string_literal>.

> code in the xsltprocfunction in pretext.py-- does this seem reasonable?

> <http://www.ece.msstate.edu/~bjones> <http://www.ece.msstate.edu/~bjones

> <http://www.ece.msstate.edu/~bjones>>
> > bjones AT ece DOT msstate DOT edu
> > voice 662-325-3149
> > fax 662-325-2298
> >
> > Our Master, Jesus Christ, is on his way. He'll show up right on
> > time, his arrival guaranteed by the Blessed and Undisputed Ruler,
> > High King, High God.
> > - 1 Tim. 6:14b-15 (The Message)
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups
> > "PreTeXt development" group.
> > To unsubscribe from this group and stop receiving emails from it,
> send an email
> > to pretext-dev...@googlegroups.com

> <mailto:pretext-dev%2Bunsu...@googlegroups.com>
> > <mailto:pretext-dev...@googlegroups.com
> <mailto:pretext-dev%2Bunsu...@googlegroups.com>>.

> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/pretext-dev/CAHhYbD-Mq%3DQGyOw2OPf1SxKwK1UbFrL6EL2cyaZrmJz7vjt%3D1g%40mail.gmail.com
> <https://groups.google.com/d/msgid/pretext-dev/CAHhYbD-Mq%3DQGyOw2OPf1SxKwK1UbFrL6EL2cyaZrmJz7vjt%3D1g%40mail.gmail.com>
>
> >

> <https://groups.google.com/d/msgid/pretext-dev/CAHhYbD-Mq%3DQGyOw2OPf1SxKwK1UbFrL6EL2cyaZrmJz7vjt%3D1g%40mail.gmail.com?utm_medium=email&utm_source=footer

> <https://groups.google.com/d/msgid/pretext-dev/CAHhYbD-Mq%3DQGyOw2OPf1SxKwK1UbFrL6EL2cyaZrmJz7vjt%3D1g%40mail.gmail.com?utm_medium=email&utm_source=footer>>.
>
> --
> You received this message because you are subscribed to the Google
> Groups "PreTeXt development" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pretext-dev...@googlegroups.com

> <mailto:pretext-dev%2Bunsu...@googlegroups.com>.

> To view this discussion on the web visit

> https://groups.google.com/d/msgid/pretext-dev/cc9caaa9-93f2-e848-cd17-85e85f48907d%40ups.edu
> <https://groups.google.com/d/msgid/pretext-dev/cc9caaa9-93f2-e848-cd17-85e85f48907d%40ups.edu>.

> https://groups.google.com/d/msgid/pretext-dev/CAHhYbD9O6F6EKNdazOFhek3giRY3Zmr5OiEjAR_1McTkusUx7Q%40mail.gmail.com
> <https://groups.google.com/d/msgid/pretext-dev/CAHhYbD9O6F6EKNdazOFhek3giRY3Zmr5OiEjAR_1McTkusUx7Q%40mail.gmail.com?utm_medium=email&utm_source=footer>.

[Jones, Bryan]

Rob,

Thanks for working on this with me!

It strikes me that the characters for markup which indicate comments are being

asked to indicate comments [Bryan: I think you mean commentary?] in some cases, but in other cases maybe they do not

need to be escaped to retain other meanings.

I don't understand -- would you provide a short example?

An idea I had after yesterday's discussion.  Author provides markup within a
"program" element that is alternately commentary and code (with element names to
be decided).  Then to create an executable program file a really simple XSL
stylesheet produces legitimate program files (such as the one you just
provided).  This would be a single command-line with our  pretext/pretext
script, which could possibly be included in the CLI.

To make sure I understand your idea: a file named foo.c.ptx contains commentary and code inside appropriate-named tags (let's say <code> and <comment> for simplicity). With that, an author could:

1. "Tangle"  it (to use Knuth's terminology) via a XSLT stylesheet to foo.c, or
2. <xi:include href="foo.c.ptx"> it, which would incorporate it into a PreTeXt document? Or simply include its contents directly inside an existing PreTeXt document, instead of in an external file, depending on the author's preferences.

Bryan

To unsubscribe from this group and stop receiving emails from it, send an email to pretext-dev...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pretext-dev/69d07824-6065-1d50-c76c-65afa70f9953%40ups.edu.

--

Bryan A. Jones, Ph.D.
Associate Professor
Department of Electrical and Computer Engineering
231 Simrall / PO Box 9571
Mississippi State University
Mississippi State, MS 39762
http://www.ece.msstate.edu/~bjones

[Rob Beezer]

On 8/10/22 15:26, Jones, Bryan wrote:
> I don't understand -- would you provide a short example?

I guess I mean that the character(s) indicating comments are "overloaded" in the
context of the definition of the language. (But here and now we want them to be
crude markup).

> To make sure I understand your idea: a file named foo.c.ptxcontains commentary

> and code inside appropriate-named tags (let's say <code>and <comment>for
> simplicity). With that, an author could:
>

> 1. "Tangle"  it (to use Knuth's terminology) via a XSLT stylesheet to foo.c, or
> 2. <xi:include href="foo.c.ptx">it, which would incorporate it into a PreTeXt

> document? Or simply include its contents directly inside an existing PreTeXt
> document, instead of in an external file, depending on the author's preferences.

Exactly. Except to the XSL processor, the two parts of (2) are nearly
identical. In other words, it is a conveniece of sorts for the autnor (and
would have no bearing on (1)). And I was trying not to say "tangle". ;-) In
part, because it is more like "reconstitution".

Rob

[Jones, Bryan]

Rob,

This seems to be identical to PreTeXt's current implementation of literate programming -- or am I missing something?

Unfortunately, the idea of using something that's not source code as the basis for literate programming seems doomed to limited acceptance from the programming community -- although Knuth claimed literate programming as one of his most important ideas, few adopted it or any of its variants. What has worked (Doxygen, Javadoc, etc.) is embedding documentation in source code, which is the approach I've taken with CodeChat. The primary argument for embedding literate programming in source code is that so many tools work directly on source code (a lot of IDEs provide the ability to rename variables globally, find the definition / calls of a function, and much more; there are also lots of static analysis tools like lint, code coverage analyzers, memory leak detection, etc.)

My proposal depends on treating source code as the input to PreTeXt, rather than an output from PreTeXt

Bryan

--
You received this message because you are subscribed to the Google Groups "PreTeXt development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pretext-dev...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/pretext-dev/ef14ae57-1150-b848-19e8-91a7ee489690%40ups.edu.

[Rob Beezer]

> This seems to be identical to PreTeXt's current implementation of literate
programming -- or am I missing something?

Any similarity may be due to the fact that we have pretty reliable procedures
for design decisions, which leads to a unified look for authors. But I have not
even reminded myself about current support throughout this discussion.

First difference: the order of the code in PreTeXt does not need to match the
order of the code to satisfy the compliler. Bottom-up, top-down,
meet-in-the-middle, or all three at once. Take your pick. Based on my
experience, I consider this the best feature of literate programming. You've
wanted to maintain an order that will satisfy a compiler.

Second difference: comments/commentary are placed between "fragments" of code
and give the full range of PreTeXt constructs to be placed in-between and
describe the code. Rather than being indented and integrated much more closely
with the adjacent code.

> The primary argument for embedding literate programming in source code

PreTeXt is about communicating ideas. The utility of having portions be used by
IDEs or linters is not one of our goals. I would expect an author to create a
program that illustrates some idea or concept, using whatever tools are
convenient or necessary. Then that can be placed in PreTeXt, with associated
markup, so it can be communicated in a variety of formats.

> My proposal depends on treating source code as the input to PreTeXt, rather
than an output from PreTeXt

So it appears that will not be possible. I'm sorry we couldn't find a way to
make it happen. Thanks for all the careful discussion.

Rob

[Jones, Bryan]

Rob,

Thanks for thinking through this with me, and taking the time to consider adding CodeChat to PreTeXt! If there's any way to make this happen in the future, I'm certainly open to it. As I understand it: the main point of departure is the requirement that CodeChat would require a Python processing step, rather than working on XSLT? That is, if I find a way to implement CodeChat in XSLT, you'd be open to integrating CodeChat into PreTeXt?

Bryan

--
You received this message because you are subscribed to the Google Groups "PreTeXt development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pretext-dev...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/pretext-dev/3624cd14-ae39-0a4b-d677-b4b299d93196%40ups.edu.

[Jones, Bryan]

Rob,

Any similarity may be due to the fact that we have pretty reliable procedures
for design decisions, which leads to a unified look for authors.  But I have not
even reminded myself about current support throughout this discussion.

 

First difference: the order of the code in PreTeXt does not need to match the
order of the code to satisfy the compliler.  Bottom-up, top-down,
meet-in-the-middle, or all three at once.  Take your pick.  Based on my
experience, I consider this the best feature of literate programming.  You've
wanted to maintain an order that will satisfy a compiler.

 

Second difference: comments/commentary are placed between "fragments" of code
and give the full range of PreTeXt constructs to be placed in-between and
describe the code.  Rather than being indented and integrated much more closely
with the adjacent code.

Good points -- I see these differences now. Thanks!

Bryan

--
You received this message because you are subscribed to the Google Groups "PreTeXt development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pretext-dev...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/pretext-dev/3624cd14-ae39-0a4b-d677-b4b299d93196%40ups.edu.