First impressions about nested and some comments

[Offray Luna]

Hi all,

I have just discovered Nested a week ago and I have being using it all the week. I really like it. Here are some comments, trying to make some bridges with another programs which could be useful for some cross-pollination of ideas, so I'll present some features that could be considered for their implementation and also some questions about usage.

= About installation =

We are using the version which appears for download in the main page (non some development release).

The installation process went smooth on Gnu/Linux (Arch) and I image that is even simpler in prepackaged distros. After installing the software works like a charm without any visible problem.

On windows we have some message about not finding some fonts (if I remember) and after installation we have problems trying to get vertical space working on empty lines. You can insert vertical spaces only if the next paragraph have something and not all the time... is an inconsistent behavior. Would be nice to have nested packaged on Windows as a portable app in a similar way to Red Notebook, who also uses txt2tags as their storage format and also exports to LaTeX, pdf and html, but is oriented towards the blog writing (and the html output is not so beautiful as in Nested). One of the nice aspects of Red Notebook is their ability to export to pdf on Windows without needing any TeX/MikTeX extra dependencies installation. May be this is because nested gives support to a superset of txt2tags including math and code highlighting, while Red Notebook keeps itself on txt2tags compliance, but would be nice to give a look to the code that makes this possible.

= On usage =

== On the general idea ==

I like the idea of having a tree as a document and small snippets of nested code in the leafs, so you can have the big picture of the doc in the tree and focus on details in the leafs (seeing the tree and the forest at the same time). This strategy of "divide and conquer" is already applied by the Leo (meta)Editor for reading and writing code and making docs in reStructuredText, but the interface and usage has a stepped learning curve and is mostly oriented towards programmers and installation on Windows is not so easy (mostly because of pyqt and qt dependencies having to be installed manually also). Is also more versatile and a nice thing is its self-referential nature, so you can use Leo to develop and extent Leo itself and some Leo docs (trees) have some automatize options implemented giving the opportunity of creating  new buttons on interface or new behaviors. I think that having development facilities like a command prompt to invoke python programs or having plugins in the form of nested docs would be a killer feature for the Nested Editor app, blurring the dived between doc writers and code writers.

== On html and pdf exports ==

I have some problems with Latin Character support on html export. This is solved by adding the "<meta charset="utf-8" />" property in the head section of the exported file. I tried using the !postproc directive:

%!postproc: (</HEAD>)	<meta name="utf-8" />\n\1

taking the http://txt2tags.org/index.t2t postproc directives as reference but seems not working. Is like if the superset of postproc directives is becoming incompatible with the original subset of txt2tags. Which would be the proper postproc usage to insert "<meta name="utf-8" />" in the header of the xhtml output?

I'm also having problems with pdf output getting this when I tried the export:

"""
! Package inputenc Error: Unicode char \u8:‐ not set up for use with LaTeX.

See the inputenc package documentation for explanation.
Type  H <return>  for immediate help.
 ...                                             
                                                 
l.128 ...iesta que "el gobierno de Bogotá 2012‐
                                                  2016
"""

and this message is displayed in a new window that takes the control over Nested until is closed. My suggestion would be to send this kind of message to a new tab, similar to the already present "Markup" and "Preview". Running pdflatex on the tex exported file works fine. It was related with the presence of hypens in dates, like in "2012-2016" so is better to use "2012 - 2016", but seems more use friendly to give LaTeX support for this kind of writing that is frequently used, to smooth the user experience.

== Possible features for the future ==

I would like to enable 3 features in the tree (inspired by Leo):

- (In)visible nodes: giving the possibility to not export some parts of the document in the outputs, like HTML, plain text or
   LaTeX/pdf.
- Transclusion: having the possibility to create a document from small sub-documents.
- Clones: Having different ways to organize the same nodes in different outputs, so a single document can have the same
  nodes in different subtrees and produce particular outputs from them. This is a complement for the idea of a single source
  and multiple targets, in the sense that, for multiple targets would be nice to not always include the same info.

Thanks for nested again and I hope to help in some way with the implementation of this ideas.

Cheers,

Offray

[Eric Forgeot]

On 31/05/2012 19:38, Offray Luna wrote:

taking the http://txt2tags.org/index.t2t postproc directives as reference but seems not working. Is like if the superset of postproc directives is becoming incompatible with the original subset of txt2tags. Which would be the proper postproc usage to insert "<meta name="utf-8" />" in the header of the xhtml output?

The official solution is to call txt2tags with the "--encoding=UTF-8" option, when using command line, __or__ add:

%!encoding: utf-8

in the header of your txt2tags document.

[Carlos Jenkins]

Hi Ofrray, 

Thank you very much for testing Nested, your feedback is very appreciated.

2012/5/31 Offray Luna <offra...@mutabit.com>

On windows we have some message about not finding some fonts (if I remember) and after installation we have problems trying to get vertical space working on empty lines. You can insert vertical spaces only if the next paragraph have something and not all the time... is an inconsistent behavior.

Windows support is quite difficult, in particular because the same version of Gtk+ behave a little different on Linux that on Windows. The font you're saying are the DejaVu fonts, that actually are packaged with Nested. It is supposed that the Windows installer should register those fonts, maybe it's not doing it or maybe on Windows a restart is required to re-index the available fonts, don't know. 

About the vertical space I would like you to explain it a little more to see if I can fix that behavior.

 

Would be nice to have nested packaged on Windows as a portable app in a similar way to Red Notebook,

That can be done, I just need to change some behavior to not create the configuration folder in user's home and instead use the installation directory. And a packager for PortableApps, or whatever platform you use.

 

who also uses txt2tags as their storage format and also exports to LaTeX, pdf and html, but is oriented towards the blog writing (and the html output is not so beautiful as in Nested). One of the nice aspects of Red Notebook is their ability to export to pdf on Windows without needing any TeX/MikTeX extra dependencies installation. May be this is because nested gives support to a superset of txt2tags including math and code highlighting, while Red Notebook keeps itself on txt2tags compliance, but would be nice to give a look to the code that makes this possible.

I know Red Notebook, their website design is inspired by a design I made for Clementine http://www.clementine-player.org/ jeje. And I stole a lot of ideas (and some code) from them jeje (with the proper attribution and copyright notice). I've already checked that feature, If I remember correctly, the just used Webkit (which is currently unavailable in Nested for Windows) facility to print a website to a pdf file, so the output is the same as if you printed a website.

I think that having development facilities like a command prompt to invoke python programs or having plugins in the form of nested docs would be a killer feature for the Nested Editor app, blurring the dived between doc writers and code writers.

Nested was conceived as a document editor exclusively, because IDEs concern is quite different and it exists already great IDEs out there (I use Geany most of the time and Eclipse for some Java EE stuffs). That design decision impacted a lot the way Nested was written, adding IDE options at this point will require a great rewrite. I'm adding command line options for Nested, so in the future you could publish a document in Nested without the need of opening the GUI.

 

== On html and pdf exports ==

I have some problems with Latin Character support on html export. This is solved by adding the "<meta charset="utf-8" />" property in the head section of the exported file. I tried using the !postproc directive:

%!postproc: (</HEAD>)	<meta name="utf-8" />\n\1

That UTF-8 problem is already fixed in the development version. Also, the postproc/preproc is kinda broke for some cases, in particular when you need to include " and ' characters, like this case... I need to fix that. The problem is Nested, not txt2tags in that case, I assume complete responsibility in screwing that up.

 

Is like if the superset of postproc directives is becoming incompatible with the original subset of txt2tags.

That's quite true. The superset is implemented before or after the txt2tags processing, so you will always have some inconsistencies and incompatibilities. The problem is that txt2tags is now centered on expanding their output catalog, not refining it (as some bugs reports have already some solutions for improving some target and are still unimplemented), nor expanding the language (in particular because expanding the language is not in their scope and one additional feature will need to be implemented in all the currently supported output languages, so... big problem). About this, I don't see any easy solution, other than fork completely txt2tags, remove unused outputs and implement the superset in it's internal workflow. txt2tags is 8000+ lines long, so it's not and easy task. Anyway, the txt2tags packaged in Nested has already a lot of modifications to refine the xhtml strict and latex output.

 

Which would be the proper postproc usage to insert "<meta name="utf-8" />" in the header of the xhtml output?

To patch txt2tags. Try also what Eric is suggesting (%!encoding: utf-8), that would be the correct way to do it in txt2tags.

 

I'm also having problems with pdf output getting this when I tried the export:

"""
! Package inputenc Error: Unicode char \u8:‐ not set up for use with LaTeX.

See the inputenc package documentation for explanation.
Type  H <return>  for immediate help.
 ...                                             
                                                 
l.128 ...iesta que "el gobierno de Bogotá 2012‐
                                                  2016
"""

Have you tried adding:

\usepackage[spanish]{babel}

To the LaTeX header (Document properties >> LaTeX >> Header)

This is the currently LaTeX header I'm using in my documents:

\usepackage[top=3cm, bottom=3cm, left=3cm, right=3cm]{geometry}

\usepackage[spanish]{babel}

\usepackage{float} % To fix images position

\usepackage{courier}

\usepackage{multicol}

% Prettify code documentation

\usepackage{color}

\definecolor{gray97}{gray}{.97}

\definecolor{gray75}{gray}{.75}

\definecolor{gray45}{gray}{.45}

\lstset{ frame=Ltb,

     framerule=0pt,

     aboveskip=0.5cm,

     framextopmargin=3pt,

     framexbottommargin=3pt,

     framexleftmargin=0.4cm,

     framesep=0pt,

     rulesep=.4pt,

     backgroundcolor=\color{gray97},

     rulesepcolor=\color{black},

     %

     stringstyle=\ttfamily,

     showstringspaces = false,

     basicstyle=\footnotesize\ttfamily,

     commentstyle=\color{gray45},

     keywordstyle=\bfseries,

     %

     numbers=left,

     numbersep=15pt,

     numberstyle=\tiny,

     numberfirstline = false,

     breaklines=true,

     xleftmargin=20px,

   }

% Spacing

\addtolength{\parskip}{\baselineskip} % Paragraph spacing

\linespread{1.15} % Lines spacing

\setlength{\plitemsep}{0.5\baselineskip} % List items spacing

\definecolor{deepblue}{RGB}{0,0,61}

\hypersetup{linkcolor=deepblue,} 

 

and this message is displayed in a new window that takes the control over Nested until is closed. My suggestion would be to send this kind of message to a new tab, similar to the already present "Markup" and "Preview".

This has changed on the development version, I developed a LaTeX compilation summary dialog (screenshot attached) which is much more user friendly.

 

Running pdflatex on the tex exported file works fine.

You can set the check "Export to PDF" in the LaTeX properties, it will run pdflatex for you on each publish (If it can find it, it assumes on Linux that pdflatex is located in /usr/bin/pdflatex). Also, in the development version, if you have rubber (compilation system for LaTeX) it will use it instead.

 

It was related with the presence of hypens in dates, like in "2012-2016" so is better to use "2012 - 2016", but seems more use friendly to give LaTeX support for this kind of writing that is frequently used, to smooth the user experience.

I think some text was missing, I'm not getting this comment.

 

== Possible features for the future ==

I would like to enable 3 features in the tree (inspired by Leo):

- (In)visible nodes: giving the possibility to not export some parts of the document in the outputs, like HTML, plain text or
   LaTeX/pdf.

That is easy, I'll add it to the TODO list.

- Transclusion: having the possibility to create a document from small sub-documents.

That is more complicated. I'll think how to do it.

 

- Clones: Having different ways to organize the same nodes in different outputs, so a single document can have the same
  nodes in different subtrees and produce particular outputs from them. This is a complement for the idea of a single source
  and multiple targets, in the sense that, for multiple targets would be nice to not always include the same info.

That is even more complicated, for now you can use target comments, so you can have some content displayed in some target and another content in other target. I know is not near what you're requiring, but maybe you can use it as a workaround.

 

Thanks for nested again and I hope to help in some way with the implementation of this ideas.

You're welcome. I invite you to try the last version directly from the repository and run it in your Arch Linux, the development version has a lot of improvements like spell checking (on Windows using LibreOffice dictionaries), LaTeX log viewer, custom LaTeX title page, better formatting of lists and link button, timestamp based undo/redo (not token based like Nested 1.2.2), search and a lot of bug fixes.

I'm in the middle of a big refactoring and implementing bibliography support for Nested, I expect to have a beta release in two months.

Kind regards

--

Carlos Jenkins

http://carlos.jenkins.co.cr/

[Carlos Jenkins]

2012/5/31 Offray Luna <offra...@mutabit.com>

"""
! Package inputenc Error: Unicode char \u8:‐ not set up for use with LaTeX.

See the inputenc package documentation for explanation.
Type  H <return>  for immediate help.
 ...                                             
                                                 
l.128 ...iesta que "el gobierno de Bogotá 2012‐
                                                  2016
"""

BTW, my LaTeX output is:

\usepackage[utf8]{inputenc}  % char encoding

Without doing anything special, I've checked the source code and all the LaTeX documents are always put as utf-8 documents.

Can you please post the header of the LaTeX document to see what is currently exported. It can be also a bug I've fixed in the development version without knowing.

[Eric Forgeot]

On 01/06/2012 02:55, Carlos Jenkins wrote:

That's quite true. The superset is implemented before or after the txt2tags processing, so you will always have some inconsistencies and incompatibilities. The problem is that txt2tags is now centered on expanding their output catalog, not refining it (as some bugs reports have already some solutions for improving some target and are still unimplemented), nor expanding the language (in particular because expanding the language is not in their scope and one additional feature will

good new: Txt2tags is now going toward a more modular approach. The next release should have this feature, and you can grab the current svn version to discover this: https://code.google.com/p/txt2tags/source/checkout

(see for example: https://code.google.com/p/txt2tags/source/detail?r=938 which is beginning to separate the targets)

need to be implemented in all the currently supported output languages, so... big problem). About this, I don't see any easy solution, other than fork completely txt2tags, remove unused outputs and implement the superset in it's internal workflow.

txt2tags has already a minimal markup which in my opinion cover most cases: specific options can be easily added by using pre and postproc (users can custom txt2tags endlessly). It might be good to add footnotes, but it would only work for LaTeX, almost, because there is no reliable way to add footnotes in html (but it could be discussed, I like the way nested displays footnotes in html)

But I wouldn't like to see, for example, superscript or subscript in txt2tags' default markup, because it's not something which is really often used and one can easily add them with preproc.

Besides, I think most wiki markup (rst, markdown, dokuwiki) don't have those extra markup, or they use html tags to add them (for example < sub > in markdown, dokuwiki).

And when it's too complicated, it couldn't really suit people's need.

Have you something in mind for additional markup which are missing in txt2tags? (I'd be in favor of a code block for example)

Such features could be discussed for "txt2tags v3".

txt2tags is 8000+ lines long, so it's not and easy task. Anyway, the txt2tags packaged in Nested has already a lot of modifications to refine the xhtml strict and latex output.

 

If this could be implemented in the default branch of txt2tags, maybe you could request to be a commiter to txt2tags ?

[Offray Luna]

Hi :),

On 31/05/12 19:55, Carlos Jenkins wrote:
> Hi Ofrray,
>
> Thank you very much for testing Nested, your feedback is very appreciated.

Thank you for your quick response.

> Windows support is quite difficult, in particular because the same version of
> Gtk+ behave a little different on Linux that on Windows. The font you're saying
> are the DejaVu fonts, that actually are packaged with Nested. It is supposed
> that the Windows installer should register those fonts, maybe it's not doing it
> or maybe on Windows a restart is required to re-index the available fonts, don't
> know.
>
> About the vertical space I would like you to explain it a little more to see if
> I can fix that behavior.

May be was a permissions problem. I don't use Windows even when is
preinstalled on a machine, so I installed Nested on a Windows machine at
University. May be we don't have all permissions there to get a proper
behavior. The next time my students use Nested I will try to see again
about the vertical space problem.

> That can be done, I just need to change some behavior to not create the
> configuration folder in user's home and instead use the installation directory.
> And a packager for PortableApps, or whatever platform you use.

Nice. That would help a lot of people carrying nested with them.

> I know Red Notebook, their website design is inspired by a design I made for

> Clementinehttp://www.clementine-player.org/ jeje. And I stole a lot of ideas

> (and some code) from them jeje (with the proper attribution and copyright
> notice). I've already checked that feature, If I remember correctly, the just
> used Webkit (which is currently unavailable in Nested for Windows) facility to
> print a website to a pdf file, so the output is the same as if you printed a
> website.

So, you're also the author of clementine! I remember using it for a
while, before going with songbird, but was a pleasant experience and is
nice to see the code sharing thing between Redmine and Nested. Maybe you
can take a look of Leo. Is kind of difficult to grasp at the beginning,
but is a really powerful outliner and having its ideas in more apps
would be really nice (the ones about invisible nodes, clones and larger
documents made from small ones came from Leo).

> Nested was conceived as a document editor exclusively, because IDEs concern is
> quite different and it exists already great IDEs out there (I use Geany most of
> the time and Eclipse for some Java EE stuffs). That design decision impacted a
> lot the way Nested was written, adding IDE options at this point will require a
> great rewrite. I'm adding command line options for Nested, so in the future you
> could publish a document in Nested without the need of opening the GUI.

Command line usage of Nested is a really killer feature (you read my
mind, even without saying anything in the first mail). This idea of
having a "end user" program and "developer" programs is kind of dystopic
to me, having programs in each extreme (Nested for "end users", Leo for
developers). We have some examples of programs that bridge the gap like
Mathpiper or Smalltalk, but they are niche products. I would like to see
Nested in that place where start to blurring that distinction.

> That UTF-8 problem is already fixed in the development version. Also, the
> postproc/preproc is kinda broke for some cases, in particular when you need to
> include " and ' characters, like this case... I need to fix that. The problem is
> Nested, not txt2tags in that case, I assume complete responsibility in screwing
> that up.

Well I have tested the development version and there is no problem with
utf-8 support in the html export.

>
> Is like if the superset of postproc directives is becoming incompatible with
> the original subset of txt2tags.
>
> That's quite true. The superset is implemented before or after the txt2tags
> processing, so you will always have some inconsistencies and incompatibilities.
> The problem is that txt2tags is now centered on expanding their output catalog,
> not refining it (as some bugs reports have already some solutions for improving
> some target and are still unimplemented), nor expanding the language (in
> particular because expanding the language is not in their scope and one
> additional feature will need to be implemented in all the currently supported
> output languages, so... big problem). About this, I don't see any easy solution,
> other than fork completely txt2tags, remove unused outputs and implement the
> superset in it's internal workflow. txt2tags is 8000+ lines long, so it's not
> and easy task. Anyway, the txt2tags packaged in Nested has already a lot of
> modifications to refine the xhtml strict and latex output.

Seems that the solution proposed by Eric about a modular txt2tags and
asking permissions for writing in their repo is the solution instead of
a complete fork. I have been thinking, before knowing of Nested about
how to extend txt2tags without breaking it severely. I don't know If
what I thought have some sense but (following the example of reST
markup) I thought of having some kind of special comments with
parameters, something like:

%! code-block
```
bash
<some bash code>
```
or

%! code-block
```
latex
<some LaTeX code>
```

or

%! code-block
```
amath
<some ASCII Math code>
```

So if classical txt2tags doesn't understand what to do with code-block
it will present the raw code with their intended syntax to the normal
user in different outputs. Something similar would happen for:

%!img: scale=50%
[myimage.png]

By the way, the code for scaling images putting their size on the name
is strange. I like the transliteration facilities of Nested, is just
strange... but anyway would be nice to have also a scale percentage for
this, say something like [60%-myimage.png]

> Which would be the proper postproc usage to insert "<meta name="utf-8" />"
> in the header of the xhtml output?
>
> To patch txt2tags. Try also what Eric is suggesting (%!encoding: utf-8), that
> would be the correct way to do it in txt2tags.

Yep I had tested this on stable release and was not working. The key was
to used the trunk version.

> I'm also having problems with pdf output getting this when I tried the export:
>
> """
> ! Package inputenc Error: Unicode char \u8:‐ not set up for use with LaTeX.
>
> See the inputenc package documentation for explanation.
> Type H<return> for immediate help.
> ...
>
> l.128 ...iesta que "el gobierno de Bogotá 2012‐
> 2016
> """
>
> Have you tried adding:
>
> \usepackage[spanish]{babel}
>
> To the LaTeX header (Document properties>> LaTeX>> Header)
>
> This is the currently LaTeX header I'm using in my documents:

[...]

Thanks. I will try it in my LaTeX and pdf exports.

>
> and this message is displayed in a new window that takes the control over
> Nested until is closed. My suggestion would be to send this kind of message
> to a new tab, similar to the already present "Markup" and "Preview".
>
> This has changed on the development version, I developed a LaTeX compilation
> summary dialog (screenshot attached) which is much more user friendly.
>
> Running pdflatex on the tex exported file works fine.
>
> You can set the check "Export to PDF" in the LaTeX properties, it will run
> pdflatex for you on each publish (If it can find it, it assumes on Linux that
> pdflatex is located in /usr/bin/pdflatex). Also, in the development version, if
> you have rubber (compilation system for LaTeX) it will use it instead.

I have being using export to pdf directly, but with the modal windows of
the LaTeX message taking control of Nested, using pdflatex was the way
to generate pdf when I had the hyphen problem. With your preamble and
the new modal non obstrusive window, I think that I will not have this
problem.

> - (In)visible nodes: giving the possibility to not export some parts of the
> document in the outputs, like HTML, plain text or
> LaTeX/pdf.
>
> That is easy, I'll add it to the TODO list.

Thanks.

> - Transclusion: having the possibility to create a document from small
> sub-documents.
>
> That is more complicated. I'll think how to do it.
>
> - Clones: Having different ways to organize the same nodes in different
> outputs, so a single document can have the same
> nodes in different subtrees and produce particular outputs from them.
> This is a complement for the idea of a single source
> and multiple targets, in the sense that, for multiple targets would be
> nice to not always include the same info.
>
> That is even more complicated, for now you can use target comments, so you can
> have some content displayed in some target and another content in other target.
> I know is not near what you're requiring, but maybe you can use it as a workaround.

Thanks. Take a look of Leo and its reStructuredText support. It uses xml
and a directed acyclic graph as a way to treat documents. I would not
like to change the storage format from clean txt2tags to clumsy xml, and
I think that, at some point is better to think in documents as graphs
and txt2tags as a plain text for its nodes and maybe YAML or JSON for
all the graph structure, but here I just speculating (hopefully this
will bring better ideas on Nested).

> You're welcome. I invite you to try the last version directly from the
> repository and run it in your Arch Linux, the development version has a lot of
> improvements like spell checking (on Windows using LibreOffice dictionaries),
> LaTeX log viewer, custom LaTeX title page, better formatting of lists and link
> button, timestamp based undo/redo (not token based like Nested 1.2.2), search
> and a lot of bug fixes.
>
> I'm in the middle of a big refactoring and implementing bibliography support for
> Nested, I expect to have a beta release in two months.

Really nice. I'm already enjoying the trunk version and will be waiting
for the beta. I can't promise anything yet, but I would like to play
with the idea of supporting txt2tags in Leo... sadly I'm not a
programmer, but seeing such nice apps is inspiring and one wants to help
with the coding thing.

Cheers,

Offray