Leo doc-generation and Wiki integration - GitIt and Pandocs
HansBKK
I have been advocating the idea of pushing Leo-derived content to DokuWiki as a platform for "wiki-publishing" to enable collaborative/community editing of content (1, 2). I've also talked about the markup syntax/doc generation tool Txt2tags (1, 2 ).
However, I have recently learned of the wiki platform Gitit, which apparently, like DW, also uses plain-text files rather than a database back-end, and integrates not only with git but mercurial (and darcs).
Gitit also incorporates the Pandoc project for its markup syntax, therefore enabling not only markdown but reST as a master source input format, while DokuWiki has its own (yet another unique) markup syntax 8-(
With the increasing likelihood that I'll be using Leo as the centerpiece of my toolchain, plus the fact that Pandoc is much more actively maintained, it's starting to look worth my while to consider switching my "master source" content syntax over from Txt2tags to reST. The only downsides are that Gitit is a Haskell project rather than Python, and one thing I like about Txt2tags is its support for conversion to AsciiDoc, rather than Pandoc's direct output to full-blown DocBook XML - but apparently even that's in the works in Pandoc's dev version.
Anyone in the Leo community using Gitit, especially for use beyond simple code documentation?
Edward K. Ream
> This is highly off-topic for most on the list, so feel free to ignore, but
> anyone using Leo for single-source documentation generation/conversion,
> including future googlers, please reply with comments or notes on your
> experiences.
No experience myself, but thanks for these interesting links.
EKR
Offray Vladimir Luna Cárdenas
El jue 19 ene 2012 21:09:26 COT, HansBKK escribió:
> This is highly off-topic for most on the list, so feel free to ignore, but
> anyone using Leo for single-source documentation generation/conversion,
> including future googlers, please reply with comments or notes on your experiences.
I'm using Leo in that scenario (for writing my Thesis and I hope my
students will use it in a similar fashion), so is not off-topic for me
and not for people who is using Leo primary for documentation.
> I have been advocating the idea of pushing Leo-derived content to DokuWiki as a
> platform for "wiki-publishing" to enable collaborative/community editing of
> content (1<../d/msg/leo-editor/fSzVi1Rh5Tg/uu85satgb9YJ>, 2
> <../d/msg/leo-editor/xf6_HBxlV_c/4RgGYdDh8ywJ>). I've also talked about the
> markup syntax/doc generation tool Txt2tags (1
> <https://groups.google.com/d/msg/leo-editor/nNEnxoohFBM/XkMPQhqhDRsJ>, 2
> <https://groups.google.com/d/msg/leo-editor/HBhBnAyVG3E/UXHC1jq50iYJ> ).
When you suggested DocuWiki I thought of MoinMoin which has also
support of plain files as storage mechanism but is also scalable to
databases if this is needed and it supports reStructuredText and is
made in python, a language that "leonizens" are familiar with.
> However, I have recently learned of the wiki platform Gitit
> <https://github.com/jgm/gitit#readme>, which apparently, like DW, also uses
> plain-text files rather than a database back-end, and integrates not only with
> git but mercurial (and darcs).
>
> Gitit also incorporates the Pandoc<http://johnmacfarlane.net/pandoc/> project
> for its markup syntax, therefore enabling not only markdown but reST as a master
> source input format, while DokuWiki has its own (yet another unique) markup
> syntax 8-(
>
> With the increasing likelihood that I'll be using Leo as the centerpiece of my
> toolchain, plus the fact that Pandoc is much more actively maintained, it's
> starting to look worth my while to consider switching my "master source" content
> syntax over from Txt2tags to reST. The only downsides are that Gitit is a
> Haskell project rather than Python, and one thing I like about Txt2tags is its
> support for conversion to AsciiDoc, rather than Pandoc's direct output to
> full-blown DocBook XML - but apparently even that's in the works in Pandoc's dev
> version.
>
> Anyone in the Leo community using Gitit, especially for use beyond simple code
> documentation?
Now that I'm using Leo + Fossil for my documentation related matters
and distributed work I certainly think that a distributed off-line
collaboration system for documentation is needed and, if MoinMoin can't
support the use of distributed wikis (and seems is not planned or in
development [1][2]) Gitit would be a nice place to start with this idea
and would offer advantages over the non-distributed and outdated Zope's
actual implementation, so, interested ones in the community could offer
an implementation of Gitit. On a related matter one of the problems I
see with actual server technology is its gigantism which concentrates
power in the people who has the resources, knowledge and time to
possess, understand and administer/intervene this technology so a
Global South Test for me about which server technology to choose is:
"it runs from a USB thumb drive?". This, for example, favors
Web2py/Smalltalk instead of Zope and Fossil instead of GitHub. May be
you should put this in the panorama when you judge GitIt or
Haskell/Pandoc. Pandoc, by the way, was for me a compelling reason to
learn Haskell[3] (but I thought that I would learn Smalltalk before)
because it deals elegantly with a problem in the diversity of markup
languages (txt2tags makes something similar but only in one way
translation) and for me the point of using Leo is having a tool to deal
consistently with diversity in the "sub-optimal distopic world of
everything is a file".
[1] http://moinmo.in/PawelPacana/MercurialBackend
[2] http://moinmo.in/MoinMoin2.0
[3] http://learnyouahaskell.com/
We could get philosophical here, and think about different programming
paradigms and languages that implement them with elegant syntaxes, like
Smalltalk, Haskell and Python versus the non elegant ones of .java, php
or ... (put your hated language here) and how this elegant syntaxes,
languages and computer using experience could cross-pollinate. If that
is the case, may be reading some about Combined Object Lambda
Architecture[4] and the comprehensive "Concepts, Techniques, and Models
of Computer Programming" by Van Roy and Haridi would be a nice reading.
Some times I dream of a world connected diversity where all the
problems of computer interaction can be solved by expressing that
diversity in fundamental constructs that respect it at the same time
that bring consistency and interface solving the apparent chaos and
noise.
[4] https://en.wikipedia.org/wiki/COLA_%28software_architecture%29
Cheers,
Offray
Largo84
Rob..........
HansBKK
On Friday, January 20, 2012 5:34:21 PM UTC+7, Offray Vladimir Luna Cárdenas wrote:
When you suggested DocuWiki I thought of MoinMoin which has also support of plain files as storage mechanism but is also scalable to databases if this is needed and it supports reStructuredText and is made in python, a language that "leonizens" are familiar with.
the "sub-optimal distopic world of everything is a file".
I certainly think that a distributed off-line collaboration system for documentation is needed and, if MoinMoin can'tsupport the use of distributed wikis (and seems is not planned or in development
On a related matter one of the problems I see with actual server technology is its gigantism which concentrates
power in the people who has the resources, knowledge and time to possess, understand and administer/intervene this technology so a Global South Test for me about which server technology to choose is: "it runs from a USB thumb drive?".
In my workflow, at the beginning of a session I first sync the relevant data and "portable apps" filesystem branches, then run everything off the local HD. At the end of a session I sync it all back - most of the time these days this is to/from a central filer, but I do have portable drives I use for this when that's required or more convenient, which then get sync'd to the filer next chance I get.
So far I've found that anything that runs under Linux is inherently portable in that sense.
This, for example, favors Web2py/Smalltalk instead of Zope and Fossil instead of GitHub.
For myself, I think mercurial would be a good fit, but my main point is that any moves toward a "distributed Leo" should IMO be VCS-agnostic, just as my plans for enabling community editing of Leo-managed content will be wiki-platform agnostic.
To me, the key enabler for that is "everything as a file". . .
Offray Vladimir Luna Cárdenas
El lun 23 ene 2012 01:18:48 COT, HansBKK escribió:
> Thanks Offray for your detailed and informative response.
Well I'm enjoying also these talks with you. I think that putting
documentation also in the center is required if we want to break the
Leo's self-fulfilled prophesy about being a "ghetto tool" for
programmers only and I want this in the best way.
[..]
>
> I for some reason missed the MoinMoin's "simple page storage" option - thanks so
> much for pointing that out. For all the reasons you cite, and most importantly
> is much more mainstream, more actively developed and well-supported than Gitit,
> I'll definitely give it a higher priority in my testing.
[...]
I enjoyed using MoinMoin for this project:
but at the end we could not intervene MoinMoin as much as we would like
because of the server permissions, that why I started to look more
integrated solutions of the development and deployment environment as
web2py or seaside, but they're not wiki engines properly but web
application frameworks (where you could build a wiki-engine if needed).
But surely
> the "sub-optimal distopic world of everything is a file".
>
>
> I personally disagree with your dislike for "everything is a file" - I see that
> principle as a fundamental part of the *nix tool philosophy, and IMO this is a
> perfect example:
>
> I certainly think that a distributed off-line collaboration system for
> documentation is needed and, if MoinMoin can't
>
> support the use of distributed wikis (and seems is not planned or in development
>
> To my mind, any wiki platform that can store the page data as plain text (as
> opposed to binary/database), in a format suitable for diff tools ("light" markup
> as opposed to html/xml) can make use of whatever VCS for distribution/replication.
You're right and I like the idea of "everything is a something" when
that something is powerful unifying idea. That's the case with Unix's
"everything is a file" or Smalltalk's "everything is an object" (in
Unix you have also every tool makes one thing and makes it right,
combined with pipes). For me these two paradigm's were the ones that,
in 70's, were fighting for the mind share about computer experience of
today and both of them won in a dystopic way, but for me "taking genius
to understand Unix simplicity"[0], was even more dystopic. When you're
trying to empower users the impedance between development and
deployment shows the dystopia, at least compared with the original
visions, so most of the "end users" cant change the tool that changes
them, so, someone else is making decisions about that changes and that
users.
[0] https://en.wikipedia.org/wiki/Unix_philosophy#Quotes
I like the simplicity of light markups and I try myself of not using
explicitly nothing as xml and I like also the idea of the light markup
being used by VCS tools. That's not where dystopia lies. The problem is
not about files or structure but about "meta-structure" (structure
talking about structure, as in meta-languages), specially
self-referential meta-structure, because self-referential
meta-structures are the key for self-directed change, as opposed with
change directed by others. When you see how the "world of everything is
a file" talks about itself, there is a lot of impedance and
discontinuity between source code, binary, apps and docs and there is a
long path for the user who is confined to using apps to create docs,
but never change the apps that could let she/he to change his/her
writing and that's why I want to use Leo this semester with
non-technical writers to explore the way that writing change the tool
that writes and not only the human who does.
For me a unified emergent meta-structure in the world of "everything is
a file" is where lies the power of Leo. You can use an outline to
change the way that Leo behaves and that's why having the Leo's
self-referential meta-structure is more powerful that the "dystopic
world of everything is a file" (in that world you don't have
meta-structure, only structure, mostly for storage purposes and the
intelligence to read/process it is mostly outside the file, in the
human reader, the compiler or the binary). What Leo does is to create
self-referentiality in the world of everything is a file by introducing
outlines that can talk/structure the files and that can talk about
outlines, i.e outlines that can talk about files and about themselves
and can reprogram the way Leo itself works, and so Leo is bridging the
gap between objects and files in a valuable and unique way. But we
need still to improve, specifically we need a more elegant way to talk
about that files, specially about their changes in time, because is in
that change where talking with the distopic world has more problems and
possibilities, and that way I'm making the Fossil/VCS experiment and
also.
Wow I think that this is the first time I have the opportunity to write
(curiously in English instead of my native tongue) about that dystopy,
because most of the time I just talk about this with my students or
friends but not as detailed and contextually, so thanks for bring this
up Hans, and thanks everyone else here who is still reading :-)
> On a related matter one of the problems I see with actual server technology
> is its gigantism which concentrates
> power in the people who has the resources, knowledge and time to possess,
> understand and administer/intervene this technology so a Global South Test
> for me about which server technology to choose is: "it runs from a USB thumb
> drive?".
>
> IMO "server" is a function, not a question of scale or complexity - the better
> question for my workflow is "does the app run portably?". I personally find
> actually running stuff from flash drives too slow and data-dangerous.
I'm agree with you. Server and gigantism have not to be equal, but
unfortunately in the "dystopic informatic world" (where the previous
dystopia is just a part) they're most of the time. Portability is the
key, not flash drives. In my context they're just a medium to ask the
same as you, but also a way to let people take the technology with
them, no matter if they have access to a "classical" server.
>
> So far I've found that anything that runs under Linux is inherently portable in
> that sense.
Agreed. Having Leo + Fossil + Laptop ( ;-P ) gives me some kind of
portability, but we need more. That's why I think that we need a self
contained version of Leo with a default discourse about file flat world
change in time (at least for Windows), but ideally would be nice to
have something like the self-contained multiplatform Pharo's One Click
Experience[1]
[1] http://www.pharo-project.org/pharo-download
> This, for example, favors Web2py/Smalltalk instead of Zope and Fossil
> instead of GitHub.
>
> I haven't any experience with these others, but note that Git does not = GitHub.
> I share your dislike for server/storage platforms out of my direct control, not
> least for privacy/security issues for many use cases. If I used Git for data
> distribution I wouldn't use GitHub, and my understanding is that even "Git for
> Windows" is already fully portable.
Oohh I don't make myself clear, sorry. Fossil compared with GitHub was
not because of the equivalence of Git and GitHub, but because of the
integration of web characteristics of GitHub in Fossil (wiki, tickets,
web interface and so on).
> For myself, I think mercurial would be a good fit, but my main point is that any
> moves toward a "distributed Leo" should IMO be VCS-agnostic, just as my plans
> for enabling community editing of Leo-managed content will be wiki-platform
> agnostic.
I fully share your opinion on that subjects, but in this case I want to
start by some specific implementation from which one start to abstract
that to think abstractly without any particular implementation in the
road, which is not your case, I just point to different implementation
strategies based on the same agnosticism/diversity as a valuable thing
to preserve.
> To me, the key enabler for that is "everything as a file". . .
For me the enabler is self-referential meta-structure
Thanks,
Offray
Offray Vladimir Luna Cárdenas
I made some errors.
In this part:
El lun 23 ene 2012 07:22:21 COT, Offray Vladimir Luna Cárdenas escribió:
>
> but at the end we could not intervene MoinMoin as much as we would
> like because of the server permissions, that why I started to look
> more integrated solutions of the development and deployment
> environment as web2py or seaside, but they're not wiki engines
> properly but web application frameworks (where you could build a
> wiki-engine if needed). But surely
>
"But surely" was not intended.
Cheers,
Offray
HansBKK
documentation also in the center is required if we want to break the Leo's self-fulfilled prophesy about being a "ghetto tool" for programmers only
Usability for clueless noobs is a lot of work, probably harder than the complex whiz-bang functionality part.
And to take something as powerful and flexible as Leo and make it accessible for noobs would require "lobotomizing" it to some extent, at least hiding those features that weren't relevant to the intended more-mainstream task at hand.
I imagine something like "application mode" flag at launch time
- Leo as a journaling tool (like Rednotebook on steroids)
- Leo as an Evernote-style note-taking brain extension
- with user-accessible tagging, perhaps multiple headings per node?
- Leo as a delicous-replacement (import/export/backup) bookmarks management tool
- Leo as a single-source multiple-output documentation management meta-organizer and conversion-supporting tool
etc - very different UX - pane layouts, menu structures etc - for each mode, but the same underlying code and data structures
but at the end we could not intervene MoinMoin as much as we would like because of the server permissions
ACLs is one of DokuWiki's strengths, as they target the corporate world (as much as a FOSS tool can 8-)
> So far I've found that anything that runs under Linux is inherently portable in> that sense.
Agreed. Having Leo + Fossil + Laptop ( ;-P ) gives me some kind of
portability, but we need more.
ideally would be nice to have something like the self-contained multiplatform Pharo's One Click
Experience[1]
integration of web characteristics of GitHub in Fossil (wiki, tickets, web interface and so on).
agnosticism/diversity as a valuable thing to preserve.
Both of these integrate well with the important VCSs, so when I finally get away from SVN and get familiar with the distributed new kids, I can keep my other tools - Redmine/Chili now has such a custom-infrastructure encrusted around it sync'ing with gcal, printing pocketmods for my calendar and @context to-do's that have become indispensable to my day-to-day life management.
"dystopic world of everything is a file" (in that world you don't have meta-structure, only structure, mostly for storage purposes and the intelligence to read/process it is mostly outside the file, in the human reader, the compiler or the binary). What Leo does is to create self-referentiality in the world of everything is a file by introducing outlines that can talk/structure the files and that can talk about outlines, i.e outlines that can talk about files and about themselves
and can reprogram the way Leo itself works, and so Leo is bridging the gap between objects and files in a valuable and unique way. But we need still to improve, specifically we need a more elegant way to talk about that files, specially about their changes in time, because is in that change where talking with the distopic world has more problems and
possibilities, and that way I'm making the Fossil/VCS experiment and also.
> To me, the key enabler for that is "everything as a file". . .
For me the enabler is self-referential meta-structure
The various levels of structural overlays as presented within Leo as "uber-manager of the metadata" can be as flexible and complex as can be, and still be stored/distributed as diffed/versionable/convertable files at whatever appropriate level of granularity to support integration with outside toolchains. To the extent design choices are made that "lock in" to a particular "higher level" technology bet, e.g. a specific database engine or DVCS, then much higher-level programming/sysadmin skills are required in order to integrate Leo into the thousands of mainstream tools that have evolved over time to support structured-plain-text.
Look at source code - after all these decades, it's still stored as plain text in a filesystem. There's a reason for that - any language that required its modules/functions/objects whatever to be stored in a "proprietary" database engine for example would have very limited uptake, because coders would have to put so much effort into infrastructure overhead work to be able to keep working with their preferred toolset. Anyway 'nuff said on that.
Matt Wilkie
> write (curiously in English instead of my native tongue) about
> that dystopy, because most of the time I just talk about this
> with my students or friends but not as detailed and contextually,
> so thanks for bring this up Hans, and thanks everyone else
> here who is still reading :-)
I am reading, and enjoying. The clouds you've placed in my mind are
making interesting shapes and I am intrigued. ;-)
-matt