import from Org-Mode

[Shay Shaked]

I'm looking around and I see a lot of Emacs references, a few less (but still a lot) Org-Mode references, but I don't see something that answers the question itself (yet): is there a way to export from Org-mode to TW. 

I started playing around with Org-mode recently and the writing experience is great. I would really like to be able to export to TW (tid) format. The closet I saw was this on tiddlywiki.com but I'm not sure how to actually use it. It doesn't font-lock (highlight) anything in the syntax really. Besides, I would really like to be able to write in Org-mode, if it exists. 

So... anyone here familiar with this? Anyone here uses Org to write and work with TW at the same time? 

[TonyM]

Shay,

Your question depends on someone knowing the emacs and org-mode references you have made (I do not), and someone may very well know what you are talking about, however if you opened this question up to include examples, such that people with extensive tiddlywiki but not emacs org experience could understand what may be needed, you will most likely get more answers and sooner.

Regards

Tony

[@TiddlyTweeter]

The full Org Mode of Emacs would require a parser for TW. Org mode is basically an "outliner" but it's actually about as sophisticated as you can get in plain text in what it does. It would need interest. Probably the central issue would be whether TW does it's functions already (in a different way). I agree its powerful plain simplicity is attractive.

[Shay Shaked]

OK, let me make an attempt at explaining better... slightly. 

Org-mode works within Emacs. It's hard to fully explain Org-mode because it has many, many features. But, for the needs of converting to .tid files, we don't need even half of it's features. 

Org-mode in its' basic for (an .org file) would look something like this: 

* Header one

** Header two

*** Header three

[[url][link]] 

/italics/

*bold*

Notice how close this already is to TW. 

The more problematic parts would be, maybe, adding tags which in Org Modes are added to header only in the form of :tag: (it's all text-based). It will be something very very simple, that switches * for !, and can "translate" links to what I described above. Org-mode does much much more, but for the purposes of writing (for me anyway) this is enough. The trick would be then, to find all the "bullet points" in TW and change them to headers; find the links and change the to TW format. 

I tried to convert from Org-mode to markdown and from markdown to TW which kinda works... but it looks like a weird way to get at it. There's tiddlywiki-mode, which I linked above. This should "plug into" Emacs, and allow to edit .tid files... just not sure how. Was hoping someone here would know. 

Hope this sheds more light. Org-mode (imo) makes a lot of sense for writing, and for those of us who already use TW on a regular basis, I think there's an obvious match here -- TW is good to present and conserve information long term, Org-mode is the dynamic, quick task-to-do and thought to capture thing I always needed from TW but couldn't use it for because it's rather static, requires revisions, and doesn't work well on phone (this is my opinion of course). The both complete each other rather well. 

[Mark S.]

Orgmode has tags ... and it has headers that have relationships to each other that would ordinarily be represented also by tags.

If I recall, you can export orgmode to html. Then you could use the slicer edition to cut the html up into tiddlers. Of course, that wouldn't give you any of the semantic tags -- just the structural ones.

-- Mark

[Mark S.]

Comment #2

The problem with orgmode was that it was too easy for one note to "leak" into the next with just a single keystroke. The Android app didn't work or look anything like the real orgmode, and the synchronization was unreliable.  And, like all emacs productions, you needed to memorize a really arcane series of key strokes for most everything. Getting it set up on different platforms could be time-consuming, and it was tricky to get it to display images. It did have a lot of features and robust user base.

-- Mark

[TonyM]

Thanks Shay,

I would find it easier to address the problems you raise with tiddlyWiki "couldn't use it for because it's rather static, requires revisions, and doesn't work well on phone " which for my uses, seems untrue.

Many conversations proceed ours about alternative markup/mark down, parsing, perhaps search and replace or the "text slicer Plugin" can sometimes solve this so let us hope someone has followed your path before.

I know it is another step but I do favour a single tool (not tiddlyWiki) that can translate any format into any other including tiddlywiki and back rather that catering for every format in TiddlyWiki, though plugins would be acceptable it would only convert a to b and b to a.

Best of luck

Tony

[LDL]

You can try this exporter: https://github.com/dfeich/org8-wikiexporters that should give you a tid file via the standard export interface of org-mode

[Shay Shaked]

This looks very promising LDL, but it gives me an error (unknown keyword: export-block). This is in line 47 in ox-tiddly.el. I'm no programmer, but it looks like "export block" is outdated and needs to be updated to whatever the new syntax is.

As for the discussion going on with Tony and Mark, it is interesting... but not sure it's part of this topic (which is just how to make it work?). 

So I will include my thoughts here, but I have a feeling this might lead us in a different direction :p

Text-slicer is news to me. It's close but doesn't exactly address the issue which is the difference in syntax. Further, my Org files are already usually rather short and built  up as tiddler, it's not a matter of breaking them down but from changing what would be a bullet point (*) to a header (!) in TW. 

There seems like there should be a way where I can export to HTML (easy enough to do with Org) and then import as tiddler with TW? Does TW do that well, assuming the HTML does not include anything too "fancy"? 

Again, all I need is handling of headers, links, and that's it mostly. I will gladly add tags (that are different on my TW anyway) and provide a name for the tiddler myself. The format is so close. It's just that typing in Org-mode is much easier and effective, for me,  and comes with the added benefits of spell checker (doesn't always work in TW). 

I can talk more about the benefits of writing in Org-mode, but I think that's outside the scope of this discussion -- to each their own. That said, I wouldn't recommend anyone try Org-mode just because they want to write in TW... that's crazy talk. There must be other reasons. 

[Shay Shaked]

So seems like I answered my own question, mostly anyway, without realizing. 

Since TW handles raw HTML rather well, it's rather easy to just export Org to HTML to a buffer (in Emacs, this means it doesn't actually save as an HTML file, it just throws the HTML code into another buffer, or another "windows"). 

In Org, one needs to include some formatting options previously according to taste. For me, I need to get rid of the table of contents (created automatically when exporting to HTML) and the numbers added to each header (again, automatically). 

In Org-mode, this is done by including these options at the head of the Org buffer: 

#+OPTIONS: toc:nil
#+OPTIONS: num:nil

Then, when I export to HTML, I am only looking for the <body> tags and copy what's between those. Paste into an empty TW window. Done. There's even the added benefit that you don't have to keep saving the wiki and overwriting your copy (which is needed in the new Firefox, even with some of the plug-ins). You simply copy-paste until you're happy with the results, and then save the wiki once.  

My only concern is that the format is completely HTML and not in TW syntax, and I'm not sure if this is a problem or not. Another issue I can think about is that I cannot link to other tiddlers and verify that the link work (in TW, it would be in italics if the tiddler doesn't exist), but that's just a small issue of one additional back and forth. Worth it for the writing experience, imho of course. 

I think that, for now, I'm satisfied with what I found. That is, until I try to write something later today or tomorrow... but I'm always the pessimistic one. 

[Shay Shaked]

I'm running out of time and I'm going to be late for work, but this work is more interesting... 

I identified the following "problem" (not a bug, a feature!). 

In TW, ~ is used to cancel CamelCase. In Org, this is used to indicate a code line (similar to ` in TW). The issue is, as you might have guessed, to exclude CamelCases while writing in Org. If I use ~, Org would start a code block later in HTML; if I don't use it, I will have a bogus CameCase link. The work-around, for now, is to use verbatim in Org (= signs) as you write, which will result in <code> blocks in html. Find them in the html and delete them manually. 

So in Org: =~CamelCase cancel= 

in TW (after taking out the <code></code>): ~CamelCase. 

I wonder if HTML has a verbatim option similar to Org that is not <code>. 

Ugh, gotta go. gotta go... hope this makes sense.  

[LDL]

I have the same problem on my machine. Looks like :export-block is no longer supported in new versions of org-mode. I opened an issue on the github project page, should be easy to fix.

[LDL]

The author of the exporter from org-mode to tiddlywiki has just fixed the problem. Works fine with the latest version from GitHub.

[Scott Kingery]

I found this tool long ago called Pandoc. https://pandoc.org/index.html Pandoc lets you convert files from one markup format into another.

Not sure how much it will help in this case (if at all) but I thought I'd drop it here in anyway.. 

[Shay Shaked]

Ha. That's a full circle right there :)

On Thu, Jun 28, 2018, 13:27 Scott Kingery <techlif...@gmail.com> wrote:

I found this tool long ago called Pandoc. https://pandoc.org/index.html Pandoc lets you convert files from one markup format into another.

Not sure how much it will help in this case (if at all) but I thought I'd drop it here in anyway.. 

--
You received this message because you are subscribed to a topic in the Google Groups "TiddlyWiki" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/tiddlywiki/BzINZCRBzvw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/dd32e2fb-b0e0-42dd-85b3-3c5432cb4989%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Martian]

This may be interesting also https://github.com/whacked/tiddlywiki-org

Though I didn't try it yet.

[Brandon Hall]

Hi, I've been thinking about this (using Org-mode and TW as mutually-beneficial systems), and I suspect that Org is a poor means of approaching TW's concerns regarding hypertext. With that said, I think that the combination of them both can be useful and easier done than you might be considering.

If you create a means of exporting a set of tiddlers (if not all) as an Org file that you read from ("Read"), then track your edits in an alternate Org file ("Write"), you can use the TODO functionality offered by Org for planning updates to your canonical Org file (Read). This would be done by cloning and refiling entries from Read to Write.

This is a workflow that I've been planning to use mostly via my phone, with the Orgzly Android app, as I've wanted an easy-ish means of accessing my TW archives and recording planned updates.

Exporting tiddlers as an Org file isn't too difficult with the Node JS version. I've made a template for exporting a list of all tiddlers' titles here, and by adding a Transclude widget, we can include the body of the tiddlers, too, as Org notes' note content: https://gitlab.com/bthall/publicwiki/blob/master/tiddlers/journals-feed.tid You then need to use a command similar to the one below to export this as an Org file:

tiddlywiki publicwiki --render "journals-feed" journals-feed.org text/plain "" exportFilter "[tag[Journal]]"

Note that with the above command, I am outputting an Org file with only posts tagged `Journal`. Journals are a poor example of a case where you'd use this functionality, but I made this as a step towards outputting Atom feeds for my posts, namely my Journal entries.

Additionally, adding tags to the output is not only easy but makes the matter of tracking which entries are duplicates (entries that have already been inputted to TW, versus those that are wholly new): if you add a dummy tag to the template, preceded and followed by colons, then append to the list of tags the tiddlers' tags followed by a colon to finish the listing of tags. For example: `:oldNews:[[actual tag]:]`

On Monday, June 25, 2018 at 6:21:30 PM UTC-7, Shay Shaked wrote:

[x-...@o-o-x.com]

I'm the author of the tiddlywiki-org repo Martian linked to above. I haven't updated the elisp files for 2+ years, but I use it every day. You *do* get the best of all worlds and skip the conversion overhead. Here are some observations looking back.

.tid is basically a container format, so it works as a superset of org files and any other markup files in your tiddlers directory. TW will happily organize them for you. As a result, my tiddlers are almost entirely .tid files of "type: text/org". I never figured out how to make the default content type "text/org" instead of vnd.tiddlywiki from the drop down. That got pretty annoying, but I almost always just created tid files from emacs.

The integration in TW using org-js is pretty good. Even the links are rewired so clicking a relative link will open the target tiddler, in the same way that C-c C-o in emacs loads the target file.

tiddlywiki-mode.el will auto narrow the tid file to hide the tiddlywiki headers so all you see is the org file. The file monitor in my TW fork is buggy but works. You save your .tid org file, and your browser or TiddlyDesktop will immediately auto-rerender the updated tiddler if it's loaded. tiddlywiki-mode.el turns on auto-revert-mode by default, so when you edit the tiddler from TW, emacs auto-refreshes. org-mode transclusion might work too.

I used this setup for maybe 1.5 years, then I stopped launching the TW server, even though all my notes are still .tid in text/org. This is mainly due to navigation, search, and sprawl. Let me elaborate. I use a quick key combo in emacs to create a new anonymous tiddler. Sometimes you want to write something down before you thought of a title. After a while, I have lots of these anonymous tiddlers, and they are all named "2017-01-06_23.03.11 anonymous-tiddler.tid" or something like that. TW's built in search is decent but not great, so takes a while to find the right tid, plus you can't escape using the mouse! If you have an Emacs window open, it's far more efficient to use the helm-org-rifle package -- which works perfectly well with org-syntax tid files.

This kind of solves navigation and search, and now you're almost completely in Emacs. What Emacs, and org-mode, don't bring, is organization of the sprawling files, because org-mode is still linearly organized. One odd thing about TW, is that while it provides non-linear navigation, it still uses a strictly linear presentation. The story view is still a 1-dimentional stream of cards. I couldn't find any good ways to use TW's rendering capabilities to augment the display of org-mode files. For example, I've tried messing with org-js so that it renders sections as individual panels within TW. It's a narrow solution, and still doesn't help with inter-file linearity.

One excellent plugin that helps with organizing sprawling tiddlers is TiddlyMap. It was great in the beginning, but since TiddlyMap uses node titles as node names, and now I have a lot of anonymous tiddlers, it no longer helps. So recently, I no longer run the TW native tools, even though all my files are tid files. I'm working on other methods to work with the tiddler graph data.

This got pretty long, but the takeaway is, if you're willing to put up with some hacking, and as long as you don't make lots of anonymous tiddlers, emacs + org + tiddlywiki is a terrific combo.

On Monday, June 25, 2018 at 6:21:30 PM UTC-7, Shay Shaked wrote:

[Felix Küppers]

Hi

> One excellent plugin that helps with organizing sprawling tiddlers is
> TiddlyMap.

thanks :)

> It was great in the beginning, but since TiddlyMap uses node titles as
> node names, and now I have a lot of anonymous tiddlers, it no longer
> helps. So recently, I no longer run the TW native tools, even though
> all my files are tid files. I'm working on other methods to work with
> the tiddler graph data.

just for the record, you can also specify that your node titles should
not be derived from "title" but from another tiddler field – in which
case you can then also use wikitext to generate the node title displayed
in the graph:
https://github.com/felixhayashi/TW5-TiddlyMap/blob/master/src/plugins/felixhayashi/tiddlymap/js/services/Adapter.js#L625
– Not sure if that helps in your case, but I think it is worth mentioning.

Best wishes
Felix