Query: Markup Parsers

[@TiddlyTweeter]

This is an initial query springing from the thread started by Jan on "Fountain" markup here--but its more general than that: https://groups.google.com/forum/#!topic/tiddlywiki/9Lf0YHfMUZk

My question may seem dumb since I don't get the underpinnings.

My FIRST question is this: Is it possible to make a new parser for mark-up without having to use JavaScript? Just a macro using Regular Expressions?

Any orientation you can give could help.
Best wishes
Josiah

[@TiddlyTweeter]

Tx PMario

Would I have to study the JavaScript code to understand what those parsers do? I'm very hazy about what the final HTML is.

Grasping that is probably beyond me. However ...

SECOND QUESTION: Are parsers bound (tied) to content types? Meaning, if we created a new parser would it need a new "content type"?

Best wishes
Josiah

@TiddlyTweeter wrote:

My FIRST question is this: Is it possible to make a new parser for mark-up without having to use JavaScript?

PMario wrote:

NO. ... There's way to much logic involved. The tiddlywiki syntax parser consists of 46 different js modules, which converts text into an internal structure, that is used to render the html output.

[TonyM]

Just a question. Why build a parser when there are ways to parse the content in a tiddlywiki already. I would have thought What is it you want to achieve is best voiced first.

Ok, maybe I don't get it but I expect the same could be said for theatrical script writing as well unless it is critical to import a different markup in which case you could work on the import and export process instead.

Food for thought?

Otherwise clueless
Tony

[@TiddlyTweeter]

TonyM

The aim is implicitly voiced in the other thread this emerged from ... https://groups.google.com/forum/#!topic/tiddlywiki/9Lf0YHfMUZk ... but it needs work to get it optimised for TW, IMO.

Looking at Fountain, that thread deals with its markup system for screenplays--I think it could be simplified for TW a lot.

The UNDERLYING ISSUE is whether a "deep level" Javascript parser (and likely a new "content type") is needed OR whether you can get away with a more surface level set of regular expressions. The RegEx I can sorta cope with. The Javascript I can't.

The BASIC markup I'm thinking is only this ...
  

:x

Each LINE started by a colon and a letter code is parsed and wrapped in different CSS classes.

Right now I'm trying to establish the tech needs for this.

PMario opined it wasn't possible without JavaScript coding.

Best wishes
Josiah

[Mark S.]

Working out what you want in a parser in terms of inputs and outputs is a good idea.

I'm wondering what the point of this exercise would be? How would you use the outputs? My experience with all things HTML is that it doesn't print out reliably. So you probably couldn't depend on it to produce physical manuscripts. So, unless you could convince directors/actors/reviewers to read scripts on their tablets (hmm, maybe they already do this?)  what would be the benefit of using TW?

Have fun,
Mark

[@TiddlyTweeter]

Ciao Mark S.

Thanks for GOOD questions.

For movie scripts outputs MUST follow the format to have any chance. The spec is precise. This is why you must have access to a physical print-out to mail that follows the convention, NOT a GUESS. OR an exactly laid out PDF could do.

I think TW can do that natively. But it needs thought yet.

Josiah

[Mark S.]

I'm skeptical about the hard-copy or PDF virtual hard-copy.

You could make a couple pages using conventional CSS/TW markup and then see if it prints out like you want. I think you might find that getting page breaks to work reasonably is challenging.

Good luck,
Mark

[PMario]

On Thursday, August 31, 2017 at 2:09:54 PM UTC+2, @TiddlyTweeter wrote:

Would I have to study the JavaScript code to understand what those parsers do?

yes.

and you can have a look at different hangouts, where Jeremy discusses parsers. .. So you may get some hints too. Open this link and search for "parser"

 

I'm very hazy about what the final HTML is.

The final html is just HTML.... So eg: ! heading  ... is translated to ... <h1>heading</h1>

 

SECOND QUESTION: Are parsers bound (tied) to content types? Meaning, if we created a new parser would it need a new "content type"?

Yes. ... If there is no "type" field, TiddlyWiki assumes "text/vnd.tiddlywiki" as the default type. Other types use other parser. eg: "text/x-markdown", which is used by the markdown plugin.

have fun!

mario

[PMario]

On Thursday, August 31, 2017 at 2:57:20 PM UTC+2, TonyM wrote:

Just a question. Why build a parser when there are ways to parse the content in a tiddlywiki already.

If you are used to a different markup, it's hard to learn tw syntax ..... and the other way around. 

And if you have existing tools, that need the "other markup", because they don't know TW markup.

-m

 

[@TiddlyTweeter]

GREAT QUESTIONS

If it can't do that then there is NO point starting.

I'm gonna look at that first, now.

tx, J.

[PMario]

On Thursday, August 31, 2017 at 3:17:48 PM UTC+2, @TiddlyTweeter wrote:

Looking at Fountain, that thread deals with its markup system for screenplays--I think it could be simplified for TW a lot.

Not really. TW-markup is different to fountain markup. Mixing things, IMO would be confusing. ... A new parser would probably be easier in the long run.

 

The UNDERLYING ISSUE is whether a "deep level" Javascript parser (and likely a new "content type")

As above. imo easier to deal with in the long run.

PMario opined it wasn't possible without JavaScript coding.

I didn't say it's not possible. ... But I don't see it. ... For me a new parser would be much simpler to understand and implement. ... There are js libraries already: https://github.com/mattdaly/Fountain.js  It should be simple to implement it, similar to the markdown stuff. ... BUT that's not the whole truth.

The magic, that's really needed, is a "sketchbook", where you write down your ideas about scenes, characters, .... and then combine the stuff to a story. ... The text parsers are just a means to an end.

have fun!

mario

[@TiddlyTweeter]

PMario wrote:

The magic, that's really needed, is a "sketchbook", where you write down your ideas about scenes, characters, .... and then combine the stuff to a story. ... The text parsers are just a means to an end.

That's easy for me. I make sure I make films every few years.

https://vimeo.com/57934864

J, x

[Jan]

Hi Josiah, Hi Mario.
I also came to the conclusion that it is not necessary have a compatibility with the fountain-markup.
For me the most important thing is that ideas developped in TW can grow to a screenplay without having to migrate the texts to another program.
In some aspects the fountain syntax won't make sense at all in TW. For Example, the title of the scene could simply be the title of the Tiddler (or an alias).
One problem I see in the normal TW-parsing that the elements have either to be opened and closed (like italic or bold) or that they have to be preceded by a blank line (like headings and lists).
This is why I put up the question whether it would be possible to modify the parsing of lists to work without a preceding blank line. I could not find how this could be done in $:/core/modules/parsers/wikiparser/rules/list.js
@Mario: Do you think it would be possible to achieve this?
If this was possible I would create stylesheets to use the existing elements (like lists) to achieve the screenplay-layout.

Yours Jan

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/ef1cccc8-f284-468a-ba13-f1cec9ec60b8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Thomas Elmiger]

Hi all

Interesting discussion, really! Not that I would intend to write a play or a film, but I have thoughts about other text production projects … I just lack time as always :–D

Page breaks are controllable in theory using CSS: https://www.w3schools.com/cssref/pr_print_pageba.asp – hope this helps for your tests Josiah!

One more thought about where TW might be a superior authoring tool: I imagine texts with transclusions used like variables. You define a character/place in a tiddler and maybe some fields. Instead of the name of the role, e.g. Andrew, you write a transclusion.

{{A}}: Let’s go home!

If you later on decide to call that role Andy you just change the text of one single tiddler and you are done. I hope you get the idea.

Have fun!
Thomas

[Mark S.]

Page breaks are controllable in theory using CSS: https://www.w3schools.com/cssref/pr_print_pageba.asp – hope this helps for your tests Josiah!

Yes, but how would you use this in practice? You'd have to do a preview print. Then, based on the print, insert MANUALLY any pagebreaks. Then repeat to make sure things still paginate. Then repeat again. Imagine doing that with a 50 page document, and then having to repeat every time there was a change to the script.

Mark

[codacoder...@outlook.com]

Hi Mark

Yes, agreed, but only where you want to print a set of tiddlers -- like the story river.  If, on the other hand, you use a macro/tiddler to "stream" the participating tiddlers into one long chunk (the "output" tiddler), you could then apply the css to the elements in the output.

I haven't tried it myself, but it principle it should work, I think.

[@TiddlyTweeter]

Let's see if Mark S. is the Last Notator of the Lost Side or not.

J.

[Thomas Elmiger]

Hi Mark

I would use this to mark the end of parts (chapters, scenes) where you know you want a page break after. The end of a chapter is the end of a chapter and stays the end of the chapter no matter how many times you rewrite the chapter. 

On the other hand you can set the value to "avoid" after the title and e.g. introduction of a chapter, where you don't want a page break. 

This should reduce the need for manual adjustments significantly. 

-t

'Mark S.' via TiddlyWiki <tiddl...@googlegroups.com> schrieb am Do. 31. Aug. 2017 um 21:44: 

Yes, but how would you use this in practice? You'd have to do a preview print. Then, based on the print, insert MANUALLY any pagebreaks. Then repeat to make sure things still paginate. Then repeat again. Imagine doing that with a 50 page document, and then having to repeat every time there was a change to the script.

--

--

Thomas Elmiger

Lorenwäldlistrasse 34

8630 Rüti ZH

[@TiddlyTweeter]

In Fountain they do ===. I think its not right. Rather the lines were right already. ITS THE CSS MATH'S honey.

[Mark S.]

Now that I'm trying it on some material I wrote, it doesn't look quite as hopeless. It might be that the output of TW is more regular than that of other web-pages. I've had various pages that would split text mid-line (I mean literally right down the horizontal middle of the line).

Mark

[Jan]

Hi Thomas,
transcluding rolenames is exactly what my plugin in http://storywriting.tiddlyspot.com/ does, together with Alain Dutechs textcompletition to complete the rolnames and insert the transclusions automatically. I think this helps you to write down your ideas faster, because you do not have to think about the name from the beinning on.

Sorry to come back to my question on parsing list-elements in TW (because the discussion switched to pagebreaks:)

The problem I see in the normal TW-parsing that the elements have either to be opened and closed (like italic or bold) or that they have to be preceded by a blank line (like headings and lists).

This is why I put up the question whether it would be possible to modify the parsing of lists to work without a preceding blank line. I could not find how this could be done in $:/core/modules/parsers/wikiparser/rules/list.js

Do you think it would be possible to achieve this?

Jan

[Jeremy Ruston]

Hi Josiah

My FIRST question is this: Is it possible to make a new parser for mark-up without having to use JavaScript? Just a macro using Regular Expressions?

As Mario has explained, it’s not currently possible to create a new parse rule without using JavaScript. It is something that I often think about, though: I like the idea of TiddlyWiki users being able to evolve their own unique domain specific markup, precisely for applications like screenwriting.

However, it’s hard to imagine a non-JavaScript approach to parsers that would allow us to replicate the complexity of something like the current table or list parser rules. The underlying rules seem sufficiently complex that some kind of programming language is required to express the logic, and that the language must necessarily be a good deal more complex than regular expressions themselves.

Instead, I’ve wondered whether we might be able to make (some of) the existing parser rules much more configurable. For example, right now there are 6 separate parser modules for each of bold, italic, strikethrough, subscript, superscript and underscore:

https://github.com/Jermolene/TiddlyWiki5/tree/master/core/modules/parsers/wikiparser/rules/emphasis

But, if you look at the individual JavaScript files, you’ll see that they only vary through a few parameters: the name of the rule, the regular expression to match at the start, and the regular expression to match at the end, and the tag of the HTML element to create.

So, one could imagine refactoring things so that there was a single JS parse rule, and that it was instantiated multiple times with different parameters.

Then, users could evolve their own notation for, say, sotto voce, and then make a new instance of the parse rule with slightly modified parameters.

Best wishes

Jeremy

[Dragon Cotterill]

This takes us into the realm of Yacc (Bison) and Lex for tokenising and parsing. Just a few Javascript versions and alternatives off hand:

https://www.npmjs.com/package/jscc-parser

http://zaach.github.com/jison/docs/

http://canna71.github.io/Jacob/

I'm sure there must be someway to sort out the tokenising of a standard Tiddler.

Hmmm, the last time I worked with Yacc/Lex was about 25 years ago. Guess I'm a little rusty on the subject these days.

[PMario]

On Friday, September 1, 2017 at 2:12:55 PM UTC+2, Jeremy Ruston wrote:

Instead, I’ve wondered whether we might be able to make (some of) the existing parser rules much more configurable. For example, right now there are 6 separate parser modules for each of bold, italic, strikethrough, subscript, superscript and underscore:

https://github.com/Jermolene/TiddlyWiki5/tree/master/core/modules/parsers/wikiparser/rules/emphasis

But, if you look at the individual JavaScript files, you’ll see that they only vary through a few parameters: the name of the rule, the regular expression to match at the start, and the regular expression to match at the end, and the tag of the HTML element to create.

There was a reason, why we splitted the emphasis rule: https://github.com/Jermolene/TiddlyWiki5/issues/701

So, one could imagine refactoring things so that there was a single JS parse rule, and that it was instantiated multiple times with different parameters.

That's right, but we can't sacrifice the existing flexibility. ....

-m

[Jeremy Ruston]

That's right, but we can't sacrifice the existing flexibility. .... 

We don’t need to; there would still be six separate filters, just that they were instantiated from the same JS filter with different parameters. It would be akin to refactoring six independent functions to each be an instance of the same base class.

Best wishes

Jeremy.

[Jan]

Dear Jeremy,
I would love to have that feature, this would give us a lot more flexibility.
For example formatting screenplays it would be great to set the format just by a mark like : preceding the line ... and it would be even better to have a fixed sequence of formats following one another
Like: <action> /n <role> <dialogue> /n <role> <dialogue>/n/n<action> etc...

Sorry for asking onnce again: 
Is it possible to modify $:/core/modules/parsers/wikiparser/rules/list.js
to parse a list without a preceding blank line?

Yours Jan

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/B06A10D8-58C6-4CE0-A133-7B5A0F1BE10C%40gmail.com.

[@TiddlyTweeter]

Ciao Jeremy

Jeremy ... I’ve wondered whether we might be able to make (some of) the existing parser rules much more configurable.

That is encouraging. I want to echo Jan's interest ...

Jan: I would love to have that feature, this would give us a lot more flexibility.

I think cinema screenplays are a great test case because of their strictness of form. But ALSO because that though a "Spec." script (the thing you submit to production companies) is formatted more tightly than a tight grip, "Shooting / Production" scripts can, in practice, vary enormously.

The ability to be able to create NEW markup rules quickly to suit specific film production needs would be a godsend.

---

Thinking about it more... once more easily customisable markup were possible things like the following could be achieved move easily ...

• Cookery book layouts
• Legal documents
• Poetry collections
• Layout of image galleries

etc ...

Best wishes

Josiah

[Mat]

On the topic of parse rules, Skeeve created the ReplacePragma which is very interesting but, if I understand the discussion it spurred, not unproblematic.

<:-)

[@TiddlyTweeter]

Mat

Tx! That is seriously interesting.

J, x

[Jan]

Hi Mat,
thanks for reminding me of the replace pragma. If I use lists and headings for the screenplay-markup the following lines could prevent that the entire layout is crashed due to a missing newline.
\replace /\n!/\n\n!/
\replace /\n:/\n\n:/
\replace /\n{{/\n\n:{{/

Next question: How can I apply them automatically to every tiddler tagged dialogue?

Yours Jan

Am 02.09.2017 um 12:29 schrieb Mat:

On the topic of parse rules, Skeeve created the ReplacePragma which is very interesting but, if I understand the discussion it spurred, not unproblematic.

<:-)

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/c3774dc5-4824-4e8e-a13c-0b93048652dc%40googlegroups.com.

[@TiddlyTweeter]

Ciao Jan & folk interested in Markup ...

INTERIM REPORT

Skeeve's Replace Pragma I have been playing with.

Its seriously dangerous, but used carefully you can demonstrate with it how TW can take raw Regular Expressions and apply them to produce zillions of different types of layout very easily.

I finish up on these experiments and report back with requests on how to achieve the same in a SAFER way.

Best wishes
Josiah

[Jan]

Hi Josiah,
Thanks for keeping this problem in mind, in the moment it is the biggest issue that I have to solve befor issuing a new version of the screenwriting-plugin.
Skeeve himself says that his pragma could be a deadly sword, but I think the danger can be handled if it is prevented from beeing applied to systemTiddlers: after adjusting all problems will disappear.

So let me repeapt task for js-coders that would have to be solved to bring forward special formats like screenplay and other documents:
It is to find a way to apply replacements with the replacePragma (http://tiddlystuff.tiddlyspot.com/#ReplacePragma) to Tiddlers based on a TW-Filter.

Yours Jan

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/63a7f76a-0b8f-4419-8055-3da60275634b%40googlegroups.com.

[@TiddlyTweeter]

Ciao Jan

Is your screenwriting plugin somewhere I can take a look so I can better understand your approach?

Regarding parsers for what we need. The more I think about it the clearer it gets to me that what is needed is a "Generic" parser that you can quickly add codes to. For instance, in what I am currently working on the markup codes are something like this ...

:s:7 INT. PARK HUT - NIGHT

:a Baxter enters carrying Perry Mason.

:c BAXTER
:p (coughs)
:d Paul, are you there? Perry had an accident.

To get those lines converted to HTML is quite easy using regular expressions. But I DON'T think the specific regexes should be individually "HARD CODED" into Javascript. Rather, we should devise a Markup that is extensible so you could apply the same method for potentially laying out many types of document. In short, we need a method by which a user could easily add new codes.

I'm also thinking there could easily be LONG-FORM codes that a user could use till they learned the SHORT-FORM (above) ... For example ...

:scene:7 INT. PARK HUT - NIGHT

:action Baxter enters carrying Perry Mason.

:character BAXTER
:parenthetical (coughs)
:dialogue Paul, are you there? Perry had an accident.

In addition, add-hoc markup could allow CONVERSION markup to be built -- Like being able to convert back & forth with Fountain markup. And also to be able to at least part convert plain text screenplays to a TW format.

I'm working on a demo using Skeeve to illustrate those issues. It will take me a few more days to finish it. I'm hoping, after that, we'll be able to say more precisely what we asking of the JavaScript department.

Just today's thoughts.

Best wishes
Josiah

[@TiddlyTweeter]

Ciao Jan

Regarding safety with Skeeve. And to reduce crashes ...

I now devise & test the regexs using an external program first: RegexBuddy. Its an inexpensive Windows program that is extremely good. It accurately emulates all versions of JavaScript. 

Best wishes
Josiah

[codacoder...@outlook.com]

Hi Joshua

Let me ask a question (apologies if this is answered somewhere above but since this thread and its "parent" thread are long, I wanted to ask it and get a straight answer) ...

What does the text look like that you wish to enter into a tiddler in EDIT mode?

Second Question...

What does the text look like that TW will render assuming you have your ideal magical tool?

Please, keep regexes, javascript et al out of this - keep it simple.

Coda

[@TiddlyTweeter]

Ciao Coda

Very encouraging to have your interest!!

Good questions.

Give me a couple of days so I don't burden you with a half-baked-ostrich.

Briefly: Input in EDIT is plain text. Output is rendered HTML after scavenged by Regex. But I need get the examples working better before I show you. On the RegEx side I know what I'm doing (for a change :).

Very best wishes
Josiah

[codacoder...@outlook.com]

Well, cool, but I don't need an exhaustive presentation of every possible combination...  Give me some examples, just a few.

I have an idea that something I'm already doing could be of benefit to you.  So gimme something to test... okay?

[@TiddlyTweeter]

C, WILL do. J.

[Jan]

Hi Coda.
My concept for formating is quite simple
I built a tool which is transclusions for the rolenames, which are automatically inserted by Alain Dutech Comptext-Plugin.
For simplicity I would use headings for the action, and the Tiddler-Ttitle (or an Alias inserted by the Alias-Plugin) for the scene-title.

Thus:

Peter and Bob are doing some Action

PETER
Now I say something.

BOB
And here I answer.

Would look like that in Editmode:

[Title]
!Peter and Bob are doing some Action
{{Peter}}Now I say something.
{{AnotherNamelaterreplacedbyBob}} And here I answer.

The issue of indent rolenames can be handled by using <dd>-Elements which should give a
:Forced negative indent Line.

My use for the replacePragma would the following to avoid breaking the list by forgetting the preceding blank line:

\replace /\n!/\n\n!/
\replace /\n:/\n\n:/
\replace /\n{{/\n\n:{{/

\replace /}}/}} /

Thanks for your help!
Jan

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/28cfb2f3-483f-446e-acbb-3a3b86dfbd7e%40googlegroups.com.

[codacoder...@outlook.com]

Thanks Jan.

It would be better if you present this DEVOID of JARGON.  I don't know (nor do I want to learn) screenwriting.  Take what you've written here, and present it like this:

I want to write something like this in the TW editor:

INPUT TEXT HERE

I want it to render (wikify) to something like this:

OUTPUT TEXT HERE

That way I can see what you are trying to achieve without getting bogged down in a bunch of terminology which, frankly, might be irrelevant.

Hope that makes sense.

It's possible my idea may work for you guys.  It's also possible it won't.  But I need something representative to test.  Okay?

[Jan]

Hi Coda,
I hope this is what you want:


InputText:

!Peter and Bob are doing some Action
{{Peter}}Now I say something.

Single Linebreaks are //okay//.

{{AnotherNamelaterreplacedbyBob}} And here I answer.

>         Here the spaces schould be displayed

OutputText:

Peter and Bob are doing some Action

PETER
Now I say something.

Single Linebreaks are okay.

BOB
And here I answer.

         Here the spaces schould be displayed


I hope this demonstration is okay.
Basically this is what I want.


Thanks for your help
Jan

Am 12.09.2017 um 16:43 schrieb codacoder...@outlook.com:

Thanks Jan.

It would be better if you present this DEVOID of JARGON.  I don't know (nor do I want to learn) screenwriting.  Take what you've written here, and present it like this:

I want to write something like this in the TW editor:

INPUT TEXT HERE

I want it to render (wikify) to something like this:

OUTPUT TEXT HERE

That way I can see what you are trying to achieve without getting bogged down in a bunch of terminology which, frankly, might be irrelevant.

Hope that makes sense.

It's possible my idea may work for you guys.  It's also possible it won't.  But I need something representative to test.  Okay?



On Tuesday, September 12, 2017 at 9:01:17 AM UTC-5, Jan wrote:

Hi Coda.
My concept for formating is quite simple
I built a tool which is transclusions for the rolenames, which are automatically inserted by Alain Dutech Comptext-Plugin.
For simplicity I would use headings for the action, and the Tiddler-Ttitle (or an Alias inserted by the Alias-Plugin) for the scene-title.

Thus:

Peter and Bob are doing some Action

PETER
Now I say something.

BOB
And here I answer.

Would look like that in Editmode:

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/d0e976f6-d5e9-4a08-a4fa-d2563cc02396%40googlegroups.com.

[@TiddlyTweeter]

Ciao Jan

Its interesting looking at that. Its more about intelligent "guessing" than the markup I'm interested in. I think it could be easy in RegEx. I'm not convinced you need ANY complex things at all to format that. So long as it stays so well structured a couple of RegExs could likely do it.

Best wishes
Josiah

[codacoder...@outlook.com]

Thanks Jan.  That's very helpful.

Okay, next question...

In principle, are you happy with using references to TW macros in the input text?  For example:

<<xp>>

Which would generate the following in the output (i.e. when viewed in TW)

PETER

NOTE:  I said "in principle" and "using references".  I'm not proposing that have to type them like that in the editor.  Also, don't get hung up on my choice of macro name (<<xp>>) it could be <<anything>> - it's not important right now.

One step at a time... :)

[Jan]

Hi Coda,
I would like to avoid expressions looking to technical in the inputText.
In my concept the macro is transcluded by the rolename:

{{Peter}} like all roles leads to a tiddler which a only macro called <<role>> as text. The macro is defined:
\define role()
<div class=role><$list filter="[{!!title}tag[emphasis]]"><hr></$list><$list filter="[{!!title}has[name]]"emptyMessage={{!!title}}>{{!!name}}</$list></div>
\end
This makes it possible to swap the names afterwards and to use uppercase in the stylesheet.

My Problems start with the following:

There are two Layouts.
One for small devices using

PETER:
Now I say something.
Single Linebreaks are okay.

And for print an big screens which should display with indent roles:

PETER:   Now I say something.
              Single Linebreaks are okay.
BOB:      And here I answer.
                          Here the spaces schould be displayed

This is why I chose to display the roles within <dd> elements.
...

You see, it is quite developped but it works if everything is entered correctlay. 
And the ReplacePragma I mentioned would solve all problems which resolve in forgetting a line-break or so...

Yours Jan

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/cc91dc72-fb34-4bd4-8737-83e9a87b9b29%40googlegroups.com.

[codacoder...@outlook.com]

Hi jan

On Tuesday, September 12, 2017 at 4:31:56 PM UTC-5, Jan wrote:

Hi Coda,

I would like to avoid expressions looking to technical in the inputText.
In my concept the macro is transcluded by the rolename:

{{Peter}} like all roles leads to a tiddler which a only macro called <<role>> as text. The macro is defined:
\define role()
<div class=role><$list filter="[{!!title}tag[emphasis]]"><hr></$list><$list filter="[{!!title}has[name]]"emptyMessage={{!!title}}>{{!!name}}</$list></div>
\end

That's completely fine (I think).  I hate typing out << and >> everywhere -- when authoring, it breaks my concentration and "flow".

My "system" doesn't care what the markup looks like (it can be any TW-valid-text -- a macro, a transclusion or any combination you can dream up).  My question was to make sure you're okay with seeing regular TW markup in the editor and have the system output something else entirely (which, when it is displayed in VIEW mode looks entirely different again).  I guess I didn't phrase the question properly.

Perhaps I should explain my system?  I hope you're running Windows...

I use a combination of AutoHotKey (AHK) and TW macros to produce markup.  I find it extraordinarily useful for oft repeated phrases which occur a lot (when authoring) and should always look the same.  So, for example, in TW, I define a single macro for each main character.  <<jack>> and <<jill>>, for example.  Then, in AHK I define another macro which triggers the TW macro reference as output -- I make sure that the AHK macro is as short as possible, made up of one or two or at most three characters unlikely to occur "naturally" -- using a hyphen at the start or end can help.

While I can't "see" exactly what it is you're trying to achieve (again, I don't know screenwriting lingo), I'm now convinced the combination of TW ~ AHK is the way to trigger anything you want to "magically appear" in the editor with one or two keystrokes.

So, final example, here's my section divider in TW:

\define break() <p><br><hr style="text-align:center;width:100px;"/><br></p>

And here is the AHK macro that triggers it:

:*:brk::   <<break>>

In TW, I type brk and out pops a call to <<break>> which, as you can see will end up with a P element, a BR element and so on.

Now, just in case I haven't explained my system good enough for you, let me say it another way...

AHK doesn't care how long or complex its output is.  With a suitably short/obscure key combo, you can output as much TW-wiki syntax as you like, identically, a thousand times over, with ease.

Any questions?  You know where I am :)

AHK Link:  https://autohotkey.com/

[TonyM]

Coda,

Off topic on Autohotkey

I am a big user of autohot key, just a few very effective key board alternatives.

I just added the following;

^,::
Send <<
return
^.::
send >>
return

So Control < and Control > give be doubles

Others include

1. NumpadEnter::Send {Down}{Home}
   Move to beginning of next line without a new line
   
2. +NumpadEnter::Send {Down}{End}
   Move to the end of next line without a new line
3. CapsLock::Shift
   Never use Capslock but making it a shift sO I DO NOT SHOUT
   

Which make my life so much easier

Tony

PS Windows 10 considers it malware.
 

[codacoder...@outlook.com]

Tony,

Try these:

;Pretty quotes
^':: SendInput “
^+':: SendInput ”

That's Ctrl ' and Ctrl-Shift '  (that's a single-quote at the end of each)

[TonyM]

“Nice”

[@TiddlyTweeter]

TonyM & Coda

Let me expose my age as its likely near yours. And since you broke this thread.

Back in the days of Dos only, what you had was "programs" (extremely boring things that only worked one way) and "TSRs" (extremely interesting gizmos that let you rule programs).

AutoHotKey(AHK) is a great evolution of the TSRs that survived. Most of them died on PCs with Windows. At one time TSRs were considered as important as applications--and often they were better than them. Windows killed most of them.

Enough nostalgia.

To the nub.

WHAT has AHK got to do with TiddlyWiki? I think VERY LITTLE. I guess it could be useful in the editor if you put the time in to get it to run. Frankly, I think its overrated--by you two. :-)

Talk-back ...

**Arguments cost 30 Euro for 5 minutes. But only 278 Euro for the full 45 minutes**

Josiah ;-)

[codacoder...@outlook.com]

On Wednesday, September 13, 2017 at 12:55:28 PM UTC-5, @TiddlyTweeter wrote:

TonyM & Coda

Let me expose my age as its likely near yours. And since you broke this thread.

Back in the days of Dos only, what you had was "programs" (extremely boring things that only worked one way) and "TSRs" (extremely interesting gizmos that let you rule programs).

Ah, yes, TSRs - recall I had a chat with Jeremy about Terminate and Stay Resident code.  Wrote a few -- some of them were even quite useful :)

 

AutoHotKey(AHK) is a great evolution of the TSRs that survived. Most of them died on PCs with Windows. At one time TSRs were considered as important as applications--and often they were better than them. Windows killed most of them.

One could argue that most modern OSs are built from TSRs.

 

Enough nostalgia.

To the nub.

WHAT has AHK got to do with TiddlyWiki? I think VERY LITTLE.

AHK has nothing to do with TW.  But in combination, they work well together.

 

Talk-back ...

I did.  :)

[TonyM]

Josiah,

I remember the TSR's as well, really they were the first gateway into multi-tasking on PC's, for the younger people the were "Terminate and Stay Resident" programs basically which were reopened with a key combination. Where they relate to Key mappings is because they were handy tools, that could help shortcut otherwise complex processes. I remember SideKick and the GEM Desktop as well. One lesson I learned is why have a popup clock when you can have one on your desk.

Early windows did hold back some of these and also made many irrelevant, but this customisability remains in the genes of Windows even today, although some have been eroded to dumb down the OS for the common person.

With that background spelt out I would actually like to defend an interest in such obscure tools, in fact they should not be so obscure. Computers have always being about automation, and as a computer specialist since the mid 80's I have seen the automation of many things, simplification and accessibility has being massive, yet far too often the tools of automation computers do not always permit self automation. Just think for yourself how often you find yourself doing some repetitive tasks on a computer, and if they include a Ctrl-alt-key combination or two you need finger gymnastics along with deep focus to do something really trivial..

I have spent my life looking for opportunity to simplify or automate.

WHAT has AHK got to do with TiddlyWiki?

In both TWC and TW5 when I am building a solution I often find a need to append or pre-pend one, and sometimes more lline's, with a piece of text. The most common example may be placing * or # at the beginning of the line. Often there is no prefix or suffix on each line you can do a search and replace on, so you need to go to the beginning and/or end of each line and type something and each time you do the cursor moves and you need to use arrow keys or mouse. This can be time consuming and painful when you miss or add a single key stroke.

With AHK my keyboard changes

1. NumpadEnter::Send {Down}{Home}
   Move to beginning of next line without a new line
   
2. +NumpadEnter::Send {Down}{End}
   Move to the end of next line without a new line
3. CapsLock::Shift
   Never use Capslock but making it a shift sO I DO NOT SHOUT

Make all the difference working with tiddlywiki, believe it or not.

My check is "in the mail"

Tony

[HansWobbe]

Tony:

You might have encountered this statement from the 1980s ...
"If you're doing the same thing over and over again on a computer, then you aren't thinking."

Enjoy your "vetran" tricks.  They save precious time.

Cheers,
Hans


On Wednesday, September 13, 2017 at 8:20:58 PM UTC-4, TonyM wrote:

...

 

Just think for yourself how often you find yourself doing some repetitive tasks on a computer, and if they include a Ctrl-alt-key combination or two you need finger gymnastics along with deep focus to do something really trivial..

...

 

Tony

[Jan]

Hi folks,
I am noticing a little desperate that this thread has drifted quite far away form its original intention.
Just two things:
There is a text-completition tool in TW http://snowgoon88.github.io/TW5-extendedit and I would strongly recommend using  and developping this instead of prehistoric plattformdependent 3rd-party-tweaks. It even has the advantage of browsing through the tiddlers for potential completitons.
And  somewhat back to topic: It is far easier to handle transclusions than macrodefinitions for the role-names.

And now back to the core of the thread:
Is there anyone working on or willing to work on adjustible parsers?


Thanks
Jan

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/0956c8d1-f866-49a7-af0c-52f1c5b25125%40googlegroups.com.

[@TiddlyTweeter]

Ciao TonyM & Coda

Its very interesting. I agree that programs (like AHK) that can interface with other programs AGNOSTICALLY (i.e. there are no dependencies) can be super useful. Saving time. And getting round limitations in the Mother program (that AHK is helping).

It deserves attention.

I often thought a site named "Applications that help me TiddlyWiki" with filmed/documented demos could be really useful. (Otherwise it would likely be too esoteric and end up as a footnote in the passing stream and looks like just a Nerd's Fetish :-)

FWIW, the one program I use most to help with TW is the Windows program "PowerGrep" --- a feature rich RegularExpression engine that has super file handling. For preparation of material for a TW its amazingly useful ... for instance: to create tiddlers in native format and inject them directly into a TW.

But all of this is for another thread. Later.

Best wishes
Josiah

[@TiddlyTweeter]

Ciao Jan

Is there anyone working on or willing to work on adjustible parsers?

I feel, as I look at this more, that we have to get precise about what we asking help for---otherwise its maybe TOO open-ended and would be a burden for others to cope with?

I'm working on breaking the issue down into PARTS--as far as I'm able--so that a more limited question (hopefully) someone might have interest in and be able to answer without burden.

Its tricky, in that the underlying issue with "auto-markup" is quite close to the central way TW works--so there are a lot of implications to deal with---and no sane helper will want to deal with all that :-).

Best wishes
Josiah

[Jan]

Hi Josiah,
I wrote it this general here, because Jeremy mentioned the idea/intention somewhat above.

My desires are quite simple:
I would like to have something which can transform the parsing of a set of tiddlers which are defined by a listfilter according to a replacePragma defined by reg-Expressions:

Transform: [tag[Scene]]

In Skeeves Pragma the desired changes would be this:

\replace /\n!/\n\n!/
\replace /\n:/\n\n:/
\replace /\n{{/\n\n:{{/
\replace /}}/}} /

Yours Jan

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/744d7001-c127-4830-9a4a-7bda014c5e92%40googlegroups.com.

[TonyM]

Jan,

I hope someone can help you get what you are asking for but I can't help but think you are somewhat partially dictating a solution and not describing the problem. I understand why Josiah was trying to open up discussion. I support his approach.

One reason I wanted a clear understanding of what was needed, what was the root problem to be solved, is because I am confident there are already solutions that will address these effectively. In TWC there are a number of opportunities to effect something similar, but I have seen them appearing in TW5 in different guises. But I am not so sure, because more specific questions are being asked.

Less specific questions may elicit answers that are more broadly applicable to others.

For one, I am keen to develop a number of "shorthand" methods while taking notes in class to enter text with Questions. Answers, Actions, Personal areas to explore, key knowledge item etc. CSS, Auto Replace, Editor Tools and more can solve these problems along with a parser.

Also digression into Auto-hot key becomes applicable in notepad, word and outside TWC/TW5, I have not looked but I am quite sure Autohotkey workes on other platforms as well, or at least has equivalents.

Lets see if someone can answer your question, but please consider why the discussion diverged.

Regards
Tony

Regards
Tony

[Jan]

Hello Tony, Hi all
to explain the problem a little wider:
I am working on a screewriting tool a part of which should of course be autoformating the text according to the necessities of o screenplay. I think I already described the standarts above.
A part of the idea is to use the standard TW markup for lists an headings (: and !) in this formating (Which interestingly is also compatible with the fountain syntax)
This makes sense because it is usefull to have a syntax which just needs a marker at the start of the line and does not have to be closed (like <dialog> </dialog>).
The problem with this syntax in TW is that the whole formatting might be broken if you forget the blank line in between. This is why I seek a parser which does not need the blank line before headings and lists.

Thanks for your interest and help.
Jan

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/fbe06126-de10-4472-a52e-b3484316bb93%40googlegroups.com.

[TonyM]

Jan,

Thanks for that clarification, the problem you face is now much simpler to solve now I know the core requirements. As I said I have a hunch the answer exists already so I will look around a little to see if I can find and answer along with helping you extend the features.

Regards
Tony

[TonyM]

Jan,,

I will endeavor to explore your requirements over coming days however

• Have you looked at TWC as TiddlyWiki Classic does not have the same blank line issues, has its own alias and other plugins?

I had to reread this whole thread again to try and clarify your requirements. Perhaps not this time, but it may help in the future you may consider the following;

I do not in anyway want to sound paternalistic, I do not have the information to know your experience, so I will just spell it out.

• Build a requirements document
• Spell out your your requirements without any solution or method
• Have a separate section if you must on possible approaches
• Be comprehensive but not long winded
• Include important details such as specific characters, their meaning, the result
• Use formatting to make it really clear
• Treat the reader including yourself as dumb or naive (in this first document)
  
• Provide selected and specific references, for example to the fountain standard that lists the special characters
• Be systematic because because if someone who knows about script writing, can't describe it systematically, how can people who do not know script witting find systematic solutions

Such a document will clarify your own requirements, set you up to succeed, as well as make the cost of participation by volunteers much lower. Such a document should work for any platform not just tiddlywiki and stands to document what was done in the end.

Just imagine if someone in your audience had all the answers they could simply read your requirements on one screen and pound out the answer on another screen and you would be able to see how they have addressed every part of your requirements.

I would also like to add that engineers, coders and many others have the instinct that drives them into solution mode way too soon. I know because I have had to battle this myself in my career. The solution I have found to do as I have suggested above otherwise the conversation wanders all over the place, but not in the direction you want, as is clear in the threads.

We can solve your problem and / or meet your requirements

Tony

On Sunday, September 17, 2017 at 8:11:34 PM UTC+10, Jan wrote: