Fountain Parser for a screenwriting tool

[Jan]

Hi all,
I am working on a tool for screenwriting (So far in german but with a
languageTiddler to configure: http://storywriting.tiddlyspot.com/)
The aim should be managing ideas, roles and storylines and of course
formatting Wikitext as a screenplay.
After hours of trying to adapt the normal rules and getting into an
awfull mess I finally decided I should follow an Idea brought up by BJ
and Josiah and implement
the fountain library https://fountain.io/
Is there anyone who has done this already?
At the moment I got no clue how this could be achieved.

Yours Jan

[@TiddlyTweeter]

I feel for you

Its a lot more difficult than it first looks.

The strictness of the screenplay form is both what is good about it and challenging.

I'm incapable of helping you with code. However I can help with conceptualisation of how to think about screenplays.

In my own thinking they are a mix of TWO fundamentally different types of things.

A script is a crossing of "vertical" sections (act, scene etc) and "horizontal" components that form the "content"  -- Dialogue, Direction, Actions, Scene instructions.

Whilst its relatively easy to think about Tiddlers to create the "vertical aspect" its not at all easy to conceptualise how to most appropriately do the "horizontal" content. Should they be separate Tiddlers? Or just Boilerplate text?

Some cases bring out more clearly the needs. For instance, would you need to be able to extract ALL but ONLY the dialogue of Eve? To be able to do that all Eve's dialogue would need to be in Tiddlers OR you gonna have to develop very smart Regular Expressions to extract it. The same applies to Scene Setting that Executive Producers &  DPs would need.

My feeling now about Fountain is its excellent if you going for the Regular Expressions extraction method. TW can actually do much more than Fountain. I think the issue is getting the right conceptual model to start from.

Just thoughts

[Jan]

Hi Josiah,
fountain (at least as it seems to me) is a markuplanguage with a freeware .js. Hopfully it could be implemented like markup is in the new release, some features are already very similar.
http://bjtools.tiddlyspot.com/#%24%3A%2Fplugins%2Fbj%2Fmarkdownlike%2Freadme
There are implementations for libre-office and mediawiki.

Developping the story of course will be a task for TW.

Yours Jan

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/d58fb788-661e-4206-ae2b-8659b5d20048%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[@TiddlyTweeter]

Ciao Jan

I looked more closely at Fountain Syntax.

It might be possible, I think, to do a reduced, but still effective, simpler subset in a TW markup. Its basically using regular expressions.

The Fountain *forced* syntax is very simple. Lines starting with special characters get converted in a consistent way. This means the parser does not to have to make "guesses" the more complex method uses. The minimal small "forced" set is only this ...

. = Scene Heading

! = Action

@ = Character

> = Transition

Character is the most complex because its a block, not just a line, i.e. its Character name plus dialogue, and sometimes parenthetic comments.

My point is that it might be just as easy, if not easier, to achieve directly in TW, rather than import a library.

Just thoughts
Josiah

[RichardWilliamSmith]

Hi Jan,

I'll just echo what Josiah has said and say that I think you can actually do a better job with tiddlywiki than you could with that library, just because Tiddlywiki already has mechanisms for handling lots of tiny bits of content, knowing what order to put them in and knowing what kind of thing they are. You might not even really need the idea of 'markup' to denote the role that a particular fragment plays because you can set your own meta data.

Much more important is how you would actually want to use the system once you have it. In my opinion, you should try to specify what operations you would like to be able to perform and hence what data structure you want/need to use to represent a 'work'. 

What I mean is that representing a play and then rendering it is quite straightforward - (I did it with some of Shakespeare's plays when I was playing with how to structure large numbers of tiddlers) but the key I think is to say what controls a writer needs when they are writing a screenplay. For example, you probably want to be able to easily edit the actual text of a line, add a new line etc. but it's probably not so important to be able to toggle a piece of content from being a line of dialogue to being a scene heading, because that isn't something you'd commonly do.

One place that i think there's room for user innovation in TW, as yet untapped, is in modifying the editing experience and I think that a playwriting template is potentially an interesting application. I hope you'll share what you make so that we can all see/help.

Regards,

Richard

[Jan]

Hi Richard, hi Josiah,
I think I will continue my efforts then. 
Seeing the syntax fountain uses, it should be the smartest way to do it interpreting "action" "charakter" etc as list-items and define a special format for them...because you do not have to close them.

The only problem i see is that in the normal parsing behaviour lists are broken if they are not preceded by other list items or an empty line
Do you think we could we change that behaviour in the $:/core/modules/parsers/wikiparser/rules/list.js?

Yours Jan

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/6ed83b13-7134-4909-97df-ccc9fbee5cd6%40googlegroups.com.

[@TiddlyTweeter]

Ciao Jan

Do lists come into this? As far as I can see all markup for screenplays is "horizontal". Lists involve nesting. I can't see the need for it in screenplay markup.

Best wishes
Josiah

[@TiddlyTweeter]

Ciao RW Smith

Totally agree with you. Only HALF the problem is format. The other half is WHAT is the best way to EDIT this?

Movie screenplays are a great test case because of the strictness of their form its possible to define and explore needs quite precisely. 

Best wishes
Josiah
 

[@TiddlyTweeter]

Ciao Jan

Just a footnote that the Fountain parser, though good, has concepts behind it that are not fully compliant with the more conservative HOLLYWOOD STANDARD.

Because of the way the film industry works its, IMO, better to embrace the "Hollywood standard". You can't really go wrong with that.

For example,

• Fountain permits complex emphasis (bold, italics, underline)--but most real screenplays still ONLY permit underline for emphasis.
  
• Fountain's idea of "Action" is rather crude. Its a kinda of bucket that anything goes into that nothing else deals with.
• Their idea of what is a heading / scene / slug is very crude. A slug could be "WATCH ON TABLE" ... their markup could not get that unless you forced it ...
  
• Their Transitions seem all right aligned. Some transitions should be left aligned.
• Their numbering of scenes does not make clear that scene numbering should ONLY be done in SHOOTING scripts, and even then not in the way they present it. They blur up "shooting scripts" and basic "speculative screenplays".
  

I think its to do with how Fountain evolved. I think its more complex than it needs to be to enable good markup. Declarative starting markup (what they call "forced") looks like the simplest way to go. BUT Fountain really has helped understand HOW to auto-format screenplays.

Best wishes

Josiah

[Mark S.]

A start. I suppose the use of iFrame makes it unusable. Trouble feeding it CSS without hard-coding. Maybe get back to it in a couple days.

[Mark S.]

A beginning wrapper for Fountain. Not quite sure if there's still interest. Oh well.

I may not have included all the css classes to render every situation that Fountain can handle. But you can write over the sample CSS file to expand it's repertoire. 

This uses an iframe wrapper rather than a direct TW-like rendering because the Fountain.js library makes HTML fairly easy but tokenizing is difficult. It hands you a load of tokens that you would have to map somehow into the TW tree, without knowing what the various tokens represent. Ach. Enough techno-babble.

Mark 

[Jan]

Hi Mark,
of course we are interested. But how do we get this to work?
Tag the Fountain-Tiddlers with a special Tag?

Yours Jan

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/dc181905-caaf-4922-8f7a-e7590b793b62%40googlegroups.com.

[Mark S.]

Oops. Forgot to mention. Use tiddler type text/fountain on Fountain markup tiddlers.

[@TiddlyTweeter]

Ciao Mark (& Jan)

Mark S: A start. I suppose the use of iFrame makes it unusable. Trouble feeding it CSS without hard-coding.

Mark S: A beginning wrapper for Fountain. Not quite sure if there's still interest. Oh well.

Oops. Forgot to mention. Use tiddler type text/fountain on Fountain markup tiddlers.

I'm very much interested in your interest in this--since you understand the underlying mechanisms I don't.

Let me pre-empt myself---I AM working on some demos that illustrate, I hope, the issues---by making a few points that in my slowness I will fairly soon hopefully be able to show...

1 - The question of HARD-CODING v. coding that enables ANY system of markup to be used at will.

2 - In fact, I'm not sure that "markup" is quite the right word needed. I suspect what is needed is more like a "text transformer" that is completely agnostic about whether "markup" is present or not.

3 - A mechanism that will allow AD-HOC "markup"---as well as RegEx transformations that require NO markup at all. The "intelligence" of RegEx can take you a long way.

4 - A clear separation of the "engine of change" from CSS "styling".

5 - I guess what I am getting at is a more GENERIC process that utilises RegEx via a global macro (limited to a new Tiddler type to prevent it "eating-my-wiki"?)

6 - In my fantasy the type could be ...

text/vnd.tiddlywiki.mss

Where "mss" stands for "manuscript/s" ...

 --- 'MSS... the original handwritten or typed version of a book, article, etc, as submitted by an author for publication'
 --- 'They appear to have fallen, with other MSS., into the hands of his executor.'
 --- 'There are other MSS., but they are all either mutilated or of much later date.'

Articulating what I am getting at is not so easy so please ask where anything is unclear. I have near finished demos of using the SAME MARKUP to present a script as both a Stage Play & a Screenplay. I hope to get it done & show-able subito.

My MAIN point is I DO NOT think this is about yet another fixed, hard-encoded markup. Though being able to generate schema that appear hard-coded could be a big plus? Maybe I'm an idiot? :-) 

Let me emphasise that your just out Fountain Markup gizmo looks interesting & very useful and I'm gonna play with it today. I don't want to sound like I am knocking that in any way, I'm not.

Very best wishes
Josiah

[@TiddlyTweeter]

Ciao Mark S. & Jan

One of the minor issues with both WikiText markup & Fountain markup (!, @, # etc) is that you can't just pass it to CSS because CSS class names only accept alpha-numerics. Its not a big deal but its not as easy as using, say, start of line markers like this ....

 :s = scene
 :a = act

With those you can use RegEx to capture the "s" or "a" very simply and pass them directly, with fewer steps, to a RegEx replace ...

<p class="mss-screen1-s">CONTENT</p>

<p class="mss-screen1-a">CONTENT</p>


Just a "bits-n-pieces" note ...

Best wishes
Josiah

[@TiddlyTweeter]

Ciao Mark S. & Jan

Another "bits-n-pieces" note ...

If "markup"  were in the form (start of line) ...

:a

or

:aa

It would allow for over 7,000 unique combinations of "markup" code. This could open the way to precision layout for multiple types of manuscript ... not just screenplays, but also novels, legal documents, poetry, and that diary format you spent 5 years developing....

If one added a "qualifier" after a second colon you could go further still ...

:s:3      (scene 3)
:pn:67  (page number 67)

I'm NOT wedded to these ideas. Just in search of a MINIMALIST, FLEXIBLE ORTHOGRAPHY that RegEx can understand in a millisecond.

Best wishes
Josiah

[@TiddlyTweeter]

Ciao Mark S. & Jan

Just a "bits-n-pieces" note ...

... that the CSS that Fountain uses BREAKS the more conservative HOLLYWOOD STANDARD. If you are writing a screenplay for actual submission to a production company you really should use the CONSERVATIVE standard to be on the safe side.

The film industry will tell you IF they need any other formatting. But IF you do excess formatting, not in the conservative standard, its used as a reason to dump your application.

Best wishes
Josiah

[@TiddlyTweeter]

Ciao Mark S. & Jan

Just, another, "bits-n-pieces" note.

Fountain markup is oriented to WHOLE SCRIPTS. It has NO CONCEPT of what WE do, which is to be able to work "from FRAGMENTS to WHOLES".

Some of the things that be-devil Fountain we can probably solve, eventually (after sweat), much better.

Like WHAT IS A PAGE?

Fountain hard-codes pages as its only method. That is unworkable until you have finished writing your script (unless you are an expert at counting 57 lines BEFORE rendering---i.e. figure out how everything will wrap--impossible).

But PAGE NUMBERS are very important in movie scripts (because 1 page = 1 minute of film). They need to be DYNAMICALLY calculated whilst WRITING so you know WHAT timing you have.

Once the script is finished they need to be STATICALLY FIXED.

Best wishes
Josiah

[Mark S.]

Hi @TiddlyTweeter, Jan, et al,

1 - The question of HARD-CODING v. coding that enables ANY system of markup to be used at will.

Yes -- that's why I posted here -- avoiding those issues ;-)
 

5 - I guess what I am getting at is a more GENERIC process that utilises RegEx via a global macro (limited to a new Tiddler type to prevent it "eating-my-wiki"?)

A macro? I can imagine a macro that you pass your text and your RegEx rules and it outputs HTML-ized text that you then modify with CSS.

Is that better or worse than a parser (or pseudo parser) that allows you to see the effect of your text in the preview window as you work?

My MAIN point is I DO NOT think this is about yet another fixed, hard-encoded markup. Though being able to generate schema that appear hard-coded could be a big plus? Maybe I'm an idiot? :-) 

But this thread was. At least at the start. Sometimes it's easiest to start with specifics and then work your way outwards.

One of the minor issues with both WikiText markup & Fountain markup (!, @, # etc) is that you can't just pass it to CSS because CSS class names only accept alpha-numerics. Its not a big deal but its not as easy as using, say, start of line markers like this ....

 :s = scene
 :a = act

Elaborate. I'm not sure what you mean by "pass it to CSS".

If "markup"  were in the form (start of line) ...

:a

or

:aa

What would the expected output of these markup tokens be? Would they be line-based, or apply to the entire MSS ?

Fountain markup is oriented to WHOLE SCRIPTS. It has NO CONCEPT of what WE do, which is to be able to work "from FRAGMENTS to WHOLES".

Some of the things that be-devil Fountain we can probably solve, eventually (after sweat), much better.

Like WHAT IS A PAGE?

Fountain hard-codes pages as its only method. That is unworkable until you have finished writing your script (unless you are an expert at counting 57 lines BEFORE rendering---i.e. figure out how everything will wrap--impossible).

But PAGE NUMBERS are very important in movie scripts (because 1 page = 1 minute of film). They need to be DYNAMICALLY calculated whilst WRITING so you know WHAT timing you have.

My assumption is that something like Fountain is useful to help a writer see how their formatting is shaping up as they work. TW shines at allowing an author to write bits and pieces of a work at a time distraction-free and then aggregating the material further down the road.

I don't understand how pages can be dynamically calculated to be 1min=1page, since a single direction item might take up more than a minute. In any event, I would think that the final pagination would best be handled in an application designed for that task. I'm really skeptical of getting browser-based automatic pagination to work. Is there a precedent?

But IF you do excess formatting, not in the conservative standard, its used as a reason to dump your application.

So this explains why, with 50,000 new scripts submitted every year, we keep getting Batman and Spiderman origination stories -- they were the only ones with the properly formatted scripts!

Let me explain about the way parsing works in TW. With standard parsing, it's not a matter of simple regular expression replacement. What happens is that regular expressions are used to prise out bits of string from the working text. The strings are analyzed by code logic and then used to build up a JSON-like internal tree.

The HTML/iframe approach I used was much simpler, because all I had to do was pass the frame the Markup text converted by Fountain.js to HTML (plus a little CSS up front). If the results of that approach are adequate (e.g. you don't hate iframe rendering) then I can imagine a more universal syntax approach. Maybe the top of the page would have instructions:

#REG/\.s\s(.+)$/<div class="something">\1</div>
#REG/\.r\s(.+)$/<div class="somethingelse">\2</div>
#REGG/something-i-want-to-replace-globally/<p class=yipes>\1</p>

I haven't worked out the full syntax obviously. It would be something like Skeeve's pragma thing, but without the controversy since we're not trying the really complicated approach of modifying the parser (which is still something like voodoo magic to me).

All for now,
Mark

[TonyM]

Jan et al,

It seems to me there is three requirements

1. Permit the Writing of scripts with the required elements
2. Permit the formatted display of scripts
3. Permit export of the scripts in a standard format

There is no reason for all three to be in the one solution. Why not use tiddlywiki to excel at 1 and 2 above, then provide an exporter or display that presents a standards formatted view (not display rendered) which can then be exported to plan text files or other fountain standard readers.

Of course Format 1 should resemble Format 3 to assist people familiar with format 3 but the point is it need not be identical, if rendering it in format 3 is simple.

What ever the formatting techniques you use to implement the display of 1 in format 2 should equally be able to be designed to display of 1 in format 3.

Just some food for thought

[@TiddlyTweeter]

Ciao Mark, Jan, TonyM & all

Other things I'll comment on by way of demonstration when I got them working well enough. I'm slow, but hopefully only be a couple more days.

Meanwhile, a short comment on pagination ...

Marks S. : I don't understand how pages can be dynamically calculated to be 1min=1page, since a single direction item might take up more than a minute.

I know it seems insane at first look. But it IS a strong convention in film-making. "The page" is fundamental to production timing & costing. For sure, 1 minute rarely equates directly to the one page per minute idealised cinematic time. But the number of pages in a screenplay DOES relate to that very tightly. IF you submitted a 30 page script for a 90 minute movie (i.e. one with longer shots) it would go in the bin post-haste. What the writer has to do is adjust to the format. It makes sense in that if you have long shots with little dialogue you have to provide more directorial description. So, in actuality it evens out.

Mark S.: I would think that the final pagination would best be handled in an application designed for that task. I'm really skeptical of getting browser-based automatic pagination to work. Is there a precedent?

I haven't seen anything that could act as a model. But I do wonder if CSS might be able to do it. One of the key things about screenplays is they use EXACT layout so maybe a CSS auto-numbering of some kind might be possible? 

Best wishes
Josiah

[Jan]

Dear Mark, dear Josiah

The HTML/iframe approach I used was much simpler, because all I had to do was pass the frame the Markup text converted by Fountain.js to HTML (plus a little CSS up front). If the results of that approach are adequate (e.g. you don't hate iframe rendering) then I can imagine a more universal syntax approach. Maybe the top of the page would have instructions:

#REG/\.s\s(.+)$/<div class="something">\1</div>
#REG/\.r\s(.+)$/<div class="somethingelse">\2</div>
#REGG/something-i-want-to-replace-globally/<p class=yipes>\1</p>

I haven't worked out the full syntax obviously. It would be something like Skeeve's pragma thing, but without the controversy since we're not trying the really complicated approach of modifying the parser (which is still something like voodoo magic to me).

I would love to have that "something-i-want-to-replace-globally"-thing working on the text of some filtered tiddlers...

I made a beta of the screenwriting-thing i am working on to make you understand wht I am after...(The UI is in german i use it for my theatre class at school...I remember you desired some German Josiah...It can be changed by replacing the dictionaryTiddler)

It can be found at storywriting.tiddlyspot.com

It has also got Textcompletition, A scene Manager (behind the curtains...), role-alias-cast manager and so forth. The formating changes between mobile-layout (because my pupils use their phone to read the texts) and a print/dektop one. Role-names are transcluded and the rolemanager in the sidebar detect which roles are used but not defined yet. A completition of rolenames while typing can be activated with the button...

Please have a look at it...
Jan

[@TiddlyTweeter]

Ciao Jan

Jan's storywriting.tiddlyspot.com

I looked at it. Its a well thought through schema. So an objective needs to be to HOW to support it.

Adding (Screeve) regex to it to help the layout is currently dangerous. Unfortunately its NOT really workable as (under the surface) it creates havoc in the text/vnd.tiddlywiki content type. Whether it could be a global macro or a Tiddler local one the same destructive issue arise. Its not a real solution.

The alternate Fountain plugin Mark S. is trying to help us with I don't think could easily work with what you need either. BUT I will comment on that in more detail later.

Best wishes
Josiah

[@TiddlyTweeter]

Ciao Mark S., TonyM, Jan & Richard WS

I just wanted to comment back into this thread--since I left my last posts kinda hanging--and then things opened in a somewhat different direction that I'd anticipated.

I want to give a big thank you to Mark S. who did a lot to help us, probably more than he realises, grasp better the direction needed.

In another thread, BJ has given the basic solution to the MARKUP problem by writing a regex component for his FLEXITYPE PLUGIN. This avoids the problems with Skeeve's useful, but rather dangerous route.

I have a few specific comments that may be of interest to you ...

1 - Mark S.' FOUNTAIN PLUGIN opened my eyes to the fact that Fountain is only really "5%" explicit markup. Its real brilliance lays in interpreting IMPLICIT markup from the simple plain text layout of screenplays.

I did tests throwing into Mark's gizmo the whole script for Apocalypse Now Redux. It got near everything in the layout correct without a single added piece of markup.  That's seriously impressive.

2 - This has completely changed my view of Fountain--as well as what is needed for formatting screenplays in TW.

3 - Now that BJ has given a workable, safe, way to "preparse" using regular expressions, without needing any direct JavaScript coding, I'm now working on a similar approach as (1) for screenplays in TW. Its just a matter of knowing what Regexes to use & the Fountain Library on Github gives many of them. So I'm hopeful I can make something good enough for most practical purposes.

4 - FWIW, what became clear to me, personally, was that in in this thread I was slightly blurring up  TWO things, albeit related: (a) a delimited issue with screenplay formatting and (b) a more generic way to have a kind of "Tiddy Regex Workshop" through which you could develop NEW ways of marking-up text without having to deal with the inherently complex TW core parser system (for which I'd be worse than stupid at). With BJ's plugin that issue is largely now manageable.

My next step is to finish-up creating a basic markup for screenplays.

The step after that will be to turn towards the issue of what is an  APPROPRIATE INTERFACE for editing screenplay Tiddlers?  Jan has done quite some work on this already. I'm hopeful its not going to be quite so complicated moving into that as sorting through to now was.

Things are looking up.

Best wishes
Josiah

@TiddlyTweeter wrote:

Other things I'll comment on by way of demonstration when I got them working well enough. I'm slow...