Shakespeare, as you like it.

[Mark S.]

One play of Bill Shakespeare rendered in TW5 form, with the "semantic unit" of one line per tiddler.

http://shakespeare.tiddlyspot.com/

Text is displayed in chunks of scenes via the "Shakespeare" tab.

Use the home-brew search engine in the table of contents to look up quotes.

Doing it at this fine level of granularity allows a user to search for a particular phrase, and then click through to the Book/Act/Scene where the phrase is used. However, the cost is that a single small play appears to take up about 1 meg of space. It might be better to load the play with semantic units of a single scene. Or perhaps at the dialog level, though much of the dialog already consists of only one or two lines. Thoughts?

Converting it at this level also posed some challenges, since every line (including stage directions) needed to be assigned an identifier reflecting its usage in the play.

Mark

[RichardWilliamSmith]

Hi Mark,

It looks great. When I did similar with Macbeth, I was able to use the spreadsheet (and a lot of futzing) to add several fields to each line of the play - which might seem like overkill but it means that you can, for example, extract all the lines spoken by a particular character etc.

The primary key here is as simple as possible - every piece of dialog and direction gets a sequential integer.

As for the total size of the finished document, my advice would be "don't panic!" - most of the web pages we load every day are much bigger than even a fully-stuffed tiddlywiki. I was reading this slide-deck just the other day which you might be interested in; http://idlewords.com/talks/website_obesity.htm

"Let's take a look at the Apple page that explains iOS on the iPad Pro. How big do you think this page is? 
Would you believe that it's bigger than the entire memory capacity of the iconic iMac? (32 MB)
In fact, you could also fit the contents of the Space Shuttle Main Computer. Not just for one Shuttle, but the entire fleet (5 MB).
And you would still have room for a tricked out Macintosh SE... (5MB).
...and the collected works of Shakespeare... (5 MB)
With lots of room to spare. The page is 51 megabytes big."

So, you see, if our file is only 10 or 20 times the size of our actual content, we are really doing quite well ;)

Regards,

Richard

[Josiah]

Ciao Mark & RichardWS

Good stuff! showing what can be done.

Mark, just looking at it made me realise how relatively easy it would be to tweak further in many ways... for instance to place all STAGE: instructions into italic or a different colour.

Richard, the depth of decomposition you go to, and adumbrating various indices, is fascinating. I'd be interested to know how much this is automated, or could be. It certainly gives potential fine grained ways of studying & using long texts.

The more I look at all this the more I think than just be an approach to presenting text, but actually to AUTHOR text. Specifically I can see how it might be developed into a SCREENPLAY writing tool.

Best wishes
Josiah

[Richard Smith]

Hi Josiah,

It's done by pummelling a spreadsheet ~ it's supposed to be automated but in practise, it takes a lot of trial and error to get the macros right. If you had a hundred of them to do, it'd be fairly automated but you'd still be cleaning up the input by hand.

This is the spreadsheet I have left over, but it seems to be missing all the macros, so I must have only kept a copy of the output, but you get the idea.

https://docs.google.com/spreadsheets/d/1Pyg99FHK8GhuPWHSLHNoLGmvypTW2Gr_wVL15Jrmeps/edit?usp=sharing

I agree that a screenplay tool would be interesting. One thing I have wondered about before is whether it makes sense sometimes to have a different, kind-of "intermediate", editing mode where an end user can edit content, but without dropping into the full wiki-text editor. Sort of like a very limited wysywig editor. We would need to create a pleasant writing environment for the creative mind.

Regards,

Richard

[Mark S.]

Hi Richard,

In my experience, there are serious performance difficulties before 10 megs with TW, especially on devices. So a 6 meg Bible will take 18 seconds to load. Once it's loaded it doesn't work too bad for reading.  Saving may be another matter. A particular problem is that on Android devices, if you go away from an app  and come back, it may arbitrarily decide to reload the page. It has to do with how Android manages memory. I wonder if the i-products suffer from the same condition?

I took a similar approach to the data, except I compressed data wherever possible. I used the spreadsheets with various formulas to massage the data. There was a lot of by-hand work to fix it up. The problem is that the source document was mostly concerned with actual speakers, but when you're importing everything, including stage directions needs to be assigned a value that says what it does and where it goes.

To reduce overhead, each line is given a 5 part id: play.act.scene.speech.line. "Speech" is a number that indicates which speech (set of lines) this is. With a js macro containing an embedded list of actor names/speech part, it would be possible to extract actor name if that was desirable. I suppose it wouldn't add too much overhead to put in a hex code or something to identify the speaker (just looked it up -- there's about 1200 speaking characters in all Shakespeare's works) .

The question is, is anyone other than an occasional english major  doing a dissertation all that interested in TW versions of Shakespeare?

That IOS page came up very fast.  I downloaded it. The entire thing with images is only about 1 meg. Maybe I'm not looking at the right page?

Thanks!
Mark

[Josiah]

Ciao Richard S.

The spreadsheet approach makes the text easier. Was that found or did you make it?

I'm looking at (thinking about, where i have competence) is Incremental RegEx with arbitrary text. One of good things with RegEx is you can test for compliance and alter next steps accordingly. Your semi-extraction of meta data from the spreadsheet is REALLY impressive. Its giving me ideas. Thank you.

The screenplay thing is interesting too. You are RIGHT that its best for specific apps to provide the Appropriate Editor. For Screenplays its, fortunately, minimal. because the constraints in the film industry for scripts are very tight, simple & doable.

Lets go on
Josiah

[Mark S.]

A couple more comments (I understand e-mail-ees don't always see updates).

The rawest form of the original data was 176k. The resulting TW size difference appears to be about 780k. So only about 4x data inflation. Unless my math is off. Which it could be. However, if I added Orator to every tiddler, I suppose that would add an estimated 52k (2824-200)*20, assuming 20 bytes per field).

This project didn't use the export-to-.json approach. Instead, there's a javascript macro that takes a tiddler full of the carefully formatted data, and converts it all into tiddlers in about 2 seconds. So the slow part is getting the data right. And writing the macro.

Mark

[RichardWilliamSmith]

Hi Mark,

I agree the ios page doesn't seem to be nearly as heavy as suggested by the slide-deck I linked to. I don't know why.

Yes - ios devices have the same annoying tendency to reload whole pages when there is too much else going on - it seems to be getting better on the newer devices but it's still an issue and I agree that if you're targeting mobile devices, this is probably more of a concern.

Have you thought about trying to get it to run as an app?

Regards,

Richard

[Mark S.]

By "app", do you mean andtidwiki? Loading times are somewhat faster and performance a little better, and the formatting flows a lot better on andtidwiki, but it still has the problem that it will sometimes "forget" and need to reload the page. Other apps reload too -- kindle can put you on a refresh screen for the longest time. So it's a platform-wide problem. What's needed is a way for apps to save a complete snapshot of their state before closing (not just the last url).

Mark