Problems using SplitTiddler

[Mike H]

Hello all,

I'm very new to TiddlyWiki, so I'm probably making some basic
mistakes....

Any guidance would be very gratefully received.

I'm trying to import a file comprising a series of individual notes
into TiddlyWiki, the objective being to have each note in the original
file end-up in its own tiddler.

I've searched my source file to ensure the character string is unique
and elected to use \n+---+\n as a separator. I set the option on
SplitTiddler to use the first line as the tiddler title.

When I activate SplitTiddler it reports (correctly) that it finds 13
items.

On clicking OK to proceed with the import, it reports 11 or 12 times
that a tiddler called '' already exists, so I cancel the import of
those items.

In the source data there is nothing that would obviously cause
SplitTiddler to believe that there was an un-named tiddler for
import. However, on completion there is indeed a 'blank' line at the
top of the list, by timeline, of the tiddlers in the system (the one
that appears in the panel on the right of the TiddlyWiki screen).
I'm unable to select or remove this rogue un-named tiddler, so the
only way out is to close the file without saving.

A sample of the data file is as follows. There are no blank likes at
the beginning or end of the file:-

UUIDs in the raid environment

From https://bugs.launchpad.net/ubuntu/+source/mdadm/+bug/107080

As far as I know the UUID in mdadm.conf is the Device UUID for md0
which is used in /etc/fstab to mount the drive. The UUID reported by
mdadm --examine is the UUID of the RAID partition which must be the
same for all members of a raid.
+---+
Copying ext3 files to Windows

To copy backup files off the USB hard drive (i.e. from an ext3
filesystem) onto Windows:-

Connect the USB drive to the Windows system
Go to Windows Control Panel
Open 'IFS Drives'
In the drive letter selector against the USB drive, select 'U'
In Explorer, open Drive U: and copy the files
+---+
Software versions

Ubunto 7.04 Feisty
Firefox 2.0.0.4 (64-bit)
Shockwave Flash 9.0 r31
flash-plugin-9.0.31.0-release.i386.tar.gz
nspluginwrapper 0.9.91.4
nspluginwrapper-0.9.91.4-1.x86_64.rpm (wrapper)
nspluginwrapper-i386-0.9.91.4-1.x86_64.rpm (viewer)
+---+
etc, etc.....

Can anyone point out where I'm going wrong?

By the way, in case my choice of note delimiter was an unwise one, I
have tried several other character strings. All produce identical
results.

I've only downloaded TiddlyWiki (the Monkey Pirate one) and the
plugins in the last couple of days, so they should all be up to date.

Thanks,

Mike

[Mike H]

OK, I've progressed this a little further....

I cut down the volume of data in my test import file to two items,
each of which should result in the creation of a tiddler. However, on
importing the data SplitTiddler reported that it would create only one
tiddler. On looking at the imported result it appears that part of my
+---+ note delimiter had disappeared.

I think the reason is that in the text of the first data item there's
a reference to use of the 'mdadm --examine' command. The two dashes
are interpreted as a TextMarkup strikethrough command. The next two
appear in my delimiter, so they're assumed to be part of the same
command and are removed.

Here's the resulting single tiddler that the import created, except
that when viewed as HTML in a browser the words 'examine' through to
the start of the delimiter are struckthrough.

UUIDs in the raid environment

(updated just now by Mike)

From https://bugs.launchpad.net/ubuntu/+source/mdadm/+bug/107080

As far as I know the UUID in mdadm.conf is the Device UUID for md0
which is used in /etc/fstab to mount the drive. The UUID reported by

mdadm examine is the UUID of the RAID Partition which must be the same
for all Members of a raid.
+-+

Copying ext3 files to Windows

etc, etc...

How is this supposed to work? It's removed the two dashes in the
'mdadm --examine' and has removed two in the subsequent delimiter,
leaving it as +-+. The text in between is all struck through.

I see that the default delimiter uses dashes, so presumably there must
be a way to tell the import mechanism to treat the incoming text as
plain text and not HTML.

What am I doing wrong?

Thanks,

Mike

[Anthony Muscio]

Just Quickly,

Some which may help.

The First line - the line at the beginning of the file, and the first line after the delimiter is where the tiddler name must come from and must be unique. Or tick first word. These words must be unique as tiddlers must be unique.

Try any-other delimiter eg; \nThis is the delimiter here and wont be mixed up with others\n

Use search and replace eg PFE32 Editor to remove any offending text that proves to give you a hastle, if you run multiple times delete the successful tiddlers or the second time they will be duplicates. You could change your delimiter to my suggestion, and also change all -- to ## and change it back later if necessary.

I have requested a tip from the author in a previous post to see if each time it runs I can use a title prefix, but he seems to have missed my post.

Remember that whatever ends up in tiddler with be interpreted, wikified according to tiddler rules, so you will possibly find a few exceptions for any particular text source.

Not much time, so I cant fix test it for you but I thought I'd give a few tips that may let you progress, or perhaps you have tried these already.

TonyM

--
Anthony Muscio

[Mike H]

Hi Tony,

Yeah, I've tried several of your suggestions and they haven't made any
difference.

Through experimentation and reading I'm slowly learning, though.

I found that if I edited the data in my source tiddler (i.e the one
into which I had dropped my original text), I could partially get
round the problem, but it then introduces problems of its own.

For instance, I tried placing <nowiki> and </nowiki> at the beginning
and end of the file. I also tried {{{ and }}}. Both solutions
prevent the tiddler being wikified, but they also trash the formatting
by removing all excess whitespace and rendering the output in a
monospace font. I can put up with the monospace font, but the removal
of whitespace makes the visual presentation of the resultant tiddler
pretty awful.

I have over 5000 tiddlers to create from this data source, so
importing and then manually editing is not an option. Also, there are
a plethora of odd character sequences that seem to have a significance
in the wiki world, so I'm reluctant to go down the route of editing
the source data because the probability of me removing all such
strings is too small.

I did see a reference in one of Eric Shulman's old posts to a plugin
called UnformattedTextPlugin which looked like it might be what I
needed, but the link in the post doesn't work and I can't find any
evidence of a plugin by that name, so maybe it's been dropped.

What I need is a version of <nowiki> </nowiki> that preserves the
original text layout and, preferably, the proportional font.

Any ideas?

Given the range of plugins for TiddlyWiki I'm certain there's a
solution to this problem - it's just that I don't know enough yet to
be able to find it in the jungle of TiddlyWiki add-ons, plugins,
scripts, core-tweaks and all the other good stuff that makes it such
an interesting application!

Mike

[Mike H]

I've read posts about how it's possible to have whole days disappear
while 'fiddling with a wiki'. Well, now I can personally vouch for
that. I've spent the entire day trying to find something that would
allow me to import my text data as plain text and split it into
individual tiddlers, but I've failed.... Miserably.

I'm getting really short of ideas on this now....

I've tried all the solutions that I can find - <nowiki>, triple
braces, triple double-quotes. I've tried placing them outside the text
to be imported and I've tried using them to create individual blocks
of text that should be immune from wikification.

None of them work.

I'm left thinking that SplitTiddler is incompatible with the use of
these 'interpret as plain text' commands.

Is that right, or am I missing something obvious.

Here's the test import data file that I've been using today:- (the
lines of hyphens were added just to emphasis the split between the
text of my post and the test file itself):-

------------------------------------------------------------------------------------------

This is the first note heading

Some text.

And some more test

Some text including double hyphens like --these
If this is presented as text these words will not be struckthrough
And here's a closing pair of double hyphens--
+----+
This is the second note heading
Yet more text
And a bit more

And even more....
+----+
This is the third note heading

This is the third note text
And this is the last line of the test file

-----------------------------------------------------------------------------------------

Can anyone tell me how to use <nowiki>, triple-braces, triple double-
quotes or anything else that will allow me to import this data using
SplitTiddler without the data being wikified?

I'm lost! Please help a newbie who's struggling!

Thanks,

Mike

[wolfgang]

Hi Mike,

> Can anyone tell me how to use <nowiki>, triple-braces, triple double-
> quotes or anything else that will allow me to import this data using
> SplitTiddler without the data being wikified?

Sorry for your difficulty. Since even <nowiki> leaves the text as it
is - I would suggest using code blocks:

//{{{
..text goes here
//}}}

- in the default distribution this comes with yellow background and
mono spaced font in a bordered box - however, this is only a matter of
a different StyleSheet applied to it.

Wouldn't know the correct styling by heart, but I can help you with my
showcase TiddlyWiki 'Changing Themes', which includes more than
hundred themes to switch. Though a bit sluggish at 3.7MB - still, if
you display a code block tiddler in its display area and use the popup
menu which appears on clicking the title 'Themes' - you can
comfortably cruise through all of them and find the kind of styling
for code blocks you need.

Here its download link: http://change.tiddlyspot.com/download

Hope this helps you further.

Regards,

W.

[Daniel Baird]

Mike, tiddlers are stored in the html as divs with a fairly simple
format, so you can always just convert your data into a bunch of
tiddler divs and use a plain old text editor to paste them into your TW.

Have a look at your TW's source to see.

If it's just a one off import (or if your data is in a spreadsheet),
that's often the easiest way.

Cheers
;Daniel

[Mike H]

Wolfgang and Daniel - thanks very much for your help.

Yet more experimentation and, at last, I think I realise what's going
on. Now that I'm beginning to understand it, it all makes much more
sense.

I tried your suggestion of code blocks Wolfgang, but I got the same
results as I did with curly braces.

Eventually I realised that SplitTiddler only handles text formatting
instructions like <nowiki>, """, {{{, etc if they are restricted to
the body of what will be an individual tiddler after import. In other
words formatting instructions CANNOT span the delimiter character
string that defines the points in the file at which a new tiddler is
to start.

Also, SplitTiddler does not recognise any text formatting that appears
to be part of what is destined to be a tiddler tittle when the file is
imported. Instead, it just treats those formatting characters as
ordinary text and appends them to the tiddle title.

The ONLY form of file layout that will work is something along the
lines of the following example. In this file I have used +----+ as
the delimiter between individual notes (each of which will become a
tiddler):-

----------------------------------------------------------------------------------------------

This is the first note heading
{{{
Some text.

And some more test

Some text including double hyphens like --these
If this is presented as text these words will not be struckthrough
And here's a closing pair of double hyphens--
}}}
+----+
This is the second note heading
{{{
Yet more text
And a bit more

And even more....
}}}
+----+
This is the third note heading
{{{

This is the third note text

And this is the end of the test file
}}}

--------------------------------------------------------------------------------------------

In this file the text formatting is specified at the level of each
individual note. For the reasons outlined above, it is NOT possible
to apply a single set of formatting commands to the whole file. This
is where I had been going wrong!

The above file imports correctly and results in three tiddlers each of
which is de-wikified, retains whitespace in the text and is displayed
in a monospace font.

This is OK, but it requires the file to be processed in order to
insert the formatting braces for each note prior to importing it into
TiddlyWiki.

It would be nice if SplitTiddler had a 'de-wikify each incoming
tiddler' setting in order to avoid having to do this. A more
sophisticated version could, perhaps, allow the user to select the
form that de-wikification would take - i.e. the ability to select the
retention of whitespace and a proportional font... but I'm dreaming
now!

So it appears that my only solution is to write a script to insert the
braces into my raw data files before submitting it to SplitTiddler.

I'll also look at Daniel's suggestion of manipulating the file and
pasting the results into the TW data file, but I'll still need to
process my original data to achieve the right formatting (along with
the sets of braces around each note).

If anyone thinks I've still misunderstood something, please let me
know. I'd love to find that there's an easier way of doing this....

Thanks,

Mike

[Eric Shulman]

> Eventually I realised that SplitTiddler only handles text formatting
> instructions like <nowiki>, """, {{{, etc if they are restricted to
> the body of what will be an individual tiddler

> ...

> de-wikified, retains whitespace in the text and is displayed
> in a monospace font

> ...

> requires the file to be processed in order to insert the

> formatting braces for each note prior to importing...
> ...

> It would be nice if SplitTiddler had a 'de-wikify each incoming
> tiddler' setting

> ...

> allow the user to select the form that de-wikification would take

> i.e. the ability to select the retention of whitespace and a
> proportional font
> ...

> write a script to insert the braces into my raw data files
> before submitting it to SplitTiddler.

> ...

> I'd love to find that there's an easier way of doing this

OK.... here you go:

I just added a new input field to the SplitTiddler form so you can
specify the desired output format for the target tiddlers. The
default format is simply "%0", where the %0 is a 'substitution marker'
that indicates the location in which the extracted content will be
inserted into the output.

For your purposes, you could use an output format of: "{{{\n%0\n}}}
\n", so that the content in each target tiddler is automatically
wrapped in a "pre" block, without requiring you to pre-process the
source input before splitting it!

Get the update (v1.3.0) here:
http://www.TiddlyTools.com/#SplitTiddler

enjoy,
-e
Eric Shulman
TiddlyTools / ELS Design Studios

[Mike H]

On Sep 10, 5:05 pm, Eric Shulman <elsdes...@gmail.com> wrote:
>
> OK.... here you go:
>
> I just added a new input field to the SplitTiddler form so you can
> specify the desired output format for the target tiddlers.  The
> default format is simply "%0", where the %0 is a 'substitution marker'
> that indicates the location in which the extracted content will be
> inserted into the output.
>
> For your purposes, you could use an output format of: "{{{\n%0\n}}}
> \n", so that the content in each target tiddler is automatically
> wrapped in a "pre" block, without requiring you to pre-process the
> source input before splitting it!
>
> Get the update (v1.3.0) here:
>    http://www.TiddlyTools.com/#SplitTiddler

Wow, Eric - thanks!

That's MOST helpful. I was thinking about doing the pre-processing in
Perl, but you've saved me the trouble.

I'm most grateful. I'll give that a go later this evening.

Mike

[Mike H]

On Sep 10, 5:05 pm, Eric Shulman <elsdes...@gmail.com> wrote:

> OK.... here you go:
>
> I just added a new input field to the SplitTiddler form so you can
> specify the desired output format for the target tiddlers.  The
> default format is simply "%0", where the %0 is a 'substitution marker'
> that indicates the location in which the extracted content will be
> inserted into the output.
>
> For your purposes, you could use an output format of: "{{{\n%0\n}}}
> \n", so that the content in each target tiddler is automatically
> wrapped in a "pre" block, without requiring you to pre-process the
> source input before splitting it!
>
> Get the update (v1.3.0) here:
>    http://www.TiddlyTools.com/#SplitTiddler

That worked beautifully, Eric. It's taken me a lot longer than
expected to post back with the results because I had around 60 files
to import. Due to the structure of the original data in InfoSelect I
had numerous duplicate note titles - something that doesn't matter in
InfoSelect, but had to be resolved before the data could be
successfully imported.

The only other problem I had was that using the FileDropPlugin with
the largest of my import data files (it was about 500K) caused the
'unresponsive script' error, followed by a Firefox crash. Changing
the script timeout made no difference. I eventually split the file
into three separate chunks and then it worked fine.

It's all finished now, and I have a TiddlyWiki with something over
5,500 tiddlers comprising the complete InfoSelect data.

Rather surprisingly, the resultant TiddlyWiki filesize is only 2.9 MB
whereas the original InfoSelect binary data file is over 9 MB.

Performance doing searches is fine. Not quite as fast as
InfoSelect,but perfectly acceptable. Searches on InfoSelect return
results pretty much instantly as each character of the search term is
entered. With TiddlyWiki the same search in the same data takes a
second or so to pop-up the results.

It now takes four or five seconds to save a new or edited tiddler but,
again, that's OK and I daresay there are ways to optimise performance
in this area.

It wouldn't benefit me personally now, but a nice extra feature in
SplitTiddler would be an option to always accept what appear to be
duplicates in an imported data set. Perhaps SplitTiddler could append
something like a timestamp to the end of any otherwise duplicate
tiddler titles that it found? This would make the import of large
amounts of data a lot simpler than at present.

Mike

[Eric Weir]

On 9/12/2008 Mike H wrote:

> Performance doing searches is fine. Not quite as fast as
> InfoSelect,but perfectly acceptable. Searches on InfoSelect return
> results pretty much instantly as each character of the search term is
> entered. With TiddlyWiki the same search in the same data takes a
> second or so to pop-up the results.

In InfoSelect it is pretty easy, even in a large set of notes such as
yours, to quickly locate the unique note you're looking for. How is
TiddlyWiki working for you in this respect? In one test with a very
small set of notes I ended up with several tiddlers opening up at the
same time, when then had to be manually scanned.

> It wouldn't benefit me personally now, but a nice extra feature in
> SplitTiddler would be an option to always accept what appear to be
> duplicates in an imported data set.

Isn't there a plugin that allows for creation of tiddlers with duplicate
titles? Perhaps some code from it could be incorporated into SplitTiddler.

Regards,
------------------------------------------------------------------------
Eric Weir
Decatur, GA USA
eew...@bellsouth.net

[Anthony Muscio]

Eric S,

Thanks for your new version of split tiddler I can use a <<tiddler splittiddler with:prefixName>>, and place the $1 in the numbered tiddler option "NewTiddler (%0)" format to read "$1 (%0)" so now all the tiddlers can be created with the prefixName.

This allows me to use split tiddler, Multiple times to generate tiddlers that are unique to say the ProjectName.

Very Helpful, good work and thanks for you quick coding responses.

Your a champion.

TonyM

--
Anthony Muscio

[Mike H]

On Sep 12, 6:05 pm, Eric Weir <eew...@bellsouth.net> wrote:
>
> In InfoSelect it is pretty easy, even in a large set of notes such as
> yours, to quickly locate the unique note you're looking for. How is
> TiddlyWiki working for you in this respect? In one test with a very
> small set of notes I ended up with several tiddlers opening up at the
> same time, when then had to be manually scanned.

I find it's OK, but using the native TW it's not quite the same as in
InfoSelect.

I tend to use the 'neural' search setting in InfoSelect. I've found
that the easiest way to get the closest to that feature is to use
Udo's YourSearchPlugin. Not only does it work in a similar way to
InfoSelect's 'neural' setting, but it also has the option of a tiddler
preview of the search results which shows enough of the tiddler text
to see if it's the one you're looking for. Sure, you can achieve the
same thing with a regular expression search, but the YourSearchPlugin
allows you to just type the two or three key words that are likely to
retrieve the tiddler you want. It effectively works pretty-well
exactly the same as a neural search in InfoSelect. For me, that's a
big advantage.

I haven't used it very much yet but, so far, I'm happy with the
results. I certainly don't plan to use InfoSelect anymore but will
probably keep it for a while as a backup.

Mike

[Eric Weir]

On 9/13/2008 Mike H wrote:

> I tend to use the 'neural' search setting in InfoSelect. I've found
> that the easiest way to get the closest to that feature is to use
> Udo's YourSearchPlugin. Not only does it work in a similar way to
> InfoSelect's 'neural' setting, but it also has the option of a tiddler
> preview of the search results which shows enough of the tiddler text
> to see if it's the one you're looking for. Sure, you can achieve the
> same thing with a regular expression search, but the YourSearchPlugin
> allows you to just type the two or three key words that are likely to
> retrieve the tiddler you want. It effectively works pretty-well
> exactly the same as a neural search in InfoSelect. For me, that's a
> big advantage.

Thanks, Mike. I briefly checked out YourSearch, too. I wondered whether
it might not turn out to work much like InfoSelect. Experience will
tell, but it looks promising.