[TW5] Should I write a typeblock parser for a "type" that doesn't exist?

[Antaeus Feldspar]

One of the regular needs I have is to deal with data that is delimited with a scheme *like* comma-separated values, but utilizing some other delimiter (one source I deal with uses tabs as a separator; another uses tildes (~).)

I can probably write a wikiparser which is similar to the core parser for "text/csv" files. The way I envision it, you would have the tokens delimiting where the block begins and ends, and immediately after the opening delimiter, an additional string which identifies (a) the major delimiter character (commas, tabs, tildes, etc.) (b) the "paired escape" characters, if any (for instance, in some variants of CSV, you can enclose an entry in double-quotes and a comma between them will not be taken as a delimiter but as part of the data) (c) the "single escape" character, if any (for instance, if you want a comma following a backslash to be read as a comma in the data rather than a delimiter.)

I have at least two questions, however:

1) This isn't an existing "type", not the way "text/csv" is, nor is it likely to be. Does this make it inappropriate to use the existing typeblock syntax for it?

2) Even if it's okay to use the typeblock syntax for a non-standard "type", does the idea of having a "control" string at the beginning defining how the rest of the block is to be read violate the intent of typeblock syntax?

A bonus question: is this the sort of discussion I should start on Github, rather than on the group?

[Tobias Beer]

Hi Antaeus,

What's the goal of the exercise?

Can you describe in more detail the workflow you have in mind?

Is it to "simply" output those data in the desired way, e.g. a table?

Best wishes,

— tb

[Jed Carty]

It may be best to create a javascript widget to display the information and just put the raw text into a tiddler. For a simple parser for using an arbirtary delimiter for a csv-like text string would probably not be too difficult. If you do this you can just have the data in raw format and have the widget put it into whatever form you want when it is displayed.

[Antaeus Feldspar]

Tobias - Yes, the intent is to display a section of tab-separated values or tilde-separated values as a table within a scrollable widget, just as TiddlyWiki already does out of the box for comma-separated values. If you put the following text in a tiddler:

$$$text/csv
,,
1,2
3,4
$$$

you see that table of comma-separated values gets neatly displayed as a table. I frequently get data that I would really like to see as a scrollable table, but it's separated by tildes or tabs rather than commas.

Jed -

What do you think would be the advantage of implementing it as a widget, rather than as a parser for a ContentType?

[Matabele]

Hi Antaeus

The easiest and quickest way is to modify: $:/core/modules/parsers/csvparser.js

replace the string in the line:

var columns = lineText.split(",");

with a regular expression:

var columns = lineText.split(/,|~|\t/);

regards

[Jed Carty]

It is much more flexible to have a widget. Being able to put <$displaycsv data=SomeTiddler options=someoptions/> where you want a table lets you have a single tiddler with the data and display some set of information from it in multiple places. It also allows you to make something separate from the core so it is more portable and easier to debug.

Part of this is selfish because I have been pushing to move away from using something special about the tiddler containing the information to determine how it is handled to making whatever uses the information be in charge of interpreting it.

[Tobias Beer]

Hi Antaeus,

Ah, I was not aware of this yet / or forgot about it / your example also does not appear to be on tiddlywiki.com...

 

Tobias - Yes, the intent is to display a section of tab-separated values or tilde-separated values as a table within a scrollable widget, just as TiddlyWiki already does out of the box for comma-separated values.  If you put the following text in a tiddler: 

 

$$$text/csv
,,
1,2
3,4
$$$ 

 

you see that table of comma-separated values gets neatly displayed as a table.  I frequently get data that I would really like to see as a scrollable table, but it's separated by tildes or tabs rather than commas.

 

Perhaps we can add options to the syntax like so:

$$$text/csv(separator:"~", option-foo:{{config!!value}}, option-bar:<<config-variable>>)
,,
1,2
3,4
$$$

 

What do you think would be the advantage of implementing it as a widget, rather than as a parser for a ContentType?

Please correct me if I'm wrong, to any wikitext markup there is always an underlying widget. In other words, the functionality would be implemented on the widget layer anyways and only then we'd be amending the wikitext syntax.

Best wishes,

— tb

[BJ]

There is a ticket for extending the type:

Introduce vocabulary parameter to wikitext content type

https://github.com/Jermolene/TiddlyWiki5/issues/345

I put in a pull request:

basic vocab stuff - includes changes to wikiparser by buggyj · Pull Request #629 ·

https://github.com/Jermolene/TiddlyWiki5/pull/629

which lead to my flexitype plugin which is on bjtools:

http://bjtools.tiddlyspot.com/

It allows options to be passed to parsers

[Antaeus Feldspar]

| It is much more flexible to have a widget. Being able to put <$displaycsv data=SomeTiddler options=someoptions/> where you want a table lets you have a single tiddler with the data and display some set of information from it in multiple places. It also allows you to make something separate from the core so it is more portable and easier to debug.

| Part of this is selfish because I have been pushing to move away from using something special about the tiddler containing the information to determine how it is handled to making whatever uses the information be in charge of interpreting it.

Well, here's the thing: one of the uses I put the TiddlyWiki to is a "to-do list", where I record (sometimes in very small snatches of time) tasks that fall into my work domain, which I get from a table that is distributed as an Excel spreadsheet. And to be clear, we're talking about "a table" in the sense of a 1990's web guru using tables for making elements line up on the page, not something that could be read programmatically.

If I have a typeblock parser which reads the TSV format you get from copy-pasting, I can copy and paste each section of the table relevant to me into a tiddler, put the typeblock markers around it, and poof! The tiddler contains a table just as readable as the one I got it from, for just a few seconds effort!

It seems like the widget style you propose would require that each task be represented by TWO tiddlers: one for the data, one for the presentation of the data. That seems overkill, because there's only one place that each section of data needs to be read and only one way that it needs to be read.