[not TW] Review of some competitors (@dev)

[Mat]

Nice comparison between Mendeley, Zotero, Evernote & Citavi

https://www.youtube.com/watch?v=dKGZlfaiE0E

IMO particularly worth noting things from a TW dev point of view but overall interesting for info-system-geeks.

<:-)

[@TiddlyTweeter]

Mat

Its interesting to me in its focus on (1) site scraping; (2) how well the discussed programs organise and present the scraped data.

The mechanism TW needs for that is part covered by TiddlyClip. I think if we can scrape the data into a well prepared form the rest would be a doddle in TW and with more flex.

Josiah

[TonyM]

Mat,

I Note the following video on Citavi divides content into References, Knowledge (links into references) and Tasks.

A Model I may look at rather than simply maintaining references.

Thanks for sharing.

Tony

[Mark S.]

The problem is images. After you use TiddlyClip to grab the basic text (and format it into TW ), you have to go back and save each image you want to keep. Then change the captured text to use your external files. There's still no quick way to create an external uri tiddler.

In addition, indexed programs like Evernote have nearly limitless capacity. You don't have to worry about putting one kind of information in one system and another in a different one to prevent memory overload.

-- Mark

[@TiddlyTweeter]

Ciao Mark S. & Mat

Both Mark's  comment and the video kinda clarified some of my thoughts on TW.

I do have an ongoing concern about SCALABILITY.

Yep. It runs in browser and there are limits. BUT I also think in order to be able to design well to use it to its best at scale one needs to understand better HOW to optimise performance.

Recently there have been some initiatives on this in the core code. But I'm practically unclear on any "ground rules" about what is an optimal approach for scaling up volume of content.

Josiah

Mark S. wrote:

... indexed programs like Evernote have nearly limitless capacity. You don't have to worry about putting one kind of information in one system and another in a different one to prevent memory overload.

[Mark S.]

I ran the video at 1.25 speed, so might have missed something.

Summary

The interest was mostly about gathering and publishing references for published texts, which is somewhat of a niche specialty.

Three of the products were aimed at gathering references. The same 3 had advanced PDF handling ability.

Evernote only offers PDF advanced handling for a premium.

There were 3 internal reference tools: Categories, Tags, Linking.

The 3 citation tools had all 3 abilities. The speaker said that Evernote does not have sub-categories (tags with sub-tags) but that is not true (I can create sub-categories on the desktop).

Comparison with TW.

They all had better web-scraping abilities than TW where a complete capture is difficult even with Tiddly Clip.

TW has no native advanced PDF handling. In particular, no way of adding annotations. Possibly someone could add a field to a _canonical_uri.

TW does not have sub-category abilities, though I imagine someone could come up with a system of naming conventions to emulate it. I didn't watch the prior video to find out why you want sub-categories, but the narrator seemed to think they were important.

TW does not have citation reporting abilities. Data would need to be captured in a structured manner. Someone could write a special edition with specialized fields that might make that available (though rules of citation format are a real pain).

For the particular task of gathering, annotating, and citing references TW is weak out-of-the-box and in a different category. Someone might still prefer it due to it's portability and simplicity, especially if the final product can be produced with some other tool and/or citation requirements are not as strict. Using a tool like TiddlyClip helps also. Being able to create quick, relative links to local resources would be a real boon for research projects.

-- Mark

[@TiddlyTweeter]

Mark S. wrote:

TW does not have citation reporting abilities. Data would need to be captured in a structured manner. Someone could write a special edition with specialized fields that might make that available (though rules of citation format are a real pain).

I have experience in "citation style". Its not that difficult IF your database it draws from is chunked appropriately. If its not chunked enough you'll have issues. On the fly conversion between, say, APA & Chicago Styles is I think doable in TW. In principle.

One issue that is MUCH easier than normal on collated bibliographic academic data is understanding citation requirements, both in print and electronically. It is incredibly well documented. Its been an issue before computers and since and been finessed & documented infinitely.

Josiah

[Ste Wilson]

Jan did some work on referencing and citations for tw.
It's somewhere here...

[BurningTreeC]

Hi all, I've made a demo where we can explore how the Text-Slicer can be used for Drag-and-Drop Import

http://textslicer-dragndrop.tiddlyspot.com/

I think it's pretty self-explanatory

When you drop something that contains an image, it isn't shown because (I think that's a bug) it add a '/' before the image-url (like /http://myimage.jpg) ... that's easy to fix for testing

I think we should explore what the text-slicer can do and what its different settings mean for the various types of content that can be dropped


all the best,
BurningTreeC

[Jeremy Ruston]

Hi BurningTreeC

On 18 Mar 2018, at 09:05, BurningTreeC <hypnotize...@gmail.com> wrote:

Hi all, I've made a demo where we can explore how the Text-Slicer can be used for Drag-and-Drop Import

http://textslicer-dragndrop.tiddlyspot.com/

That’s a great little demo, and very useful, congratulations.

I’m delighted you’ve been able to dig into the text-slicer plugin; I sometimes worry that it’s quite unapproachable.

As you note, the Text-Slicer plugin is now a generic tool for importing XML-like data. I use it to import XHTML and MS Word documents, and plan to migrate the current bibtex plugin to it. The process for transforming document elements into tiddlers is now a declarative JSON-based format that’s easy to hack. That means that it is feasible to consider converting any sane XML-like format — which represents a big chunk of the data out there.

Best wishes

Jeremy

[BurningTreeC]

That’s a great little demo, and very useful, congratulations.

I’m delighted you’ve been able to dig into the text-slicer plugin; I sometimes worry that it’s quite unapproachable.

Thank you Jeremy,

About the Text-Slicer ... it's great, but  it makes the  impression "this is expert stuff only" - it isn't...
There's only an easy interface needed so that people can explore

As you note, the Text-Slicer plugin is now a generic tool for importing XML-like data. I use it to import XHTML and MS Word documents, and plan to migrate the current bibtex plugin to it. The process for transforming document elements into tiddlers is now a declarative JSON-based format that’s easy to hack. That means that it is feasible to consider converting any sane XML-like format — which represents a big chunk of the data out there.

Can we put a list together of all possible XML-like formats?
Could be something for the text-slicer readme

BTC

[BurningTreeC]

I'm just adding two ideas here that should be possible with the text-slicer plugin:

1. There are music scores in XML format (https://en.wikipedia.org/wiki/MusicXML)
   
2. Pdfs can be converted easily to html or xml (on linux / mac / windows 10 (linux subsystem) there's a commandline tool called pdftohtml or pdftohtmlEX)

LaTeX is able to render music scores, I'm wondering if KaTeX can do it with an extension. Then imported text-slicer chunks can be used to create a tw-native musescore from an imported/dropped musicXML file

Alternativelly, there's for example abcjs that renders text-notation into musical scores

For pdfs: Converting pdf to html: pdftohtml or pdftohtmlEX (I tried both, pdftohtmlEX can create a very accurate pdf copy in html ... but I think that's too much for the use-case)

With the new Windows Linux subsystem (or how it's called) both pdftohtml and pdftohtmlEX can be used on Windows, too

When the Text-Slicer handles images right and maybe has an option "pdf" that sets the needed text-slicer settings automatically and lets one specify the image-location to keep them external ...

[BurningTreeC]

If rendering music scores could be done or implemented, in combination with the timer plugin tw would be an app for all the orchestras out there
Even annotation would be easy
Could replace a whole bunch of mediocre software
The timer plugin can switch pages or lines of the music-scores with the playing-tempo
(I've made such an auto-switcher some weeks ago for a simulation of an entrance exam at a local university where questions get presented as projections on the wall and stay there for a limited time)

Sorry all for the slightly off-topic posts!

[@TiddlyTweeter]

BTC: Would this give a piano that could play? See your previous:  https://groups.google.com/d/msg/tiddlywiki/7IRkFw0Xb0c/zb-C-gyrCQAJ

[BurningTreeC]

BTC: Would this give a piano that could play? See your previous:  https://groups.google.com/d/msg/tiddlywiki/7IRkFw0Xb0c/zb-C-gyrCQAJ

yes, not within tiddlywiki but with a real piano player playing it in front of a tiddlywiki

I was referring to notation of musical notes - musical scores - sheets of virtual tiddlywiki paper with notes on it

The idea was slicing a musicXML file (a musical score) with the text-slicer and have something else render the lines of the score
letting something like the timer plugin reveal one line after the other, in the speed of the score (like 120 bpm overall - line contains 32 beats - show next line after ~15s)

yea well, with the "autoplay" on the html <audio> element it could also automatically play the whole score using sound files on a local network server (so that it doesn't delay that much)
if someone has the time and will to implement that we'd have an autoplay piano

BTC

[@TiddlyTweeter]

Jeremy, a footnote on this...

... I sometimes worry that it’s quite unapproachable.

I do NOT find it unapproachable.

I think its far more likely we don't have enough people who would use it who know it exists. Once you get into the area of bespoke importing to TW we talking about a far smaller audience.

IMO if we could raise the number of users broadly you'd find that all the "special editions" would come into wider use.

Just my opinion.
Josiah

[BurningTreeC]

By the way, that remembers me of http://spritzinc.com/

Which could be done within TW, too

[@TiddlyTweeter]

BurningTreeC wrote:

I was referring to notation of musical notes - musical scores - sheets of virtual tiddlywiki paper with notes on it

Got it. IF that could be in a format that could also be fed to synthesiser so much the better. There has been quite a lot of wok along those lines. But I'm not sure there is "universal standard" per se?

[@TiddlyTweeter]

BurningTreeC wrote:

By the way, that remembers me of http://spritzinc.com/

That is a neat idea of "modern reading". Well suited to miniscule screens.

FWIW, I'd be very interested in converting novels to use that approach.

[BurningTreeC]

@Jeremy

The process for transforming document elements into tiddlers is now a declarative JSON-based format that’s easy to hack

What are the tiddlers to look for if one wants to hack it?

BTC

[BurningTreeC]

@TiddlyTweeter

This abcjs library https://abcjs.net seems very interesting for writing sounds and having them played by web-midi

I didn't know that something like soundfonts do even exist - they're basically made like woff or truetype fonts
Can be saved within a tiddlywiki and used by the abcjs library to play the notes one types or generates in human-readable form

I don't know about the synthesizer part, but the basic concept could be realized with this

[BurningTreeC]

See a demo here: http://musicsheets.tiddlyspot.com/

I'll have to solve the audio/ogg decoding errors, then it will play the notes using a specified sountfont

[@TiddlyTweeter]

Whoa! You did it again!

Can I suggest you create a new thread just for this ...

J, x

BurningTreeC wrote:

This abcjs library https://abcjs.net seems very interesting for writing sounds and having them played by web-midi

[Jeremy Ruston]

Hi BTC

The process for transforming document elements into tiddlers is now a declarative JSON-based format that’s easy to hack

What are the tiddlers to look for if one wants to hack it?

Have a look at the rulesets that come in the plugin:

https://github.com/Jermolene/TiddlyWiki5/tree/master/plugins/tiddlywiki/text-slicer/modules/rules

Here’s the code that processes the rules:

https://github.com/Jermolene/TiddlyWiki5/blob/master/plugins/tiddlywiki/text-slicer/modules/slicer.js

The key methods are onOpenTag() and onCloseTag().

Best wishes

Jeremy.

BTC

--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/7fa077b4-c4ec-46f0-94a0-afddd8cd9e22%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mark S.]

In the generated text, what is the trick that makes

<<toc-selective-expandable """MyLabeldocument""">>

macro work, despite their being no actual tag "MyLabeldocument", though there is a tag "MyLabel" ??

Thanks
-- Mark

[iain]

Really this video is a rather long winded and verbose comparison between three oranges and a lemon. The oranges seems to be citation managers and the lemon is Evernote which is in my view more likely a "competitor" to TW in that they are at heart databases of notes rather than citations.

As I actually work as an archaeologists and professional historian and use the tools on a daily basis I junked Mendeley and Zoltero as they just seemed orientated towards site scraping from other databases and far too focused on the "Cloud". The desktop version of Mendeley in particular kept crashing. So I migrated back to a paid version of EndNotes to handle my data (which is getting on to over 7000 pdf's).

Evernote is hanging in there on my phone but I am not sure why. It was ok once but the free version now had numerous limitations. If I decide to take the MS poisoned chalice and upgraded to their on-line products I suspect Evernote will be replaced by One-Note.

I have always liked the simplicity of TW and its hyperlinking but feel that there are far too many developers and half-finished, non-maintained products out there which is daunting for someone who uses this software as tools to undertake my work not as an end in themselves.

Iain