[TW5] Json Mangler Plugin v2.0.8 - The Over 9000 Csv Update!

[Joshua Fontany]

Hi Everybody,

I'm very excited about the new release of Json Mangler. I'm calling this the Over 9000 Csv Update!

(I hope enough of the audience gets that...)

This update turns Json Mangler into a swiss-army-knife-data-conversion tool. Nearly all databases can export to csv tables, and now all your base are belong to tiddlywiki.

;D

• New $csv widget parses and renders any tiddler with type = 'application/csv'!
• Simply drag/drop your *.csv text-file, complete the import, (type is auto-set to 'application/csv')!!
• OR use a 'tiddlywiki.files' file to import whole folders of Csv data at once!!!
• CSV does not mean COMMA seperated values, it means CHARACTER separated values (set your own - no UI on first release)!!!!
• Fully configurable access to the included PapaParse library via a "state" tiddler for each $csv widget onscreen!!!!!
• Peek/preview to limit the rows loaded, skip-to-page to jump around LARGE datasets, a whole bunch of cool options!!!!!!
• Sophisticated CSV Import - like, whoa!!!!!!!
• Import the whole thing as one JSON Array tiddler, or import each row of the Csv as a Json or regular Tiddler!!!!!!!!
• Dynamic creation of "simplified" field-names when importing individual tiddlers based on column index# or a "slugged" version of the original column name!!!!!!!!

I have included a test Csv in the new example wiki (below), which is a database of video game sale from a few years ago. It has 16,600 rows (including an empty row, and a headers row), so you can play around with all of the peek, display, and import options.

Thanks very much for any feedback!

Github repo: https://github.com/joshuafontany/TW5-JsonMangler

Github release: https://github.com/joshuafontany/TW5-JsonMangler/releases/tag/2.0.8

Example wiki: https://joshuafontany.github.io/TW5-JsonMangler/

This plugin is a work in progress; any suggestions or bug reports are very welcome.

Please report any issues on GitHub: https://github.com/joshuafontany/TW5-JsonMangler/issues

If you find this useful and would care to donate, please use my PayPal: https://paypal.me/JoshuaFontany

Best,
Joshua Fontany

[Mohammad]

Love this!

Thank you Josuha, this means we can also import husge excel sheet as cvs into Tiddlywiki!

I just checked the example and looks great still the example UI can be polished I think!

--Mohammad

[Joshua Fontany]

Hi Everybody,

There have been a couple of bugs found, and are working on getting fixed.

• Unchecking "Peek" does not actually allow importing of all rows (FIXED)
• The browser locks up upon importing large csv(s) (15,000+ rows) as individual tiddlers.
• Peek/Preview max row number count can be thrown off by new-lines in the data.
  (Looks like we need to actually papa-parse the full file to get that count. Can we do that on import and cache it?)
  

I think it is worth examining how I am importing the individual rows. It almost doesn't make sense to use the "usual" $:/Import process, especially for large-numbers-of-individual-tiddlers-at-once. One possibility I thought of was bundling them all as a plugin, thus making each row a Shadow Tiddler that could be overwritten/replaced, etc. Open to suggestions.

Best,
Joshua Fontany