Importing Entries (csv/json) from the Zotero Bibliography Tool.

328 views
Skip to first unread message

Jan

unread,
May 13, 2016, 8:17:18 PM5/13/16
to TiddlyWiki
Hallo,
has anyone discovered a workflow of exporting entries from a Zotero-Library and reimporting them as Tiddlers.
Although Zotero is able to create a json file, this is not packing the interesting Data like autor and year, but useless information.
I had big trouble importing the CSVs, but finally the render as Tables.
Unfortunately they crash the MAS/csv2json.js-plugin...

-Is there a way of generating tiddlers from a table within a tiddler?
-Should i use a different format than json and csv to start with?
-Has anyone experience with the Zotero  xul-editor to produce a better .json-export?
Jan

Mark S.

unread,
May 14, 2016, 11:37:27 AM5/14/16
to TiddlyWiki
So how did the csv2json crash? I imagine if you try to convert too much at once there will be problems.

A quick look online suggests that the zotero RDF format may have more fields than the json export. RDF is (usually) just a text format (XML) so you could browse it and see if it contains everything you want.

If you have your data in the form of a table, what some people have done is to load it into a spreadsheet. Then add columns and rows to "decorate" the data so that it looks like a JSON file. Then copy the whole thing as text (usually need "copy as" functionality) and then paste into a text file to create your json file to import.

HTH
Mark

prog...@assays.tv

unread,
May 14, 2016, 1:09:37 PM5/14/16
to TiddlyWiki
Ciao Mark & Jan

This is an interesting issue. Personally I found that creating simple JSON's using GREP tools, rather than relying on a programme's own routines, is more successful for importable files. i dont know that particular application. What i do know is that we should not expect Tiddlywiki to natively cope with too much complexity. The simpler and more "flat-file" the import the better in my experience.

Best wishes
Josiah

Jan

unread,
May 14, 2016, 6:26:07 PM5/14/16
to tiddl...@googlegroups.com
Dear Mark,
It crashed when I used 6 arrays with 5 entries...and it still crashed when I pasted a single line.
The errorcode is: TypeError: araLine[j] is undefined
Dear Josiah,
I tried out various programms to convert csv to json. Both formats are to be quite vulnerable if you have to convert them manually...If you got a nice 'n stable solution I' d be thankfull.
I'd be glad if i coud avoid the detour vía json.
I wrote an xml to make Zotero export csv-compatible strings. I attached it though it is not yet perfect for all subjects (The Template for websites and articles in journals are not ready yet... )
Just pull it on firefox or Zotero to install  (and do not mind the error message)...btw: Zotero can be found on https://www.zotero.org/ and is very practical for scientific work.

The quotations look like this:

"motion_picture","Peter Greenaway", "A Zed & Two Noughts", "115 min", "25.05.1990"

"motion_picture","Steven Spielberg", "E.T. the Extra-Terrestrial", "115 min", "11.06.1982"

"motion_picture","Terry Gilliam", "Fear and Loathing in Las Vegas", "118 min", "22.05.1998"

"book","John Howard Lawson", "Film: the creative process: the search for an audio-visual language and structure", "New York", "1964"

"book","Felicity Colman", "Deleuze and cinema: the film concepts", "Oxford", "2011"


I would like to create templates like
"director", "title", "length", "date"   for the type motion_picture
to parse the strings and create the tiddlers and fields directly out of them.

I guess this way is safer and easier than vía json...

Jan
--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.
To view this discussion on the web visit https://groups.google.com/d/msgid/tiddlywiki/f6b9987d-40d0-49c5-9830-78bcb35fdfb3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

export_with_type_csv.csl

Mark S.

unread,
May 14, 2016, 7:22:42 PM5/14/16
to TiddlyWiki
Can you share the lines that crashed it?  I would guess that you either have an empty field or a blank line somewhere.

Mark

prog...@assays.tv

unread,
May 14, 2016, 8:51:42 PM5/14/16
to TiddlyWiki
Jan

I feel for you. One of the limitations with TW is when things fail is its quite difficult to pin-down why. The interface gives virtually no feedback.

JSON is a much better format IMO because its not ambiguous in the way CSV can be. But I don't think I can help you on specifics because I  use a GREP engine (https://www.powergrep.com/) that requires understanding how to use complex Regular Expressions to prep text into JSON form.

Maybe I can help a bit if you can show me a fragment of what a native JSON the program makes looks like?

It may be that Mark S. is in the right direction on CSV. Its difficult to be sure. Anyway, I will happily look at any JSON file if you want.

Best wishes
Josiah

Jan

unread,
May 14, 2016, 9:32:22 PM5/14/16
to tiddl...@googlegroups.com
Hi Mark,
the last string I tested was:

title,tags,text,birth,death
"John Smith","English male,Therapist","Explorer, Artist","1900","1981"

because I wanted to see it work...
This means I copypasted it right from the mail to the field...

Yours Jan
--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+...@googlegroups.com.
To post to this group, send email to tiddl...@googlegroups.com.
Visit this group at https://groups.google.com/group/tiddlywiki.

Mark S.

unread,
May 14, 2016, 9:38:11 PM5/14/16
to TiddlyWiki
Although you copied it from the list, when you pasted it in you already had a line in place. Or you selected an extra line feed. So if you check, you'll see that there is one "invisible" line at the bottom of your input tiddler that is choking the program. When/If I do an update, I'll have to add checking code to toss out bad lines.

HTH
Mark

Mal

unread,
May 15, 2016, 3:30:24 AM5/15/16
to TiddlyWiki
Just as another option, I tend to use spreadsheets to create json files suitable for importing. I would load the CSV data into the spreadsheet and then use cell formulas to create lines of json text that can be cut and pasted into a text editor and saved as a json file.

Regards,

Mal

Jan

unread,
May 15, 2016, 7:11:46 PM5/15/16
to tiddl...@googlegroups.com
Hallo Mark,
Your Example works fine now, the code does not cause crashes anymore, but it gives me your error-message for this:

type,director,title,length,launch,tags
"motion_picture","Hans Richter","Ghosts Before Breakfast","6 min","14.07.1928","Neuro,Gestalttheorie,Movie"
"motion_picture","Monty Pythons Flying Circus","30 min","05.10.1969","Neuro,Gestalttheorie,Movie"
"motion_picture","John Mc Tiernan","Predator","107 min","12.06.1987","Neuro,Gestalttheorie,Movie"
"motion_picture","Paul Verhoeven","Robo Cop","102 min","17.07.1987","Neuro,Gestalttheorie,Movie"
"motion_picture","Peter Greenaway","The Draughtsmans Contract","108 min","30.06.1983","Neuro,Gestalttheorie,Movie"
"motion_picture","James Cameron","The Terminator","107 min","26.10.1984","Neuro,Gestalttheorie,Movie"

I have no clue why...

Thanks for your endurance,
Jan

Mark S.

unread,
May 15, 2016, 7:33:09 PM5/15/16
to TiddlyWiki
The entry with "Monty Pythons Flying Circus" is missing a title (or a director, depending on if you meant for MPFC to be director or title).

All fields need to be present, though they can be empty. An empty field would represented by empty double quotes. So that particular entry might be represented as:

"motion_picture","Monty Pythons Flying Circus","","30 min","05.10.1969","Neuro,Gestalttheorie,Movie"

(Assuming MPFC was the 'directory').

HTH
Mark

Jan

unread,
May 17, 2016, 4:56:18 PM5/17/16
to tiddl...@googlegroups.com
Hi Mark, dear scientific users of TW.
In the search of a way to create Tiddlers from a Zotero/Mendeley/...-Bibliography I found a solution.
In the attachment you find a csl-template, wich supports the export into something, that can manually be transformed into a Json-file which will create Tiddlers for each chosen Film or Videostream.
That's my UseCase for now. I will do other formats later or quicker on demand If someone asks.
The description is in the Readme.

Cheers Jan

PS:
Dear Mark
your were right in your analysis. Your Plugin now works fine. Despite I decided that CSVs as export format are to vulnerable for my usecase.
J
json-export.csl
German version-Gebrauchsanweisung und Vorlage.txt
Readme and Template.txt

Mark S.

unread,
May 17, 2016, 6:30:22 PM5/17/16
to TiddlyWiki
Congratulations Jan. The good thing about the name Zotero, is that anyone looking for your solution will have an excellent chance of finding it.

Steven Schneider

unread,
Oct 18, 2016, 1:07:23 PM10/18/16
to TiddlyWiki
Hello, sorry to be late to the party. 

For high-quaity csv-to-json conversion see http://www.convertcsv.com/csv-to-json.htm

I to am very interested in this, for the same reason -- bibliographic entries -- and am trying to get JabRef to generate a JSON file for importing (ideally, without an intermediate csv step. If you are willing to take that step, see my workflow:


or more generally, 


Let me know if you are still working on this; perhaps we can put our minds together...:)

//steve.
Reply all
Reply to author
Forward
0 new messages