Access Zotero from Emacs
David Maus
After asking a question on the usage of Zoteros internal functions to
insert citations and bibliographies that can be used by an outside
program[1] I was directed here.
As a Emacs user something that bugged me the whole time was the
inability to access or reference my literature in Zotero
directly. BibTeX as a intermediate format works but with a loss of
information (Zotero knows more item types than bibtex does) and no
easy way to link a bibtex entry back to it's Zotero item.[2] In
addition I'd be able to insert citations and references into arbitrary
textfiles without the hassle to support different styles.
After reading the documentation, playing with the SOAP interface and
realizing it seems to be gone I decided to use MozRepl to get access
to the Zotero database because I don't want to learn JavaScript that
much (i.e.: writing a plugin for mozilla) and especially don't want to
deal with different operating systems types of the integration pipe.
The current state of this project is experimenting with Zotero's
internals and putting together clumsy elisp functions that perform
simple tasks[3].
As the question I asked in the forum are answered after digging
through the source I'd like to ask if anyone who knows more on
javascript has an idea how to return the results of operations back to
the outside program aka Emacs. As far as I understood there are no
sockets in JavaScript and writing to a named pipe is not platform
independent. The only solution seems to be using temporary files -- am
I right on this?
Regards
-- David
[1] http://forums.zotero.org/discussion/10200/need-some-hints-on-accessing-zotero-2x-via-mozrepl/#Item_2
[2] So if I reference an item, I don't reference the Zotero item
but the BibTeX item. This is good enough for now but, well, it is
not what I want.
[3] http://gist.github.com/252402 -- currenty there's only one
function that performs a search and exports the result //to a
temporary file.
Frank Bennett
There is a new CSL processor in development, intended for deployment
in Zotero post-2.0, which can be run independently of the browser, and
has the potential, at least, to provide a BibTeX replacement. The
idea would be to embed the processor in a utility that scans and
analyzes the *.aux file produced by LaTeX, fetches reference items
directly from Zotero (or whereever), and writes cites in LaTeX markup
to a *.bbl file. Setting such a thing up may be more work than you
have in mind, but (speaking as the author of the processor) it seems
like it might be an interesting possibility. The tool would be able
to use the full Zotero style repository natively, and it would provide
a number of facilities (such as name disambiguation and complex
backreferencing) that may be more limited in current LaTeX.
For further information, the processor manual is here:
http://gsl-nagoya-u.net/http/pub/citeproc-doc.html
Several projects are working on adapting the processor for use as a
web service and whatnot. There is a discussion group for integrators
here:
http://groups.google.com/group/citeproc-js
The output format for the processor is all defined in one source file, here:
http://bitbucket.org/fbennett/citeproc-js/src/tip/src/formats.js
The one slightly sticky issue is that the processor relies on
externally supplied positioning hints (for ibid, etc.), and that logic
would need to be implemented in the utility. Zotero's current
positioning machinery is in the file integration.js, which is here:
https://www.zotero.org/svn/extension/trunk/chrome/content/zotero/xpcom/integration.js
(search for getCitationPositions to find the main function for this purpose)
Hope this helps!
Frank Bennett
>
> Regards
>
> -- David
>
> [1] http://forums.zotero.org/discussion/10200/need-some-hints-on-accessing-zotero-2x-via-mozrepl/#Item_2
>
> [2] So if I reference an item, I don't reference the Zotero item
> but the BibTeX item. This is good enough for now but, well, it is
> not what I want.
>
> [3] http://gist.github.com/252402 -- currenty there's only one
> function that performs a search and exports the result //to a
> temporary file.
>
> --
>
> You received this message because you are subscribed to the Google Groups "zotero-dev" group.
> To post to this group, send email to zoter...@googlegroups.com.
> To unsubscribe from this group, send email to zotero-dev+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/zotero-dev?hl=en.
>
>
>
Bruce D'Arcus
On Dec 17, 2009, at 5:49 PM, Frank Bennett <bierc...@gmail.com>
wrote:
But he didn't say he was working with LaTeX files. I guess a lot
depends on exactly what sort of use cases/workflows he's looking to
accomodate.
Erik Hetzner
Hi David -
Just a shot in the dark, but you might have a look at Fireforg, a
Firefox extension for integration with org-mode.
Docs here: http://orgmode.org/worg/org-devel.php
Repo here: http://repo.or.cz/w/org-fireforg.git
It is sort of backwards because it uses emacsclient as the program to
use in Firefox for org-protocol:// links, whereas you want to get data
out of Zotero. But Fireforg seems to make it possible to send BibTeX
entries from Zotero to emacs via org-protocol, so I assume you might
be able to do something similar.
If you can advance integration between org-mode and Zotero so that I
can use org-mode for notes & Zotero for my bibliography maintenance, I
will be happy!
-Erik
Dan Stillman
> As the question I asked in the forum are answered after digging
> through the source I'd like to ask if anyone who knows more on
> javascript has an idea how to return the results of operations back to
> the outside program aka Emacs. As far as I understood there are no
> sockets in JavaScript and writing to a named pipe is not platform
> independent. The only solution seems to be using temporary files -- am
> I right on this?
>
Bear in mind that you have more than just JavaScript proper at your
disposal�you have the entire Mozilla Application Framework, which does
have socket support (network sockets, at least) that you could use, but
anything beyond HTTP with XMLHTTPRequest would likely have a learning
curve. Initiating connections from Firefox via MozRepl would let you
avoid a separate persistent plugin in Firefox, but you'd of course need
an endpoint on the Emacs side, and it might be just as easy to restore
the previous integration HTTP server (using SOAP or otherwise) in a
Zotero plugin.
So if you don't want to do that, a temporary file is probably the
easiest way to go.
Frank Bennett
True. Shows the one-trick pony in me, I'm afraid. :)
David Maus
Actually the proposed CSL process is indeed far to specific: Im
currently more on the basic part of the integration, that is to get
something that allows making a reference to a Zotero item. My personal
wishlist looks like that:
,----
| ** Intended functionality
|
| - insert references while writing a LaTeX2e document using RefTeX
|
| - insert references while writing an Orgmode document that will be
| expanded on export
|
| - insert citations and bibliography in arbitrary files
|
| - annotate Zotero items
`----
So my very first milestone is simple: let the user select an Zotero
item and store a reference to it's ID in the Zotero database.
On the question how to communicate with Zotero I looks like using the
MozRepl way I don't need some kind of a communication (xmlrpc etc.)
because I don't request information of a external program but running
parts of my program inside firefox. And while MozRepl is the
unidirectional connection elisp -> javascript what I am looking for is
a the other way round.
I have to admit that it feels strange to have a program that operates
in two different languages and two different environments; especially
if there's no "backchannel".[1] At the moment it seems a good idea to
separate the two contexts and use two independed programs (Emacs and a
Firefox plugin) but that on the other hand would mean to learn more
JavaScript, the whole Mozilla framework and develope kind of a
communication protocol.
Maybe I should start with a "hybrid": define JavaScript functions and
objects that perform simple tasks, execute them using MozRepl and let
them talk back to Emacs using emacsclient analogous to fireforg and
org-protocol (thanks for the hint!). This would "only" require to
think of communication protocol and handling asynchronous
communication on the Emacs side.
Sorry for the rambling, I suppose I should put question of the
protocol design on top of my list.
Regards
-- David
[1] Of cause I could process the MozRepl buffer in Emacs to detect
errors or JavaScript responses.
--
OpenPGP... 0x316F4BE4670716FD
Jabber.... dmj...@jabber.org
Email..... maus....@gmail.com
ICQ....... 241051416
Bruce D'Arcus
On Dec 20, 2009, at 5:35 AM, David Maus <maus....@gmail.com> wrote:
> Thanks a lot for the comments on my project.
>
> Actually the proposed CSL process is indeed far to specific: Im
> currently more on the basic part of the integration, that is to get
> something that allows making a reference to a Zotero item. My personal
> wishlist looks like that:
>
> ,----
> | ** Intended functionality
> |
> | - insert references while writing a LaTeX2e document using RefTeX
> |
> | - insert references while writing an Orgmode document that will
> be
> | expanded on export
> |
> | - insert citations and bibliography in arbitrary files
> |
> | - annotate Zotero items
> `----
>
> So my very first milestone is simple: let the user select an Zotero
> item and store a reference to it's ID in the Zotero database.
Not per we aimed at you, but I feel the need to keep harping on this,
since it keeps coming up.
Using db ids is the same approach that developers continue to take,
but it's a pretty broken one, because it fundamentally ties documents
to a particular zotero library.
This isn't an easy problem to resolve, but it's an important one.
David Ross
few skimmed others. No time for more, but wanted to offer a more direct
integration approach.
> -----Original Message-----
> From: zoter...@googlegroups.com [mailto:zoter...@googlegroups.com]On
> Behalf Of David Maus
> Sent: Sun, Dec 20, 2009 05:36 AM
> To: zoter...@googlegroups.com
> Subject: Re: Access Zotero from Emacs
>
>[snip]
> On the question how to communicate with Zotero I looks like using the
> MozRepl way I don't need some kind of a communication (xmlrpc etc.)
> because I don't request information of a external program but running
> parts of my program inside firefox. And while MozRepl is the
> unidirectional connection elisp -> javascript what I am looking for is
> a the other way round.
Since Zotero stores data in a database (SQLite/PostgreSQL) it can be
accessed directly in Elisp. Pg.el provides socket level integration with
PostgreSQL. There is one or more ODBC drivers for PostgreSQL.
http://www.emacswiki.org/emacs/PostGreSQL
An Alternative would be ejacs/javascript/JDBC/ODBC.
http://www.emacswiki.org/emacs/Ejacs
Dynamic Database Access from Client-Side JavaScript:
http://onjava.com/onjava/2002/01/23/javascript.html
Freex mode is a useful integration model based on yet another language
Python/Pymacs. Elisp/Perl is also available.
http://search.cpan.org/~jtobey/Emacs-EPL-0.7/
http://search.cpan.org/~seano/Sepia-0.991/lib/Sepia.pm
> I have to admit that it feels strange to have a program that operates
> in two different languages and two different environments; especially
> if there's no "backchannel".[1] At the moment it seems a good idea to
> separate the two contexts and use two independed programs (Emacs and a
> Firefox plugin) but that on the other hand would mean to learn more
> JavaScript, the whole Mozilla framework and develope kind of a
> communication protocol.
>
> Maybe I should start with a "hybrid": define JavaScript functions and
> objects that perform simple tasks, execute them using MozRepl and let
> them talk back to Emacs using emacsclient analogous to fireforg and
> org-protocol (thanks for the hint!). This would "only" require to
> think of communication protocol and handling asynchronous
> communication on the Emacs side.
>
> You received this message because you are subscribed to the
> Google Groups "zotero-dev" group.
> To post to this group, send email to zoter...@googlegroups.com.
> To unsubscribe from this group, send email to
> zotero-dev+...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/zotero-dev?hl=en.
>
If data access is the primary concern a direct approach has advantages.
RefDB uses databases. Your application could unified view/data model free
from storage/access details.
Hope something above is helpful.
david