Zeroconf or other way to access Zotero library in local network

[Rönkkö Mikko]

Zotero used to have zeroconf support and an item exists for this in the Firefox preferences, but it does not seem to do anything. Is this feature currently functional?

Also if not, is there currently any other way to access libraries that are stored on a local computer? The reason why I am asking this is that I would like to add a feature of downloading Zotero library and the attached files from a local computer instead of Zotero server in ZotPad. This would be faster, save bandwidth, and enable viewing attached files even if a user did not have the files stored on the Zotero server or a webdav server.

Mikko

[Avram Lyon]

On Sat, Nov 26, 2011 at 2:40 AM, Rönkkö Mikko <mikko....@aalto.fi> wrote:
> Zotero used to have zeroconf support and an item exists for this in the Firefox preferences, but it does not seem to do anything. Is this feature currently functional?

I think it's broken-- but it's been over a year since I last looked into it.

> Also if not, is there currently any other way to access libraries that are stored on a local computer? The reason why I am asking this is that I would like to add a feature of downloading Zotero library and the attached files from a local computer instead of Zotero server in ZotPad. This would be faster, save bandwidth, and enable viewing attached  files even if a user did not have the files stored on the Zotero server or a webdav server.

You mean short of copying the Sqlite database?

The clean way to do this is probably to extend the connector server:
http://www.zotero.org/support/dev/client_coding/connector_http_server

That'd require a new extension on the Zotero/Firefox/Standalone side
of things, but you could have it generate output much like the current
read API does on the server. If you go this route, I'd be glad to work
with you to build the extension, since I could use it for Zandy as
well.

In fact, I've been thinking about writing a support extension for some
time, mainly to support handling file uploads; my thought is to upload
files (images, PDFs) to an intermediary server, then have the
extension on the desktop attach them to the right items. It'd be good
if things like this were compatible so that iOS and Android users
don't need to install different support extensions on the desktop.

Avram

[Kieren Diment]

On 27/11/2011, at 9:53 AM, Avram Lyon wrote:

> On Sat, Nov 26, 2011 at 2:40 AM, Rönkkö Mikko <mikko....@aalto.fi> wrote:
>> Zotero used to have zeroconf support and an item exists for this in the Firefox preferences, but it does not seem to do anything. Is this feature currently functional?
>
> I think it's broken-- but it's been over a year since I last looked into it.
>
>> Also if not, is there currently any other way to access libraries that are stored on a local computer? The reason why I am asking this is that I would like to add a feature of downloading Zotero library and the attached files from a local computer instead of Zotero server in ZotPad. This would be faster, save bandwidth, and enable viewing attached files even if a user did not have the files stored on the Zotero server or a webdav server.
>
> You mean short of copying the Sqlite database?
>
> The clean way to do this is probably to extend the connector server:
> http://www.zotero.org/support/dev/client_coding/connector_http_server

I concur. The clean way to implement this is to write endpoints that serialise the data as json for the collection and item viewers so that you can have client side templating on the other end that renders the json into a view.

I have some fairly old code lying around that does this with server side javascript and a firefox embedded web server (my use case was for creating annotated bibliographies from notes), but it needs some architectural work to get it functioning within the connector server, a better dispatch process than just an old style cgi-bin layout, and to make it a bit more general purpose. Also I think that client side javascript templating libraries have matured a bit since I wrote this stuff.

https://github.com/singingfish/zotero-browser

[Rönkkö Mikko]

On Nov 27, 2011, at 00:53, Avram Lyon wrote:

> On Sat, Nov 26, 2011 at 2:40 AM, Rönkkö Mikko <mikko....@aalto.fi> wrote:
>
>> Also if not, is there currently any other way to access libraries that are stored on a local computer? The reason why I am asking this is that I would like to add a feature of downloading Zotero library and the attached files from a local computer instead of Zotero server in ZotPad. This would be faster, save bandwidth, and enable viewing attached files even if a user did not have the files stored on the Zotero server or a webdav server.
>
> You mean short of copying the Sqlite database?

Yes.

>
> The clean way to do this is probably to extend the connector server:
> http://www.zotero.org/support/dev/client_coding/connector_http_server
>
> That'd require a new extension on the Zotero/Firefox/Standalone side
> of things, but you could have it generate output much like the current
> read API does on the server. If you go this route, I'd be glad to work
> with you to build the extension, since I could use it for Zandy as
> well.
>
> In fact, I've been thinking about writing a support extension for some
> time, mainly to support handling file uploads; my thought is to upload
> files (images, PDFs) to an intermediary server, then have the
> extension on the desktop attach them to the right items. It'd be good
> if things like this were compatible so that iOS and Android users
> don't need to install different support extensions on the desktop.

I agree that the same extension should serve both Android and iOS users and can help with the development.

What I would like such extension to do is primarily to act as a file server for downloading files from Zotero and uploading modified versions back. A webdav server implemented inside Firefox would be one option, but I do not know if this was technically feasible or an optimal solution. Reading and writing item metadata would be useful as well, and this probably would not be too difficult to implement. A direct port of the server API to Zotero client would be one option.

In the ideal case a user would not need to configure anything to use this support extension with ZotPad, but ZotPad would find support extension enabled Zoteros in the local network and try to authenticate with them. If it could authenticate with a support extension, then it would use this exclusively for data and upload all changes to the local support extension. These would be then synced to the Zotero server by the Zotero client.

Mikko

>
> Avram
>
> --
> You received this message because you are subscribed to the Google Groups "zotero-dev" group.
> To post to this group, send email to zoter...@googlegroups.com.
> To unsubscribe from this group, send email to zotero-dev+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/zotero-dev?hl=en.
>

[Rönkkö Mikko]

I gave this issue some thought over Sunday and came to the conclusion that it would be best if the communication between Zandy, ZotPad, or some other mobile client and a local Zotero client would use the same protocol as the server APIs. If this was the case, it would probably make more sense for the Zotero server and the Zotero client to share at least some of the API code.

Where is the Zotero server API code located at and do the Zotero developers have some thoughts on this issue?

Also this would mean that we probably should wait until Zotero server supports editing attachment files.

Mikko

[Dan Stillman]

On 11/28/11 3:35 AM, R�nkk� Mikko wrote:
> I gave this issue some thought over Sunday and came to the conclusion that it would be best if the communication between Zandy, ZotPad, or some other mobile client and a local Zotero client would use the same protocol as the server APIs. If this was the case, it would probably make more sense for the Zotero server and the Zotero client to share at least some of the API code.
>
> Where is the Zotero server API code located at and do the Zotero developers have some thoughts on this issue?

I think trying to use the same API makes sense. It wouldn't be trivial,
but the client and server have very similar data layer code, so you'd
mostly have to port Atom-specific code and controller code. A lot of
stuff related to caching and permissions could be ignored. (The client
would generate an API key, and that key would just have full access.)

Here's the most relevant code:

model/Atom.inc.php
model/Items.inc.php (Zotero_Items::convertItemToAtom() and
Zotero_Items::search() in particular)
controllers/ApiController.php
model/API.inc.php

I'd start with Zotero_Atom::createAtomFeed() and work backwards, not
worrying about the controller code until you're successfully generating
Atom feeds. (And those convert[Object]ToAtom() functions should probably
be toAtom() methods in their respective objects instead of static
methods.) For unsupported things (e.g., 'html' mode, probably), return a
501 Unimplemented.

You could do this initially as a separate extension, but I'd say it's
likely we'd integrate this if it's successful, so you might want to just
work off of a fork of the main code.

[Rönkkö Mikko]

On Nov 28, 2011, at 11:16, Dan Stillman wrote:

Would it be possible to import this code from the server repository the same way as CSL code is imported form the CSL repository so that in the long term there would not be a need to maintain two branches (server and client) of the same code?

Mikko

[Avram Lyon]

On Mon, Nov 28, 2011 at 1:54 AM, Rönkkö Mikko <mikko....@aalto.fi> wrote:
> Would it be possible to import this code from the server repository the same way as CSL code is imported form the CSL repository so that in the long term there would not be a need to maintain two branches (server and client) of the same code?

Probably not -- the server is in PHP, and the client is in JavaScript.
But the API is pretty well-defined, so this isn't a moving target like
the sync protocol.

Avram

[Rönkkö Mikko]

A couple of question relating to this:

Are there any specifications for modifying attachments through the write API?

Is there a way to get a list of valid API keys for the user in the Zotero client? This would be useful in authentication between a mobile client and a local computer. Sending an API key in local network directly is not secure because a mobile client would have no way of knowing if the local computer is authenticated with the users credentials. If if was possible to get a list of valid API keys in the client, these keys could be used as shared secret in challenge-response authentication without ever needing to send the key over the network.

Mikko

On Nov 28, 2011, at 11:16, Dan Stillman wrote:

[Avram Lyon]

On Tue, Nov 29, 2011 at 7:33 AM, Rönkkö Mikko <mikko....@aalto.fi> wrote:
> Are there any specifications for modifying attachments through the write API?

We can probably assume that attachments will be modified just like
items and notes-- by POSTing a new JSON block with an etag in the
headers.

File uploads are another matter; I'm eager to hear what the team is
thinking about, although that smaller detail could be tweaked to match
the server when the latter gets upload capabilities.

Avram

[Dan Stillman]

On 11/29/11 10:33 AM, R�nkk� Mikko wrote:
> Is there a way to get a list of valid API keys for the user in the Zotero client? This would be useful in authentication between a mobile client and a local computer. Sending an API key in local network directly is not secure because a mobile client would have no way of knowing if the local computer is authenticated with the users credentials. If if was possible to get a list of valid API keys in the client, these keys could be used as shared secret in challenge-response authentication without ever needing to send the key over the network.

No, the client doesn't have access to API keys, and I don't think this
shouldn't be dependent on a server API key anyway�mobile clients should
be able to talk to the Zotero client without a server account.*

There can be a pref pane in the Zotero client that shows incoming
registration attempts and provides a key to type into the mobile client.
On success, the clients store a key (longer than the typed string) for
that device, and either client can deregister the link.

I wouldn't worry about this for now, though. This can be added at the end.

* The Zotero client will generate local URIs if it hasn't been synced.
For this and for cases of switched sync accounts, mobile clients should
probably check to see if the incoming URIs (from any "server") match
what they already have, and probably prompt to clear all local data if
they're different.

[Dan Stillman]

On 11/29/11 2:19 PM, Dan Stillman wrote:
> No, the client doesn't have access to API keys, and I don't think this
> shouldn't be dependent on a server API key anyway�mobile clients
> should be able to talk to the Zotero client without a server account.*
>

> [...]

>
> * The Zotero client will generate local URIs if it hasn't been synced.
> For this and for cases of switched sync accounts, mobile clients
> should probably check to see if the incoming URIs (from any "server")
> match what they already have, and probably prompt to clear all local
> data if they're different.

Actually, this part doesn't make sense, since the mobile client will
need to make API requests for something to begin with. So Zotero would
have to do something like pass a list of valid URI prefixes (which might
be local or global) for the available libraries. But let's worry about
this later.

[Rönkkö Mikko]

I gave this some more thought. The most likely use case for this would be for downloading files from a local computer to a mobile device. The number of people who do not want to use the Zotero server at all is probably much smaller than the number who do not want to use it to store files. It might be thus sufficient to implement just transfer of files over local network.

A request could be encoded by using the username as a shared and including the key of the attachment file and the attachment filename as parameters that are used to identify what file Zotero will return to the mobile client.

[Rönkkö Mikko]

Now that the first version of ZotPad is out, I had some time to do a bit of prototyping. It seems that implementing a file server in javascript is difficult, so the way to go would be to have the mobile client implement a file server (e.g. WebDAV) that a Zotero extension then connects to. Moreover, running a server might not be allowed on centrally managed PCs, but is most likely not blocked on the mobile client.

Having a two-way file syncing between Zotero and mobile clients regardless of what file storage solutions (Zotero file storage, webdav, dropbox) a user has seems like a high-priority thing for many uses. Any comments on this idea?

Mikko

[Rönkkö Mikko]

I wrote a post about different ways to synchronize files between Zotero and ZotPad

http://www.zotpad.com/node/14

If anyone on this mailing list has any opinions on this matter, I would be happy to hear about these.

Mikko

[Dan Stillman]

If you're going to do this, I don't see any reason to implement it on
the mobile client side (for each client). You have the entire Mozilla
framework at your disposal, not just plain JavaScript. I'm also not
clear what "platform dependent code" you're referring to in that blog post.

[Rönkkö Mikko]

I did some research and could not find a file server implemented in javascript. Also I did not find any file server related pages on MDN. This lead to the conclusion that the way to do this using Mozilla frameworks would be to start with an HTTP server (e.g. http://mxr.mozilla.org/mozilla/source/netwerk/test/httpserver/httpd.js) and write my own WebDAV extension for that. At this point this is too much.

So my second thought was to find a light-weight file server and embed this as executable that is launched when Firefox lauches. This is what I mean with platform dependent code.

Are there some other options that I could use for implementing a file server inside Firefox that I might not be aware of? The server would need to support both uploading and downloading.

Mikko