2.0 plans
Bruce D'Arcus
please document somewhere what the plans for 2.0? I'm mostly asking for
high-level use cases and requirements.
I see Zotero torn in two somewhat contradictory directions: towards the
web, and down to the desktop. I'm a big proponent of the first, of
course. But I really don't know how the team will plan to address these
tensions, and what the goals really are.
I'm asking this in part b/c I'm seriously considering writing my own
custom CMS to mange note-taking, writing, etc. I'd really like to be
able to integrate Zotero 2.0 services within it, but I don't honestly
know if that's even in the plans. So much of the design of Zortero 1.0
essentially is based on the notion of a monolithic application (witness
how citation processing works), and I'm not sure if that thinking will
also impact/limit 2.0.
So some details, please?
Bruce
Fitzgerald Steele
how I will be able to use Zotero to support my own citiation and
writing in the future. As a part-time-supporter-and-developer, I am
anxious to learn new how I will have to update my existing plugins,
and what functionality will be available to extend its capabilities.
High level use-cases and requirements would be an excellent first step
to get an understanding of the type of problems and solutions the
Zotero team is prioritizing and seeking to address.
David Botschinsky
Zotero's intentions
Bruce D'Arcus
> I agree; both contributors and users need concrete information about
> Zotero's intentions
FWIW, I think Dan C. will get to this, but has been traveling.
Bruce
Daniel Cohen
Thanks to everyone who wrote in about 2.0. As Bruce notes, I've been
traveling (and alas, I'm off again tomorrow through Thursday for yet
another conference). It might help me respond better if in the
meantime everyone can ask some more specific questions that I might be
able to answer--i.e., about whether particular functionality will be
included, will that functionality exist on the client or server, etc.
Or provide hypotheticals or use cases.
Thanks, and very sorry for the delayed response--
Dan
Bruce D'Arcus
> It might help me respond better if in the
> meantime everyone can ask some more specific questions that I might be
> able to answer--i.e., about whether particular functionality will be
> included, will that functionality exist on the client or server, etc.
> Or provide hypotheticals or use cases.
Well, I was asking you to document *your* "hypotheticals or use cases."
I know about some of them, but I think it's fair to say this project
remains thin on details. I don't really know how you can develop the
sort of (developer, in particular) community I'm sure you'd like to
develop if you aren't more open about plans so that people can see how
they fit within them (or not).
My use cases?
1) Just as I can easily add a link roll from delicious or magnolia to a
weblog or site, I'd like to do the same with items in Zotero (as I think
about it, a citation is just a special kind of "link" or "bookmark"
after all).
2) I'm not sure about this yet, but I might like to be able to integrate
(public) notes as well into something like a tumbelog
3) To be able to integrate full citation processing into any web
application.
4) every user has a profile page that can also incorporate some of the
above, as well as their publications; sort of like a dynamic CV, and
accessible as FOAF data
All of the above are really about the data and API story, of course. #1
could be easily implemented even with feeds (I'd vote Atom and//or RSS
1.0), as perhaps could #2. #3 is a little more tricky.
But I'm hoping these kind of web-oriented use cases are guiding the design.
Zotero 1.0 was about pulling the web into the desktop. I'd like for
Zotero 2.0 to be about pulling the desktop back out onto the web.
Moreover, ANY feature or improvement that gets added should be
consistent with this goal. So, for example, 1980s-era rich text is out
unless it can be made to work with 21st century web standards and best
practices (e.g. @rel and @class at minimum).
On the details, I'm also unclear on the place of technology like:
- identity and authentication (will Zotero 2.0 support openid?)
- search (SPARQL? OpenSearch? both?)
- Dan S. mentioned exposing RDF; handled via content-negotiation?
That's all for now.
Bruce
PS - I was looking again at Open Library. It's pretty damned cool that I
can add/edit data about me and my book there! I've mentioned this
before, but I think Zotero 2.0 ought to learn from OL. They've also got
a full open source technology stack underneath of course.
Matthias Steffens
> Daniel Cohen wrote:
>
> > It might help me respond better if in the meantime everyone can
> > ask some more specific questions that I might be able to
> > answer--i.e., about whether particular functionality will be
> > included, will that functionality exist on the client or server,
> > etc. Or provide hypotheticals or use cases.
>
> Well, I was asking you to document *your* "hypotheticals or use
> cases." I know about some of them, but I think it's fair to say
> this project remains thin on details. I don't really know how you
> can develop the sort of (developer, in particular) community I'm
> sure you'd like to develop if you aren't more open about plans so
> that people can see how they fit within them (or not).
I must admit that I second this feeling. As a third-party developer,
I'm *very* interested in working with (and to the advantage of)
Zotero, especially w.r.t. the upcoming server plans. But ATM I don't
feel that there's much interest from the Zotero people to grow a
thriving developer community. Of course, I'd be more than happy to
be proven wrong, though. And sorry if we're just too impatient.
Bruce already gave some good use cases. In addition I'd like to add
that I'd love to see the Zotero server offer an open, documented and
standards-based sync API which could be used by third-party tools to
send/fetch data from/to Zotero. This would e.g. allow users to push
record additions/updates to Zotero.org from another supporting
desktop client, or from within their own institutional repository.
And they can then pull (sync) these data from the server back to
their Zotero desktop app.
W.r.t. standards-based sync APIs, these come to mind:
SRU Record Update:
<http://www.loc.gov/standards/sru/record-update/>
SWORD (Simple Web-service Offering Repository Deposit),
a profile of the Atom Publishing Protocol:
<http://www.ukoln.ac.uk/repositories/digirep/index/SWORD>
I appreciate the discussion,
Matthias
Gerald
is my question:
As we are using Zotero in an international research project, we need
to share collections online, we need to enable every team member to
add/change bibliographic data and add tags and notes, and we would
like to hold the documents online, too.
Will be a feature like that in 2.0? When about is the release of
version 2.0 planned?
Thank you
Gerald
Daniel Cohen
On Mar 11, 2008, at 7:22 AM, Bruce D'Arcus wrote:
>
> Daniel Cohen wrote:
>
>> It might help me respond better if in the
>> meantime everyone can ask some more specific questions that I might
>> be
>> able to answer--i.e., about whether particular functionality will be
>> included, will that functionality exist on the client or server, etc.
>> Or provide hypotheticals or use cases.
>
> Well, I was asking you to document *your* "hypotheticals or use
> cases."
> I know about some of them, but I think it's fair to say this project
> remains thin on details. I don't really know how you can develop the
> sort of (developer, in particular) community I'm sure you'd like to
> develop if you aren't more open about plans so that people can see how
> they fit within them (or not).
A brief overview that might be helpful. When you look at our plans for
the server (feeds, groups, APIs, etc) you realize that almost all of
the heavy lifting is in getting all of the clients synched to the
server. Once that's done, we can, e.g., export in whatever formats
(RSS, Atom, SWORD, whatever) the community thinks would be most
helpful. I suspect given that it was helpful for the client to provide
multiple in/out methods, the server will be similar, but we're open to
suggestions/discussion over the right forms for this.
The sync is no small feat. Dan and the dev team have been working on
it for months, doing a number of different tests that, while it might
seem like smoke filled back-room dealing, really is just to make sure
we don't release something that's not ready for prime time (think of
the scale of all of the transactions the server will deal with) and
has lots of people pounding away at it while Dan et al are working on
it. A couple of quick notes: 1) re: standards like Atom for the sync:
besides being problematically much more verbose than we would like
(we're instead using a tight XML transmission), the sync is more about
a diff and really needs the client and server on each side (note the
potential for problems, e.g., if a deletion occurs on either side); 2)
remember that a good bit of the server will, in effect, reside in the
client (i.e., a communications/sync layer), and so that can be
hijacked/reused by other developers. This will open up the possibility
for better plugins, two-way syncs to other services, etc.
>
>
> My use cases?
>
> 1) Just as I can easily add a link roll from delicious or magnolia
> to a
> weblog or site, I'd like to do the same with items in Zotero (as I
> think
> about it, a citation is just a special kind of "link" or "bookmark"
> after all).
You can do this now with the client's contextual drag and drop, or
with a plugin that uses the client API. But you'll also be able to
pull citations from the server for reuse elsewhere.
>
>
> 2) I'm not sure about this yet, but I might like to be able to
> integrate
> (public) notes as well into something like a tumbelog
Again, doable in multiple ways.
>
>
> 3) To be able to integrate full citation processing into any web
> application.
This would be nice. But probably a big side project that should
involve Simon and a few others. We have been contacted by Adobe
Buzzword, among others, so maybe a working group could be put together.
>
>
> 4) every user has a profile page that can also incorporate some of the
> above, as well as their publications; sort of like a dynamic CV, and
> accessible as FOAF data
>
> http://zotero.org/user/doej
Yes, in the plan. We're also exploring the possibility of additionally
using another group's dedicated web app for the CV functionality
(i.e., universities could deploy a dedicated dynamic CV app for their
faculty and students).
>
>
> All of the above are really about the data and API story, of course.
> #1
> could be easily implemented even with feeds (I'd vote Atom and//or RSS
> 1.0), as perhaps could #2. #3 is a little more tricky.
>
> But I'm hoping these kind of web-oriented use cases are guiding the
> design.
>
> Zotero 1.0 was about pulling the web into the desktop. I'd like for
> Zotero 2.0 to be about pulling the desktop back out onto the web.
>
> Moreover, ANY feature or improvement that gets added should be
> consistent with this goal. So, for example, 1980s-era rich text is out
> unless it can be made to work with 21st century web standards and best
> practices (e.g. @rel and @class at minimum).
Agreed.
>
>
> On the details, I'm also unclear on the place of technology like:
>
> - identity and authentication (will Zotero 2.0 support openid?)
OpenID, yes. We're looking at authentication but have promised
plugable auth to our funders.
>
> - search (SPARQL? OpenSearch? both?)
Not sure yet; open to suggestions.
>
> - Dan S. mentioned exposing RDF; handled via content-negotiation?
We saw RDF as another feed type; maybe you can explain further what
you're looking for here.
>
>
> That's all for now.
>
> Bruce
>
> PS - I was looking again at Open Library. It's pretty damned cool
> that I
> can add/edit data about me and my book there! I've mentioned this
> before, but I think Zotero 2.0 ought to learn from OL. They've also
> got
> a full open source technology stack underneath of course.
As part of the Zotero-IA partnership we plan to do a tight integration
with OL (indeed, beginning next week). So that will be another way for
people in import/export/change data.
>
>
Daniel Cohen
>
> On 11-Mar-08 at 07:22 -0400 Bruce D'Arcus wrote:
>
>> Daniel Cohen wrote:
>>
>>> It might help me respond better if in the meantime everyone can
>>> ask some more specific questions that I might be able to
>>> answer--i.e., about whether particular functionality will be
>>> included, will that functionality exist on the client or server,
>>> etc. Or provide hypotheticals or use cases.
>>
>> Well, I was asking you to document *your* "hypotheticals or use
>> cases." I know about some of them, but I think it's fair to say
>> this project remains thin on details. I don't really know how you
>> can develop the sort of (developer, in particular) community I'm
>> sure you'd like to develop if you aren't more open about plans so
>> that people can see how they fit within them (or not).
>
> I must admit that I second this feeling. As a third-party developer,
> I'm *very* interested in working with (and to the advantage of)
> Zotero, especially w.r.t. the upcoming server plans. But ATM I don't
> feel that there's much interest from the Zotero people to grow a
> thriving developer community. Of course, I'd be more than happy to
> be proven wrong, though. And sorry if we're just too impatient.
No problem, and a fair enough criticism. I too have felt that we
should be doing a better job helping out the surrounding developer
community. To this end we have just added to the team Raymond Yee of
Berkeley's School of Information (and the author of a book on mashups
and APIs) to serve as a dedicated developer to help cultivate outside
connections to the Zotero server and client, as well as third-party
plugins. Raymond is currently getting up to speed on our code base and
exisiting initiatives, but I hope that he'll soon be available to
answer questions, help with functionality, etc.
>
>
> Bruce already gave some good use cases. In addition I'd like to add
> that I'd love to see the Zotero server offer an open, documented and
> standards-based sync API which could be used by third-party tools to
> send/fetch data from/to Zotero. This would e.g. allow users to push
> record additions/updates to Zotero.org from another supporting
> desktop client, or from within their own institutional repository.
> And they can then pull (sync) these data from the server back to
> their Zotero desktop app.
>
> W.r.t. standards-based sync APIs, these come to mind:
>
> SRU Record Update:
> <http://www.loc.gov/standards/sru/record-update/>
>
> SWORD (Simple Web-service Offering Repository Deposit),
> a profile of the Atom Publishing Protocol:
> <http://www.ukoln.ac.uk/repositories/digirep/index/SWORD>
See my note to Bruce about the sync that's part of the client/server
relationship and other possibilities for import/export.
>
>
> I appreciate the discussion,
Thanks--
Dan
>
>
> Matthias
>
> >
Bruce D'Arcus
Thanks for the explanation. A few quick points:
On Tue, Mar 11, 2008 at 2:15 PM, Daniel Cohen <dco...@gmu.edu> wrote:
...
> > My use cases?
> >
> > 1) Just as I can easily add a link roll from delicious or magnolia
> > to a
> > weblog or site, I'd like to do the same with items in Zotero (as I
> > think
> > about it, a citation is just a special kind of "link" or "bookmark"
> > after all).
>
> You can do this now with the client's contextual drag and drop, or
> with a plugin that uses the client API. But you'll also be able to
> pull citations from the server for reuse elsewhere.
Yeah, drag-and-drop is not a solution. I'm thinking more like
embedding a little javascript or some such.
> > 2) I'm not sure about this yet, but I might like to be able to
> > integrate
> > (public) notes as well into something like a tumbelog
>
> Again, doable in multiple ways.
Good. Having addressable over HTTP, feedable, and so forth would obviously help.
> > 3) To be able to integrate full citation processing into any web
> > application.
>
> This would be nice. But probably a big side project that should
> involve Simon and a few others. We have been contacted by Adobe
> Buzzword, among others, so maybe a working group could be put together.
Yeah, I was just talking to someone at Sun about this too ;-)
I have a post on my blog that links to some work that the RefDB
developer has been doing, in which he pleads for a single API to write
to. I have in mind something a little different, but in any case, it's
a tough problem that needs a solution.
> > - search (SPARQL? OpenSearch? both?)
>
> Not sure yet; open to suggestions.
I'm not sure either, but the two I mentioned ought to be on the radar.
The nice thing about SPARQL is it's designed for data integration (you
can query muttiple endpoints simultaneously)..
> > - Dan S. mentioned exposing RDF; handled via content-negotiation?
>
> We saw RDF as another feed type; maybe you can explain further what
> you're looking for here.
And so using RSS 1.0 presumably? That might work, but I was meaning
something different.
The question came from a chat I had with Aaron Schwartz. I was looking
at the HTML markup of their content and seeing some problems, and so
asked if they might clean that up and consider layering in RDFa
support. His response: they're planning full RDF/XML support via
content negotation.
In other words, you ping <http://zotero.org/items/1234> requesting
HTML, you get a web page. Ping the same URI requesting RDF/XML, you
get back RDF/XML.
And going back to SPARQL, you can presumably expect to be able to load
up a list of resources in a query.
Bruce
Daniel Cohen
On Mar 11, 2008, at 9:59 AM, Gerald wrote:
>
> as Dan noted, he would like to have some more specific questions. Here
> is my question:
>
> As we are using Zotero in an international research project, we need
> to share collections online, we need to enable every team member to
> add/change bibliographic data and add tags and notes,
Just what the server will be perfect for.
> and we would
> like to hold the documents online, too.
We are examining options for document storage, including in concert
with our partnership with IA. We're going to start with metadata
synching and then roll out a bunch of options for storage. If the docs
are web pages, the IA's cache might be the best option.
>
Daniel Cohen
On Mar 11, 2008, at 3:11 PM, Bruce D'Arcus wrote:
> The question came from a chat I had with Aaron Schwartz. I was looking
> at the HTML markup of their content and seeing some problems, and so
> asked if they might clean that up and consider layering in RDFa
> support. His response: they're planning full RDF/XML support via
> content negotation.
OL is still finalizing their read/write API/RDF/XML (from what I've
heard). We are talking to the OL people next week to see how we can
start integrating OL services.
>
>
> In other words, you ping <http://zotero.org/items/1234> requesting
> HTML, you get a web page. Ping the same URI requesting RDF/XML, you
> get back RDF/XML.
One question that I think will come up is which server do you want to
ping for different purposes. For some of the things you discuss, it
might be best to send a request to OL (e.g., for canonical refs); for
others, the Zotero server (e.g., collaborative possibilities; saved
searches). We'll sort all of this out over the next few months.
Dan
Bruce D'Arcus
>
>
> On Mar 11, 2008, at 3:11 PM, Bruce D'Arcus wrote:
>
> > The question came from a chat I had with Aaron Schwartz. I was looking
> > at the HTML markup of their content and seeing some problems, and so
> > asked if they might clean that up and consider layering in RDFa
> > support. His response: they're planning full RDF/XML support via
> > content negotation.
>
> OL is still finalizing their read/write API/RDF/XML (from what I've
> heard). We are talking to the OL people next week to see how we can
> start integrating OL services.
Great.
> > In other words, you ping <http://zotero.org/items/1234> requesting
> > HTML, you get a web page. Ping the same URI requesting RDF/XML, you
> > get back RDF/XML.
>
> One question that I think will come up is which server do you want to
> ping for different purposes. For some of the things you discuss, it
> might be best to send a request to OL (e.g., for canonical refs); for
> others, the Zotero server (e.g., collaborative possibilities; saved
> searches). We'll sort all of this out over the next few months.
Yes , I think the more we envision distributed webs of linked data and
services, the more possibilities we open up.
BTW, to give a hint of this, as part of some experiments, I just set
up a SPARQL endpoint using a few lines of PHP code as discussed here:
<http://arc.semsol.org/docs/v2/endpoint>
See:
<http://darcusb.geo.muohio.edu/data/>
So the data is stored in a MySQL-based RDF store, and using SPARQL,
once could query the data I have there (which is minimal) and mix it
up with queries from other endpoints.
Bruce
Raymond Yee
the Zotero project and working on getting up to speed on many
interesting aspects of the Zotero code base and initiatives. I've
remained quiet so far in order to listen and learn carefully to avoid
inadvertently injecting any misinformation!
Thanks, Dan, for answering questions -- and thanks, Bruce and
Matthias, for your deep interest. Thanks for your patience while I get
up to speed in this discussion.
-Raymond
David Botschinsky
overlooked -
a) on the desktop
Most users (at least in terms of scientists) rely on the ability to
easily insert and format citations. Zotero will be compared to the
likes of EndNote and reference manager and will be abandoned if it is
not possible satisfy style criteria set by publishers.
Zotero has come a long way with handling citations in Word and
openoffice but it is not there yet; some of what is missing is
documentation, examples and templates.
It would also be great if Zotero could deal with more complex
citations e.g. produce a citation by combining the information from a
section or chapter record with the information from a book entry:
"Bernstein, D. 1995, 'Transportation planning' in The Civil
Engineering Handbook, ed. W.F.Chen, CRC Press, Boca Raton."
b) Database
A useful feature would be the ability to have more than one database
and easely switch between them; it would allow unrelated topics to be
kept separate.
The next bit may sound negative, it is not meant to be. There has
been a number of times where I have toyed with the idea of
implementing my own version of linking records together, of storing
additional data in external tables or as custom fields, even
implementing a crude synchronisation feature. However, each attempt
always got stuck when it came to using the API; I'm not sure how much
of it is lack of documentation and how much is a genuine shortfall of
the API, but it would be great if the Zotero team could start to
address this issue. Zotero has a lot of flexibility, why not make it
easier for third party developers to use it?
It is great that the Zotero team is working on a server that would
allow two or more users to share their records; I hope you will soon
release a beta.
Kieren Diment
When it's ready, do we get to look at and deploy the code for the
zotero server if we don't want to use the zotero service - that would
seem to be in line with the "free, forever" philosophy.
When will zotero work with firefox 3, and what kind of speed
improvement should we see?
Some responses:
On 14 Mar 2008, at 10:02, David Botschinsky wrote:
> Most users (at least in terms of scientists) rely on the ability to
> easily insert and format citations. Zotero will be compared to the
> likes of EndNote and reference manager and will be abandoned if it is
> not possible satisfy style criteria set by publishers.
>
I don't know to what extent this is a furphy. I'm thinking that a
lot of the time, when people use software a lot, they get used to its
deficiencies. So it's important to ask what bits of the other
software are eccentricities that users are just used to, and what's a
valuable feature. Sometimes these two things do become combined to
increase the user's structural inertia. Most users don't make use fo
their software very effectively - for example the proportion of
people who use Word's styles features properly compared to those who
don't.
One useful thing that zotero can do is take the significant and
surprisingly large academic community who don't use reference
management software, and make the case so compelling for them to do
so, that there's no reason not to. I think zotero is part-way there
with this.
>
> b) Database
> A useful feature would be the ability to have more than one database
> and easely switch between them; it would allow unrelated topics to be
> kept separate.
>
I agree, this sounds useful
> The next bit may sound negative, it is not meant to be. There has
> been a number of times where I have toyed with the idea of
> implementing my own version of linking records together, of storing
> additional data in external tables or as custom fields, even
> implementing a crude synchronisation feature. However, each attempt
> always got stuck when it came to using the API; I'm not sure how much
> of it is lack of documentation and how much is a genuine shortfall of
> the API, but it would be great if the Zotero team could start to
> address this issue. Zotero has a lot of flexibility, why not make it
> easier for third party developers to use it?
>
I personally struggle with documentation in javascript already, and
documentation with examples for the Zotero API would be extremely
useful for me, because I could "do stuff" easily rather than
struggling, or "wishing I could do stuff" . Alternatively an
"Advanced Zotero" book with a few walkthrough examples of extension
development would be very useful, and a potential profit centre.
>
Elena Razlogova
On Mar 17, 2008, at 8:36 PM, Kieren Diment wrote:
>
> Two questions:
>
> When it's ready, do we get to look at and deploy the code for the
> zotero server if we don't want to use the zotero service - that would
> seem to be in line with the "free, forever" philosophy.
>
> When will zotero work with firefox 3, and what kind of speed
> improvement should we see?
Dan Stillman said on forum about Zotero in FF3 vs FF2:
> Actually about 13x faster at last count. After some optimizations, a
> quicksearch on a test library with 7,000 items has gone from 82
> seconds to 6 seconds.
Elena
David Botschinsky
> > easily insert and format citations. Zotero will be compared to the
> > likes of EndNote and reference manager and will be abandoned if it is
> > not possible satisfy style criteria set by publishers.
>
> I don't know to what extent this is a furphy. I'm thinking that a
> lot of the time, when people use software a lot, they get used to its
> deficiencies. So it's important to ask what bits of the other
> software are eccentricities that users are just used to, and what's a
> valuable feature. Sometimes these two things do become combined to
> increase the user's structural inertia. Most users don't make use fo
> their software very effectively - for example the proportion of
> people who use Word's styles features properly compared to those who
> don't.
or Reference Manager to format the references for their publications,
be it their thesis or the latest article accepted by a journal. For
senior scientists who have collected thousands of references other
features are also important, but for students and younger scientists,
the ability to easily insert and format references in Word is the main
reason for using the software. We also have a librarian who has to
ensure the appropriate styles are available in EndNote.
Bruce D'Arcus
> All the scientists at the research Institute where I work use EndNote
> or Reference Manager to format the references for their publications,
> be it their thesis or the latest article accepted by a journal. For
> senior scientists who have collected thousands of references other
> features are also important, but for students and younger scientists,
> the ability to easily insert and format references in Word is the main
> reason for using the software. We also have a librarian who has to
> ensure the appropriate styles are available in EndNote.
It's worth noting, though, that the styling language Zotero uses (which
I happened to design) is more powerful than that in Endnote/RefMan/etc.,
and that at least some of the styles available for use in Zotero are in
fact more correct and robust than those of its counterparts.
The only real limitation is the lack of an easy-to-use GUI for creating
new styles. That's a problem that needs to be solved.
Also, if I had to guess, I'd say that manual citation formatting remains
far more common than automated tools. A lot of people find applications
like Endnote more trouble than they're worth.
I think the goal should not be to match Endnote in a battle-of-features,
but rather to be better all around: easier to use, more powerful, more
flexible. Often that should mean doing things differently.
Bruce
Kent Tenney
>
>
> On Mar 11, 2008, at 7:22 AM, Bruce D'Arcus wrote:
>
> >
> > Daniel Cohen wrote:
> >
> >> It might help me respond better if in the
> >> meantime everyone can ask some more specific questions that I might
> >> be
> >> able to answer--i.e., about whether particular functionality will be
> >> included, will that functionality exist on the client or server, etc.
> >> Or provide hypotheticals or use cases.
> >
> > Well, I was asking you to document *your* "hypotheticals or use
> > cases."
> > I know about some of them, but I think it's fair to say this project
> > remains thin on details. I don't really know how you can develop the
> > sort of (developer, in particular) community I'm sure you'd like to
> > develop if you aren't more open about plans so that people can see how
> > they fit within them (or not).
>
> A brief overview that might be helpful. When you look at our plans for
> the server (feeds, groups, APIs, etc) you realize that almost all of
> the heavy lifting is in getting all of the clients synched to the
> server.
Evidently it's not feasible to generically sync an sqllite database on the
client with one on the server.
:-(
If that was the solution, the code would be of great interest to many projects.
Thanks,
Kent
Bess Sadler
On Mar 20, 2008, at 2:27 PM, Kent Tenney wrote:
> Evidently it's not feasible to generically sync an sqllite database
> on the
> client with one on the server.
> :-(
> If that was the solution, the code would be of great interest to
> many projects.
>
> Thanks,
> Kent
>
This would be of great interest to the projects we support at
University of Virginia as well. Is anyone looking at implementing
this as a stepping stone on the path towards Zotero Commons? It seems
to me that if Zotero can write to a local sqllite database it could
also be made to write to a centrally hosted mysql database. Even some
kind of sqllite --> mysql export function would be a step in the
right direction.
Bess Sadler
University of Virginia
Bruce D'Arcus
I have to wonder if relying on a relational database is really the right
long-term strategy.
Bruce
Bess Sadler
Please, propose something else if there's a better way to go. But
really I'm not seeing this as a long-term strategy, but rather as a
short term win. Faculty members want the ability to collaboratively
maintain citation databases NOW, and some of the ones I support are
planning to do seriously kludge-y things to get that ability. Asking
them to wait for Zotero commons is going to be a hard sell. An easier
sell would be to give them a university maintained central database
for citation management, a great client like Zotero to use for adding
and editing citations, and a migration path once Zotero commons is
ready for prime time. That's what I meant by stepping stone.
Is the Zotero team planning to or interested in working on something
like this? How open is Zotero development and would this be the kind
of thing another group could develop? (I know people are developing
plugins for Zotero. Maybe this could work as a plug-in?)
Kent, is this the kind of thing you were talking about? How much
interest would there be in a project like this?
Bess
Kieren Diment
On 21 Mar 2008, at 08:31, Bruce D'Arcus wrote:
> I have to wonder if relying on a relational database is really the
> right
> long-term strategy.
Why, what would you see the alternatives for a multiuser situation?
Here's my quick and dirty perl solution to some of this problem. The
only reason that it's dirty is that I've spent a grand total of 10
minutes on it.
Anyway, the perl modules SQL::Translator and DBIx::Class will handle
transitions of schemas between databases quite nicely.
The script below nearly works. There is some mysql specific metadata
that needs to be supplied but isn't. This is preventing complete
success at this moment. The approach I'd use to fix that would be to
dump the sqlite databases to a set of DBIx::Class schemas, and edit
them to include the mysql specific metadata, then deploy the
DBIx::Class schema from the DBIx::Class data into mysql (or any other
back end that SQL::Translator supports - pg, db2, oracle, csv ... ).
The script below is definately picking up relationships from the
sqlite schema, but it would need someone familiar with the zotero
schema to ensure that everything was being caught (or better, writing
some tests ;) ).
Not bad for 10 minutes work eh?
#!/usr/bin/perl
use warnings;
use strict;
use DBIx::Class::Schema::Loader qw/ make_schema_at /;
# load the database info into memory using the sqlite schema
make_schema_at(
'Zotero::SQLite',
{ debug => 1,
relationships => 1,
},
[ 'dbi:SQLite:dbname=zotero.db','','' ],
);
# dump it out to mysql
my $user='root'; my $pass = '';
my $schema = Zotero::SQLite->connect('dbi:mysql:database=zotero',
$user, $pass);
$schema->deploy({ add_drop_table => 1, });
Richard Karnesky
> also be made to write to a centrally hosted mysql database.
http://forums.zotero.org/discussion/994/can-i-use-a-external-sql-database/
and others.
Zotero's ability to write to sqlite is based entirely on the
mozstorage API.
I must say that the mere ability to sync two databases with the same
schema is very small-minded and it is depressing to see encouragement
of this. Far better would be tackle the problem of synchronizing
bibliographic databases. This wouldn't let you use the tool to sync
another mozstorage/sqlite file (or other database) that is designed
for a different purpose, but I don't think that this "limitation"
means that a solution is "too specific."
It would be "general" in the sense that it would be usable by other
databases with a different schema (I can imagine it being adopted by
other free/open source reference managers for sure & possibly some
proprietary ones). This would provide a nicer means of syncing a
local zotero to Connotea or refbase or to sync the zotero server to
Bookends, etc. Synchronization of snapshots & other files may already
be an important need for Zotero that other database applications
might not have.
Such an API could also be used for not only simple relational
databases, but also used by programs that use flat-files (bibtex), xml
databases, etc.
--Rick
Bruce D'Arcus
> Why, what would you see the alternatives for a multiuser situation?
I suppose it is a bit of a tease to ask a big question like this but
not provide a hint of an answer. I'm not sure I DO have the answer (at
least for Zotero), whic is exactly why I on;y raised the question, but
...
My personal solution for Zotero 1.x is to simply use ryncs for the
sqlite dbs. No ,that doesn't work for a multi-user setup, but Zotero
1.x isn't multi-user. There are a ton of problems to solve to get this
right, and at the end of the day you're still stuck with an inflexible
data model for data which requires flexibility.
More broadly, I've not had time to work on it, but it took me all of
about 10 minutes to set up this:
<http://darcusb.geo.muohio.edu/data/>
What do we have here?
It's a SPARQL endpoint sitting on top of an RDF store (which happens
to be implemented on MySQL). I can load and query it using standard
protocols. I can add any data I want, so long as it conforms to RDF. I
can also trivially partition the data into different chunks, while
also able to merge them together equally trivally.
Just saw Rick's post; I think what I'm talking about is in the same
spirit. If I get farther with something functional, I'll be sure to
update the list.
Bruce
Dan Stillman
> On Mar 20, 2008, at 2:27 PM, Kent Tenney wrote:
>
>> Evidently it's not feasible to generically sync an sqllite database
>> on the
>> client with one on the server.
>> :-(
>> If that was the solution, the code would be of great interest to
>> many projects.
>>
>> Thanks,
>> Kent
>>
>>
>
> This would be of great interest to the projects we support at
> University of Virginia as well. Is anyone looking at implementing
> this as a stepping stone on the path towards Zotero Commons?
Just a quick terminology correction: "Zotero Commons" refers to the
collaboration between Zotero and the Internet Archive for making
documents from within Zotero publicly available in the IA and will
likely be part of Zotero 2.0. Server-based syncing, which will allow for
a basic form of collaborative editing (by using a shared account), will
be available in Zotero 1.5. As Dan Cohen said a few days ago, we're
testing syncing internally now and will be releasing a beta in the near
future.
> It seems
> to me that if Zotero can write to a local sqllite database it could
> also be made to write to a centrally hosted mysql database.
Not really. As Rick just wrote, Zotero can write to SQLite databases
because Firefox includes mozStorage, a native interface to SQLite. MySQL
support would require someone reimplementing mozStorage using the MySQL
client API, which would be no small feat. Of course, if someone wanted
to do that, it would have tremendous appeal in the Mozilla community far
beyond Zotero, but we have no plans to work on that. Writing to a remote
database alone--or some sort of manual SQL-based sync--also doesn't
really give you collaborative editing beyond what you can already do
with something like rsync, since currently running Zotero processes
wouldn't have any awareness of changes and could easily get corrupted if
the underlying data store changed during runtime. Actual sync logic is
really quite a bit more complicated, with all sorts of problematic edge
cases. But again, we're in the final stages of internal testing before
releasing a beta. And, in fact, it works.
Elizabeth Sadler
On 20-Mar-08, at 7:22 PM, Dan Stillman wrote:
> But again, we're in the final stages of internal testing before
> releasing a beta. And, in fact, it works.
Yippee!! Okay, I am pacified. I will wait for the beta with bated
breath. Thank you.
Bess
Kent Tenney
>
> Hi, Bruce.
>
> Please, propose something else if there's a better way to go. But
> really I'm not seeing this as a long-term strategy, but rather as a
> short term win. Faculty members want the ability to collaboratively
> maintain citation databases NOW, and some of the ones I support are
> planning to do seriously kludge-y things to get that ability. Asking
> them to wait for Zotero commons is going to be a hard sell. An easier
> sell would be to give them a university maintained central database
> for citation management, a great client like Zotero to use for adding
> and editing citations, and a migration path once Zotero commons is
> ready for prime time. That's what I meant by stepping stone.
>
> Is the Zotero team planning to or interested in working on something
> like this? How open is Zotero development and would this be the kind
> of thing another group could develop? (I know people are developing
> plugins for Zotero. Maybe this could work as a plug-in?)
>
> Kent, is this the kind of thing you were talking about?
My use case is much more trivial than what this thread assumes.
During any day I find myself working at a number of computers,
home workstation, laptop, several machines at work ...
All I want is the ability to access the same zotero.sqllite file
wherever I happen to be, IE I want to collaborate with myself.
My comment about generic sqllite sync was assuming this use case.
Like http://foxmarks.com, except I'd be syncing zotero.sqllite
instead of bookmarks.sqllite (or whatever the filenames actually are)
Thanks,
Kent
Kent Tenney
Cool, can't wait.
My comment was referring to simple
web based raid-1 mirroring of sqllite files.
Clearly you are implementing a superset of this.
Thanks,
Kent
>
>
>
> >
>
trevor.j...@gmail.com
> My use case is much more trivial than what this thread assumes.
>
> During any day I find myself working at a number of computers,
> home workstation, laptop, several machines at work ...
>
> All I want is the ability to access the same zotero.sqllite file
> wherever I happen to be, IE I want to collaborate with myself.
ways people are using Zotero over a network.
http://www.zotero.org/documentation/zotero_over_a_network
or some of the portable solutions people are using
http://www.zotero.org/documentation/zotero_portable_solutions
David Botschinsky
> I happened to design) is more powerful than that in Endnote/RefMan/etc.,
> and that at least some of the styles available for use in Zotero are in
> fact more correct and robust than those of its counterparts.
> The only real limitation is the lack of an easy-to-use GUI for creating
> new styles. That's a problem that needs to be solved.
> Also, if I had to guess, I'd say that manual citation formatting remains
> far more common than automated tools. A lot of people find applications
> like Endnote more trouble than they're worth.
formatting.
> I think the goal should not be to match Endnote in a battle-of-features,
> but rather to be better all around: easier to use, more powerful, more
> flexible. Often that should mean doing things differently.
it is achieved. The user will look at a piece of software; if it
works, if it is easy to tailor to specific needs then the software
will be used.
David
Bruce D'Arcus
<d.bots...@imperial.ac.uk> wrote:
...
> > Also, if I had to guess, I'd say that manual citation formatting remains
> > far more common than automated tools. A lot of people find applications
> > like Endnote more trouble than they're worth.
> Wrong. The sofware is not great but users find it better than manual
> formatting.
"Wrong"?!
I'm basing this statement on personal experience talking to colleagues
and graduate students, few of whom use Endnote or Zotero. How can you
possibly tell me that's "wrong"?
Just because your experience has been different doesn't mean it's universal.
> > I think the goal should not be to match Endnote in a battle-of-features,
> > but rather to be better all around: easier to use, more powerful, more
> > flexible. Often that should mean doing things differently.
> Somebody who is looking for formatting of citations will not care how
> it is achieved. The user will look at a piece of software; if it
> works, if it is easy to tailor to specific needs then the software
> will be used.
Um, sure, but define "works" and then we get into the challenges
projects like this are up against.
Endnote, for example, never worked very well for me. I couldn't
reliably store my data in it, and it couldn't do something pretty
basic that Zotero can: switch between author-date and note-based
styles without my having to substantially edit my documents.
Bruce
David Botschinsky
bigger than it is (especially as the style repository has been
expanding the past few months).
The formatting tool is an important feature of Zotero. Whatever needs
to be done to improve this tool should form part of any roadmap/
enhancement list.
Bruce D'Arcus
> The formatting tool is an important feature of Zotero. Whatever needs
> to be done to improve this tool should form part of any roadmap/
> enhancement list.
Right :-)
FWIW, there are potentially different angles to attack here.
You asked for documentation, presumably on CSL. The best we have ATM is
this:
<http://dev.zotero.org/csl_syntax_summary>
Beyond that, I personally think the place where we want to get to is
that people never, ever, have to think about citation styling again:
that if they need a new style, it's simply available to them.
This in turn requires making it trivially easy to create new styles, and
for users to access them, so everybody isn't forced to reinvent the wheel.
There are, I'd say, three possibilities to tackle this, all of which
have been thought about or worked on, and none of which are necessarily
mutually exclusive:
1) import Endnote style files. This has some short-term advantages, but
I personally think is really problematic. I'm not going to go into why ATM.
2) expand csledit.xul to be a full-blown CSL editor. This has one
obvious disadvantage: it's not trivial to do. There's also a
less-obvious disadvantage, which I think is illustrated by the third
option ...
3) A simple-to-use web app for creating and hosting styles; outlined here:
<http://community.muohio.edu/blogs/darcusb/archives/2008/02/13/makecsl-a-proposal-2>
... with what I'd call an "early prototype" here:
<http://community.muohio.edu/blogs/darcusb/archives/2008/02/25/from-proposal-to-example-csl-gallery>
I obviously think 3 is the most promising approach. If done right, it
would be trivial for anyone to create a new style; easier than even the
2 option above. And because of that, and the fact that it's a public
repository, I think the number of styles would grow really quickly.
But, alas, I doubt I can do it alone; I have neither the time nor the
skill (I think the hard part for me is a nice wizard-based interface,
including a way to easily add and reorder the macros in a citation and
bibliographic entry). Nevertheless, I'll keep plugging away as time permits.
Bruce
Kent Tenney
Likely one of those approaches would do the trick,
I'm lazy enough to wait for 2.0
Thanks,
Kent
>
>
>
>
>
> >
>
David Botschinsky
mean by time and skill.
> There are, I'd say, three possibilities to tackle this, all of which
> have been thought about or worked on, and none of which are necessarily
> mutually exclusive:
>
> 1) import Endnote style files. This has some short-term advantages, but
> I personally think is really problematic. I'm not going to go into why ATM.
>
> 2) expand csledit.xul to be a full-blown CSL editor. This has one
> obvious disadvantage: it's not trivial to do. There's also a
> less-obvious disadvantage, which I think is illustrated by the third
> option ...
>
> 3) A simple-to-use web app for creating and hosting styles; outlined here:
>
> ... with what I'd call an "early prototype" here:
>
Shel
I work also. From my POV, being able to format citations in different
ways is a minor consideration. The thing is, if you want to keep a
personal list of references and format it for a paper, EndNote already
does that quite well. So while I see that function as an important
aspect of Zotero's development, it doesn't seem like a priority. As
long as we can export our references to EndNote or some other software
to do that stuff, we're happy.
If Zotero supported even fairly rudimentary collaborative features,
and completely got rid of the word processing integration tools, I
expect that Zotero would quickly become the standard at my workplace.
We are looking for ease of collaboration + ease of collecting online
material -- and this combination is surprisingly hard to find.
Zotero has the excellent interface, but without the collaborative
features, most researchers I know are happy to stick with the tools
they're already familiar with.
CiteULike and Connotea have the collaborative features, which some
researchers are using... but both sites' "scraping" abilities are
severely limited, and collaboration can be difficult when people find
the collaboration tool cumbersome...
A lot of researchers that I work with are desperate for something like
Zotero + collaboration. In the meantime, we continue to develop
EndNote-based kludges. There just isn't any reason to push a team of
social science researchers with varying degrees of computer savvy to
learn to use Zotero. It doesn't make collaboration any easier, and
that's the only reason for someone who is already used to EndNote to
learn something new.
Just sayin'.
Rachelle
(Snip)
Bruce D'Arcus
...
> There just isn't any reason to push a team of social science researchers with
> varying degrees of computer savvy to learn to use Zotero. It doesn't make
> collaboration any easier, and that's the only reason for someone who is
> already used to EndNote to learn something new.
There may not be with Zotero 1.x, (though even that is debatable; I
have a somewhat computer-phobic colleague who loves Zotero, having
migrated to it from ProCite) but I think you'll likely find a
different answer with 1.5 and (particularly) 2.0.
Also, I'd like to think that we'll soon start to see other projects
implementing the same kinds of technology such that we'll begin to
reasonably expect the kind of seamless interoperability we've never
seen in this space. When we see more of that, the value proposition
will be really clear.
Bruce
Shel
It's just frustrating because the potential value *is* so great...
and it's not here yet :-)
In the meantime research teams have to make decisions about how much
time to invest in developing alternatives, and it's a hard call. The
delivery date for collaborative features has been pushed back a couple
of times -- ah well, that's how it always is with software
development, and kudos to you all for not rushing out sucky releases.
Plus, what you've already given us is not only free, but fabulous.
Best interface for collecting information online, bar none, and thank
you very much for that.
I just keep hoping for the next piece, because that's where the real
value will be for us.
Rachelle
On Mar 25, 11:20 am, "Bruce D'Arcus" <bdar...@gmail.com> wrote:
> On Tue, Mar 25, 2008 at 1:53 PM, Shel <joralemonshe...@gmail.com> wrote:
>
> ...