Possible read/unread solution
Frank Bennett
http://forums.zotero.org/discussion/1787/3/simple-marking-of-items/#Item_14
Implicit tagging might provide an unintrusive solution to this. If items added via a Web translator are assigned a "New" tag by the translator engine, the user can get a quick view of all newly added items. I've hacked this into MLZ, and it seems to serve the purpose well. I no longer worry about where new items are landing in Zotero, since I can readily fetch them from the library view for reading, commenting and tagging. The changes are here:
https://github.com/fbennett/zotero/commit/5274b6ce5935c526ceb3f11220c541ff9130bb78
https://github.com/fbennett/zotero/commit/24083d7f977ecf907e6959eba09b04c06880cce2
Frank
Aurimas Vinckevicius
I was thinking that, in addition to automatic tags, we should also introduce "system" (or w/e you want to call them) tags, which would be used for such purposes like read/unread, ratings, starring, ZotFile marking files sent to tablet, etc. The system tags would not be displayed in either the tag selector or the item tag list by default (could be enabled via hidden pref).
Aurimas
--
You received this message because you are subscribed to the Google Groups "zotero-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zotero-dev+...@googlegroups.com.
To post to this group, send email to zoter...@googlegroups.com.
Visit this group at http://groups.google.com/group/zotero-dev?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
Patrick Mineault
If you guys settle on a way of implementing system tags let me know.
Brenton Wiernik
Dan Stillman
> I've been digging through the Zotero functions, and I don't see much
> that explicitly relies on tag types other than display rules for
> showing versus not showing automatic tags. Is there anything (perhaps
> that I missed in the server functions) that would restrict use of a
> tagType=2 for "system" tags?
pretty easily — that field was obviously designed with the possibility
of future tag types in mind. But can we talk more about how these would
be used, and how they would be treated differently in various places
(client UI, website, import/export)? I think certainly they wouldn't
display to other users on the website. I'm less sure about not
displaying them in the client by default. (Granted, with group libraries
the distinction between private and public is blurred a bit.) But maybe
invisibility is the defining characteristic? That is, they're a way for
extensions to bind their own internal data (up to 255 chars, at least)
to items without hacking stuff into the Extra field or maintaining a
separate (potentially inconsistent) database, and it's up to the
extensions to manage that data, act on it appropriately, and communicate
its presence to users as necessary?
Aurimas Vinckevicius
--
You received this message because you are subscribed to the Google Groups "zotero-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zotero-dev+unsubscribe@googlegroups.com.
To post to this group, send email to zoter...@googlegroups.com.
Visit this group at http://groups.google.com/group/zotero-dev.
Brenton Wiernik
The specific use I am thinking of is creating item-specific filename suffixes for Zotfile. The plugin needs a way to store this information with an item and have it persist. I would rather use the existing tag architecture than hack something in.
Emiliano Heyns
On 2/24/14, 12:12 PM, Brenton Wiernik wrote:
> I've been digging through the Zotero functions, and I don't see much
> that explicitly relies on tag types other than display rules for
> showing versus not showing automatic tags. Is there anything (perhaps
> that I missed in the server functions) that would restrict use of a
> tagType=2 for "system" tags?
I imagine something would break,
Brenton Wiernik
Frank Bennett
For project tags, I just masked them from sync altogether. Given what they do, that's not much of an issue. If you need tags other than type 0 or 1 to sync, I think it would need an adjustment on the Zotero sync servers. The code for masking, in case it's relevant, is here, at lines 3621-3623 and 3629-3631:
https://github.com/fbennett/zotero/blob/multi-4.0/chrome/content/zotero/xpcom/sync.js#L3595
There is a bunch of jiggery-pokery in tagselector.xml for display and editing of project tags. Simply masking them from view would be simpler. The starting point would be to add a constraint in the makeClickableTag() function:
https://github.com/fbennett/zotero/blob/multi-4.0/chrome/content/zotero/bindings/tagselector.xml#L857
To screen the 10000 tags completely from the right-hand tagsbox, I hacked a constraint into an SQL call in getTags():
https://github.com/fbennett/zotero/blob/multi-4.0/chrome/content/zotero/xpcom/data/item.js#L4457
The one thing to guard against would be tags that share a value with a visible tag, since the selection machinery begins with the assumption that all tags of the same name are to be selected as a single batch.
Hope that helps -- let me know if anything in the patches looks puzzling (I'm not the greatest coder, so that could well happen).
Frank
jl
I agree that there should be a better solution to store item-related information that syncs for zotero extensions. But I am not sure that hidden tags are the solution. Is the google scholar citations addon supposed to add tags like cite_3, cite_34 etc to items? How can the user see this information? When I define the zotfile tags '_tablet' and '_tablet_modified' as hidden, can the user still assign a color to them so that they are easily visible? What is about saved searches?
The good thing with the current '_tablet' tags is that they a) sync, b) can have colors so that the user can easily highlight the items, and c) can be part of saved searches. The bad thing is that the user can use the number shortcuts to add and remove the tags, which confuses zotfile. The google scholar citations addon has a different set of requirements (mainly that the information is visible to the user both as an optional column in the item tree and as a field in the item info). Could hidden tags solve all these problems?
The more general problem is that I don't see a good way for addons to store item specific information that syncs and can be visible to users. I am using attachment notes for information that should not be visible to the user. The google scholar addon hijacks the extra field, which can be annoying because some citation styles add that information to the bibliography. A separate database is neither visible to the user nor syncs (at least I think so). So it doesn't work for zotfile and google scholar citations. Other solutions would be greatly appreciated. I thought about an additional database field that stores extension data as json such as {"zotfile":..., "gscholar": ...} but not sure whether that is a good solution.
Emiliano Heyns
I agree that there should be a better solution to store item-related information that syncs for zotero extensions.
But I am not sure that hidden tags are the solution. Is the google scholar citations addon supposed to add tags like cite_3, cite_34 etc to items? How can the user see this information? When I define the zotfile tags '_tablet' and '_tablet_modified' as hidden, can the user still assign a color to them so that they are easily visible? What is about saved searches?
The good thing with the current '_tablet' tags is that they a) sync, b) can have colors so that the user can easily highlight the items, and c) can be part of saved searches. The bad thing is that the user can use the number shortcuts to add and remove the tags, which confuses zotfile. The google scholar citations addon has a different set of requirements (mainly that the information is visible to the user both as an optional column in the item tree and as a field in the item info). Could hidden tags solve all these problems?
The more general problem is that I don't see a good way for addons to store item specific information that syncs and can be visible to users. I am using attachment notes for information that should not be visible to the user. The google scholar addon hijacks the extra field,
which can be annoying because some citation styles add that information to the bibliography.
A separate database is neither visible to the user nor syncs (at least I think so).
So it doesn't work for zotfile and google scholar citations. Other solutions would be greatly appreciated. I thought about an additional database field that stores extension data as json such as {"zotfile":..., "gscholar": ...} but not sure whether that is a good solution.
Frank Bennett
On Thu, May 1, 2014 at 9:49 AM, jl <jp...@nyu.edu> wrote:
I agree that there should be a better solution to store item-related information that syncs for zotero extensions.
The only real solution can come from the Zotero team itself, who manages the sync infrastructure.But I am not sure that hidden tags are the solution. Is the google scholar citations addon supposed to add tags like cite_3, cite_34 etc to items? How can the user see this information? When I define the zotfile tags '_tablet' and '_tablet_modified' as hidden, can the user still assign a color to them so that they are easily visible? What is about saved searches?
The good thing with the current '_tablet' tags is that they a) sync, b) can have colors so that the user can easily highlight the items, and c) can be part of saved searches. The bad thing is that the user can use the number shortcuts to add and remove the tags, which confuses zotfile. The google scholar citations addon has a different set of requirements (mainly that the information is visible to the user both as an optional column in the item tree and as a field in the item info). Could hidden tags solve all these problems?
I've been told by the Zotero team (forgot who) that using the tags this way without constraint could cause performance problems.The more general problem is that I don't see a good way for addons to store item specific information that syncs and can be visible to users. I am using attachment notes for information that should not be visible to the user. The google scholar addon hijacks the extra field,
I take exception to this formulation. My addon similarly "hijacks" the extra field, in exactly the same way that zotfile "hijacks" the tags.which can be annoying because some citation styles add that information to the bibliography.
And tags, likewise, are added to the bibliography by most citation styles, and could likewise cause annoyance. Using charged words to describe a given workaround in the face of external restrictions (that there is no way to sync addon-data is certainly not by choice of the addon developers) is not really helpful.A separate database is neither visible to the user nor syncs (at least I think so).
It isn't (unless the addon provides a GUI for it), and it won't sync, no. The complex solution would be to store the info in the extra field or notes and to add/strip automatically before/after sync. But that's very fragile.
So it doesn't work for zotfile and google scholar citations. Other solutions would be greatly appreciated. I thought about an additional database field that stores extension data as json such as {"zotfile":..., "gscholar": ...} but not sure whether that is a good solution.
The additional database field likewise won't sync. I've looked at various ways to get this done, and without structural support from the Zotero team, all these are fragile. Good options are available for non-shared libraries, using sync infrastructure from Dropbox, Google, Simplenote, etc, but with shared libraries, things get very hairy, very fast. The best solution, I think, would be for Zotero to abandon SQL as a storage facility and go for a document store instead, but that'd have major infrastructural impact; perhaps not something the zotero team wants to get into right now.Emile
--
You received this message because you are subscribed to the Google Groups "zotero-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zotero-dev+...@googlegroups.com.
To post to this group, send email to zoter...@googlegroups.com.
Visit this group at http://groups.google.com/group/zotero-dev.
For more options, visit https://groups.google.com/d/optout.
Emiliano Heyns
This is how I sync field variants in Multilingual Zotero, for the present. It is transparent to the user (apart from the clutter of header offset values and JSON content visible in the Extra field on zotero.org), and has proven to be quite reliable; but putting it together took a *lot* of work, and involved numerous changes to core code in sync.js and elsewhere. I don't think it could be done in a plugin.
- Syncing extra data via XML files in WebDAV storage
- JSON objects on synced media such as DropBox
- PouchDB
- Dropbox DataStore (Google also has something like this -- I keep forgetting it's name)
- Simperium
- CardDAV
- SyncML
jl
The more general problem is that I don't see a good way for addons to store item specific information that syncs and can be visible to users. I am using attachment notes for information that should not be visible to the user. The google scholar addon hijacks the extra field,
I take exception to this formulation. My addon similarly "hijacks" the extra field, in exactly the same way that zotfile "hijacks" the tags.
Dan Stillman
This is still a little ways off, but it's something I'm planning to work on after async DB access and API syncing.
Emiliano Heyns
On 5/2/14, 3:08 AM, jl wrote:Abandoning SQL seems unrealistic but why not an additional 'extension data' field in the core database that is synced? Of course, that requires chances in the core but it's less complex. Addons just have to agree how to use this field and json seems to be a good solution with {"gscholar": ..., "zotfile": ...}.
Yeah, we're not going to be abandoning SQL anytime soon,
- Sync is a hard problem, and with hard problems I usually look first to how others have solved it (CouchDB, SyncML, where sync is a first-class class citizen). Zotero does a queued sync it seems, where syncers have to wait for a slot.
- I am a heavy note-taker in my PDFs, and the left-or-right choice when two PDFs conflict during sync makes me sad. There's really no way for me to resolve the conflict at that point without data loss. Here' I prefer what dropbox et al do; notify me, and give me ample time to resolve the matter.
but Zotero already has synced 'relations' triples, which are how we track deleted items and inter-library drags. Those are currently for internal use only and limited to a few specific predicates (among other reasons because the current implementation is awkward and inefficient), but we're planning to expand on that support, which could, among other things, open the door to semantic relations, extensible identifiers, and custom fields, defined by plugins or otherwise.
This is still a little ways off, but it's something I'm planning to work on after async DB access and API syncing.
Dan Stillman
On Fri, May 2, 2014 at 10:42 AM, Dan Stillman <dsti...@zotero.org> wrote:
Yeah, we're not going to be abandoning SQL anytime soon,
I didn't quite expect that would be on the table, no :) But there are other reasons I'm toying with alternative technologies:
- Sync is a hard problem, and with hard problems I usually look first to how others have solved it (CouchDB, SyncML, where sync is a first-class class citizen). Zotero does a queued sync it seems, where syncers have to wait for a slot.
2. I am a heavy note-taker in my PDFs, and the left-or-right choice when two PDFs conflict during sync makes me sad. There's really no way for me to resolve the conflict at that point without data loss. Here' I prefer what dropbox et al do; notify me, and give me ample time to resolve the matter.
I'm not sure what this has to do with anything else. Dropbox-style on-disk file conflict resolution has never been a design goal. If it were, we could implement it.
I also think that SQL for document fragment storage (because that's what Zotero does) is an artifact from SQLs ubiquity, not its fit-for-purpose, but as long as it's here and it's reliable, I could see reason for its continued use.
2) SQLite is actually an excellent fit in many ways. For example, when we load the items list (which, for JS post-processing and sorting purposes, requires all values in the visible columns), we can pull out just the necessary fields. If we used a document store, we'd have to load every single document in the database and extract the necessary fields.
That's the British "a little ways off", I take it :) As in, "don't hold your breath". No offence intended; for entirely understandable reasons, Zotero moves rather slowly.but Zotero already has synced 'relations' triples, which are how we track deleted items and inter-library drags. Those are currently for internal use only and limited to a few specific predicates (among other reasons because the current implementation is awkward and inefficient), but we're planning to expand on that support, which could, among other things, open the door to semantic relations, extensible identifiers, and custom fields, defined by plugins or otherwise.
This is still a little ways off, but it's something I'm planning to work on after async DB access and API syncing.
Emiliano Heyns
On May 2, 2014 7:30 PM, "Dan Stillman" <dsti...@zotero.org> wrote:
>
>> 2. I am a heavy note-taker in my PDFs, and the left-or-right choice when two PDFs conflict during sync makes me sad. There's really no way for me to resolve the conflict at that point without data loss. Here' I prefer what dropbox et al do; notify me, and give me ample time to resolve the matter.
>
>
> I'm not sure what this has to do with anything else. Dropbox-style on-disk file conflict resolution has never been a design goal. If it were, we could implement it.
Don't take things so personally. What it has to do with anything else, specifically, is that I am toying with different technologies, because the current sync does things differently than I'd like it to. This is what I said, and it doesn't make any claim on you. It doesn't mean everyone must share my wants or needs. I acknowledge you and the zotero team can legitimately seek to serve different needs.
> 1) SQLite is the technology available to us. There's no point considering things like CouchDB because we can't use them. If we wanted a document store, we could implement it via SQLite. (And that's exactly what Mozilla does for IndexedDB.)
That just rephrases what I said.
> 2) SQLite is actually an excellent fit in many ways. For example, when we load the items list (which, for JS post-processing and sorting purposes, requires all values in the visible columns), we can pull out just the necessary fields. If we used a document store, we'd have to load every single document in the database and extract the necessary fields.
That is simply not true. I don't know what document stores you're familiar with, but any nosql store can do map-reduce or similar. Pouch is, as I noted, unproven, but it includes an sql like query language in addition to MapReduce.
>> That's the British "a little ways off", I take it :) As in, "don't hold your breath". No offence intended; for entirely understandable reasons, Zotero moves rather slowly.
>
>
> Well, it's not "don't hold your breath" — I'm saying it's the next big thing I plan to work on after async DB (which I'm currently working on) and API syncing. But yes, it will take a while.
Errr... In what way are we in disagreement, then? I didn't say it wasn't a priority for you. I only said it's nothing we should expect within a time frame that would make holding ones breath survivable.
Again: I think it is entirely legitimate that zotero moves at a careful pace. This is also known as "slow". There's nothing, at all, wrong with being slow and steady, certainly when you're screwing about with people's research data (texstudio crashed on me yesterday, causing me to lose an hours work. Not great) of many people (as zotero does).
None of this negates that there are things I want zotero to do differently, and because I happen to know a little programming, I can tinker around to see what works. No reason to get any panties in a knot.
Regards,
Emile
Sebastian Karcher
> > Well, it's not "don't hold your breath" — I'm saying it's the next
> big thing I plan to work on after async DB (which I'm currently
> working on) and API syncing. But yes, it will take a while.
>
> Errr... In what way are we in disagreement, then? I didn't say it
> wasn't a priority for you. I only said it's nothing we should expect
> within a time frame that would make holding ones breath survivable.
>
>
breath" has a strong negative connotation and often implies that
something is not going to happen at all
http://en.wiktionary.org/wiki/don%27t_hold_your_breath
It doesn't just mean "it'll take a long time". So Dan clarified what he
meant.
I'm not sure if this is on purpose, but you're sounding oddly
belligerent in this whole exchange and I don't quite understand why,
since content-wise this is all very useful.
Emiliano Heyns
And if you look at the etymology, you'll see that it says exactly what I said.
From my point of view, I listed a few technologies I was tinkering with, and the reason why I was doing so. Dan seemed to take exception to this, so I clarified why I was doing this. Belligerent? Not any more than Dan was, I suppose. Email is a limited medium through which to express sarcasm. I still don't see what the hoopla is about. More flexible sync will come at some point, and when it does, I'll tap into it. Until that time, and I do fully expect that to take a long time, for, again, entirely legitimate reasons, we'll have to make do with workarounds. Much more trivial changes took forever to get integrated; on just being realistic. Zotero doesn't move fast, and that's perfectly fine, and does right by the bulk of its users. I don't know how many times this must be repeated for it to sink in that I am not sarcastic about this.
--
You received this message because you are subscribed to a topic in the Google Groups "zotero-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/zotero-dev/alLe1Vr9Hns/unsubscribe.
To unsubscribe from this group and all its topics, send an email to zotero-dev+...@googlegroups.com.
Dan Stillman
On May 2, 2014 7:30 PM, "Dan Stillman" <dsti...@zotero.org> wrote:
>
>> 2. I am a heavy note-taker in my PDFs, and the left-or-right choice when two PDFs conflict during sync makes me sad. There's really no way for me to resolve the conflict at that point without data loss. Here' I prefer what dropbox et al do; notify me, and give me ample time to resolve the matter.
>
>
> I'm not sure what this has to do with anything else. Dropbox-style on-disk file conflict resolution has never been a design goal. If it were, we could implement it.Don't take things so personally.
> 2) SQLite is actually an excellent fit in many ways. For example, when we load the items list (which, for JS post-processing and sorting purposes, requires all values in the visible columns), we can pull out just the necessary fields. If we used a document store, we'd have to load every single document in the database and extract the necessary fields.That is simply not true. I don't know what document stores you're familiar with, but any nosql store can do map-reduce or similar. Pouch is, as I noted, unproven, but it includes an sql like query language in addition to MapReduce.
And if you look at the etymology, you'll see that it says exactly what I said.
Beyond that, like adamsmith I'm totally baffled by your tone here. Everyone else here is trying to have a discussion about specific technical solutions for improving Zotero. I assumed you also were, which is why I was providing clarifications about what is and isn't realistic in terms of future Zotero improvements. But now I'm not sure your goal isn't just to argue with people.
Emiliano Heyns
On 5/2/14, 1:57 PM, Emiliano Heyns wrote:Huh? I just provided a technical clarification that this is a totally separate issue from what the thread is about. File syncing doesn't even have anything to do with data syncing.
>
> I'm not sure what this has to do with anything else. Dropbox-style on-disk file conflict resolution has never been a design goal. If it were, we could implement it.Don't take things so personally.
Unless specific fields are indexed, which wouldn't be the case in the example I gave, MapReduce doesn't keep the database from having to load and parse every document in order to extract specific values. MapReduce is ideally parallelizable, but SQLite is almost certainly a better fit for this use case.
I think you're going to have to take this from a native English speaker — the phrase has a specific connotation, and one that was incorrect in the context you used it.And if you look at the etymology, you'll see that it says exactly what I said.
Beyond that, like adamsmith I'm totally baffled by your tone here. Everyone else here is trying to have a discussion about specific technical solutions for improving Zotero. I assumed you also were, which is why I was providing clarifications about what is and isn't realistic in terms of future Zotero improvements. But now I'm not sure your goal isn't just to argue with people.