Multilingual Zotero news
Frank Bennett
those brave souls who wish to test the experimental multilingual
version of Zotero. This version provides support for sorting on
alternative field keys (needed for Asian language support), and for
translations and transliterations in rendered citations.
Those wishing to run the multilingual version should take note of the
following:
Caveats
- This is not an official release of Zotero. It is a product of
branch development, and by installing it you effectively sign yourself
up as a member of a separate branch development team. The core Zotero
developers are not responsible for the code, and should not be
expected to provide support.
- Before running this version against any existing Zotero database,
special care should be taken to back up and protect your data. Local
backups of data should be taken regularly in any case, but that is
particularly true here. Close attention should be paid to any
unexpected or anomalous behavior. Problems should be reported to the
Zotero forums with the word "multilingual" in the subject header, so
that they will be noticed and picked up by other participants in
multilingual development.
Overview
- This version of multilingual Zotero is based on Zotero 2.1b4. As
development of the Zotero 2.1 branch proceeds, periodic updates to the
multilingual version will be announced here, but you will need to
install any such updates manually; the multilingual version will not
update automatically.
- The multilingual version is available only as a Firefox extension.
It should be possible to build a standalone version as well, but that
is left as an exercise for the user (please post a note here on zotero-
dev if you get one going).
- In contrast to the previous multilingual XPI release, this version
permits synchronization with the Zotero data server. Only "headline
data" (the non-multilingual fields used in ordinary Zotero items) will
be synchronized at present, however.
- When run against a database created with the previous multilingual
XPI version, multilingual data will be converted to a new storage
format. This is a complex process, and you should inspect entries
carefully after conversion to be sure no data is lost or corrupted.
- Some notes on how the multilingual interface works are available in
following document:
http://gsl-nagoya-u.net/http/pub/zotero-multilingual-overview.html
Link to version 2 multilingual XPI
http://gsl-nagoya-u.net/http/pub/zotero-multi-2.xpi
Enjoy!
Frank Bennett
Nagoya University
Frank Bennett
separate branches of the source code) I've resurrected a duplicates
detection interface that I put together at the end of last summer, and
added it to the zotero-multi-2.xpi (still at the same URL,
http://gsl-nagoya-u.net/http/pub/zotero-multi-2.xpi).
Duplicates screening is enabled by default, and you'll notice that all
items added via a web translator get a yellow highlight. You can
screen new items periodically by right-clicking (I think) over My
Library or a shared group, and selecting the "Duplicates view". This
will show only suspected duplicates, color-coded according to whether
they have no preexisting partner (color code red), have a preexisting
partner with which they might be merged (color code yellow), or have
no partner at all (color code green). The context menu over each item
gives a list of options appropriate to the state of the item.
Merging of items is currently only half-smart. Merging is blocked if
the selected item has attachments (so you must delete them by hand or
move them across manually if they are to be preserved). Merging to a
"master" item will move the master into any collections in which the
selected item existed. However, no reference link is preserved to the
selected item; it is simply deleted, and any documents that referenced
it will break.
That last issue should be fixed, but I'm putting the interface out
there in its current form so that people can take a look and form an
opinion as to whether it looks viable.
Frank
Avram Lyon
> As a little added extra (and because I'm too lazy to maintain two
> separate branches of the source code) I've resurrected a duplicates
> detection interface that I put together at the end of last summer, and
> added it to the zotero-multi-2.xpi (still at the same URL,
> http://gsl-nagoya-u.net/http/pub/zotero-multi-2.xpi).
This is starting to look like the souped-up branch of Zotero, and I'm
now planning on moving to it for my primary Zotero usage. I'd like to
start building multilingual support into translators as I write and
fix them. Is there a reliable flag I can use to detect whether the
Zotero instance is multilingual enabled? Perhaps a function in the
Zotero.Translate object that is defined?
It is also probably time to start seeing about getting Florian's MARC
work incorporated as well, using similar tricks to support
multilingual and monolingual.
Avram
Frank Bennett
> 2011/1/15 Frank Bennett <biercena...@gmail.com>:
> I'd like to
> start building multilingual support into translators as I write and
> fix them. Is there a reliable flag I can use to detect whether the
> Zotero instance is multilingual enabled? Perhaps a function in the
> Zotero.Translate object that is defined?
Zotero.Utilities.setMultiField() and
Zotero.Utilities.setMultiCreator(), which are used to add field and
creator data to the translator sandbox item. I'll write a little
manual when I can get to it, but it will probably just say something
like this:
setMultField(item, fieldName, value, langTag);
- item (Object) is a Zotero sandbox item instance.
- fieldName (String) is the name of the field to be updated.
- value (String) is the field content.
- langTag is a validatable language tag.
If the base field on the item is empty, the value will be set as the
base value, and any langTag value will be recorded as its language
(this doesn't currently turn up in the interface anywhere, but the
value will be stored). If the base field is non-empty, a value for
langTag must be provided.
setMultiCreator(baseCreator, newCreator, langTag, creatorType);
- baseCreator (Object) is an existing creator (which may be an empty
object -- see below).
- newCreator (Object) is a creator to be added to the base creator.
- langTag is a validatable language tag.
- creatorType (String) is the usual creatorType value used to specify
creators.
If the base creator has no lastName value, the newCreator object will
be used as the source of field content. The handling and constraints
of the langTag value are the same as for setMultiField().
For base values, the use of setMultiField() and setMultiCreator() is
not required; those values can be set directly in the usual way.
However, base values must be set for a given field or creator object
before adding multilingual content to it.
Hope it helps! If these helper functions need adjustment, or if there
are other utilities that will make the task of building multilingual
translators easier, just give a shout.
Frank
Frank Bennett
source, which can be used to build a multilingual standalone from an
SVN checkout under Linux. It seems to work with Google Chrome okay in
my environment here.
The script has runtime query options for the two Linux flavors and for
Windows. I wasn't sure how the xulrunner for MacOSX was meant to be
acquired, so that one is disabled for the present.
Just in case, the SVN repo is here:
https://www.zotero.org/svn/extension/branches/trunk-multilingual/
Frank
skornblith
SVNPATH in build_sa; you should be able to pass branches/trunk-
multilingual as the first argument to the script. At the moment, the
only way to get a build of xulrunner 2.0 for OS X is to copy the
bundle from the Zotero standalone distribution, or to apply the patch
for Mozilla bug 600435 and build it yourself. (See also
https://www.zotero.org/trac/wiki/Mozilla_bugs_affecting_Zotero).
Simon
Avram Lyon
When I install this over Zotero 2.1b4, I get an SQL error:
https://gist.github.com/781665
This causes the usual dead Zotero error:
http://www.gimranov.com/avram/research/zotero/multi-2-error.png .
It would appear that the database upgrade is choking.
Avram
2011/1/15 Frank Bennett <bierc...@gmail.com>:
> --
> You received this message because you are subscribed to the Google Groups "zotero-dev" group.
> To post to this group, send email to zoter...@googlegroups.com.
> To unsubscribe from this group, send email to zotero-dev+...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/zotero-dev?hl=en.
>
>
Avram Lyon
http://www.gimranov.com/research/zotero/multi-2-error.png
2011/1/16 Avram Lyon <ajl...@gmail.com>:
Frank Bennett
first here, but I'll take a look at the 2.1b4 source and rework the
schema update steps this evening to patch things up. Sorry about the
glitch; but happily it looks like a small thing.
Frank
Frank Bennett
Just checked in some changes that should get it going for you.
Frank
Avram Lyon
2011/1/16 Frank Bennett <bierc...@gmail.com>:
> Just checked in some changes that should get it going for you.
I'm now running from SVN, since it looks like the patch hasn't made it
into a new XPI. But the database upgrade works.
Most everything is working, except that one of my collections won't
display-- I can select it, and nothing appears in the middle pane,
just "Loading items list..." I can still create a report from it, and
it's certainly there. The only error of note is:
[JavaScript Error: "bCreators[bPos] is undefined" {file:
"chrome://zotero/content/xpcom/itemTreeView.js" line: 1146}]
Debug output for selecting that collection is at https://gist.github.com/782055
There are also issues with setting nicknames for languages -- they
don't seem to stick. I'll look into it some more and post a screencast
+ debug output if necessary.
Thanks again for pushing this along.
- Avram
Frank Bennett
> Most everything is working, except that one of my collections won't
> display-- I can select it, and nothing appears in the middle pane,
> just "Loading items list..." I can still create a report from it, and
> it's certainly there. The only error of note is:
>
> [JavaScript Error: "bCreators[bPos] is undefined" {file:
> "chrome://zotero/content/xpcom/itemTreeView.js" line: 1146}]
have two or more creators, the first of which are equal, and none of
which are author, editor, or contributor. That seems to cause the
first-cut sort key (sortCreators, in multilingual, which replaces
firstCreator for generating hidden sort keys, in multilingual) to show
a match, and invokes more fine-grained names resolution code recently
added by Dan. That code does not pick up the list position of
creators that are non-author, non-editor, and non-contributor, and you
get a list position of "false", which produces the undefined error.
It sounds like this condition cannot arise in 2.1b4, so I'll take a
look at that code and map a fix into the multilingual version. Should
be better soon.
Frank
Frank Bennett
This turned out to be due to a sloppy table join, in the SQL that
generates the sortCreator key. I've checked in a fix, generated a
fresh copy of zotero-multi-2.xpi, and checked that the fix works in
the multilingual standalone. See how it goes for you.
One thing to watch is that there is a schema version discrepancy
between the xpi and generated standalone, and the vanilla svn. The
build scripts for the xpi and standalone clients mixes in the
duplicates patch, which bumps the schema version a step, to add a
table for tracking newly added items. If you get a version mismatch
error when running the vanilla svn source against a database built by
the xpi or standalone, it's safe to just decrement the schema version
a step (to 79). Repeating the upgrade won't hurt anything, at least
for testing purposes; it will just wipe out any record of newly added
entries from that special table.
On the nicknames issue, they don't show up immediately in the item
data panel for me (although they should do), but they are recorded
okay, and take effect when Zotero is restarted. Do you get the same
behavior?
Frank
Avram Lyon
2011/1/17 Frank Bennett <bierc...@gmail.com>:
> This turned out to be due to a sloppy table join, in the SQL that
> generates the sortCreator key. I've checked in a fix, generated a
> fresh copy of zotero-multi-2.xpi, and checked that the fix works in
> the multilingual standalone. See how it goes for you.
> ..
> One thing to watch is that there is a schema version discrepancy
> between the xpi and generated standalone, and the vanilla svn. The
> ..
I noticed that, so I really didn't want to use the vanilla SVN.
> On the nicknames issue, they don't show up immediately in the item
> data panel for me (although they should do), but they are recorded
> okay, and take effect when Zotero is restarted. Do you get the same
> behavior?
Some nicknames disappeared when I added a language variant based on
them; the nicknames reverted to what they previously had been (not
necessarily to the language tag-- in one case I had renamed the
nickname twice, then it lost its most recent name). If you can't
reproduce this, I'll try to do it again and produce debug output for
it.
Finally, and on a somewhat separate note, I see that new items are
marked as unreviewed, but I don't see how to run the duplicate
detection system to deal with them.
Avram
Frank Bennett
> Frank,
>
> 2011/1/17 Frank Bennett <biercena...@gmail.com>:
> > One thing to watch is that there is a schema version discrepancy
> > between the xpi and generated standalone, and the vanilla svn. The
> > ..
>
> I noticed that, so I really didn't want to use the vanilla SVN.
>
> > On the nicknames issue, they don't show up immediately in the item
> > data panel for me (although they should do), but they are recorded
> > okay, and take effect when Zotero is restarted. Do you get the same
> > behavior?
>
> Some nicknames disappeared when I added a language variant based on
> them; the nicknames reverted to what they previously had been (not
> necessarily to the language tag-- in one case I had renamed the
> nickname twice, then it lost its most recent name). If you can't
> reproduce this, I'll try to do it again and produce debug output for
> it.
>
> Finally, and on a somewhat separate note, I see that new items are
> marked as unreviewed, but I don't see how to run the duplicate
> detection system to deal with them.
option to show the duplicates view. It will show the items in color-
coded form. The context menu over each item gives a set of duplicate
handling options appropriate to its status.
You might want to reload zotero-multi-2.xpi. I've just made some
changes to citeproc-js to allow the display of transliterated author
names in the original script, and tied it into the UI. This one
pretty well rounds out the feature set for the present, I think.
Frank
>
> Avram
Frank Bennett
awhile back, that works in a way similar to the duplicates view. It
was very helpful when I had it running, so I'll probably resurrect
that and stir it in as well.
>
> Frank
>
>
>
> > Avram
Frank Bennett
I have come across an item here that strangely hangs the browser when
a particular Japanese value is entered into the Publisher field. I'm
not at all sure of the cause: it might be a bug in my code, or it
might be a bug in Firefox. Possibly it has nothing at all to do with
multilingual processing. It's triggered even when there is only that
one entry in the library, and only one reference in the target
document. That, plus the fact that disabling a bug-workaround to cope
with locale breakage affecting some Japanese sorts, leads me to
suspect that it is not connected with sorting. A very puzzling one.
More news when I figure this one out, but if you run across any
similar symptoms, do give a shout.
Frank
Frank Bennett
Please reinstall the client from the zotero-multi-2.xpi.
Frank
>
> Frank
Avram Lyon
>> Some nicknames disappeared when I added a language variant based on
>> them; the nicknames reverted to what they previously had been (not
>> necessarily to the language tag-- in one case I had renamed the
>> nickname twice, then it lost its most recent name). If you can't
>> reproduce this, I'll try to do it again and produce debug output for
>> it.
>
> Thanks. Fingers crossed.
Don't have this one yet, but my usage today revealed the following issue:
1. Create a new item.
2. Add an author.
3. Press shift-enter to get another entry line. Nothing happens.
4. Click + to get another line. New line appears, but in single-field mode!
5. After entering a name, text entry fields stop defocusing, the entry
is broken.
6. Select another collection and you'll be presented with the message
that an error has occurred and Firefox will need to be restarted.
I don't have debug output for this yet, but the errors are at
https://gist.github.com/784712 . Since my description above isn't
terribly clear, you can watch my low-quality screencast to see it in
action at http://www.gimranov.com/research/zotero/multi-fields-error.ogv
I also am coming to believe more firmly that it should be possible to
set the language of each headline item entry; it just feels wrong to
not have that information when I'm so careful with everything else.
Also, it will matter for transliteration when the time comes.
Avram
Avram Lyon
> Don't have this one yet, but my usage today revealed the following issue:
> I don't have debug output for this yet, but the errors are at
> https://gist.github.com/784712 . Since my description above isn't
> terribly clear, you can watch my low-quality screencast to see it in
> action at http://www.gimranov.com/research/zotero/multi-fields-error.ogv
There was one more error that I ran into while at the library today--
I saved the error report and posted it at
https://gist.github.com/784723, but I can't recall precisely what
triggered it. I think it had to do with setting creator roles, but I
haven't tried to replicate it yet.
Avram
Avram Lyon
> There was one more error that I ran into while at the library today--
And now one more:
[JavaScript Error: "libraryID is not defined" {file:
"chrome://zotero/content/xpcom/data/creator.js" line: 378}]
This is when I attempt to drag an item to a group library. Loss of
groups support might be a deal-breaker in my mad attempt to use
experimental software in my real research life.
Avram
Frank Bennett
reinstalling zotero-multi-2.xpi.
>
> Avram
Frank Bennett
of zotero-multi-2.xpi, uploaded a short while ago.
The database and JS data layer support language tags on the main
creator and text field entries; it just needs to be hooked up to the
UI (probably with a little debugging). I'll set that up after clearing
the issue with keypress creator entry, etc.
Frank
>
> Avram
Frank Bennett
everything in the thread so far, apart from setting language tags on
may text and creator fields. I need this also, and have a plan for
what I think will be a comprehensive solution. Should be reading in a
few days.
Separately, I am thinking that, if we have good control over language
tags, there is no need to distinguish name and title
transliterations. What would you say to merging them to just a
"Transliteration" heading in the preferences menu, to companion with
"Translation", "Sort" and "Zotero UI"?
FB
Avram Lyon
I am glad to report that the build is working nicely for me now,
although I haven't been working the multilingual parts too hard yet.
2011/1/19 Frank Bennett <bierc...@gmail.com>:
> Separately, I am thinking that, if we have good control over language
> tags, there is no need to distinguish name and title
> transliterations. What would you say to merging them to just a
> "Transliteration" heading in the preferences menu, to companion with
> "Translation", "Sort" and "Zotero UI"?
That sounds good to me.
My most recent issue is a very small one, and I'm not sure that it's
specific to the branch; when I ask Zotero to start up in debug mode,
it starts up without debug output and the checkbox is not checked.
Avram
Frank Bennett
> Frank and all,
>
> I am glad to report that the build is working nicely for me now,
> although I haven't been working the multilingual parts too hard yet.
fresh copy of zotero-multi-2.xpi that includes it. It works the same
as the duplicates view, from a right-click over the target library or
group. As with duplicates, it can be canceled explicitly with another
right-click selection, or by clicking on any other collection or
group.
Will now start looking into sprucing up language tag management, with
support for setting a language on the main entry.
Frank
Frank Bennett
I've just put up a fresh build of the xpi, at
http://gsl-nagoya-u.net/http/pub/zotero-multi-2.xpi
The new build fixes a problem that I stumbled across when giving a
fresh install a try: the UI in the item pane was breaking when no
language settings were defined. Oops. It also deploys a completely new
(and, I hope, sufficiently comprehensive) set of menus for managing
language tags and multilingual field data. There are some small
changes, but you should be able to figure out how things work with a
bit of right- and left-clicking over the field labels.
The client install will upgrade the database another notch. The
upgrade process scrubs the two SQL tables used to store language tags
for headline entries. Be sure (as always) to have a backup to hand,
but the affected tables weren't hooked up to the UI before, so nothing
should be lost.
Lots of new code in this release. If you notice further breakage, let
me know. I'll try to look into the sorting problem and implement
language-selectable center-pane listings for titles and creators
sometime in the coming week or so.
Frank