Translators location for running them on translation-server

[Sonali Gupta]

I am trying to write translators and test them on translation-server. What will be the place to make changes in translators to see the effect? Running the command
docker run --rm -p 1969:1969 translation-server
to run the container show Loading translators from /opt/translation-server/app/translators but I am unable to understand where should I add new translator files locally and test them. I have my clone of translators repo at a location and I tried adding that location in cofing.js as
pref("translation-server.translatorsDirectory", "complete address of the translator directory");
but the docker run command shows no such directory for that path. Maybe this way only works for the manual way of development. Moreover, for the command to test changes without rebuilding the container
./build.sh && docker run -ti --rm -v `pwd`/build/app/:/opt/translation-server/app/ translation-server
gives a long list of error as follows.
cp: cannot stat ‘/home/sonali/Videos/translation-server/modules/zotero/build/chrome/content/zotero/xpcom/rdf’: No such file or directory
cp: cannot stat ‘/home/sonali/Videos/translation-server/modules/zotero/build/chrome/content/zotero/xpcom/citeproc-prereqs.js’: No such file or directory
cp: cannot stat ‘/home/sonali/Videos/translation-server/modules/zotero/build/chrome/content/zotero/xpcom/citeproc.js’: No such file or directory
cp: cannot stat ‘/home/sonali/Videos/translation-server/modules/zotero/build/chrome/content/zotero/xpcom/cookieSandbox.js’: No such file or directory
cp: cannot stat ‘/home/sonali/Videos/translation-server/modules/zotero/build/chrome/content/zotero/xpcom/date.js’: No such file or directory
cp: cannot stat ‘/home/sonali/Videos/translation-server/modules/zotero/build/chrome/content/zotero/xpcom/debug.js’: No such file or directory
cp: cannot stat ‘/home/sonali/Videos/translation-server/modules/zotero/build/chrome/content/zotero/xpcom/file.js’: No such file or directory
cp: cannot stat ‘/home/sonali/Videos/translation-server/modules/zotero/build/chrome/content/zotero/xpcom/http.js’: No such file or directory
cp: cannot stat ‘/home/sonali/Videos/translation-server/modules/zotero/build/chrome/content/zotero/xpcom/openurl.js’: No such file or directory
cp: cannot stat ‘/home/sonali/Videos/translation-server/modules/zotero/build/chrome/content/zotero/xpcom/server.js’: No such file or directory
cp: cannot stat ‘/home/sonali/Videos/translation-server/modules/zotero/build/chrome/content/zotero/xpcom/utilities.js’: No such file or directory
cp: cannot stat ‘/home/sonali/Videos/translation-server/modules/zotero/build/chrome/content/zotero/xpcom/utilities_internal.js’: No such file or directory
cp: cannot stat ‘/home/sonali/Videos/translation-server/modules/zotero/build/chrome/content/zotero/xpcom/utilities_translate.js’: No such file or directory
cp: cannot stat ‘/home/sonali/Videos/translation-server/modules/zotero/build/chrome/content/zotero/xpcom/xregexp’: No such file or directory
cp: cannot stat ‘/home/sonali/Videos/translation-server/modules/zotero/build/chrome/content/zotero/tools/testTranslators/translatorTester.js’: No such file or directory

I changed (as I found it suitable)
EXTENSION_DIR="$SCRIPT_DIR/modules/zotero/build" to EXTENSION_DIR="$SCRIPT_DIR/modules/zotero"
in the build.sh file and this was finally successful when I ran the build.sh again. But then
docker run -ti --rm -v `pwd`/build/app/:/opt/translation-server/app/ translation-server
throws the error
JavaScript error: resource://zotero/bluebird.js, line 5: Error: Module `bluebird/promise` is not found at resource://zotero/bluebird/promise.js

I want to know where to add the location of the translators directory in which I am making changes locally such that they are reflected in the docker container or how to modify the docker run command to let the container pick that directory.

[Dan Stillman]

On 8/9/17 7:59 PM, Sonali Gupta wrote:

Moreover, for the command to test changes without rebuilding the container
./build.sh && docker run -ti --rm -v `pwd`/build/app/:/opt/translation-server/app/ translation-server
gives a long list of error as follows.

You have to compile the client code in modules/zotero first. I've updated the README to make this clearer.

You could then edit translators in modules/zotero/translators and re-run the line above.

If you just want to edit translators, though, you could skip the client build part and mount a host translators directory on top of /opt/translation-server/app/translators with -v.

[Sonali Gupta]

./build.sh && docker run -ti --rm -v `pwd`/build/app/:/opt/translation-server/app/ translation-server



Running the docker container after making changes and rebuilding with this command did work but this method doesn't allow to make a query through curl outside the container. So one needs to enter root@id:opt/translation-server through

docker exec -it id /bin/bash



and use apt-get update and install curl every time to test a change. Moreover the changes modules/zotero/translators which updates the in build/app/translators aren't taking effect, the translation is taking place through Embedded Metadata instead of the translator for an example url. 





Sonali Gupta

Vice Chairperson- Computer Society of India,Student Chapter

member, The Literary Committee

Undergraduate- Computer Science

The LNM Institute of Information Technology, Jaipur

contact- +91-9462900363

--
You received this message because you are subscribed to the Google Groups "zotero-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to zotero-dev+unsubscribe@googlegroups.com.
To post to this group, send email to zoter...@googlegroups.com.
Visit this group at https://groups.google.com/group/zotero-dev.
For more options, visit https://groups.google.com/d/optout.

[Dan Stillman]

On 8/14/17 6:12 PM, Sonali Gupta wrote:

./build.sh && docker run -ti --rm -v `pwd`/build/app/:/opt/translation-server/app/ translation-server

Running the docker container after making changes and rebuilding with this command did work but this method doesn't allow to make a query through curl outside the container.

It does — it just needs the `-p 1969:1969` from the other example to bind the port. I'll add that to the README.

[Sonali Gupta]

Thank you for the help. If anyone can find time to reproduce this issue that I am facing in testing changes in translators and can let know how to make it work, it would be great. Following are the steps:

1) Get the updated translator code from this PR.
2) Overwrite TV by the numbers.js present at translation-server/modules/zotero/translators and save it.

3) Update the image and docker container and run the server with ./build.sh && docker run -p 1969:1969 -ti --rm -v `pwd`/build/app/:/opt/translation-server/app/ translation-server

4) Pass the query
curl -d '{"url":"http://tvbythenumbers.zap2it.com/daily-ratings/monday-final-ratings-july-3-2017/","sessionid":"abc123"}' \
       --header "Content-Type: application/json" \
       127.0.0.1:1969/web

5) Compare the translation output provided by server with the output generated by Scaffold as provided (Since these test cases are generated through Scaffold) and notice the differences. The translation output on console I am receiving is:

[{"itemKey":"ILYUXUHT","itemVersion":0,"itemType":"blogPost","creators":[],"tags":[],
"title":"‘American Ninja Warrior’ adjusts up: Monday final ratings",
"date":"2017-07-06T19:14:36+00:00",
"url":"http://tvbythenumbers.zap2it.com/daily-ratings/monday-final-ratings-july-3-2017/",
"abstractNote":"Final broadcast primetime live + same-day ratings for Monday, July 3, 2017
 The top show on a rerun-filled Monday night saw its adults 18-49 rating grow from the preliminary 
numbers to the finals. &…","accessDate":"CURRENT_TIMESTAMP","shortTitle":"‘American Ninja Warrior’
 adjusts up"}]

Doubt: Is there any error in the approach mentioned above that is causing different outputs?

Sonali Gupta

Vice Chairperson- Computer Society of India,Student Chapter

member, The Literary Committee

Undergraduate- Computer Science

The LNM Institute of Information Technology, Jaipur

contact- +91-9462900363

--

[Dan Stillman]

On 8/18/17 8:24 AM, Sonali Gupta wrote:

5) Compare the translation output provided by server with the output generated by Scaffold as provided (Since these test cases are generated through Scaffold) and notice the differences. The translation output on console I am receiving is:

[{"itemKey":"ILYUXUHT","itemVersion":0,"itemType":"blogPost","creators":[],"tags":[],
"title":"‘American Ninja Warrior’ adjusts up: Monday final ratings",
"date":"2017-07-06T19:14:36+00:00",
"url":"http://tvbythenumbers.zap2it.com/daily-ratings/monday-final-ratings-july-3-2017/",
"abstractNote":"Final broadcast primetime live + same-day ratings for Monday, July 3, 2017
 The top show on a rerun-filled Monday night saw its adults 18-49 rating grow from the preliminary 
numbers to the finals. &…","accessDate":"CURRENT_TIMESTAMP","shortTitle":"‘American Ninja Warrior’
 adjusts up"}]

OK, yeah, this was broken — it was only using generic translators (in this case Embedded Metadata). Try again with the latest commit.

(We haven't pushed a new build of translation-server to Docker Hub yet since the 5.0 upgrade, and the one on Docker Hub is still the recommended one for production use (though this might fix many of the tests that were broken in the 5.0 version that we hadn't yet looked into, so thanks!). You can always compare against the Docker Hub build by pulling that image and overlaying an updated translators directory on top of /opt/translation-server/app/ with -v.)