[okapi-devel] Pensieve REST API

158 views
Skip to first unread message

Yves

unread,
May 8, 2010, 10:41:54 AM5/8/10
to okapi-devel
I like the potential notation you show in the wiki, Asgeir.

A few notes:

-- Wouldn't it be more logical to place the 'search' first? as:
GET /search/{tm}/{sourceLang}/{targetLang}/?q=etc...

-- What is the format of the text queried? Plain text, or a single
simple tagged format?
If we use an abstract tagging: Wouldn't this affect the match quality?
as, I think, there is a penalty for different inline codes. I'm just
not sure if it goes to the level of the original code.

-ys

Asgeir Frimannsson

unread,
May 9, 2010, 12:51:53 AM5/9/10
to okapi...@googlegroups.com
On Sun, May 9, 2010 at 12:41 AM, Yves <yv...@opentag.com> wrote:
I like the potential notation you show in the wiki, Asgeir.

A few notes:

-- Wouldn't it be more logical to place the 'search' first? as:
GET /search/{tm}/{sourceLang}/{targetLang}/?q=etc...

Yeah, perhaps. I think my motivation for this was to have all operations for a single TM under the same url namespace. 

Create a TM:
PUT /{tm}

Import e.g. a TMX:
POST /{tm}

I'm wondering about the implications of multiple physical TMs though. I think it would be more practical to have a single lucene index and in some way 'shard' the index based on a field which is the TM id. Any thoughts?

With a single physical TM it would be possible to also easily search across multiple TMs, something like:

/{tm1};{tm1};{tm3}/search/...
 
-- What is the format of the text queried? Plain text, or a single
simple tagged format?
If we use an abstract tagging: Wouldn't this affect the match quality?
as, I think, there is a penalty for different inline codes. I'm just
not sure if it goes to the level of the original code.

hm... need to chew on this one.

cheers,
asgeir

Asgeir Frimannsson

unread,
May 9, 2010, 1:23:41 AM5/9/10
to okapi...@googlegroups.com
Hi all,

I've imported an initial version of a REST server to ./okapi/tm/pensieve-server.

You can run it through 'mvn tomcat:run-war' which will launch it in an embedded tomcat server. ..or you can configure a tomcat 6.x server in eclipse. ...or you can do 'mvn package' and deploy ./target/pensieve.war to your tomcat instance.

You can then navigate to:


cheers,
asgeir

Yves Savourel

unread,
May 9, 2010, 9:48:36 AM5/9/10
to okapi...@googlegroups.com
> You can run it through 'mvn tomcat:run-war' which will
> launch it in an embedded tomcat server. ..or you can
> configure a tomcat 6.x server in eclipse. ...or you
> ...
Thanks for working on this Asgeir.

I've tried to run the example using 'mvn tomcat:run-war':

Everything compiles fine, and (after Maven amuses itself by downloading about half of the Web on my machine) the Coyote server seems to start. However, there is an exception:

SEVERE: Exception sending context initialized event to listener instance of class org.jboss.resteasy.plugins.server.servlet.ResteasyBootstrap.java.lang.RuntimeException: Unable to instantiate MessageBodyReader

And any call to http://localhost:8080/search/EN-US/FR-FR/?q=Elephants%20cannot%20fly. or http://localhost:8080/ or any variation get me a 404: no such resource.

:(

I've also tried to deploy the war file on GlassFish, the default server that comes with JDK 5 EE. The deployment seems to work fine, but any call to the service results either in a 404 (resource not available) or 503 (service not available). I assume the war should work with any server, right?

-ys




From: okapi...@googlegroups.com [mailto:okapi...@googlegroups.com] On Behalf Of Asgeir Frimannsson
Sent: Saturday, May 08, 2010 11:24 PM
To: okapi...@googlegroups.com
Subject: Re: [okapi-devel] Pensieve REST API

Hi all,

I've imported an initial version of a REST server to ./okapi/tm/pensieve-server.

Yves Savourel

unread,
May 9, 2010, 11:33:59 AM5/9/10
to okapi...@googlegroups.com
> I'm wondering about the implications of multiple physical
> TMs though. I think it would be more practical to have a
> ...
> With a single physical TM it would be possible to also
> easily search across multiple TMs, something like:
> /{tm1};{tm1};{tm3}/search/...

+1

The simple current attribute support that should make this relatively easy.

-ys

Yves

unread,
May 9, 2010, 1:08:02 PM5/9/10
to okapi-devel
> And any call tohttp://localhost:8080/search/EN-US/FR-FR/?q=Elephants%20cannot%20fly. or
> http://localhost:8080/or any variation get me a 404: no such resource.
> ...
> I've also tried to deploy the war file on GlassFish, the default server that comes with JDK 5 EE.
> The deployment seems to work fine, but any call to the service results either in a 404 (resource
> not available) or 503 (service not available). I assume the war should work with any server, right?

I've tried on Ubuntu. I'm getting a few warnings only, when starting
the the server:

INFO: Starting Servlet Engine: Apache Tomcat/6.0.26
159 [main] INFO org.jboss.weld.Version - WELD-000900 1.0.1 (Final)
303 [main] INFO org.jboss.weld.Bootstrap - WELD-000101 Transactional
services not available. Injection of @Inject UserTransaction not
available. Transactional observers will be invoked synchronously.
856 [main] WARN org.jboss.interceptor.model.InterceptionTypeRegistry -
Class 'javax.ejb.PostActivate' not found, interception based on it is
not enabled
857 [main] WARN org.jboss.interceptor.model.InterceptionTypeRegistry -
Class 'javax.ejb.PrePassivate' not found, interception based on it is
not enabled

But they are not critical, because the queries are working :-)
It's strange that it's not working on Windows. I don't run the same
VM, maybe that's the problem.

-ys

Yves

unread,
May 9, 2010, 5:31:02 PM5/9/10
to okapi-devel
Continuation of pensieve-server testing:

I've tried to run under another Linux server (One that serves our MTs,
a Hardy Heron running with XFCE) and when the server starts I get:

SEVERE: Error configuring application listener of class
org.jboss.weld.environment.servlet.Listener
java.lang.NoClassDefFoundError: org/slf4j/ext/XLoggerFactory

then the start continues.
But, as for Windows, invoking the search service results in 404.

-ys

Asgeir Frimannsson

unread,
May 9, 2010, 5:52:34 PM5/9/10
to okapi...@googlegroups.com
strange, I just tested in on Win7 and get the same error. I'll investigate.

cheers,
asgeir

Jim Hargrave

unread,
May 10, 2010, 12:30:36 PM5/10/10
to okapi...@googlegroups.com
a single tm as it grows larger tends to not be as performant as
multiple tms - but this is based on my experiencd with db
implementations - perhaps lucene is different

for search its the size of the index that matters along with the
number of search terms - so with a single very large index it could be
slower than searching multiple small index especially if we throw in
threading

Yves

unread,
May 10, 2010, 12:51:22 PM5/10/10
to okapi-devel
Things are working (with warnings at start time) on linux know, thanks
to the addition of the missing dependency.

But I still have the error about "Unable to instantiate
MessageBodyReader" on Windows tough.
Does anyone tried a "mvn tomcat:run-war" and got things working on
Windows?

The problem seems to come from:

Caused by: java.lang.RuntimeException: Illegal to inject a message
body into a singleton
at
org.jboss.resteasy.core.MessageBodyParameterInjector.inject(MessageBodyParameterInjector.
java:157)

For reference I've copied the trace I get below.

I've delete most of the local .m2 repository to be sure to get fresh
dependencies, but without any success.

--- trace:

May 10, 2010 9:53:06 AM org.apache.catalina.core.StandardContext
listenerStart
SEVERE: Exception sending context initialized event to listener
instance of class
org.jboss.resteasy.plugins.server.servlet.ResteasyBootstrap
java.lang.RuntimeException: Unable to instantiate MessageBodyReader
at
org.jboss.resteasy.spi.ResteasyProviderFactory.registerProvider(ResteasyProviderFactory.java:
728)
at
org.jboss.resteasy.spi.ResteasyProviderFactory.registerProvider(ResteasyProviderFactory.java:
710)
at
org.jboss.resteasy.spi.ResteasyDeployment.registerProvider(ResteasyDeployment.java:
371)
at
org.jboss.resteasy.spi.ResteasyDeployment.registration(ResteasyDeployment.java:
249)
at
org.jboss.resteasy.spi.ResteasyDeployment.start(ResteasyDeployment.java:
185)
at
org.jboss.resteasy.plugins.server.servlet.ResteasyBootstrap.contextInitialized(ResteasyBootstrap.java:
28)
at
org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:
3972)
at
org.apache.catalina.core.StandardContext.start(StandardContext.java:
4467)
at
org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
at
org.apache.catalina.core.StandardHost.start(StandardHost.java:785)
at
org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
at
org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
at org.apache.catalina.startup.Embedded.start(Embedded.java:
825)
at
org.codehaus.mojo.tomcat.AbstractRunMojo.startContainer(AbstractRunMojo.java:
533)
at
org.codehaus.mojo.tomcat.AbstractRunMojo.execute(AbstractRunMojo.java:
239)
at
org.apache.maven.plugin.DefaultPluginManager.executeMojo(DefaultPluginManager.java:
490)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoals(DefaultLifecycleExecutor.java:
694)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeStandaloneGoal(DefaultLifecycleExecutor.java:
569)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoal(DefaultLifecycleExecutor.java:
539)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeGoalAndHandleFailures(DefaultLifecycleExecutor.java:
387)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.executeTaskSegments(DefaultLifecycleExecutor.java:
348)
at
org.apache.maven.lifecycle.DefaultLifecycleExecutor.execute(DefaultLifecycleExecutor.java:
180)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:
328)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:
138)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:362)
at
org.apache.maven.cli.compat.CompatibleMain.main(CompatibleMain.java:
60)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:
25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.codehaus.classworlds.Launcher.launchEnhanced(Launcher.java:315)
at org.codehaus.classworlds.Launcher.launch(Launcher.java:255)
at
org.codehaus.classworlds.Launcher.mainWithExitCode(Launcher.java:430)
at org.codehaus.classworlds.Launcher.main(Launcher.java:375)
Caused by: java.lang.RuntimeException: Illegal to inject a message
body into a singleton
at
org.jboss.resteasy.core.MessageBodyParameterInjector.inject(MessageBodyParameterInjector.java:
157)
at
org.jboss.resteasy.core.ConstructorInjectorImpl.injectableArguments(ConstructorInjectorImpl.java:
63)
at
org.jboss.resteasy.core.ConstructorInjectorImpl.construct(ConstructorInjectorImpl.java:
129)
at
org.jboss.resteasy.spi.ResteasyProviderFactory.getProviderInstance(ResteasyProviderFactory.java:
1002)
at
org.jboss.resteasy.spi.ResteasyProviderFactory.addMessageBodyReader(ResteasyProviderFactory.java:
460)
at
org.jboss.resteasy.spi.ResteasyProviderFactory.registerProvider(ResteasyProviderFactory.java:
724)
... 33 more

--- end trace

Asgeir Frimannsson

unread,
May 10, 2010, 3:53:08 PM5/10/10
to okapi...@googlegroups.com
Yes, I'm getting this on Windows as well, I didn't have time to chase it down last night (Don't have a Windows machine at work to try it on!). It seems to be a problem with the json provider. 

cheers,
asgeir

Asgeir Frimannsson

unread,
May 10, 2010, 5:45:28 PM5/10/10
to okapi...@googlegroups.com
Yves,

Could you confirm that the latest commit fixes the deployment problem on Windows.

You should still see a few warnings (which are there because I didn't include some dependencies in order to save us from downloading the last 1% of the Internet.).

cheers,
asgeir

Yves Savourel

unread,
May 10, 2010, 6:06:37 PM5/10/10
to okapi...@googlegroups.com
> Could you confirm that the latest commit
> fixes the deployment problem on Windows.

It works beautifully now :)


> You should still see a few warnings (which
> are there because I didn't include some
> dependencies in order to save us from downloading
> the last 1% of the Internet.).

Much appreciated :)
I'll work on debugging the connector.
-ys


Reply all
Reply to author
Forward
0 new messages