Session OpenRefine

[Massimo Imparato]

Hi all,

I hope someone can help me; I wish I know how OpenRefine manage session objects.

It's very important because I must porting Refine from jetty to Apache

Thank in advance

www.eng.it

Max

[Tom Morris]

On Fri, Nov 15, 2013 at 5:31 AM, Massimo Imparato <massimo....@gmail.com> wrote:

Hi all,

I hope someone can help me; I wish I know how OpenRefine manage session objects.

It's very important because I must porting Refine from jetty to Apache

I'm not sure I understand the question.  What problem are you running into?  Does Apache mean Apache Tomcat or something else?

All server dependencies should be isolated to the source tree rooted at server/src which is pretty small.  The Refine server is mostly stateless, so doesn't use sessions.  Long running operations are managed internally with IDs that the client can use to poll to find out status.

If you run into a problem and have a specific question, let us know.

Tom

[Massimo Imparato]

Thanks so much for response.

I must use OpenRefine like webapp and not like standalone application.

I m looking for some advice or tutorial.

How can change port 3333 to 8080

And Ho can port my openRefine under Apache Tomcat?

[Sergio Fernández]

I'd be willing to help working on that. For us (LMF) we are running
Refine in Tomcat, that's why we created the WAR:

http://markmail.org/message/zkodesocwzkaeui4

And there we are completely missing any session concept, and would be
nice to work on that direction.

The first think I'll do, hopefully by Monday, is update the WAR to the
current version.

Cheers,

> --
> You received this message because you are subscribed to the Google Groups
> "OpenRefine Development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to openrefine-de...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.



--

Sergio Fernández <ser...@wikier.org>

[Tom Morris]

Hi Sergio.  I'd forgotten that you had already done work on this.  I know I integrated your patch for relative path addressing cleanups.  Is there anything else missing?  If folks want to create a branch to host this work, I'd be happy to look at integrating any necessary changes.

Tom

[Tom Morris]

p.s. I don't suppose we could talk you into maintaining a mirror on Github (perhaps using this technique https://blog.glyphobet.net/essay/2029).  That would make it a lot easier to integrate changes.

[Massimo Imparato]

Thanks a lot!

Thank Sergio and thank Tom, I will study your patch

I will write you beetwen few days

THANKS

Il giorno venerdì 15 novembre 2013 16:15:13 UTC+1, Tom Morris ha scritto:

p.s. I don't suppose we could talk you into maintaining a mirror on Github (perhaps using this technique https://blog.glyphobet.net/essay/2029 ).  That would make it a lot easier to integrate changes.

[Sergio Fernández]

Long time... so I'd need to take a look what we did there:

https://bitbucket.org/wikier/google-refine

Because I remember you applied many patches that appears on those days
regarding changing the port or the context name. But all the maven
stuff not, and the WAR was mainly based on it.

So, I'll work on porting all necessary stuff on my fork at github, and
I'll come back to you as soon as I have it running again.

Have a nice weekend!

Cheers,

[Sergio Fernández]

Hi,

finally I found some time for porting the work done few months ago to
the current version of OpenRefine.

Using maven as build system, wrapping the current ant build, you are
able to run it embedded in tomcat7 or package a proper WAR archive, by
respectively executing:

mvn tomcat7:run
mvn package

Currently you can find it available in my fork at github:

http://github.com/wikier/OpenRefine

because I don't want to push a pull request until we clarified some issues:

* I'm not a fan of ant, so we'd need to agree on a migration to maven
at some point or re-implement the war packaging using that build
system. Personally I think it could bring some benefits, but of course
that is something to discuss.

* One of the reasons I do not like ant it's the dependencies issue.
Right now we have the binaries of all dependencies in the source code
repository. And, for instance, I realized the code does not compile
anymore against the latest snapshot of opencsv (commit
36a9c354f498be32bd4c1b7084432b35a2d910fe) . So this would one of the
things to improve in upcoming versions.

* The gzip compression is now using jetty's implementation. I'd need
to know if there is a portable way to do it.

* I have an issue with the relative path to the extensions, that's why
I have to patch butterfly.properties when packaging (commit
1aed66b47a6c1f103899b8a10979d748264c0604)

And for the moment that's it. Feedback is welcomed.

Best,

--
Sergio Fernández
http://redlink.co

[Tom Morris]

Thanks for updating this, Sergio!

On Mon, Nov 18, 2013 at 6:40 AM, Sergio Fernández <ser...@wikier.org> wrote:

* I'm not a fan of ant, so we'd need to agree on a migration to maven
at some point or re-implement the war packaging using that build
system. Personally I think it could bring some benefits, but of course
that is something to discuss.

Maven's dependency management is magical when it works and a nightmarish hell when it doesn't.  I'm using it for another project and I don't think I've ever done a clean build from scratch in a single try.  Usually it's a matter of retrying the same command several times to get around ibibio's arbitrary blocking requests, but other times it involves finding and manually download jars.

I'm willing to consider switching to it, but it's not a clear win in my mind.

 

* One of the reasons I do not like ant it's the dependencies issue.
Right now we have the binaries of all dependencies in the source code
repository. And, for instance, I realized the code does not compile
anymore against the latest snapshot of opencsv (commit
36a9c354f498be32bd4c1b7084432b35a2d910fe) . So this would one of the
things to improve in upcoming versions.

"Lastest" is a bit of a tricky concept in a distributed version control system.  OpenRefine 2.6 is build against a jar built from the tip of https://github.com/OpenRefine/opencsv/tree/openrefine-2.6 (although it should really have a release tag).  I'm not sure there's a way to do this kind of forked private build in Maven.

* The gzip compression is now using jetty's implementation. I'd need
to know if there is a portable way to do it.

I'd have to look at what you're using it for, but there's obviously Java's native Gzip implementation available on all JVMs if it's needed for something internal and web servers can can be configured to use (or not Gzip) if it's just for HTTP transport purposes.

 

* I have an issue with the relative path to the extensions, that's why
I have to patch butterfly.properties when packaging (commit
1aed66b47a6c1f103899b8a10979d748264c0604)

I'll try looking at this again, but from a glance at Github, it looks like an entirely new file, so it's hard to tell what the issue is or what you had to change (although I certainly would believe there are additional relative path issues lurking).

Thanks again for doing all this work Sergio!  I'd love to get everything integrated if it all possible.

Tom 

[Sergio Fernández]

Hi Tom,

On 20 November 2013 03:38, Tom Morris <tfmo...@gmail.com> wrote:
> Thanks for updating this, Sergio!

Everything I can help... :-)

> Maven's dependency management is magical when it works and a nightmarish
> hell when it doesn't. I'm using it for another project and I don't think
> I've ever done a clean build from scratch in a single try. Usually it's a
> matter of retrying the same command several times to get around ibibio's
> arbitrary blocking requests, but other times it involves finding and
> manually download jars.

Well, using two basic principles (use always artifacts from maven
central and avoid using snapshots) you should not have major problems.
I manage large projects (over 100 modules) with Maven, including
release management, reporting and so on without too many problems.

> I'm willing to consider switching to it, but it's not a clear win in my
> mind.

The current setup (keep ant and use maven for the workflow around it)
has the benefit of keeping you guys working with the same tool, but
could be a mess if we keep it for so long...

>> * One of the reasons I do not like ant it's the dependencies issue.
>> Right now we have the binaries of all dependencies in the source code
>> repository. And, for instance, I realized the code does not compile
>> anymore against the latest snapshot of opencsv (commit
>> 36a9c354f498be32bd4c1b7084432b35a2d910fe) . So this would one of the
>> things to improve in upcoming versions.
>
> "Lastest" is a bit of a tricky concept in a distributed version control
> system. OpenRefine 2.6 is build against a jar built from the tip of
> https://github.com/OpenRefine/opencsv/tree/openrefine-2.6 (although it
> should really have a release tag). I'm not sure there's a way to do this
> kind of forked private build in Maven.

I see...

The same principles I described above apply: working with a snapshot
version could be a mess, specially when is not periodically published
by official builds.

Therefore, what I've just done, is to make a "release"
net.sf.opencsv:opencs:2.4-refine-20130712 with the head in our fork. I
uploaded the artifact to our nexus, so everyone could fetch it with
the configuration of the current pom.

But this is a patch, an such kind of forks are not really a good idea.
But for the moment it works for us.

>> * The gzip compression is now using jetty's implementation. I'd need
>> to know if there is a portable way to do it.
>
> I'd have to look at what you're using it for, but there's obviously Java's
> native Gzip implementation available on all JVMs if it's needed for
> something internal and web servers can can be configured to use (or not
> Gzip) if it's just for HTTP transport purposes.

I'm not using it, the filter (org.mortbay.servlet.GzipFilter) was
already declared in our web.xml.

>> * I have an issue with the relative path to the extensions, that's why
>> I have to patch butterfly.properties when packaging (commit
>> 1aed66b47a6c1f103899b8a10979d748264c0604)
>
> I'll try looking at this again, but from a glance at Github, it looks like
> an entirely new file, so it's hard to tell what the issue is or what you had
> to change (although I certainly would believe there are additional relative
> path issues lurking).

I'd need to work on a proper patching strategy, but the issue is just
a path one:

https://github.com/wikier/OpenRefine/blob/master/patches/butterfly.patch

> Thanks again for doing all this work Sergio! I'd love to get everything
> integrated if it all possible.

For the moment I'd prefer to keep it unofficial until the community
actually agree on some of this changes.

If Massimo could give it a try would also help to discover issues in a
early stage.

Best,

[Massimo Imparato]

Hi dear

this link not run :(

[2] https://dl.dropbox.com/u/608222/refine.war
THANK A LOT

[Sergio Fernández]

On 25 November 2013 11:27, Massimo Imparato <massimo....@gmail.com> wrote:
> this link not run :(
> [2] https://dl.dropbox.com/u/608222/refine.war

Even if it would, it'll be a very old version...

Checking out the code at my fork, you could build it by executing:

mvn clean package

I've just did it, you have a fresh (not tested) WAR at:

https://dl.dropboxusercontent.com/u/608222/tmp/openrefine.war

Tell me if it works for you.

Cheers,

--

Sergio Fernández <ser...@wikier.org>

[Sergio Fernández]

please, be sure you get the latest version of the war
md5sum 67646fbd085138ea9c3202147260ceaa
where I quickly fixed (new) issues calling commands when deploying
under a named context (i.e., not root)
https://github.com/wikier/OpenRefine/commit/f9d5b62b361058a223650cfe464c7b76f25bee68

--

Sergio Fernández <ser...@wikier.org>

[Massimo Imparato]

Sergio, you are a great man, it work very well!

THANKS a lot

I have done some extension, I would like create a new war like yours

Have u some suggest to me for compile all project in one war?

Thank in advance

Cheers

MAX

[Tom Morris]

You should be able to use Maven with the pom.xml file from Sergio's project to build a war.

Tom

--

[Sergio Fernández]

On 26 November 2013 11:41, Massimo Imparato <massimo....@gmail.com> wrote:
> Sergio, you are a great man, it work very well!
> THANKS a lot

np

> I have done some extension, I would like create a new war like yours
> Have u some suggest to me for compile all project in one war?

just clone my repo (fork)
put your extensions in the extensions directory as usual
and, in the root ot the project, execute:

mvn clean package

and at the target directory you would have a war build

of course it requires to have maven and a jdk installed in your system

--

Sergio Fernández <ser...@wikier.org>

[Sergio Fernández]

Tom, now the question is: what do we actually want to do with that fork?

--

Sergio Fernández <ser...@wikier.org>

[Massimo Imparato]

Sergio, 

may you send here the link to your fork repo?

Sorry for my english :) 

[Tom Morris]

On Tue, Nov 26, 2013 at 10:56 AM, Sergio Fernández <ser...@wikier.org> wrote:

Tom, now the question is: what do we actually want to do with that fork?

The last thing I saw was this:

So, what does the community think?  Any other developers want to weigh in?  

Certainly the patch to make the I18N command references relative paths is worthy of an immediate pull request.

The sentiment "only use official releases" from earlier in the thread is nice in theory, but in practice, when patches haven't been integrated after a couple of years, as is the case with OpenCSV, sometimes you just need to move on.  I don't really want the hassle of hosting our own Maven repository or the uncertainty of depending on a repository we don't control, but if people are interested in moving to Maven, I think we can finesse this by having developers install the jar in their local repositories.

Tom

[Thad Guidry]

Maven (or Gradle, or whatever is newish...) does solve the dependencies issues, which is a big hurdle and pain, I would say, with our current build process.  True, our build process is not THAT complicated, but it could be made a bit easier regarding dependencies, as Sergio says.

+1 for moving to Maven, or whatever the bulk of developers have comfort with.  In the end, the winner is what makes everyone more productive and not less so.

--

You received this message because you are subscribed to the Google Groups "OpenRefine Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openrefine-de...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

-Thad

+ThadGuidry
Thad on LinkedIn

[Sergio Fernández]

Thad, although there is a question I still didn't have time to think
about how to solve in Maven: extensions.

Right now it's dependency manages it's how classpath, which could be
fine moving extensions as dependencies in Maven. But then your
wouldn't be able to install extensions as It's done right now... I
have to think about it.

The question comes more about taking the decision of switching to
Maven than actually doing. The current fork only outlines some things,
but I would expect some work before such option would be fully
feasible.

Sergio Fernández <ser...@wikier.org>

[Tom Morris]

Thad - can you expand on this?

On Tue, Nov 26, 2013 at 12:04 PM, Thad Guidry <thadg...@gmail.com> wrote:

Maven (or Gradle, or whatever is newish...) does solve the dependencies issues, which is a big hurdle and pain, I would say, with our current build process.

I thought the only burden was on the engineer updating the dependencies (typically me).  I didn't realize that having the dependencies checked in was also creating an additional burden on those who just want to build things

Tom 

[Thad Guidry]

Yes, I am only referring to the engineer.  Or the engineer who might be using other tools like Netbeans, Eclipse, etc.

(personally, it is not a pain point for me at the Windows command console and running "ant build" or "refine build", etc...)

I have been told that Maven, Gradle, etc... solve and manage dependencies much better than Ant.  That was the highlight of my input.

--

You received this message because you are subscribed to the Google Groups "OpenRefine Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to openrefine-de...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Massimo Imparato]

Hi All dear friend.

I wish thank you for alla support.

I use refine and with your help I finish my work.

Thank u all.

Max

[Sergio Fernández]

great!

[Massimo Imparato]

Buddy,

I would like make a question for you two openrefine guru.

Do you know if I can retrive in gerl (window: transform cell ) the project title?

look like jquery:

$("#project-title").show()

THANK A LOT AS USUALLY

[Sergio Fernández]

I guess the scope of GREL is restricted to the data itself (the
spreadsheet), not to the project.

Sergio Fernández <ser...@wikier.org>

[Massimo Imparato]

Thank you so much for your help buddie