Help to test the DB extension

[qi]

Hi All,

Tony finished the DB extension for OpenRefine:

https://github.com/tcbuzor/openrefine-db-extension.git

Is there anyone can help to test it out by clone the database extension repo? The instruction to use it is in home page. I am going to install and try it later today.

Cheers

Jacky

[Owen Stephens]

Hi Jacky,

Just tried this:

* The installation instructions point at a very old version of Refine on Google Code

* On trying to build with ./refine build I get:

OpenRefine-master >>./refine build

You have 2508M of free memory.

Your current configuration is set to use 1400M of memory.

OpenRefine can run better when given more memory. Read our FAQ on how to allocate more memory here:

https://github.com/OpenRefine/OpenRefine/wiki/FAQ:-Allocate-More-Memory

Buildfile: /Users/damyantiandowen/Downloads/OpenRefine-master/build.xml

build_server:

build_webapp:

build_extensions:

build:

     [echo] Building extensions

build_java:

build:

build_java:

build:

build_java:

build:

build_java:

build:

     [echo] JAVA_HOME is set to = /Library/Java/JavaVirtualMachines/jdk1.8.0_72.jdk/Contents/Home

build_java:

    [mkdir] Created dir: /Users/damyantiandowen/Downloads/OpenRefine-master/extensions/database/module/MOD-INF/classes

    [javac] Compiling 26 source files to /Users/damyantiandowen/Downloads/OpenRefine-master/extensions/database/module/MOD-INF/classes

    [javac] warning: [options] bootstrap class path not set in conjunction with -source 1.7

    [javac] /Users/damyantiandowen/Downloads/OpenRefine-master/extensions/database/src/com/google/refine/extension/database/DatabaseUtils.java:10: error: package org.codehaus.jackson does not exist

    [javac] import org.codehaus.jackson.JsonGenerationException;

    [javac]                            ^

    [javac] /Users/damyantiandowen/Downloads/OpenRefine-master/extensions/database/src/com/google/refine/extension/database/DatabaseUtils.java:11: error: package org.codehaus.jackson does not exist

    [javac] import org.codehaus.jackson.JsonParseException;

    [javac]                            ^

    [javac] /Users/damyantiandowen/Downloads/OpenRefine-master/extensions/database/src/com/google/refine/extension/database/DatabaseUtils.java:52: error: cannot access Versioned

    [javac]                   mapper.writerWithDefaultPrettyPrinter().writeValue(new File(filename), sc);

    [javac]                         ^

    [javac]   class file for org.codehaus.jackson.Versioned not found

    [javac] /Users/damyantiandowen/Downloads/OpenRefine-master/extensions/database/src/com/google/refine/extension/database/DatabaseUtils.java:59: error: cannot access JsonParseException

    [javac]             SavedConnectionContainer savedConnectionContainer = mapper.readValue(new File(filename), SavedConnectionContainer.class);

    [javac]                                                                       ^

    [javac]   class file for org.codehaus.jackson.JsonParseException not found

    [javac] /Users/damyantiandowen/Downloads/OpenRefine-master/extensions/database/src/com/google/refine/extension/database/DatabaseUtils.java:62: error: cannot find symbol

    [javac]         } catch (JsonParseException e) {

    [javac]                  ^

    [javac]   symbol:   class JsonParseException

    [javac]   location: class DatabaseUtils

    [javac] /Users/damyantiandowen/Downloads/OpenRefine-master/extensions/database/src/com/google/refine/extension/database/DatabaseUtils.java:35: error: cannot access JsonProcessingException

    [javac]         try {

    [javac]         ^

    [javac]   class file for org.codehaus.jackson.JsonProcessingException not found

    [javac] /Users/damyantiandowen/Downloads/OpenRefine-master/extensions/database/src/com/google/refine/extension/database/DatabaseUtils.java:103: error: cannot access JsonNode

    [javac]              SavedConnectionContainer savedConnectionContainer = mapper.readValue(new File(savedConnectionFile), SavedConnectionContainer.class);

    [javac]                                                                        ^

    [javac]   class file for org.codehaus.jackson.JsonNode not found

    [javac] /Users/damyantiandowen/Downloads/OpenRefine-master/extensions/database/src/com/google/refine/extension/database/DatabaseUtils.java:106: error: cannot access ObjectCodec

    [javac]              mapper.writerWithDefaultPrettyPrinter().writeValue(new File(savedConnectionFile), savedConnectionContainer);

    [javac]                    ^

    [javac]   class file for org.codehaus.jackson.ObjectCodec not found

    [javac] /Users/damyantiandowen/Downloads/OpenRefine-master/extensions/database/src/com/google/refine/extension/database/DatabaseUtils.java:108: error: cannot find symbol

    [javac]          } catch (JsonGenerationException e1) {

    [javac]                   ^

    [javac]   symbol:   class JsonGenerationException

    [javac]   location: class DatabaseUtils

    [javac] /Users/damyantiandowen/Downloads/OpenRefine-master/extensions/database/src/com/google/refine/extension/database/DatabaseUtils.java:111: error: incompatible types: JsonMappingException cannot be converted to Throwable

    [javac]          } catch (JsonMappingException e1) {

    [javac]                   ^

    [javac] /Users/damyantiandowen/Downloads/OpenRefine-master/extensions/database/src/com/google/refine/extension/database/DatabaseUtils.java:113: error: cannot find symbol

    [javac]              e1.printStackTrace();

    [javac]                ^

    [javac]   symbol:   method printStackTrace()

    [javac]   location: variable e1 of type JsonMappingException

    [javac] /Users/damyantiandowen/Downloads/OpenRefine-master/extensions/database/src/com/google/refine/extension/database/DatabaseUtils.java:143: error: cannot access JavaType

    [javac]              SavedConnectionContainer savedConnectionContainer = mapper.readValue(new File(savedConnectionFile), SavedConnectionContainer.class);

    [javac]                                                                        ^

    [javac]   class file for org.codehaus.jackson.type.JavaType not found

    [javac] /Users/damyantiandowen/Downloads/OpenRefine-master/extensions/database/src/com/google/refine/extension/database/DatabaseUtils.java:147: error: cannot access JsonGenerator

    [javac]              mapper.writerWithDefaultPrettyPrinter().writeValue(new File(savedConnectionFile), savedConnectionContainer);

    [javac]                                                     ^

    [javac]   class file for org.codehaus.jackson.JsonGenerator not found

    [javac] /Users/damyantiandowen/Downloads/OpenRefine-master/extensions/database/src/com/google/refine/extension/database/DatabaseUtils.java:149: error: cannot find symbol

    [javac]          } catch (JsonGenerationException e1) {

    [javac]                   ^

    [javac]   symbol:   class JsonGenerationException

    [javac]   location: class DatabaseUtils

    [javac] /Users/damyantiandowen/Downloads/OpenRefine-master/extensions/database/src/com/google/refine/extension/database/DatabaseUtils.java:152: error: incompatible types: JsonMappingException cannot be converted to Throwable

    [javac]          } catch (JsonMappingException e1) {

    [javac]                   ^

    [javac] /Users/damyantiandowen/Downloads/OpenRefine-master/extensions/database/src/com/google/refine/extension/database/DatabaseUtils.java:154: error: cannot find symbol

    [javac]              e1.printStackTrace();

    [javac]                ^

    [javac]   symbol:   method printStackTrace()

    [javac]   location: variable e1 of type JsonMappingException

    [javac] /Users/damyantiandowen/Downloads/OpenRefine-master/extensions/database/src/com/google/refine/extension/database/DatabaseUtils.java:172: error: cannot access TypeReference

    [javac]              SavedConnectionContainer savedConnectionContainer = mapper.readValue(new File(savedConnectionFile), SavedConnectionContainer.class);

    [javac]                                                                        ^

    [javac]   class file for org.codehaus.jackson.type.TypeReference not found

    [javac] /Users/damyantiandowen/Downloads/OpenRefine-master/extensions/database/src/com/google/refine/extension/database/DatabaseUtils.java:192: error: cannot find symbol

    [javac]          } catch (JsonGenerationException e1) {

    [javac]                   ^

    [javac]   symbol:   class JsonGenerationException

    [javac]   location: class DatabaseUtils

    [javac] /Users/damyantiandowen/Downloads/OpenRefine-master/extensions/database/src/com/google/refine/extension/database/DatabaseUtils.java:195: error: incompatible types: JsonMappingException cannot be converted to Throwable

    [javac]          } catch (JsonMappingException e1) {

    [javac]                   ^

    [javac] /Users/damyantiandowen/Downloads/OpenRefine-master/extensions/database/src/com/google/refine/extension/database/DatabaseUtils.java:197: error: cannot find symbol

    [javac]              e1.printStackTrace();

    [javac]                ^

    [javac]   symbol:   method printStackTrace()

    [javac]   location: variable e1 of type JsonMappingException

    [javac] 20 errors

    [javac] 1 warning

BUILD FAILED

/Users/damyantiandowen/Downloads/OpenRefine-master/build.xml:154: The following error occurred while executing this line:

/Users/damyantiandowen/Downloads/OpenRefine-master/extensions/build.xml:16: The following error occurred while executing this line:

/Users/damyantiandowen/Downloads/OpenRefine-master/extensions/database/build.xml:52: Compile failed; see the compiler error output for details.

Total time: 1 second

Error: Error while running ant task 'build'

[Owen Stephens]

So it looks like in .classpath this is using <classpathentry kind="lib" path="/grefine/webapp/WEB-INF/lib/jackson-core-asl-1.9.12.jar"/> - which has been replaced with jackson-core-2.9.1.jar in OpenRefine https://github.com/OpenRefine/OpenRefine/pull/1264/files

So some dependency issues that look like they need sorting before we can test?

Owen

[qi cui]

I just sent an email to Tony. should be a quick fix. will update when he is done.

--
You received this message because you are subscribed to a topic in the Google Groups "OpenRefine" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/openrefine/OQKAvx_hBVI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to openrefine+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Owen Stephens]

Thanks Jacky,

I did try to update the code to use the com.fasterxml.jackson - but I hit another issue with messages like:

    [javac]                   mapper.writerWithDefaultPrettyPrinter().writeValue(new File(filename), sc);

    [javac]                         ^

    [javac]   class file for org.codehaus.jackson.Versioned not found

I don't understand this, but a post here https://github.com/klout/brickhouse/issues/103 indicates it could be a Java 8 vs 7 issue - you may want to check Tony is building on 8?

Owen

[qi cui]

No worry. I think he is using 8. Will confirm though.

[Thad Guidry]

He was using 8.  I told him to directly.

-Thad

[Tonio O]

Try to clone the repository (https://github.com/tcbuzor/openrefine-db-extension.git) again. I added the missing jackson-core-asl-1.9.13.jar to the module/lib folder. 

[qi cui]

Build is good now. But got an exception when run query. I created an issue in your repo.

[Owen Stephens]

Build is good for me too now. I will report issues on the repo

Owen

To unsubscribe from this group and all its topics, send an email to openrefine+...@googlegroups.com.

[Antonin Delpeuch (lists)]

I can see you have changed the Jsonizable interface - why did you rename
that method? If that is required (but I struggle to see why it would),
it's clearly a change that would deserve to be discussed in its own PR,
as it is going to break basically every OR extension (and it changes
hundreds of files, so the diff of your PR is flooded by that rename).

The way JSON is currently handled in OR is clearly obsolete and there
are lots of ways we could improve it (most of which will indeed break
extensions), but just renaming that method is a very curious move (as
far as I can tell).

As the diffs are huge (and spammed by this spurious change) it's hard to
exactly understand what you did, but as I understand it you are storing
the whole Data Package metadata in a special new slot, similar to how
the OverlayModel works. This slot can contain Data Package metadata but
also other types of project metadata such as CSVW. Can you confirm that
this is the case?

Antonin

On 18/12/2017 10:11, Owen Stephens wrote:
> Build is good for me too now. I will report issues on the repo
>
> Owen
>
> On Friday, December 15, 2017 at 9:52:37 PM UTC, qi wrote:
>
> Build is good now. But got an exception when run query. I created an
> issue in your repo.
>
> On Fri, Dec 15, 2017 at 1:37 PM, Tonio O <tcb...@gmail.com
> <javascript:>> wrote:
>
> Try to clone the repository
> (https://github.com/tcbuzor/openrefine-db-extension.git

> <https://github.com/tcbuzor/openrefine-db-extension.git>) again.

> <https://groups.google.com/d/topic/openrefine/OQKAvx_hBVI/unsubscribe>.

> To unsubscribe from this group and all its topics, send an email

> to openrefine+...@googlegroups.com <javascript:>.

> For more options, visit https://groups.google.com/d/optout

> <https://groups.google.com/d/optout>.
>
>
> --
> You received this message because you are subscribed to the Google
> Groups "OpenRefine" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to openrefine+...@googlegroups.com
> <mailto:openrefine+...@googlegroups.com>.

[qi cui]

Thanks Antonin. The reason I renamed it is to avoid a generic name which confused with other interface introduced by metadata/data package. But you are right that we should keep the interface compatible with other extensions if possible. I will rename it back.

> an email to openrefine+unsubscribe@googlegroups.com
> <mailto:openrefine+unsub...@googlegroups.com>.

> For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to a topic in the Google Groups "OpenRefine" group.

To unsubscribe from this topic, visit https://groups.google.com/d/topic/openrefine/OQKAvx_hBVI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to openrefine+unsubscribe@googlegroups.com.