Problems building lingual 1.2.1
23 views
Skip to first unread message
Benjamin Ross
Nov 16, 2015, 3:57:28 PM11/16/15
to Lingual User
I wanted to take a look at the lingual source. Building it resulted in the following error:
╰─$ gradle idea --stacktrace 1 ↵
FAILURE: Build failed with an exception.
* Where:
Build file '/home/bross/src/lingual/lingual-hadoop/build.gradle' line: 38
* What went wrong:
A problem occurred evaluating project ':lingual-hadoop'.
> Cannot change dependencies of configuration ':lingual-hadoop:providedCompile' after it has been resolved.
* Try:
Run with --info or --debug option to get more log output.
* Exception is:
org.gradle.api.GradleScriptException: A problem occurred evaluating project ':lingual-hadoop'.
at org.gradle.groovy.scripts.internal.DefaultScriptRunnerFactory$ScriptRunnerImpl.run(DefaultScriptRunnerFactory.java:76)
at org.gradle.configuration.DefaultScriptPluginFactory$ScriptPluginImpl$1.run(DefaultScriptPluginFactory.java:149)
at org.gradle.configuration.DefaultScriptPluginFactory$ScriptPluginImpl.apply(DefaultScriptPluginFactory.java:157)
at org.gradle.configuration.project.BuildScriptProcessor.execute(BuildScriptProcessor.java:39)
at org.gradle.configuration.project.BuildScriptProcessor.execute(BuildScriptProcessor.java:26)
at org.gradle.configuration.project.ConfigureActionsProjectEvaluator.evaluate(ConfigureActionsProjectEvaluator.java:34)
at org.gradle.configuration.project.LifecycleProjectEvaluator.evaluate(LifecycleProjectEvaluator.java:55)
at org.gradle.api.internal.project.AbstractProject.evaluate(AbstractProject.java:488)
at org.gradle.api.internal.project.AbstractProject.evaluationDependsOn(AbstractProject.java:556)
at org.gradle.api.internal.project.AbstractProject.evaluationDependsOn(AbstractProject.java:548)
at org.gradle.api.internal.BeanDynamicObject$MetaClassAdapter.invokeMethod(BeanDynamicObject.java:232)
at org.gradle.api.internal.BeanDynamicObject.invokeMethod(BeanDynamicObject.java:127)
at org.gradle.api.internal.CompositeDynamicObject.invokeMethod(CompositeDynamicObject.java:150)
at org.gradle.groovy.scripts.BasicScript.methodMissing(BasicScript.java:79)
at build_4tkhx7rma4f1jky5i6t8rofsk.run(/home/bross/src/lingual/lingual-client/build.gradle:26)
at org.gradle.groovy.scripts.internal.DefaultScriptRunnerFactory$ScriptRunnerImpl.run(DefaultScriptRunnerFactory.java:74)
at org.gradle.configuration.DefaultScriptPluginFactory$ScriptPluginImpl$1.run(DefaultScriptPluginFactory.java:149)
at org.gradle.configuration.DefaultScriptPluginFactory$ScriptPluginImpl.apply(DefaultScriptPluginFactory.java:157)
at org.gradle.configuration.project.BuildScriptProcessor.execute(BuildScriptProcessor.java:39)
at org.gradle.configuration.project.BuildScriptProcessor.execute(BuildScriptProcessor.java:26)
at org.gradle.configuration.project.ConfigureActionsProjectEvaluator.evaluate(ConfigureActionsProjectEvaluator.java:34)
at org.gradle.configuration.project.LifecycleProjectEvaluator.evaluate(LifecycleProjectEvaluator.java:55)
at org.gradle.api.internal.project.AbstractProject.evaluate(AbstractProject.java:488)
at org.gradle.api.internal.project.AbstractProject.evaluate(AbstractProject.java:86)
at org.gradle.execution.TaskPathProjectEvaluator.configureHierarchy(TaskPathProjectEvaluator.java:47)
at org.gradle.configuration.DefaultBuildConfigurer.configure(DefaultBuildConfigurer.java:35)
at org.gradle.initialization.DefaultGradleLauncher$4.run(DefaultGradleLauncher.java:143)
at org.gradle.internal.Factories$1.create(Factories.java:22)
at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:62)
at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:47)
at org.gradle.initialization.DefaultGradleLauncher.runBuildOperation(DefaultGradleLauncher.java:189)
at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:140)
at org.gradle.initialization.DefaultGradleLauncher.access$200(DefaultGradleLauncher.java:35)
at org.gradle.initialization.DefaultGradleLauncher$1.create(DefaultGradleLauncher.java:104)
at org.gradle.initialization.DefaultGradleLauncher$1.create(DefaultGradleLauncher.java:97)
at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:62)
at org.gradle.initialization.DefaultGradleLauncher.runRootBuildOperation(DefaultGradleLauncher.java:184)
at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:97)
at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:86)
at org.gradle.launcher.exec.InProcessBuildActionExecuter$DefaultBuildController.run(InProcessBuildActionExecuter.java:93)
at org.gradle.tooling.internal.provider.ExecuteBuildActionRunner.run(ExecuteBuildActionRunner.java:28)
at org.gradle.launcher.exec.ChainingBuildActionRunner.run(ChainingBuildActionRunner.java:35)
at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:43)
at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:27)
at org.gradle.launcher.exec.ContinuousBuildActionExecuter.execute(ContinuousBuildActionExecuter.java:72)
at org.gradle.launcher.exec.ContinuousBuildActionExecuter.execute(ContinuousBuildActionExecuter.java:44)
at org.gradle.launcher.exec.DaemonUsageSuggestingBuildActionExecuter.execute(DaemonUsageSuggestingBuildActionExecuter.java:50)
at org.gradle.launcher.exec.DaemonUsageSuggestingBuildActionExecuter.execute(DaemonUsageSuggestingBuildActionExecuter.java:27)
at org.gradle.launcher.cli.RunBuildAction.run(RunBuildAction.java:40)
at org.gradle.internal.Actions$RunnableActionAdapter.execute(Actions.java:170)
at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:237)
at org.gradle.launcher.cli.CommandLineActionFactory$ParseAndBuildAction.execute(CommandLineActionFactory.java:210)
at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:35)
at org.gradle.launcher.cli.JavaRuntimeValidationAction.execute(JavaRuntimeValidationAction.java:24)
at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:206)
at org.gradle.launcher.cli.CommandLineActionFactory$WithLogging.execute(CommandLineActionFactory.java:169)
at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:33)
at org.gradle.launcher.cli.ExceptionReportingAction.execute(ExceptionReportingAction.java:22)
at org.gradle.launcher.Main.doAction(Main.java:33)
at org.gradle.launcher.bootstrap.EntryPoint.run(EntryPoint.java:45)
at org.gradle.launcher.bootstrap.ProcessBootstrap.runNoExit(ProcessBootstrap.java:54)
at org.gradle.launcher.bootstrap.ProcessBootstrap.run(ProcessBootstrap.java:35)
at org.gradle.launcher.GradleMain.main(GradleMain.java:23)
Caused by: org.gradle.api.InvalidUserDataException: Cannot change dependencies of configuration ':lingual-hadoop:providedCompile' after it has been resolved.
at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration.validateMutation(DefaultConfiguration.java:592)
at org.gradle.api.internal.artifacts.configurations.DefaultConfiguration$2.run(DefaultConfiguration.java:141)
at org.gradle.api.internal.DefaultDomainObjectCollection.assertMutable(DefaultDomainObjectCollection.java:267)
at org.gradle.api.internal.DefaultDomainObjectCollection.add(DefaultDomainObjectCollection.java:158)
at org.gradle.api.internal.DelegatingDomainObjectSet.add(DelegatingDomainObjectSet.java:81)
at org.gradle.api.internal.artifacts.dsl.dependencies.DefaultDependencyHandler.doAdd(DefaultDependencyHandler.java:84)
at org.gradle.api.internal.artifacts.dsl.dependencies.DefaultDependencyHandler.methodMissing(DefaultDependencyHandler.java:119)
at org.gradle.api.internal.BeanDynamicObject$MetaClassAdapter.invokeMethod(BeanDynamicObject.java:232)
at org.gradle.api.internal.BeanDynamicObject.invokeMethod(BeanDynamicObject.java:127)
at org.gradle.api.internal.CompositeDynamicObject.invokeMethod(CompositeDynamicObject.java:166)
at org.gradle.api.internal.artifacts.dsl.dependencies.DefaultDependencyHandler_Decorated.invokeMethod(Unknown Source)
at build_durcnbdylcg8fhcjk2yfcymp9$_run_closure2.doCall(/home/bross/src/lingual/lingual-hadoop/build.gradle:38)
at org.gradle.api.internal.ClosureBackedAction.execute(ClosureBackedAction.java:67)
at org.gradle.util.ConfigureUtil.configure(ConfigureUtil.java:130)
at org.gradle.util.ConfigureUtil.configure(ConfigureUtil.java:91)
at org.gradle.api.internal.project.AbstractProject.dependencies(AbstractProject.java:843)
at org.gradle.api.internal.BeanDynamicObject$MetaClassAdapter.invokeMethod(BeanDynamicObject.java:232)
at org.gradle.api.internal.BeanDynamicObject.invokeMethod(BeanDynamicObject.java:127)
at org.gradle.api.internal.CompositeDynamicObject.invokeMethod(CompositeDynamicObject.java:150)
at org.gradle.groovy.scripts.BasicScript.methodMissing(BasicScript.java:79)
at build_durcnbdylcg8fhcjk2yfcymp9.run(/home/bross/src/lingual/lingual-hadoop/build.gradle:27)
at org.gradle.groovy.scripts.internal.DefaultScriptRunnerFactory$ScriptRunnerImpl.run(DefaultScriptRunnerFactory.java:74)
... 62 more
Andre Kelpe
Nov 16, 2015, 4:30:44 PM11/16/15
to lingua...@googlegroups.com
For lingual 1.x, you have to use gradle 1.x. You seem to be using 2.x
- André--
You received this message because you are subscribed to the Google Groups "Lingual User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lingual-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
--
Benjamin Ross
Nov 16, 2015, 6:02:27 PM11/16/15
to Lingual User
Thanks Andre,
A few more quick questions, if you don't mind?
1. One of the reasons I was hoping to use cascading lingual is because it has tez support, but it seems that's only available in wip-2.0. How mature is wip-2.0?
2. In particular, I'd like to write data flows that invoke sql on avro files stored in hadoop and running on tez, or as a fallback, on map reduce. The reason I want to do this is because I need to also support ad-hoc SQL queries on a different SQL variant... suffice it to say, they're my requirements :). Based on the documentation, it looks like I need to write a ProviderInterface in order to do this, yet there's no ProviderInterface in 1.2, and it seems like that whole infrastructure has been supplanted by PlatformBroker. Looking at TezPlatformBroker, it looks like it doesn't support avro (i.e., implementation of getFileTypeFor). Is this correct?
3. Registration of a table in tez requires execution of a command-line tool. Is there an API I can call to do this as well? Or DDL support? I figure I could also just invoke the shell main manually...
Thanks again,
Ben
Andre Kelpe
Nov 16, 2015, 8:22:06 PM11/16/15
to lingua...@googlegroups.com
On Mon, Nov 16, 2015 at 3:02 PM, Benjamin Ross <benjam...@gmail.com> wrote:
Thanks Andre,A few more quick questions, if you don't mind?1. One of the reasons I was hoping to use cascading lingual is because it has tez support, but it seems that's only available in wip-2.0. How mature is wip-2.0?
It should work, we just haven't gotten to release 2.0 yet. Please give it a try.
2. In particular, I'd like to write data flows that invoke sql on avro files stored in hadoop and running on tez, or as a fallback, on map reduce. The reason I want to do this is because I need to also support ad-hoc SQL queries on a different SQL variant... suffice it to say, they're my requirements :). Based on the documentation, it looks like I need to write a ProviderInterface in order to do this, yet there's no ProviderInterface in 1.2, and it seems like that whole infrastructure has been supplanted by PlatformBroker. Looking at TezPlatformBroker, it looks like it doesn't support avro (i.e., implementation of getFileTypeFor). Is this correct?
The best way to do that is wrapping the AvroTap and make it a new Format:
https://github.com/ScaleUnlimited/cascading.avro (not sure what the Cascading 3.0 support status is here. I'd ask Ken) then follow what we do here:
https://github.com/Cascading/cascading-jdbc/blob/3.0/cascading-jdbc-core/src/main/java/cascading/jdbc/JDBCFactory.java or here:
https://github.com/Cascading/cascading.hbase/blob/3.0/cascading-hbase-hadoop/src/main/shared/cascading/hbase/HBaseFactory.java
https://github.com/ScaleUnlimited/cascading.avro (not sure what the Cascading 3.0 support status is here. I'd ask Ken) then follow what we do here:
https://github.com/Cascading/cascading-jdbc/blob/3.0/cascading-jdbc-core/src/main/java/cascading/jdbc/JDBCFactory.java or here:
https://github.com/Cascading/cascading.hbase/blob/3.0/cascading-hbase-hadoop/src/main/shared/cascading/hbase/HBaseFactory.java
See here as well for more information: http://docs.concurrentinc.com/lingual/2.0/index.html#_creating_a_data_provider
3. Registration of a table in tez requires execution of a command-line tool. Is there an API I can call to do this as well? Or DDL support? I figure I could also just invoke the shell main manually...
The catalog is only required for the JDBC part. If you have a cascading flow and a Tap with fields + types, you don't need a catalog. There is currently no API to populate a catalog, but it is a simple JSON document, so you could generate that.
- Andre
Cyrille Chépélov
Nov 17, 2015, 4:11:29 AM11/17/15
to lingua...@googlegroups.com
Le 17/11/2015 02:22, Andre Kelpe a écrit :
>
> The best way to do that is wrapping the AvroTap and make it a new Format:
>
> https://github.com/ScaleUnlimited/cascading.avro (not sure what the
> Cascading 3.0 support status is here. I'd ask Ken) then follow what we
> do here:
> https://github.com/Cascading/cascading-jdbc/blob/3.0/cascading-jdbc-core/src/main/java/cascading/jdbc/JDBCFactory.java
> or here:
> https://github.com/Cascading/cascading.hbase/blob/3.0/cascading-hbase-hadoop/src/main/shared/cascading/hbase/HBaseFactory.java
>
Might want to give
>
> The best way to do that is wrapping the AvroTap and make it a new Format:
>
> https://github.com/ScaleUnlimited/cascading.avro (not sure what the
> Cascading 3.0 support status is here. I'd ask Ken) then follow what we
> do here:
> https://github.com/Cascading/cascading-jdbc/blob/3.0/cascading-jdbc-core/src/main/java/cascading/jdbc/JDBCFactory.java
> or here:
> https://github.com/Cascading/cascading.hbase/blob/3.0/cascading-hbase-hadoop/src/main/shared/cascading/hbase/HBaseFactory.java
>
https://github.com/ScaleUnlimited/cascading.avro/pull/44 a spin
Benjamin Ross
Dec 9, 2015, 4:17:07 PM12/9/15
to Lingual User
Hey all,
It turns out that we decided to go with lingual, because the alternatives (spark and hive) both are insufficient for our needs. The zeroconfig nature of lingual is really appealing.
Anyhow, I'm writing my avro format provider, and am running into a class not found exception. I've been able to debug it using the --debug option and stepping through the code in lingual-client.
Basically, it comes down to a url class loader not being able to load the class I'm providing it. The class name is correct, and I'm providing a shaded jar, so as far as I understand there should be no issues loading the class. I can't step into the class loader itself.
Any suggestions for things to try?
Thanks in advance,
Ben
Benjamin Ross
Dec 9, 2015, 4:46:25 PM12/9/15
to Lingual User
Nevermind - user error.
Reply all
Reply to author
Forward
0 new messages