java.lang.ClassCastException on Enumerable linq4.

[Richard Whitcomb]

Anyone seen this before?  On anything that parses to an OrderBy I seem to get this error.

So this query will fail "select * from table limit 5" and "select * from table order by id"

Is there a way to work around this?

Caused by: java.lang.ClassCastException: net.hydromatic.linq4j.EnumerableDefaults$12 cannot be cast to cascading.lingual.optiq.enumerable.CascadingFlowRunnerEnumerable

at cascading.lingual.flow.SQLPlanner.resolveTails(SQLPlanner.java:116)

at cascading.flow.planner.FlowPlanner.resolveAssemblyPlanners(FlowPlanner.java:151)

at cascading.flow.planner.FlowPlanner.resolveTails(FlowPlanner.java:138)

at cascading.flow.hadoop.planner.HadoopPlanner.buildFlow(HadoopPlanner.java:246)

... 39 more

Thanks.  Rich

[Andre Kelpe]

The first one  fails due to the problem with LIMIT, which is currently not supported. The second one I have never seen. Can you share your table structure?

- André

--
You received this message because you are subscribed to the Google Groups "Lingual User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lingual-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
André Kelpe
an...@concurrentinc.com
http://concurrentinc.com

[Richard Whitcomb]

The order by is just attempting to read a tsv file in cascading in hadoop mode.  I followed the example on the lingual page but I am not using SQLTypedTextDelimited. I am instead using Scheme from elephant-bird that loads a lzo file.  Does SQLTypedTextDelimited do anything special that could help avoid this error?

I also noticed that Optiq does strange parsing of sql queries that contain order by.  They are parsed as an orderBy object which doesn't seem to be a subclass of selectBy.  Not sure if that would cause this but I thought that was odd.

[Andre Kelpe]

If you use a different Scheme, you will have to implement a provider for it, package it in a jar and tell your table to use that as the format: http://docs.cascading.org/lingual/1.1/#provider

If you have questions about writing a provider, let me know.

- André

--

You received this message because you are subscribed to the Google Groups "Lingual User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lingual-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Richard Whitcomb]

Sorry misspoke, I am using SequenceFile Scheme from Cascading.  Do I need to create a provider for that also?

On Tuesday, August 12, 2014 2:52:43 PM UTC-4, Richard Whitcomb wrote:

[Andre Kelpe]

Yes, because otherwise lingual does not know how to read that file. These are the defaults: https://github.com/Cascading/lingual/blob/wip-1.2/lingual-hadoop/src/main/java/cascading/lingual/platform/hadoop/HadoopDefaultFactory.java

- André

--

You received this message because you are subscribed to the Google Groups "Lingual User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lingual-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Richard Whitcomb]

Actually tracked down the problem with order by.  The casting issue is different and only with limit.  With order by I get this error:

Caused by: java.lang.NullPointerException

at cascading.lingual.optiq.enumerable.CascadingFlowRunnerEnumerable.createEnumerator(CascadingFlowRunnerEnumerable.java:145)

at cascading.lingual.optiq.enumerable.CascadingFlowRunnerEnumerable.enumerator(CascadingFlowRunnerEnumerable.java:124)

at net.hydromatic.linq4j.EnumerableDefaults.toLookup_(EnumerableDefaults.java:1769)

at net.hydromatic.linq4j.EnumerableDefaults.orderBy(EnumerableDefaults.java:1170)

at net.hydromatic.linq4j.DefaultEnumerable.orderBy(DefaultEnumerable.java:471)

at Baz.bind(Unknown Source)

at net.hydromatic.optiq.prepare.OptiqPrepareImpl$OptiqPreparingStmt$2.bind(OptiqPrepareImpl.java:755)

at net.hydromatic.optiq.prepare.OptiqPrepareImpl.prepare2_(OptiqPrepareImpl.java:359)

at net.hydromatic.optiq.prepare.OptiqPrepareImpl.prepare_(OptiqPrepareImpl.java:250)

at net.hydromatic.optiq.prepare.OptiqPrepareImpl.prepareSql(OptiqPrepareImpl.java:195)

at cascading.lingual.flow.SQLPlanner.resolveTails(SQLPlanner.java:114)

at cascading.flow.planner.FlowPlanner.resolveAssemblyPlanners(FlowPlanner.java:151)

at cascading.flow.planner.FlowPlanner.resolveTails(FlowPlanner.java:138)

at cascading.flow.hadoop.planner.HadoopPlanner.buildFlow(HadoopPlanner.java:246)

... 39 more

Its because this line returns null.

  public PlatformBroker getPlatformBroker()

    {

    return flowHolder.branch.platformBroker;

    }

Any reason platformBroker would be null here?

On Tuesday, August 12, 2014 2:52:43 PM UTC-4, Richard Whitcomb wrote:

[Andre Kelpe]

Can you post your catalog and the SQL query? Then I can take a look. A test case would even be better.

Thanks!

- André

--

[Richard Whitcomb]

Looks like it happens with the FlowExample if you add an order by.  I modified the flow example to look like this:

package cascading.lingual.examples.foodmart;

import java.io.IOException;

import cascading.flow.Flow;

import cascading.flow.FlowDef;

import cascading.flow.local.LocalFlowConnector;

import cascading.lingual.flow.SQLPlanner;

import cascading.lingual.tap.local.SQLTypedTextDelimited;

import cascading.tap.SinkMode;

import cascading.tap.Tap;

import cascading.tap.local.FileTap;

import cascading.tuple.TupleEntryIterator;

/** A trivial Flow example */

public class FlowExample

  {

  public static void main( String[] args ) throws Exception

    {

    new FlowExample().run();

    }

  public void run() throws IOException

    {

    String statement = "select *\n"

      + "from \"example\".\"sales_fact_1997\" as s\n"

      + "join \"example\".\"employee\" as e\n"

      + "on e.\"EMPID\" = s.\"CUST_ID\"\n"

      + "order by e.\"EMPID\"";

    Tap empTap = new FileTap( new SQLTypedTextDelimited( ",", "\"" ),

        "src/main/resources/data/example/employee.tcsv", SinkMode.KEEP );

    Tap salesTap = new FileTap( new SQLTypedTextDelimited( ",", "\"" ),

      "src/main/resources/data/example/sales_fact_1997.tcsv", SinkMode.KEEP );

    Tap resultsTap = new FileTap( new SQLTypedTextDelimited( ",", "\"" ),

      "build/test/output/flow/results.tcsv", SinkMode.REPLACE );

    FlowDef flowDef = FlowDef.flowDef()

      .setName( "sql flow" )

      .addSource( "example.employee", empTap )

      .addSource( "example.sales_fact_1997", salesTap )

      .addSink( "results", resultsTap );

    SQLPlanner sqlPlanner = new SQLPlanner()

      .setSql( statement );

    flowDef.addAssemblyPlanner( sqlPlanner );

    Flow flow = new LocalFlowConnector().connect( flowDef );

    flow.complete();

    TupleEntryIterator iterator = resultsTap.openForRead( flow.getFlowProcess() );

    while( iterator.hasNext() )

      System.out.println( iterator.next() );

    iterator.close();

    }

  }

When you build the fatjar and run in the way documented in the README you see the same error:

/usr/local/Cellar/hadoop/2.2.0/bin/hadoop jar /Users/rwhitcomb/Documents/twitter/github/lingual/lingual-examples/build/libs/lingual-examples-1.2.0-wip-dev-fatjar.jar cascading.lingual.examples.foodmart.FlowExample

14/08/14 20:32:15 INFO property.AppProps: using app.id: C3AE880B1A174EAE97EF85D1F2D26E52

14/08/14 20:32:15 INFO tap.TapSchema: adding table on schema: example, table: employee, fields: 'EMPID', 'NAME' | int, String, identifier: src/main/resources/data/example/employee.tcsv

14/08/14 20:32:15 INFO tap.TapSchema: adding table on schema: example, table: sales_fact_1997, fields: 'CUST_ID', 'PROD_ID' | int, int, identifier: src/main/resources/data/example/sales_fact_1997.tcsv

14/08/14 20:32:15 INFO tap.TapSchema: adding table on schema: null, table: results, fields: UNKNOWN, identifier: build/test/output/flow/results.tcsv

Exception in thread "main" cascading.flow.planner.PlannerException: could not build flow from assembly: [null]

at cascading.flow.planner.FlowPlanner.handleExceptionDuringPlanning(FlowPlanner.java:577)

at cascading.flow.local.planner.LocalPlanner.buildFlow(LocalPlanner.java:108)

at cascading.flow.local.planner.LocalPlanner.buildFlow(LocalPlanner.java:40)

at cascading.flow.FlowConnector.connect(FlowConnector.java:459)

at cascading.lingual.examples.foodmart.FlowExample.run(FlowExample.java:70)

at cascading.lingual.examples.foodmart.FlowExample.main(FlowExample.java:40)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:601)

at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

Caused by: java.lang.NullPointerException

at cascading.lingual.optiq.enumerable.CascadingFlowRunnerEnumerable.createEnumerator(CascadingFlowRunnerEnumerable.java:141)

at cascading.lingual.optiq.enumerable.CascadingFlowRunnerEnumerable.enumerator(CascadingFlowRunnerEnumerable.java:124)

at net.hydromatic.linq4j.EnumerableDefaults.toLookup_(EnumerableDefaults.java:1769)

at net.hydromatic.linq4j.EnumerableDefaults.orderBy(EnumerableDefaults.java:1170)

at net.hydromatic.linq4j.DefaultEnumerable.orderBy(DefaultEnumerable.java:471)

at Baz.bind(Unknown Source)

at net.hydromatic.optiq.prepare.OptiqPrepareImpl$OptiqPreparingStmt$2.bind(OptiqPrepareImpl.java:755)

at net.hydromatic.optiq.prepare.OptiqPrepareImpl.prepare2_(OptiqPrepareImpl.java:359)

at net.hydromatic.optiq.prepare.OptiqPrepareImpl.prepare_(OptiqPrepareImpl.java:250)

at net.hydromatic.optiq.prepare.OptiqPrepareImpl.prepareSql(OptiqPrepareImpl.java:195)

at cascading.lingual.flow.SQLPlanner.resolveTails(SQLPlanner.java:114)

at cascading.flow.planner.FlowPlanner.resolveAssemblyPlanners(FlowPlanner.java:151)

at cascading.flow.planner.FlowPlanner.resolveTails(FlowPlanner.java:138)

at cascading.flow.local.planner.LocalPlanner.buildFlow(LocalPlanner.java:81)

... 9 more

On Tuesday, August 12, 2014 2:52:43 PM UTC-4, Richard Whitcomb wrote:

[Andre Kelpe]

One remark: The example is using the local platform, not hadoop, so you should run it with

java -cp ....

I took a look and the problem only occurs in the SQLPlanner. If you use the JDBC Driver, it works. (see JdbcExample). I will add a story to our internal bugtracker to find out, why that is happening.

Thanks for letting us know!

- André

--

You received this message because you are subscribed to the Google Groups "Lingual User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lingual-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Richard Whitcomb]

Thanks.  I am working on a small project to integrate scalding with lingual so the FlowExample is closest to the solution I need to get everything tied together nicely.  I can avoid order by for now though.

Let me know if you have an idea why this is happening and I can give a shot of tracking it down also.    

On Tuesday, August 12, 2014 2:52:43 PM UTC-4, Richard Whitcomb wrote:

[Andre Kelpe]

Hi Richard,

we have fixed the issue. Can you give version 1.2.0-wip-42 a try and let us know if it causes any more issues for you.

Thanks!

- André

--

You received this message because you are subscribed to the Google Groups "Lingual User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to lingual-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.