Getting the following error in EMR

[Pravanjan Choudhury]

System - 

EMR - hadoop 1.03 hbase 0.92.1

Crux - Aggregation build with hbase0.92 target

Tomcat throws the following error - 

2013-03-08 14:56:56,440 DEBUG freemarker.beans.debug:81 - Key 'location' was not found on instance of java.lang.reflect.InvocationTargetException. Introspection information for the class is: {getClass=public final native java.lang.Class java.lang.Object.getClass(), localizedMessage=java.beans.PropertyDescriptor@cca3bf96, cause=java.beans.PropertyDescriptor@b3fc2a0, getCause=public java.lang.Throwable java.lang.reflect.InvocationTargetException.getCause(), getMessage=public java.lang.String java.lang.Throwable.getMessage(), equals=public boolean java.lang.Object.equals(java.lang.Object), java.lang.Object@5fa6fb3e=freemarker.ext.beans.MethodMap@70e69696, hashCode=public native int java.lang.Object.hashCode(), class=java.beans.PropertyDescriptor@af7b4974, fillInStackTrace=public synchronized native java.lang.Throwable java.lang.Throwable.fillInStackTrace(), targetException=java.beans.PropertyDescriptor@d5d19429, stackTrace=java.beans.PropertyDescriptor@39ce5b2b, message=java.beans.PropertyDescriptor@33b9608f, java.lang.Object@64f007ad={public void java.lang.Throwable.setStackTrace(java.lang.StackTraceElement[])=[Ljava.lang.Class;@576f8789, public boolean java.lang.Object.equals(java.lang.Object)=[Ljava.lang.Class;@5b2b6037, public java.lang.Throwable java.lang.reflect.InvocationTargetException.getCause()=[Ljava.lang.Class;@7cb9e9a3, public java.lang.StackTraceElement[] java.lang.Throwable.getStackTrace()=[Ljava.lang.Class;@119a0c4e, public java.lang.String java.lang.Throwable.getMessage()=[Ljava.lang.Class;@189ddde9, public final native java.lang.Class java.lang.Object.getClass()=[Ljava.lang.Class;@2453f89f, public java.lang.String java.lang.Throwable.getLocalizedMessage()=[Ljava.lang.Class;@4e7a15b, public synchronized native java.lang.Throwable java.lang.Throwable.fillInStackTrace()=[Ljava.lang.Class;@3125fe1, public synchronized java.lang.Throwable java.lang.Throwable.initCause(java.lang.Throwable)=[Ljava.lang.Class;@2b54ae5a, public native int java.lang.Object.hashCode()=[Ljava.lang.Class;@4482acef, public java.lang.Throwable java.lang.reflect.InvocationTargetException.getTargetException()=[Ljava.lang.Class;@5f27a20c, public java.lang.String java.lang.Throwable.toString()=[Ljava.lang.Class;@22fff2a2}, getStackTrace=public java.lang.StackTraceElement[] java.lang.Throwable.getStackTrace(), setStackTrace=public void java.lang.Throwable.setStackTrace(java.lang.StackTraceElement[]), getTargetException=public java.lang.Throwable java.lang.reflect.InvocationTargetException.getTargetException(), toString=public java.lang.String java.lang.Throwable.toString(), initCause=public synchronized java.lang.Throwable java.lang.Throwable.initCause(java.lang.Throwable), printStackTrace=freemarker.ext.beans.MethodMap@51927ba1, getLocalizedMessage=public java.lang.String java.lang.Throwable.getLocalizedMessage()}

Any pointers? A great project btw.

[Sonal Goyal]

Did you build yourself? What version of tomcat are you on? 

Best Regards,
Sonal
Real Time Analytics for BigData
Nube Technologies

--
You received this message because you are subscribed to the Google Groups "cruxUsers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cruxUsers+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Pravanjan Choudhury]

Tried both on 7.0.14 and 7.0.23. Anyways, the cdh4 fresh compile did work. But have some other problem now. When I put an equal clause on my composite query (orgid, userid), it works. However, when I put a equal on the first id and put filters on columns (essentially when it initializes a scanner), the regionserver lease is expiring after sometime.

Is there anything specially to be done for emr hbase 0.92.1 ? 

Pravanjan

[Sonal Goyal]

We havent tested with EMR, but I dont think there should be any difference, unless AWS has some different hbase build of configs. What do the Crux tomcat logs say? Did you try with a start and stop scanner (= and >= on composite key and = and < on composite key)? Basically with start and stop keys ?

[Pravanjan Choudhury]

Actually no.. I thought it will do automatically if I put an equal to on the first part. Will try it out and let you know. BTW, I didn't see the sum, round feature etc. When I click on any variable on the report, the functions tab says none. Anything I need to do to enable it ?

Best,
Pravanjan

[Pravanjan Choudhury]

Initially it didn't work with 

diff --git a/src/main/java/co/nubetech/crux/server/QueryExecutor.java b/src/main/java/co/nubetech/crux/server/QueryExecutor.java

index 0266636..fce3e92 100755

--- a/src/main/java/co/nubetech/crux/server/QueryExecutor.java

+++ b/src/main/java/co/nubetech/crux/server/QueryExecutor.java

@@ -79,11 +79,12 @@ public class QueryExecutor {

                                RangeFilters rangeFilters = getRangeFilters(report, mapping);

                                setRangeScan(scan, mapping, rangeFilters);

                                FilterList columnFilters = HBaseFilterFactory.getColumnFilters(report);

-                               FilterList rowFilters = HBaseFilterFactory.getRowFilters(report, mapping, rangeFilters);

+/*                             FilterList rowFilters = HBaseFilterFactory.getRowFilters(report, mapping, rangeFilters);

                                for (Filter columnFilter: columnFilters.getFilters()) {

                                        rowFilters.addFilter(columnFilter);

                                }

-                               scan.setFilter(rowFilters);

+                               scan.setFilter(rowFilters);*/

The above fixed it by removing rowfilter in the scan. I am not sure it is the right way but it was adding ROW FILTER along with startrow and endrow..

{"timeRange":[0,9223372036854775807],"batch":-1,"startRow":"\\x00\\x00\\x00\\x92\\x00Ny\\xAE","stopRow":"\\x00\\x00\\x00\\x92\\x00Ny\\xBA","totalColumns":2,"cacheBlocks":true,"families":{"identity":["firstname","mobile"]},"maxVersions":1,"filter":"FilterList AND (1/1): [RowFilter (EQUAL, \\x00\\x00\\x00\\x92)]","caching":-1}

Removing the row filter worked fine.. i have not tested other things though.. just got it to work somehow. 

[Sonal Goyal]

Sorry, can you please explain what you are doing? Are you doing a range scan or just providing a start key? 

[Pravanjan Choudhury]

Just providing the start key was working since the start. The range scan wasn't working earlier. The final scan object was something like this - 

{"timeRange":[0,9223372036854775807],"batch":-1,"startRow":"\\x00\\x00\\x00\\x92\\x00Ny\\xAE","stopRow":"\\x00\\x00\\x00\\x92\\x00Ny\\xBA","totalColumns":2,"cacheBlocks":true,"families":{"identity":["firstname","mobile"]},"maxVersions":1,"filter":"FilterList AND (1/1): [RowFilter (EQUAL, \\x00\\x00\\x00\\x92)]","caching":-1}

The start key and end key was proper but there was additional RowFilter EQUAL (probably on the first of the two composite key) due to which no result was being returned. This was on latest cloudera hbase 0.94.2. 

[Sonal Goyal]

Thanks for the explanation. Could you please file a bug?

[Pravanjan Choudhury]

Will do. One more clarification - Is the coprocessor feature of crux available for Amazon EMR hbase - The version is 0.92.0 there. Can you please let me know what needs to be done for enabling aggregation for any hbase cluster?