Issues making MapR (6.0) compatibility changes recommended in CDAP JIRA 13103

[rvill...@ucera.org]

Talking with MapR support about certain changes recommended for MapR 6.0 compatibility in CDAP JIRA (https://issues.cask.co/browse/CDAP-13103?focusedCommentId=31799&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-31799), they seem to warn that making some of these changes are not possible or would at least cause problems with MapR's normal operation.

For reference, the recommended changes in the CDAP JIRA are...

1. The MapR cluster should include HBase/MapR-DB Common (1.1)

2. Need to remove the property from /opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop/core-site.xml

<property>
  <name>hbase.table.namespace.mappings</name>
  <value>*:/</value>
</property>

CDAP-7613

3. Update the following property in /opt/mapr/hbase/hbase-1.1.8/conf/hbase-site.xml

<property>
  <name>mapr.hbase.default.db</name>
  <value>hbase</value>
</property>

4. Start HBase by running the following

./opt/mapr/hbase/hbase-1.1.8/bin/start-hbase.sh

My notes on these changes from speaking to MapR support are as follows:

* For #2:

This property means that we map all the tables and MapRDB tables. 
This is to specify the default location where MapRDB tables will be created (more info can be found in the docs: https://mapr.com/docs/home/UpgradeGuide/MappingTableNamespace-HBase-DBbinary.html?hl=mapping%2Ctable%2Cnamespace%2Cbetween%2Capache%2Chbase%2Ctables%2Cmapr%2Ctables).

By default, when you create/access an Apache HBase table, you just specify the table name. You do not specify the complete HDFS path to the table (and hence wyou not have "/" while specifying the table). For example:

create 'tableHBase','c'

will create an HBase table (if you have a cluster with HBase master and HBase regionserver installed).
The table will be created inside '/hbase' volume.

However, in case of MapRDB table, you can specify the complete HDFS path while creating and accessing the table. Example:

create '/tmp/maprdb/tablesample','c'

The above command will create a MapRDB table in '/tmp/maprdb/' folder. Hence the basic assumption is that if you have '/', it will be a MapRDB table and not an HBase table. This is also the case with REST calls.

The property hbase.table.namespace.mappings="*:/" is configurable as described in the documentation. Background processes will continue to work based on the configured value. However, if you have some user applications that are already using this default location, then changing this will cause issues to those specific processes.

Since from MapR 6.0, they only support MapRDB, following is default setting:
hbase.table.namespace.mappings="*:/"
Prior to MapR 6.0, if you had Apache HBase installed in the cluster, then the default location for Apache HBase will be '/hbase'.

* #3 Is a more base issue. 

Talking with MapR support about the recommended changes, they essentially say that in mapr 6.0 hbase is removed and mapr 6.0 comes with only mapr db (though the mapr-hbase binaries exist, we cannot change the mapr.hbase.default.db value to mapr.hbase.default.db=hbase). And I have been explicitly told that it is thus not possible to install CDAP.
So it appears as though the comment in the JIRA that "Since MapR dropped support for HBase in MapR 6.0, we need to add a comment in our docs that mentions we don't support MapR DB." (https://issues.cask.co/browse/CDAP-13103?focusedCommentId=32463&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-32463) what it really means it that CDAP does not support MapR 6.0+.

Would anybody be able to speak to these issues? Was really hoping to be able to try CDAP on our cluster.

Thank you

[Derek Wood]

Hi,

I would summarize as follows:

CDAP has always required HBase and not MaprDB primarily due to usage of HBase coprocessors.  When CDAP first starts up, it programmatically creates several HBase tables as in your first example (names without slashes).  The problem we had was that a default-configured MapR cluster would still create these as MaprDB tables instead of HBase tables.  The configurations described in CDAP-13103 document how we were able to force it to create HBase tables, and was used to get CDAP running on MapR 6.0 for our original certification.  It is entirely possible these settings could adversely affect other non-CDAP use-cases on a cluster.  It's also possible that there may be less forceful configurations that can provide this default behavior solely for CDAP, but i'm not familiar with any (this may be a good question for MapR support).

It's clear MapR is moving further away from HBase, and so this hasn't been an area of focus for us.  However, there is a project now underway to decouple HBase from CDAP entirely, so that may be something to watch out for.  See https://wiki.cask.co/display/CE/Decoupling+CDAP+System+Storage+from+Hadoop

HTH,

-Derek

This electronic message is intended only for the named
recipient, and may contain information that is confidential or
privileged. If you are not the intended recipient, you are
hereby notified that any disclosure, copying, distribution or
use of the contents of this message is strictly prohibited. If
you have received this message in error or are not the named
recipient, please notify us immediately by contacting the
sender at the electronic mail address noted above, and delete
and destroy all copies of this message. Thank you.

--
You received this message because you are subscribed to the Google Groups "CDAP User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cdap-user+...@googlegroups.com.
To post to this group, send email to cdap...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cdap-user/f0adbb13-67ca-4aae-a7de-60c312e6d237%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Reed Villanueva]

I see.

That helps my understanding of the current situation a bit more.

Thanks for the link.

To view this discussion on the web visit https://groups.google.com/d/msgid/cdap-user/CAFPuCGM-PxqUo0RAucgQe9x1GdtkTOWFbtY1AKR_zufiLsPOog%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.