HBaseAdmin1_0 class may implement HBase 2.x Admin interface, leading to NoClassDefFoundError at runtime

[ssal...@twitter.com]

Hey y'all,

I've been experimenting with the usage of JanusGraph on top of Google Cloud Bigtable over the past few, and I've managed to get it working with the following set of configuration options:

storage.backend=hbase
storage.hbase.ext.hbase.client.connection.impl=com.google.cloud.bigtable.hbase2_x.BigtableConnection
storage.hbase.ext.google.bigtable.project.id=<Google Cloud Platform project id>
storage.hbase.ext.google.bigtable.instance.id=<Bigtable instance id>

However, when I set these configuration values to use an HBase 1.x-style Bigtable connection instead, as documented here: 

https://docs.janusgraph.org/storage-backend/bigtable/

I end up receiving the following exception at runtime:

java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/client/TableDescriptor
	at org.janusgraph.diskstorage.hbase.HConnection1_0.getAdmin(HConnection1_0.java:43)
	at org.janusgraph.diskstorage.hbase.HBaseStoreManager.getAdminInterface(HBaseStoreManager.java:978)
	at org.janusgraph.diskstorage.hbase.HBaseStoreManager.ensureTableExists(HBaseStoreManager.java:719)
	at org.janusgraph.diskstorage.hbase.HBaseStoreManager.getLocalKeyPartition(HBaseStoreManager.java:537)
	at org.janusgraph.diskstorage.hbase.HBaseStoreManager.getDeployment(HBaseStoreManager.java:376)
	at org.janusgraph.diskstorage.hbase.HBaseStoreManager.getFeatures(HBaseStoreManager.java:418)
	at org.janusgraph.graphdb.configuration.builder.GraphDatabaseConfigurationBuilder.build(GraphDatabaseConfigurationBuilder.java:51)
	at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:161)
	at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:132)
	at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:122)
	at org.janusgraph.core.JanusGraphFactory$Builder.open(JanusGraphFactory.java:261)

Delving into the problem further, I found that this problem can be reproduced by simply constructing the org.janusgraph.diskstorage.hbase.HBaseAdmin1_0 class with null in place of an instantiated Admin class. Viewing the respective class bytecode via javap -c -p HBaseAdmin1_0.class, it appears that some of the methods on this class may be linked against an HBase 2.x interface at compile time:

  public void createTable(org.apache.hadoop.hbase.HTableDescriptor) throws java.io.IOException;
    Code:
       0: aload_0
       1: getfield      #2                  // Field adm:Lorg/apache/hadoop/hbase/client/Admin;
       4: aload_1
       5: invokeinterface #42,  2           // InterfaceMethod org/apache/hadoop/hbase/client/Admin.createTable:(Lorg/apache/hadoop/hbase/client/TableDescriptor;)V
      10: return

  public void createTable(org.apache.hadoop.hbase.HTableDescriptor, byte[], byte[], int) throws java.io.IOException;
    Code:
       0: aload_0
       1: getfield      #2                  // Field adm:Lorg/apache/hadoop/hbase/client/Admin;
       4: aload_1
       5: aload_2
       6: aload_3
       7: iload         4
       9: invokeinterface #43,  5           // InterfaceMethod org/apache/hadoop/hbase/client/Admin.createTable:(Lorg/apache/hadoop/hbase/client/TableDescriptor;[B[BI)V
      14: return

This looks like it may be a bug and I'm investigating a fix at the moment, as when I run javap against the HBaseAdmin1_0.class built from the current JanusGraph master, its interface invocations match that from above.

I was wondering if y'all would like me to file an issue for this problem, and if anyone else here on the list has run into similar issues.

Thanks much for the consideration and all the best!

[ssal...@twitter.com]

Upon further investigation, it looks like this issue can be worked around at the moment by building JanusGraph with the following flags:

-Dhbase.profile -Phbase1

It looks like the Admin class expected at HBaseAdmin1_0 class initialization time, when using the hbase2 Maven profile, is that from the HBase 2.x library, which includes dependencies on 2.x classes such as TableDescriptor, leading to the problem I ran into. Reading over the release notes, I think this is likely not a bug, but I'd be glad to drop a quick PR to update the docs.