flushCache

[Андрей Валиев]

Hello.

I'm using Hadoop 2.7(hortonworks) and Hadoop 3.3(bigtop)

I'm configured waggle-dance on two clusters hadoop.

Server waggle-dance running on Hadoop 2.7 (hortonworks) Hive version:  1.2.1000.2.6.1.0-129

Hadoop 3.3. Hive version: Hive 3.1.2

I'm connect Hive 3 on Hive metastore 1.2  - hive --hiveconf hive.metastore.uris=thrift://hostname:48869

Connecting is OK.

I get error in logs on hive 1.2  - (2022-07-20T10:11:33,285 ERROR org.apache.thrift.ProcessFunction:41 - Internal error processing get_tables_by_type

org.apache.hadoop.hive.metastore.api.NoSuchObjectException: Primary metastore does not have database @hive#default)

and when doing show databases;

get errror in logs on hive 1.2 - (2022-07-20T10:12:34,608 ERROR org.apache.hadoop.hive.metastore.RetryingHMSHandler:204 - org.apache.thrift.TApplicationException: Invalid method name: 'flushCache')

in the opposite direction, all bases are correctly displayed, when i'm connecting Hive 1.2 - Hive 3.

Help please, 

Sorry for my English :))

[Андрей Валиев]

Waggle dance version: waggle-dance-rpm-3.10.6 

среда, 20 июля 2022 г. в 10:28:47 UTC+3, Андрей Валиев:

[Mass Dosage]

I'm afraid Hive 3 is not backwards compatible with Hive 1 or Hive 2. There are actually quite a few changes required to move over to it. There is a branch and PR that have been opened as a first attempt at supporting Hive 3 which you can look at here:

https://github.com/ExpediaGroup/waggle-dance/pull/219

I haven't looked at it in a while but I think it should work for Hive 3 but won't work for earlier versions. So if you have a mix of Hive versions then WaggleDance will probably be unable to federate them. There might be some workarounds that could be done in WaggleDance to allow this to work but we haven't had the opportunity to try this out.

--
You received this message because you are subscribed to the Google Groups "Waggle Dance User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to waggle-dance-u...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/waggle-dance-user/004e2ded-6daa-4f8b-8173-be0d8886a624n%40googlegroups.com.