Joining two tables in different metastores

26 views
Skip to first unread message

KB

unread,
Jun 5, 2019, 4:51:49 AM6/5/19
to Waggle Dance User
Hi,

Where does the join happen when joining two tables which reside in different hive instances? Is there a way to route join to a specific hive instance?

Patrick Duin

unread,
Jun 5, 2019, 5:56:06 AM6/5/19
to Waggle Dance User
Hi,

WD only proxies metadata, data is process in the cluster that fires the query.

Example:
I spin up an EMR cluster that I point to WD that federates across two metastores. The EMR cluster will be where all the data is crunched. So it needs to have access to HDFS or s3 buckets depending where your data is coming from.

Hope that helps,
 Patrick

kbee...@gmail.com

unread,
Jun 5, 2019, 7:51:45 AM6/5/19
to Waggle Dance User
Got it, thanks!

--
KB
Reply all
Reply to author
Forward
0 new messages