Parallel execution of hive queries on the Hive Editor

[Anandha Loganathan]

Hi, 

I am using Hue 3.11.  

Can we run the hive queries parallel on the single editor?   It seems on the single editor, we are able to run the queries sequentially and not parallel. 

Can we run more than one query in parallel or concurrently in a single Hive editor.

Thanks
Anand

[Romain Rigaux]

Are you using Tez?

Note about Tez:

1 2 3 4

[beeswax]  # Hue will use at most this many HiveServer2 sessions per user at a time.  # For Tez, increase the number to more if you need more than one query at the time, e.g. 2 or 3 (Tez as a maximum of 1 query by session).  max_number_of_sessions=1

http://gethue.com/hadoop-hue-3-on-hdp-installation-tutorial/

--
You received this message because you are subscribed to the Google Groups "Hue-Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hue-user+unsubscribe@cloudera.org.

[AnandaLoganathan]

Yes, we are using  Tez as a query engine. 

What is the reason for a single session for Tez query engine?

Thanks

Anand

[AnandaLoganathan]

Romain,

I am using 3.11 version and I don't see this in my config but I see this in 3.12.  

Is that mean, can we not change our config and run parallel queries?

 

https://github.com/cloudera/hue/blob/release-3.12.0/apps/beeswax/src/beeswax/conf.py

MAX_NUMBER_OF_SESSIONS = Config(
	key="max_number_of_sessions",
	help=_t("Hue will use at most this many HiveServer2 sessions per user at a time"),
	type=int,
	default=1
​​	) ​​

[Romain Rigaux]

You would need to ask the Tez people why they don't support more than one query at the time :)

Yes, you would need to upgrade or backport the commit to have it.

[Rick Bernotas]

Tez only supports running one DAG per TezSession and each HiveSession will only use one TezSession.  This is different from MapReduce, the following page gives some discussion about the differences:  https://hortonworks.com/blog/introducing-tez-sessions/

The addition of the max_number_of_sessions configuration and associated code to Hue allows for Hue to manage multiple HiveSessions.  I recommend to upgrade, we have been using this feature for a while now and it works very well if you are using Hive on Tez.