1. I believe you are correct. After a complete restart of the
cluster, I saw the expected behavior when
mr3.container.localize.python.working.dir.unsafe is true.
2. Thanks for your .yaml example. It is similar to another example I
was trying to use but yours had a few extra things that might or might
not have been important. I also had some other issues before I
realized run-hive.sh was creating/recreating the workdir pv/pvc. It's
all working now, though, and I successfully ran a Python, transform
sctipt.
3. Our IT finally reconfigured the VM nodes for me so I can tackle the
last, main bit of my Hive/MR3 evaluation. That is trying to get
perofmance similar to that of Trino for simple queries. Our analysts
like to run simple lookups and aggregations interactively when
investigating issues. We started using PrestoDB long ago for those
due to pre-LLAP Hive being unacceptably slow.
What follows is an example of what we might like to run interactively.
The dns_ip_answer table is stored in ORC files with two levels of
partitions. The first partition is called 'day' and represents the
date the data was collected. The second partition represents the
source of the data and is not used in this sample query. For the
sample quere below, there are typically 4 to 8 distinct sources per
day and an average of about 10000000 total rows per day.
Here is the sample query I'm running.
select count(distinct ip),
sum(statistic_occurs64bit),
min(statistic_timefirstseen),
max(statistic_timelastseen)
from dns_ip_answer
where day between '2022-05-01' and '2022-05-31'
and ip between unhex('01010101') and unhex('01010101') ;
Here is a section of conf/hive-site.xml configuring the pods.
<property>
<name>hive.mr3.containergroup.scheme</name>
<value>all-in-one</value>
</property>
<property>
<name>hive.mr3.container.max.java.heap.fraction</name>
<value>0.7f</value>
</property>
<property>
<name>hive.mr3.resource.vcores.divisor</name>
<value>100</value>
</property>
<property>
<name>hive.mr3.map.task.memory.mb</name>
<value>8192</value>
<!--<value>24576</value>-->
</property>
<property>
<name>hive.mr3.map.task.vcores</name>
<value>100</value>
<!--<value>300</value>-->
</property>
<property>
<name>hive.mr3.reduce.task.memory.mb</name>
<value>8192</value>
<!--<value>24576</value>-->
</property>
<property>
<name>hive.mr3.reduce.task.vcores</name>
<value>100</value>
<!--<value>300</value>-->
</property>
<property>
<name>hive.mr3.all-in-one.containergroup.memory.mb</name>
<value>98304</value>
</property>
<property>
<name>hive.mr3.all-in-one.containergroup.vcores</name>
<value>1200</value>
</property>
With our test cluster, I can get 6 mr3worker pods running with this
configuration. In short, I have 6 nodes with about 96 GB and 12 cores
available to run mr3 workers.
With the hiveserver2, mr3master and 6 mr3worker pods already running,
the sample query consistently takes about 45-50 seconds. I've tried
all (I think) of the suggestions found at the locactions below and
none of them seemed to make any difference.
https://mr3docs.datamonad.com/docs/k8s/performance/performance-tuning/
https://mr3docs.datamonad.com/docs/k8s/performance/performance-tuning-k8s/
In contrast, when running Trino with 16 GB workers, the same query
only takes about 12 seconds with 2 workers and about 5 seconds with 6
workers. I realize the absolute difference between 5-12 to 45-50
seconds isn't all that great but the 4x to 10x slower seems to hold
for slightly more comples queries.
Is it possible to get Hive/MR3 closer to Trino for this type of query?
We could continue using Trino/PrestoDB but it would be nice if we
could settle on just Hive/MR3 and not have to maitain two sets of UDFs
and train people on the different dialects of SQL.
David
> To view this discussion on the web visit
https://groups.google.com/d/msgid/hive-mr3/26eb2321-448e-407d-b890-2f5a98bce2fdn%40googlegroups.com.
--
David Engel
da...@istwok.net