Problem with collect_set()

[David Engel]

While running some comparisons between Hive on MR3 and Hadoop, I ran
into a problem with a moderately large aggregation query. I believe
I've narrowed the problem down to an issue with collect_set(). Here
are the smallest set of queries I've come up with to reproduce the
problem.

drop table if exists t ;
create table t ( k string, b binary, s string ) ;
insert into table t values
('0', unhex('00'), '0'),
('0', unhex('0001'), '0a' ),
('1', unhex('01'), '1') ;
select k, collect_set(b) as bset, collect_set(s) as sset
from t group by k ;

Attached are the resulting logs from running these right after a
restart. This is on Kubernetes using the
mr3project/hive:4.0.0.mr3.2.0 image.

David
--
David Engel
da...@istwok.net

[Sungwoo Park]

This is a problem related with Java 17, introduced in a recent commit to Hive.

Let me upload the fixed version later.

--- Sungwoo

[Sungwoo Park]

Problem fixed. I just uploaded a new Docker image: mr3project/hive:4.0.0.mr3.2.0

--- Sungwoo

--
You received this message because you are subscribed to the Google Groups "MR3" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hive-mr3+u...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/hive-mr3/590956f2-80d3-4c95-ab77-0f4322a638d0n%40googlegroups.com.

[David Engel]

I confirmed your fix is good here. Thanks!

David

> > <https://groups.google.com/d/msgid/hive-mr3/590956f2-80d3-4c95-ab77-0f4322a638d0n%40googlegroups.com?utm_medium=email&utm_source=footer>
> > .

> >
>
> --
> You received this message because you are subscribed to the Google Groups "MR3" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to hive-mr3+u...@googlegroups.com.

> To view this discussion visit https://groups.google.com/d/msgid/hive-mr3/CAKHFPXDMVXqfdd_Bb%2B7FqdNqSis-DEbu5yW0Bp2VBw3f8tbe7g%40mail.gmail.com.

--
David Engel
da...@istwok.net