Evaluating hRaven

[Douglas M]

Hi,

We're looking to evaluate hRaven for resource utilization management, chargeback and reporting on several clusters running HDP 2.3 (Hadoop 2.7, YARN, TEZ, Spark, MapReduce, MR2) type workloads.

Is hRaven a suitable tool for these purposes?

is hRaven still in use at Twitter and inMobi?

If not, are there recommendations?

Many thanks!

Douglas

[Vrushali Channapattan]

Hi Douglas,

Yes, hRaven is still in use at Twitter. We use it for all that you've mentioned : application resource utilization, reporting on chargeback as well as ad-hoc reporting for several clusters. But it presently supports YARN in a limited way. For instance only mapreduce jobs on yarn are captured by hRaven. Spark job stats are captured but that is a twitter_only spark version, that code is available on the "twitter_only" branch on git for hRaven. Also, we are not super actively enhancing hRaven further.

Currently, there is active development going on at https://issues.apache.org/jira/browse/YARN-2928 to build the next generation timeline service within YARN itself, so that it can be available as part of an hadoop release and deploy.

Thanks

Vrushali

--
You received this message because you are subscribed to the Google Groups "hraven-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hraven-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.