Hive-MR3 in HiDataPlus.
Sungwoo Park
https://mp.weixin.qq.com/s/65bgrnFpXtORlb4FjlPMWA
English translation:
Before, we released HDP 3.3.2.0-004 (x86_64), and since the changes were not significant, we did not notify. This time, version 005 has some more substantial changes, so we need to spread the word (^_^).
I. Major Changes:
1.1 New Components
Alluxio: Serving as a data orchestration layer, it brings speed, agility, and cost reduction to big data and AI workloads, enabling users to migrate to updated storage solutions like object storage. In simpler terms, we believe it mainly involves data caching and a unified directory function, with corresponding commercial versions.
Celeborn: Intermediate data service for big data computing engines, a universal Remote Shuffle Service aimed at improving the performance, stability, and elasticity of big data computing engines.
Gluten: Empowers Spark engine execution with a Native Vectorized engine, doubling the speed of Spark SQL.
MR3: Hive on MR3, the best way to run Apache Hive in production. It's not open-source software, but it's said to replace Hive directly, significantly improving performance. It can be used for free, but computing resources are limited to below 5T. Those interested can research on the official website: https://www.datamonad.com.
*** Several newly added components might be unfamiliar to many, and their usage is not straightforward. We will write topics later to introduce them.
1.2 Bug Fixes
- Ambari Metrics Collector out log's unlimited growth issue.
II. Current Component Versions
III. Operating System Correspondence:
IV. Download Method:
Cloud directory:
x86_64 version: HiDataPlus/HDP3.3.2/x86_64/3.3.2.0-005/
aarch64 version: It will definitely be available before the 2024 Spring Festival.
Download Method: Follow HiDataPlus public account.
Leave a message "download" to get the latest cloud link. When a new version is released, we will also notify through the public account.
Finally, we hope everyone can help promote (post on social media, etc.). Since we have limited experience with the newly added components, if there are any issues during usage, please feel free to message us. If it causes any inconvenience, we ask for your understanding.
If any companies are interested in cooperation, feel free to add our WeChat: HiDataPlus.