Alluxio catalog sync is not deleting the older partitions

14 views
Skip to first unread message

Saroj Kumar Mishra

unread,
Feb 14, 2023, 1:22:06 PM2/14/23
to Alluxio Users
We have collocated presto and alluxio on the same cluster to get the best performance. Also we have integrated alluxio catalog service to query the data using presto.

We have attached the hiveDb into alluxio catalog and running one cron job to sync the metadata into alluxio catalog tables. The cron job is running every 5 mins to sync the metadata.

We have seen alluxio is not deleting the older partitions from its metastore even we don't have those partitions on the hive metastore table.

When we are trying to run a simple  query select * from alluxio_catalog.alluxio_cached_schema limit 10; its still pointing to older partition which is already deleted from the hive metastore table.
ERROR :
Partition location does not exist: alluxio://zk-hostname/mnt/mkt_dl/l2_layer/mkt_mvw__days/loaddate=2023-02-13/orgid=180079

Please help me to fix this issue. I am kind of blocked with this issue.

Thanks,
Saroj
Reply all
Reply to author
Forward
0 new messages