Problem writing to HDFS - java.io.IOException: Could not rename

10 views

Skip to first unread message

Luis

unread,

Oct 18, 2022, 2:27:51 PM10/18/22

to sdc-user

Hello friends !

I have a pipeline reading from Kafka and writing on HDFS using DRIFT format, some days ago this problem start in my pipeline

Pipeline Status: RUNNING_ERROR: HADOOPFS_13 - Error while writing to HDFS: java.io.IOException: Could not rename '/user/hive/warehouse/sample.db/table_drift/year=2022/month=10/day=18/_tmp_sdc-8000-ecbf44d0-3a4e-11eb-8db0-590dab45b396_0' to '/user/hive/warehouse/sample.db/table_drift/year=2022/month=10/day=18/sdc-8000-ecbf44d0-3a4e-11eb-8db0-590dab45b396_13d345b9-2fc2-40b0-b2ef-5b4a98fd7a69'

...

Caused by: java.io.IOException: Could not rename '/user/hive/warehouse/sample.db/table_drift/year=2022/month=10/day=18/_tmp_sdc-8000-ecbf44d0-3a4e-11eb-8db0-590dab45b396_0' to '/user/hive/warehouse/sample.db/table_drift/year=2022/month=10/day=18/sdc-8000-ecbf44d0-3a4e-11eb-8db0-590dab45b396_13d345b9-2fc2-40b0-b2ef-5b4a98fd7a69'

...

Any ideas ?

Thanks for your help!

Luis

Reply all

Reply to author

Forward

0 new messages