Missing File in Iceberg Table

9 views

Skip to first unread message

David Engel

unread,

Jul 13, 2025, 6:55:08 PMJul 13

to MR3

Our client system was in a bad state last night and required a hard
reboot. This appears to have resulted in a missing file in an Iceberg
table. New data was probably being inserted at the time of the
reboot. Here's a parital log from trying to query the table now.

Caused by: java.io.FileNotFoundException: File does not exist: /hivemr3/warehouse/shield.db/sflow/data/day=2025-07-11/00538-2-hive_20250713154920_2634fa29-4e3b-43bd-9d72-33767d1d7ec8-job_73511_0000-3-00001.orc

Whast is the best way to recover from this error? Will rolling back
to the previous automatic snapshot suffice?

David
--
David Engel
da...@istwok.net

Sungwoo Park

unread,

Jul 14, 2025, 3:00:33 PMJul 14

to MR3

Not sure what would be the best way to fix it. I believe it is an Iceberg problem, so perhaps the Iceberg community is a better place to ask the question.

(Although the support for Iceberg in Apache Hive is quite mature, you can see from the log of Hive commits that several bugs are still being fixed and also new features are introduced. I guess the story is the same for other engines like Spark and Trino, but it's just my guess.)