WAL corruption data

285 views
Skip to first unread message

Oliver

unread,
Feb 7, 2023, 7:36:38 AM2/7/23
to Prometheus Users
Hello,

In the last days, I get a lot of messages of:

ts=2023-02-07T07:00:06.577Z caller=compact.go:519 level=info component=tsdb msg="write block" mint=1675742400020 maxt=1675749600000 ulid=01GRNAS88AEMGDZW1H8591G88S duration=4.646391237s
ts=2023-02-07T07:00:06.882Z caller=head.go:1213 level=info component=tsdb msg="Head GC completed" caller=truncateMemory duration=227.73045ms
ts=2023-02-07T07:00:06.937Z caller=checkpoint.go:100 level=info component=tsdb msg="Creating checkpoint" from_segment=12 to_segment=14 mint=1675749600000
ts=2023-02-07T07:00:09.313Z caller=db.go:908 level=error component=tsdb msg="compaction failed" err="WAL truncation in Compact: create checkpoint: read segments: corruption in segment /prometheus/wal/00000013 at 97484800: unexpected non-zero byte in padded page"


WAL folder grows until prometheus pod is restarted and WAL folder is compacted on PVC.

Where can I search about "unexpected non-zero byte in padded page".

Best regards,
Oliver.

Reply all
Reply to author
Forward
0 new messages