|
HBASE-12859 adds reporting for the last (oldest) major compaction timestamp per region and per table. We can build on top of this to determine when we can know that all potential writes for a given failed or invalidated transaction have been cleared.
As a prerequisite, we will need to change TransactionProcessor so that if fails compaction attempts if it is unable to load a transaction state snapshot or if the loaded snapshot is too old. This gives us a deterministic boundary for the possible invalid transactions that would have been cleared.
With this in place, we should be able to:
-
query the oldest major compaction timestamp for all tables with transactions enabled
-
drop any entries from the invalid list older than: oldest timestamp - transaction snapshot loading window - transaction timeout.
This will work for "short" transactions at least, which are subject to timeouts. "Long" transactions, however, will still cause a problem for this, as they do not time out. One possible solution would be to have a separate timeout for long transactions, for example 1 day, and use that as the transaction timeout in the calculation.
|
Major
