[ISSUE] (TEPHRA-55) Provide a command to manually clear IDs from invalid list

[Gary Helmling (JIRA)]

Gary Helmling commented on an issue

Re: Provide a command to manually clear IDs from invalid list A few comments on the proposal: instead of overloading the method to prune a set of IDs vs. prune by time, let's use distinct method names – both for the java methods and thrift changes I would replace "prune" with "truncate" why have the prune by time call the prune by set method? couldn't it update the list directly and more efficiently by just checking that id < pruneTxId? should we do some sanity check and not allow the pruning by time if there are any in progress transactions with id < the calculated pruneTxId? Add Comment

Tephra / TEPHRA-55 Provide a command to manually clear IDs from invalid list As a basic first step to support pruning invalid transactions from the invalid list, we can provide a command which will tell the TransactionService to clear a set of transaction IDs from the invalid list. This would support the case where an administrator has manually clear all the data for an invalid transaction, or knows that the cluster has pruned ...

This message was sent by Atlassian JIRA (v6.1.5#6160-sha1:a61a0fc)

[Poorna Chandra (JIRA)]

Poorna Chandra commented on an issue

Re: Provide a command to manually clear IDs from invalid list

Thanks for the review Gary!

instead of overloading the method to prune a set of IDs vs. prune by time, let's use distinct method names – both for the java methods and thrift changes

Agreed. I'll call them truncateInvalidTxnsBefore(long time) and truncateInvalidTxns(Set<Long> invalidTxnIds).

I would replace "prune" with "truncate"

Agreed.

why have the prune by time call the prune by set method? couldn't it update the list directly and more efficiently by just checking that id < pruneTxId?

I was planning on collecting all ids to be removed, and calling removeAll() method on the invalid list, so that one array resize operation would be performed.

should we do some sanity check and not allow the pruning by time if there are any in progress transactions with id < the calculated pruneTxId?

Good point. I'll include it in the revised proposal.

Add Comment

[Poorna Chandra (JIRA)]

Poorna Chandra edited a comment on an issue

Re: Provide a command to manually clear IDs from invalid list

Thanks for the review Gary!

* instead of overloading the method to prune a set of IDs vs. prune by time, let's use distinct method names -- both for the java methods and thrift changes ** Agreed. I'll call them {{ truncateInvalidTxnsBefore truncateInvalidTxBefore (long time)}} and {{ truncateInvalidTxns truncateInvalidTx (Set<Long>  invalidTxnIds  invalidTxIds )}}. * I would replace "prune" with "truncate" ** Agreed. * why have the prune by time call the prune by set method?  couldn't it update the list directly and more efficiently by just checking that id < pruneTxId? ** I was planning on collecting all ids to be removed, and calling {{removeAll()}} method on the invalid list, so that one array resize operation would be performed. * should we do some sanity check and not allow the pruning by time if there are any in progress transactions with id < the calculated pruneTxId? ** Good point. I'll include it in the revised proposal.

Add Comment

[Poorna Chandra (JIRA)]

Poorna Chandra updated an issue

Tephra / TEPHRA-55 Provide a command to manually clear IDs from invalid list

Attaching Truncateinvalidtransactionlist-manualapproach.pdf with changes suggested. Change By: Poorna Chandra Attachment: Truncateinvalidtransactionlist-manualapproach.pdf

Add Comment

[Gary Helmling (JIRA)]

Gary Helmling commented on an issue

Re: Provide a command to manually clear IDs from invalid list

why have the prune by time call the prune by set method? couldn't it update the list directly and more efficiently by just checking that id < pruneTxId?

I was planning on collecting all ids to be removed, and calling removeAll() method on the invalid list, so that one array resize operation would be performed.

Ok, sounds good. New design doc looks good to me.

Add Comment

Tephra / TEPHRA-55 Provide a command to manually clear IDs from invalid list

As a basic first step to support pruning invalid transactions from the invalid list, we can provide a command which will tell the TransactionService to clear a set of transaction IDs from the invalid list. This would support the case where an administrator has manually clear all the data for an invalid transaction, or knows that the cluster has pruned ...

[Poorna Chandra (JIRA)]

Poorna Chandra assigned an issue to Poorna Chandra

Tephra / TEPHRA-55 Provide a command to manually clear IDs from invalid list

Change By: Poorna Chandra Assignee: Gary Helmling Poorna Chandra

Add Comment

[Poorna Chandra (JIRA)]

Poorna Chandra commented on an issue

Re: Provide a command to manually clear IDs from invalid list PR https://github.com/caskdata/tephra/pull/36

Add Comment

Tephra / TEPHRA-55 Provide a command to manually clear IDs from invalid list

As a basic first step to support pruning invalid transactions from the invalid list, we can provide a command which will tell the TransactionService to clear a set of transaction IDs from the invalid list. This would support the case where an administrator has manually clear all the data for an invalid transaction, or knows that the cluster has pruned ...

[Poorna Chandra (JIRA)]

Poorna Chandra updated an issue

Tephra / TEPHRA-55 Provide a command to manually clear IDs from invalid list

Change By: Poorna Chandra Fix Version/s: 0.3.5

Add Comment

[Poorna Chandra (JIRA)]

Poorna Chandra updated an issue

Tephra / TEPHRA-55 Provide a command to manually clear IDs from invalid list

Change By: Poorna Chandra Affects Version/s: 0.3.4

Add Comment

[Gary Helmling (JIRA)]

Gary Helmling resolved an issue as Fixed

This has been merged

Tephra / TEPHRA-55 Provide a command to manually clear IDs from invalid list

Change By: Gary Helmling Status: In Progress Resolved Resolution: Fixed

Add Comment