Any way to set TTL per row for KeyValueTable

[Tongqing Qiu]

First, is TTL per database supported for KeyValueTablet? Second, any way to set TTL per row for KeyVaueTable now or on the roadmap? 

I know it is based on HBase, and hbase only has cf TTL support. 

Any suggest to implement ttl per (key,value) pair?

Thanks,

Tongqing

[Gary Helmling]

Hi Tongqing,

No, we do not currently support per row TTL for any datasets.  TTL can only be set at the dataset level.

HBase has very recently added support for per-cell TTLs (see http://hbase.apache.org/book/ttl.html and https://issues.apache.org/jira/browse/HBASE-10560), but we do not yet support this. I added https://issues.cask.co/browse/CDAP-1108 to cover support for per-cell TTLs, but it would require work in both CDAP and Tephra to support, and even then would only work with HBase 0.98+.

In the meantime, you could implement your own version of KeyValueTable, which, in addition to storing the value in a column, would store an additional column (say "ttl"), with the TTL to apply for a given row ("key" + "value").  When reading data out, you could consult the stored TTL to see whether or not the given value should be returned.  This would filter "expired" data on read, but you would still need to implement a background job to actually cleanup the TTL'd data.  You could do this by setting up a CDAP workflow that runs a map reduce job periodically to scan the full table and delete any data that has exceeded its configured TTL.

Gary

--
You received this message because you are subscribed to the Google Groups "CDAP User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cdap-user+...@googlegroups.com.
To post to this group, send email to cdap...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/cdap-user/638f8709-221f-48a5-83e8-3f6a0ac178b0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.