Per-Database and Per-Table Property to control the replication

[Zheng Shao]

We have a strong requirement for Hive replication: To let users decide what database/tables to replicate (vs not - for example, temporary database/tables).

In order to fulfill that requirement, I am planning to utilize DBProperty and TBLProperty.

So the following Hive commands will enable the replication:

ALTER DATABASE <db> SET DBPROPERTY ("replication" = "true");

ALTER TABLE <table> SET TBLPROPERTY ("replication" = "true");

One possibility is to add the logic to TableCompareWorker.processTable().

Is this something that the Reair project will take on?  Anybody else needs this?

--

Zheng

[Paul Yang]

Hey Zheng,

We had a similar need, but only on the incremental replication side. There's an interface 

https://github.com/airbnb/reair/blob/61d433de146c4e9f04fa324bb3b8794c0ad42911/main/src/main/java/com/airbnb/reair/incremental/filter/ReplicationFilter.java

for users to create custom filtering classes and configuration options for the user to specify the class name. I think a similar option would be useful to have in batch replication.

Cheers,

Paul

--
You received this message because you are subscribed to the Google Groups "reair" group.
To unsubscribe from this group and stop receiving emails from it, send an email to airbnb-reair+unsubscribe@googlegroups.com.
To post to this group, send email to airbnb...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/airbnb-reair/CAAguJ7qV%3DWYRFWFxkJ9GpdRKVn9AgQUJLGQk8pkwzJUgNRUYkA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.