Groups
Groups
Sign in
Groups
Groups
reair
Conversations
About
Send feedback
Help
reair
Contact owners and managers
1–28 of 28
Mark all as read
Report group
0 selected
Dayanand Lature
5/6/19
Hive Metastore hook for single insert into hive there is an two event occuring.
Hi Team, When i was tried to insert into hive table as a single record by using insert into statement
unread,
Hive Metastore hook for single insert into hive there is an two event occuring.
Hi Team, When i was tried to insert into hive table as a single record by using insert into statement
5/6/19
VG
,
Paul Yang
10
10/24/18
Incremental sync with specific set of tables
Will check, thanks On Wednesday, October 24, 2018 at 3:47:41 PM UTC-7, Paul Yang wrote: i used the
unread,
Incremental sync with specific set of tables
Will check, thanks On Wednesday, October 24, 2018 at 3:47:41 PM UTC-7, Paul Yang wrote: i used the
10/24/18
rei sivan
, …
Taikun Liu
11
8/3/18
ReAir with kerberos
Hi Rei, Sorry, we couldn't share the internal doc unless we writing a very formal public blog. On
unread,
ReAir with kerberos
Hi Rei, Sorry, we couldn't share the internal doc unless we writing a very formal public blog. On
8/3/18
loran...@gmail.com
,
Paul Yang
10
7/10/18
reair - batch
Can you elaborate on why your source / destination nameservices are identical, and what the
unread,
reair - batch
Can you elaborate on why your source / destination nameservices are identical, and what the
7/10/18
Yann Barraud
,
Paul Yang
3
4/30/18
Typical architecture to setup ReAIR
Ok so average machine in between clusters should do the trick. Thanks. On Mon, Apr 30, 2018, 21:10
unread,
Typical architecture to setup ReAIR
Ok so average machine in between clusters should do the trick. Thanks. On Mon, Apr 30, 2018, 21:10
4/30/18
Ernest Libertucci
3/26/18
Map Reduce Job fails
Kerberos enabled clusters with HA. Not sure if that makes a difference, but I'm having some
unread,
Map Reduce Job fails
Kerberos enabled clusters with HA. Not sure if that makes a difference, but I'm having some
3/26/18
Vishwanath Sharma
, …
Paul Yang
7
10/25/17
Reair is not able to delete partitions from destination side which are removed from source side.
This is unexpected. Before creating the metadata in Hive, there are verification steps that check to
unread,
Reair is not able to delete partitions from destination side which are removed from source side.
This is unexpected. Before creating the metadata in Hive, there are verification steps that check to
10/25/17
Vishal Agrawal
,
Taikun Liu
2
8/8/17
Getting Exception in argument --table-list. Its not finding that input file which I am passing to it
What is the argument you passed to table-list? Can you try file://yourlocalpath ? On Monday, August 7
unread,
Getting Exception in argument --table-list. Its not finding that input file which I am passing to it
What is the argument you passed to table-list? Can you try file://yourlocalpath ? On Monday, August 7
8/8/17
rei sivan
,
Paul Yang
3
6/26/17
ReAir doesn't copy indexes properly
The index case hasn't been tested extensively, but last time I looked at it, they were created as
unread,
ReAir doesn't copy indexes properly
The index case hasn't been tested extensively, but last time I looked at it, they were created as
6/26/17
pgo...@uber.com
,
Paul Yang
3
6/19/17
Requirement of CLIAuditLogHook
Thanks a lot Paul for this information!!! On Mon, Jun 12, 2017 at 2:04 PM, Paul Yang <paul.yang@
unread,
Requirement of CLIAuditLogHook
Thanks a lot Paul for this information!!! On Mon, Jun 12, 2017 at 2:04 PM, Paul Yang <paul.yang@
6/19/17
rei sivan
,
Paul Yang
3
6/8/17
ReAir Batch Job sync hive DWH between two cdh clusters.
Thanks for your quick reply, my bad regarding the internal_ip I should be more clear... I meant to
unread,
ReAir Batch Job sync hive DWH between two cdh clusters.
Thanks for your quick reply, my bad regarding the internal_ip I should be more clear... I meant to
6/8/17
adityak...@exadatum.com
,
Paul Yang
4
5/15/17
ReAir Batch job to sync the two warehouses for a specific window period
In that case, you can list all the partitions that should be copied over and use the batch
unread,
ReAir Batch job to sync the two warehouses for a specific window period
In that case, you can list all the partitions that should be copied over and use the batch
5/15/17
adityak...@exadatum.com
, …
Zheng Shao
5
4/10/17
How to make schema change on source reflected in the destination ?
Hi Aditya, Paul is right. Starting from the main class is the way to go. For the batch job, here is
unread,
How to make schema change on source reflected in the destination ?
Hi Aditya, Paul is right. Starting from the main class is the way to go. For the batch job, here is
4/10/17
pras.sr...@glassdoor.com
,
Paul Yang
5
4/3/17
HDFS to S3 Data movement issue
As it is now, ReAir doesn't support S3 connectivity well. Unfortunately, we don't have plans
unread,
HDFS to S3 Data movement issue
As it is now, ReAir doesn't support S3 connectivity well. Unfortunately, we don't have plans
4/3/17
pavel...@lamoda.ru
, …
Paul Yang
4
3/29/17
java8
My guess is that you'll run into compatibility issues as the code makes use of Java 8 language
unread,
java8
My guess is that you'll run into compatibility issues as the code makes use of Java 8 language
3/29/17
Zheng Shao
2
1/13/17
A set of small improvements
There are 2 more (both are very small): https://github.com/airbnb/reair/pull/57 https://github.com/
unread,
A set of small improvements
There are 2 more (both are very small): https://github.com/airbnb/reair/pull/57 https://github.com/
1/13/17
Zheng Shao
1/12/17
Make Batch replication more efficient/consistent for partition append (insert into instead of insert overwrite)
Background: Today, in Batch replication, we have 3 jobs: * In first job, copyData = whether a
unread,
Make Batch replication more efficient/consistent for partition append (insert into instead of insert overwrite)
Background: Today, in Batch replication, we have 3 jobs: * In first job, copyData = whether a
1/12/17
Zheng Shao
,
Paul Yang
9
1/12/17
Reair - master-master replication
Hi Paul, I just made the warning messages more clear for debugging. Can you take a look again? https:
unread,
Reair - master-master replication
Hi Paul, I just made the warning messages more clear for debugging. Can you take a look again? https:
1/12/17
Zheng Shao
1/5/17
Oozie integration with Reair Batch?
Hi everyone, Did anyone work on / need Oozie integration with Reair Batch mode? We are thinking about
unread,
Oozie integration with Reair Batch?
Hi everyone, Did anyone work on / need Oozie integration with Reair Batch mode? We are thinking about
1/5/17
Scott Wallace
, …
zh...@uber.com
8
1/3/17
Bulk Replication: Infinite Loop: WARN [main] com.airbnb.reair.batch.BatchUtils: Not renaming
Hi Scott, I believe this pull-request (already merged by Jingwei) would solve your problem: https://
unread,
Bulk Replication: Infinite Loop: WARN [main] com.airbnb.reair.batch.BatchUtils: Not renaming
Hi Scott, I believe this pull-request (already merged by Jingwei) would solve your problem: https://
1/3/17
Zheng Shao
,
Paul Yang
2
12/6/16
Per-Database and Per-Table Property to control the replication
Hey Zheng, We had a similar need, but only on the incremental replication side. There's an
unread,
Per-Database and Per-Table Property to control the replication
Hey Zheng, We had a similar need, but only on the incremental replication side. There's an
12/6/16
Don Krapohl
, …
Paul Yang
6
11/4/16
Bulk replication in HDFS with HA
Beautiful. We got it working about the time you posted. So others can shortcut it, to get replication
unread,
Bulk replication in HDFS with HA
Beautiful. We got it working about the time you posted. So others can shortcut it, to get replication
11/4/16
Zheng Shao
,
Paul Yang
4
11/3/16
MetastoreReplicationJob 's Step3's input
Just worked out permission issues with Jingwei. On Thu, Nov 3, 2016 at 4:53 PM, Jingwei Lu <
unread,
MetastoreReplicationJob 's Step3's input
Just worked out permission issues with Jingwei. On Thu, Nov 3, 2016 at 4:53 PM, Jingwei Lu <
11/3/16
Don Krapohl
,
Paul Yang
6
11/3/16
Does not replicate impala inserts/deletes
We've decided to just use batch replication as our max 2-hour replication load would be something
unread,
Does not replicate impala inserts/deletes
We've decided to just use batch replication as our max 2-hour replication load would be something
11/3/16
Zheng Shao
2
11/2/16
Ability to resume progress / avoid copying identical files in Batch Replication
By the way, the logic in TaskEstimator checking the whole (unpartitioned) table, and checking a
unread,
Ability to resume progress / avoid copying identical files in Batch Replication
By the way, the logic in TaskEstimator checking the whole (unpartitioned) table, and checking a
11/2/16
Zheng Shao
,
Paul Yang
2
11/2/16
How do we compare com.airbnb.reair.batch.hdfs.ReplicationJob and DistCp?
ReplicationJob was a tool that was developed to address some of the shortcomings of DistCp when
unread,
How do we compare com.airbnb.reair.batch.hdfs.ReplicationJob and DistCp?
ReplicationJob was a tool that was developed to address some of the shortcomings of DistCp when
11/2/16
Manasee Kamble
,
Paul Yang
7
10/27/16
Hive to hive copy between different HDP versions and from HDFS to S3
Unfortunately, it looks like you'll have to make some code changes to work with 1.7.0. My guess
unread,
Hive to hive copy between different HDP versions and from HDFS to S3
Unfortunately, it looks like you'll have to make some code changes to work with 1.7.0. My guess
10/27/16
Joe Meadows
,
Paul Yang
7
8/3/16
Hive metastore not copied by batch update
Pull request sent. Thanks! Joe
unread,
Hive metastore not copied by batch update
Pull request sent. Thanks! Joe
8/3/16