SST progress

[Graham Green]

I have a question about monitoring progress of an SST.

We're using rsync as our SST method with synchronization occurring over a WAN connection. We'd like to be able to get some visibility into where the process is at, whether that be a % of data transferred or the file currently being synced etc.

Is there something built into galera that can give us that information? If not how have others solved this problem?

Thanks in advance

Graham

[Alex Yurchenko]

On 2013-06-20 03:07, Graham Green wrote:
> I have a question about monitoring progress of an SST.
> We're using rsync as our SST method with synchronization occurring
> over a
> WAN connection. We'd like to be able to get some visibility into where
> the
> process is at, whether that be a % of data transferred or the file
> currently being synced etc.
> Is there something built into galera that can give us that
> information?

As you already indicated, you use rsync for SST. Last time I checked
rsync does not provide any progress indication. So that's the answer to
the first question.

Regards,
Alex

[Jan Kirchhoff]

Am 20.06.2013 10:20, schrieb Alex Yurchenko:
> On 2013-06-20 03:07, Graham Green wrote:
>> I have a question about monitoring progress of an SST.
>> We're using rsync as our SST method with synchronization occurring over a
>> WAN connection. We'd like to be able to get some visibility into where
>> the
>> process is at, whether that be a % of data transferred or the file
>> currently being synced etc.
>> Is there something built into galera that can give us that information?
>
> As you already indicated, you use rsync for SST. Last time I checked
> rsync does not provide any progress indication. So that's the answer to
> the first question.
>

AFAIK rsync does not know its progress itself, it checks if a file has
to be transferred not before it actually hits it when traversing trough
the directory tree.

I think you could patch the SST script and do one of these 2 things:

- run once with dry-run, collect the statistics (how many bytes/files
would have been transferred), run again without dry-run but with
--progress and calculate the progress with a simple sed/awk or perl
logic that you pipe STDOUT to.
- use "find" or something like that to find the number of files that
need to be copied before starting rsync and then simply run rsync -v,
pipe into perl (or something else), count the line numbers and output
something like "242/534 files transferred = 22%" to a log file.

any of these should be implemented within less than an hour. let me know
if you need further input of any kind.

I didn't look deeper into it as I use the xtrabackup SST.

Jan

[Graham Green]

I had a feeling that was the case, I just know if there ever comes a time when we need to do an SST in production we'll have some middle-level manager saying "Is it done yet? How long is going to take? What do you mean you don't know? Is it done yet? How about now?" ;)

[Vadim Tkachenko]

Graham,

You can do "du -sh datadir" on source node and compare wiht
"du -sh datadir" on destination node. This will give some sense of progress.

> --
> You received this message because you are subscribed to the Google Groups
> "codership" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to codership-tea...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>



--
Vadim Tkachenko, CTO, Percona
Phone +1-925-400-7377, Skype: vadimtk153
Schedule meeting: http://meetme.so/VadimTkachenko

Looking for Replication with Data Consistency?
Try Percona XtraDB Cluster!

[Graham Green]

Thanks Vadmin, I should have pointed out we had done something similar. Thank you for the suggestion.

[Raghavendra D Prabhu]

Hi,

I have created
https://bugs.launchpad.net/percona-xtradb-cluster/+bug/1193215 for this.
It shouldn't be too hard to add.

>
>




Regards,
--
Raghavendra Prabhu
Percona LLC. - http://www.percona.com / Blog: http://www.mysqlperformanceblog.com/
Skype: percona.raghavendrap
GPG: 0xD72BE977
Contact: http://wnohang.net/contact

Make plans to attend Percona Live London MySQL Conference
2013.<http://www.percona.com/live/london-2013/>

[Alain Craven]

I would like to see the results of the rsync in the mysql log.

Currently I can run

rsync -avv --progress source dest

in the CLI and see progress.

Can that not be part of the SST and results written to log?