Speeding HDR ontape archive-restore

[red_valsen]

I am streaming an ontape archive from an HDR primary server over my
network and restoring it to a secondary server to start replication.
Do the onconfig file's TAPE-related parameters affect the speed of
the
transfer even though a physical tape isn't in use?

Are there any other steps I can take to improve the performance of the
archive-restore?

Using HPUX 11.23 and IDS 11.50.FC3.

[Art Kagel]

Block size seems to have an effect on archive/restore performance even to/from files, though obviously not as severe as when performing IO to/from tape.  The ideal block size will depend on how the filesystem the files lives on was built.  Matching the stripe size of the filesystem would be ideal.

Art

Art S. Kagel
Advanced DataTools (www.advancedatatools.com)
IIUG Board of Directors (a...@iiug.org)
Blog: http://informix-myview.blogspot.com/

Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Advanced DataTools, the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference.  Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves.

_______________________________________________
Informix-list mailing list
Inform...@iiug.org
http://www.iiug.org/mailman/listinfo/informix-list

[Cesar Inacio Martins]

Maybe this isn't valid for your environment...

What I already used to improve performance over archive/restore to
configure HDR/RSS and I get great improve over total execution time (30
minutes to 7 minutes), is using the pigz utility, where it will
parallelize your CPU , force more I/O throughput (ontape reading) and
will optimize your network transfer.

http://www.zlib.net/pigz/

- It isn't an Informix utility
- I don't know if is easy to compile it for HP-UX (I used it only with
Linux)
- Will help only if you have multicore machine and great I/O throughput.

Regards
Cesar

[Cesar Inacio Martins]

I forgot, the command will be something like:

ontape -s -d -v -L 0 -t STDIO | pgiz -1c | \
ssh informix@otherserver ". myenvs.sh ; pigz -cd | ontape -p -t STDIO"

[mos...@wellsfargo.com]

> I forgot, the command will be something like:
>
> ontape -s -d -v -L 0 -t STDIO | pgiz -1c | \
> ssh informix@otherserver ". myenvs.sh ; pigz -cd | ontape -p -t
> STDIO"
>
>
> On 12/08/2010 11:07 AM, Cesar Inacio Martins wrote:

Along the same lines, but maybe easier:
I tried using STDIO and piping the output across the network to an "ontape -p" process on the other server, but found that our old way of doing the transfer was still much faster overall. That process involved breaking the archive into smaller disk files (for historical reasons, we were using 2GB for TAPESIZE). As each piece was completed, it was sent off to a background script to be compressed and then sent via 'scp' over to the other server. Thus, as the archive progresses, there can be several compressed pieces being sent over the network in parallel. Also, having the archive write directly to disk files meant that the archive itself was not being slowed down by waiting for network throughput. Now, of course, YMMV, depending on server horsepower (the compresses use a lot of CPU), I/O capacity and throughput, and mainly network bandwidth. But, for us, it really paid off to parallelize the file transfers over the network.

HTH,
Paul Mosser

[Neil Truby]

>> "Art Kagel" <art....@gmail.com> wrote in message
>> news:mailman.677.12917493...@iiug.org...

Block size seems to have an effect on archive/restore performance even
to/from files, though obviously not as severe as when performing IO to/from
tape. The ideal block size will depend on how the filesystem the files
lives on was built. Matching the stripe size of the filesystem would be
ideal.

I don't know the answer to this, so it's a genuine question: might not the
outbound block size have at least as much impact on the network performance
as the stripe size for restore ...?

[Art Kagel]

If you are backing up over the network, then yes, blocksize should be coordinated with the system's network drivers.  IP packets can be from 2K to 32K but most drivers can't handle 32K packets and break them up into smaller packets.

That said, the larger the actual packets that are sent are the less waste the IP and TCP packet headers will represent and the better the throughput.

Art

Art S. Kagel
Advanced DataTools (www.advancedatatools.com)
IIUG Board of Directors (a...@iiug.org)
Blog: http://informix-myview.blogspot.com/

Disclaimer: Please keep in mind that my own opinions are my own opinions and do not reflect on my employer, Advanced DataTools, the IIUG, nor any other organization with which I am associated either explicitly, implicitly, or by inference.  Neither do those opinions reflect those of other individuals affiliated with any entity with which I am affiliated nor those of the entities themselves.